the relevant physics you should be reading about is called the conservation of momentum...
this means that in any collision (in this case, the collision of the firing mechanism of the rifle with the bullet), the total amount of what we call momentum is the same before and after the collision
this is one of the foundations of all physics understanding...
so, what is momentum...it is the massxvelocity of an object
what is the momentum of the bullet before it is fired...zero...and, the conservation of momentum tells you that the total momentum will stay zero AFTER collision...
ok, how can this be? the bullet will come out a high (very high) speeds, and so it will carry lots of momentum with it...
true, but the "kickback" of the rifle carries momentum in the opposite direction, and...it turns out the rifle has exactly the same momentum backward as the bullet does forward, so the total amount adds to zero
this gets us to what determines the "kick" in the kickback...let's say the bullet travels off with a speed v and a small mass m, its momentum is mv
the rifle kicks back with the same momentum, and let's call its speed V and its mass M
we have that MV=mv or, with a little division
V=mv/M
we see that the bigger we make the mass of the rifle, the smaller the speed it has in kicking back...and this is what you are experiencing...a real life version of conservation of momentum
it has to do with the mass of the gun, not the pull of gravity on the gun