this is a case of impact force. it is complicated to go into detail but lets see the basics.
the time of impact is usually very small say 0.02 seconds . depends on other factors
the brick falls on the man
now the impact force is given by impact force * time of collision = change in momentum
impact force = mv/impact time assuming the man absorbs all the enrgy and the surface of the man is not elastic for the brick to bounce up
so the impact force depends on the velocity of the brick when it hits the man
so if you drop it from a higher building, impact force will be greater since it hits with a higher velocity
similarly a heavier stone will also lead to a greater impact force.
increasing number of bricks will cause 4 different impacts on him and it may cause him more pain
now imagine you throw something which bounces off his head the change in momentum will be higher since it will be mv - (-mv1) since it bounces off the surface in the opposite direction
nowimagine the man and brick do not bounce, but he is moving up with a velocity and brick is falling down with a velocity, again the impact force will be higher. because the relative velocity is higher
the mass of the man will not affect the impact force i think
there is more
hope this helps,