Imagine you stand on the surface of a sphere. It has no water, is perfectly smooth, and no magnetic field. Let's let it rotate like the Earth (for "poles" and "equator"). We'll place you north of the equator, a line we draw on the surface appropriately.
You start walking "due east", and without any course corrections, you will cross the equator twice before ending up back where you started. So the "equator" is like the center-of-mass of the Sun, and you are like Earth. You "orbit" the center of mass of the Sun, without the action of a force (neglecting the one that holds you to the surface of the sphere).
The rubber sheet analogy is supposed to get you thinking about physical mechanisms, and really, does more harm than good. Spacetime is not "stuff".
"Curvature" is supposed to describe the force free path of either light or matter, that deviates from "flat" / Euclidian space. It describes the motions we see around us, without the action-at-a-distance that Newton did not like in his theory of gravitation.
I doubt that this will give you too much more insight, but here goes:
http://www.physics.adelaide.edu.au/~dkoks/Faq/Relativity/GR/gravity.html
Cross tying to duplicate question:
https://answersrip.com/question/index?qid=20110211193411AAZEk2d