It's great that you're using a thought experiment, they are the sort of things that geniuses like Einstein often used. However, as has already been pointed out, your experiment seems impractical. And if you imagined a bullet that was not affected by matter or any forces in the universe, well, it would be like it's not part of the universe at all.
Still, you imaginary bullet was not the key idea, what I think you're really interested in is if the universe has an end, and if so, what is after it's end.
These are great questions, and astronomers have studied this for years. The theory that currently seems most likely based on our understanding of physics and the measurements we have taken, is that the universe is infinite and has no end. (Note comparing the distance of two objects as far apart from each other as we can see in our best telescopes is a distance that really is *not* the same as the size of the universe.) Our best measurements are supporting the idea that the universe is not curved (though space-time is of course curved locally near massive objects like stars).
There are other theories about what the universe is like, one is a somewhat saddle shaped universe. While that model of the universe is curved, it is both infinite and unbounded. The other one is like a sphere (or as already pointed out, more like the surface of a hypersphere). This is the classic "balloon" idea of the universe many people are exposed to. It can be blown up showing it is expanding. But if you are trapped inside the surface of the balloon, if you moved in a straight line you would eventually come back to where you started. This model seems less likely based on astronomical observations made in the last decade.
So what's past all of it? Well, maybe there is nothing past it if it's truly infinite as most astronomers currently believe.
In principle, your bullet could keep going forever. In practice, it would likely run into something or come close enough to something that it's course would be altered even if it wasn't stopped.
But if the bullet kept going, it would just keep going without end.
If space is infinite, why do you think there has to be something surrounding it? Just imagine a Cartesian coordinate system. If you write it on paper, it is of course limited by the size of your paper. But as you know, such a system is meant to go on forever, and you're just modeling a small section of the overall with what you have written.
Or there maybe could be something around it. Imagine for a moment that our universe is two dimensional, a bit like a piece of paper, except the page has infinite height and width, but no thickness. Then you could imagine a whole set of other universes the same way, and have something like a book of these pages. Each page is surrounded by other pages, other universes. While not using this model, there are theories like the many worlds theory which could mean there are other universes out there.
You ask how can space be infinite? I ask why couldn't it be infinite? I mean we don't have a lot of other universes that we have seen and can compare ours too. We're in this universe, that's all we know about. Infinite may not seem intuitive to you since pretty much everything around us seems finite. But numbers are infinite, curves on graphs may be infinite, you even have an infinite number of points on a line segment of finite length. Infinity exists in mathematics, and most of our theories about the physical laws of the universe give us mathematical equations that describe things. So really, why couldn't the universe be infinite?
In my sources, I am including a few links you might find interesting. The first two are about the shape of the universe, while the last one is an old book that may help you understand three dimensional space better by the way it looks at a fictional two dimensional universe.