It doesn't NEED anything to keep it moving 'sideways'.
Remember Newton's first law: An object maintains constant uniform motion UNLESS there's a force to make it slow down or turn.
In space, gravity is usually the only force acting on anything, and gravity acts toward the other object, not sideways.