No. That isn't specifically how time travel works.
When you time travel, you are taking your body from your current time and traveling through universal zones to go back to the specific time you wanted. You would just be absent from the present time as if you were at home, or something.
You would go back into time and you would have to be able to keep yourself hidden enough, because if anyone recognized you (such as family), it would alter time itself.
Along with, if you killed your grandfather then your parent (from such grandfather), may or may not exist, which could alter your own existence, changing time and space itself, altering the universe itself.
So no, it would not be an infinite loop. You would just be absent from your present time until you decided to return back to it.