Plymouth is located in Massachusetts.
The pilgrims originally wanted to settle in Virginia, not Massachusetts. They ended up getting in a bad storm on the way to America and landed on Cape Cod. They then spent the rest of the winter on the Mayflower, as it was too cold to do anything productive.
The Atlantic Ocean is to the east of New England.
The ocean that borders New England is the Atlantic Ocean, specifically the Northern Atlantic. It is the second-largest ocean in the world.