Do whatever you want to do! Nobody should ever dictate where you want your life to go- that's for you and you alone to decide. Whether or not you do become a pescatarian, life will always have things that people like or dislike. We're all different, so one point of view may be on the other side of the spectrum compared to another person's point of view.
Hope you decide and live your life to the fullest either way! =)