No, you do not have to own stocks yourself to be impacted by the change of the markets. Anybody who owns stocks AND run businesses that YOU go too will impact YOU dramatically. If stock prices drop, the amount of money they have will drop considerably, which means they have less money for merchandise. If they don't have merchandise, the businesses will go out, and you will not have anyplace to go too for your needs (for food, medicine, etc)
hope this helps