There are many reasons. For example, he's a shady guy who probably deals in criminal activities because he works together with Gatsby who was a bootlegger. It is also stated that he fixed the state series meaning he also did something illegal. In addition, people like Tom dislike him because of his religion since there is a wave of Americans who were against Jewish people at the time.