Northern workers were opposed to the abolition of slavery because of the founded fear that free slaves could take their jobs. They knew that slaves will work for free so, they represented an economic threat to them. The shared believe was that most probably managers were going to hire these slaves to take up their jobs.
It is easy to imagine the consternation among the free workers of the north, to see their livelihood being taking away by the arrival of these new free slaves if the abolition of slavery when through.
Also, the release of these slaves presented a reduction of status for white workers who would have to compete with free slaves that probably were willing to work for less money. We can see now, that the concern was real for northern workers due to the increase competition that they were about to face.