Factory owners could set the terms for work because there was a surplus of labor.
When more people work at a factory, it is easier for the owner(s) to dictate payments, working conditions, equity distribution, and much more. This cannot be done with little amounts of workers because it would be recognized easily.