Answer:
This happened more than likely due to the fact that although these men were free, black people were still seen as less than human even in parts of the northern U.S. This resulted in what we know as Jim Crowe laws later in the late 1800s and early 1900s.