It was mostly about wealth. Without slavery, the southern prosperity would end. The unionists wanted to remain in the union but were not completely prepared to abolish slavery, while the secessionists demanded a clear separation from the union and an establishment of an independent confederation of southern states if others were willing to join them. Not only would they lose wealth, but also political power because the state would become a lot weaker. They would lose much more than they gained in both ways so it was problematic.