The main difference between a society with slaves to a slave society is what role the selling and use of slaves has on the economy.
The United States shifted between the two when the selling of slaves became an economic driver onto itself, instead of just a source of "free" labor.