Popular views of property rights and marriage created a situation in which women and slaves were not seen as truly independent people. Therefore, they were not really party to the social contract laid out in the Declaration of Independence.
The idea of the Declaration of Independence is that all white men are created equal. These are people who are responsible for themselves. They can enter into a social contract with the government because they are free agents. Therefore, they are to enjoy all of the freedoms set out in the Declaration.