Food - Without an efficient way of producing and gathering food in a way that actively keeps up with the population demand, would cause a difficulty in supporting the population, thus stopping it from increasing.
Weather or environment - Harsh weather conditions can diminish basic needs like maintaining a warm, dry shelter or producing crops without them being destroyed for example.
Some other limiting factors would be Diseases or predators and the ability to fight them off, fresh water sources, warmth, temperatures, location of the settlements, etc.
Hope you found it helpful.