If deer population continues to grow, there will be a point where there is an oversaturation of deer in any given possible environment. Environmental factors such as loss of food or shelter will then cut down the deer population. Other factors such as disease, which its spread is amplified in high concentrations of organism saturation, can also cut down the population.
The ecosystem will until then sustain damage to its plant life, which can have effects on shelter, food, and possibly water provision for other organisms.