Note: this only applies to humans and other mammals. Other groups have different systems, and the logic wouldn't necessarily apply.
In mammals, males have one X chromosome and females have two. This is unusual. For all other chromosomes, the number never varies amongst healthy, normal individuals. But because of this natural variation within the species, we've had to evolve ways to deal with having different numbers of X chromosomes while still being healthy. This is the problem of "dosage compensation", and the details don't matter. The point is that, unlike with all autosomal chromosomes, we have systems in place to deal with changing numbers of X chromosomes. This means that we're much better able to survive having abnormal X chromosome numbers than we are any other chromosome.