Because male sex chromosome is XY which has only one X. It will become dominant in male and cause colour blindness while female sex chromosome is XX in which if she has one allele that carries the colorblindess gene, it will only become carrier. Please remember that this alelle like hemophilic, thalassemia and colorblindess only stick with X alelle, never in Y. Hope it helps!