To answer this question we need two equations and allele frequency.
Hardy-Waynberg: p²+2pq+p²=1
Allele frequency, being p= dominant and q= recessive, so p+q=1
Now we have set our alleles and equation we proceed to obtain proportions
Total population= 200 individuals =100%
Brown mice (dominant phenotype)= 168 individuals = 84%
gray mice (recessive phenotype)= 32 individuals= 16%
So we have that
p² + 2pq= 0.84
q² = 0.16
So we obtain q
q=√0.16 = 0.4
Now by the allele frequency formula, we obtain p
p=1-0.4 = 0.6
Now we use hardy weinberg to obtain dominant homozygous and heterozygous
p²+2pq+p²=1
0.6²+2(0.6)(0.4)+0.4²=1
0.36+2(0.48)+0.16=1
Now we have the proportion of homozygous represented by p² and p² we add one to each other and we have:
0.36+0.16= 0.52
Therefore the homozygous proportion is 52% option A.