Answer:
To solve this problem, we need to use the concept of hypergeometric distribution, which gives the probability of selecting a certain number of objects with a specific characteristic from a population of known size without replacement. We will use the formula:
P(X = k) = [ C(M, k) * C(N - M, n - k) ] / C(N, n)
where:
P(X = k) is the probability of selecting k objects with the desired characteristic;
C(M, k) is the number of ways to select k objects with the desired characteristic from a population of M objects;
C(N - M, n - k) is the number of ways to select n - k objects without the desired characteristic from a population of N - M objects;
C(N, n) is the total number of ways to select n objects from a population of N objects.
In our case, we want to select 5 rats out of a population of 80, and we want exactly 3 of them to have 2 genetic mutations. We can calculate this probability as follows:
P(3 rats have exactly 2 mutations) = [ C(12, 3) * C(68, 2) ] / C(80, 5)
where:
M is the number of rats that have exactly 2 mutations, which is the sum of the rats that have mutations AB, AC, AD, BC, BD, and CD, or M = 5 + 6 + 4 + 3 + 3 + 1 = 22;
N - M is the number of rats that do not have exactly 2 mutations, which is the remaining population of 80 - 22 = 58 rats;
n is the number of rats we want to select, which is 5.
We can simplify this expression as follows:
P(3 rats have exactly 2 mutations) = [ C(12, 3) * C(68, 2) ] / C(80, 5)
= [ (12! / (3! * 9!)) * (68! / (2! * 66!)) ] / (80! / (5! * 75!))
= 0.03617
Therefore, the probability that if 5 rats are selected at random, 3 will have exactly 2 genetic mutations is 0.03617 (rounded to five decimal places).