If the smaller number is x, then the larger number has to be 20 - x in order for the arithmetic mean to be 10, as their sum must be 20.
If the geometric mean is 8 the product of the numbers has to be 64 and so the larger number is 64/x
Solving 64/x = 20 - x gives the quadratic x^2 -20x +64 = 0 so x = 4 or 16.
The two numbers are 4 and 16.