Let x be the first number. Since these numbers are in geometric progression, there is some constant r such that the second number is xr and the third number is xr•r = xr ².
Their sum is 65 :
x + xr + xr ² = 65
Their product is 3375 :
x • xr • xr ² = 3375
We can simplify the first equation as
x (1 + r + r ²) = 65
and the second one as
x ³ r ³ = (xr )³ = 3375
Since 15³ = 3375, it follows that xr = 15. Then x = 15/r. Substitute this into the second equation and solve for r :
15/r (1 + r + r ²) = 15/r + 15 + 15r = 65
15 + 15r + 15r ² = 65r
15 - 50r + 15r ² = 0
3 - 10r + 3r ² = 0
I'll complete the square:
3r ² - 10r = -3
3 (r ² - 10/3 r ) = -3
3 (r ² - 10/3 r + 100/36) = -3 + 300/36
3 (r - 10/6)² = 16/3
(r - 10/6)² = 16/9
r - 10/6 = ± √(16/9) = ± 4/3
r = 10/6 ± 4/3
r = 3 or r = 1/3
Both cases give the same numbers:
If r = 3, then x = 15/3 = 5. Then the 3 numbers are 5, 15, 45.
If r = 1/3, then x = 15/(1/3) = 45. Then the 3 numbers are the same but in reverse order: 45, 15, 3.