When the rock is suspended in the air, the net force on it is
∑ F₁ = T₁ - m₁g = 0
where T₁ is the magnitude of tension in the string and m₁g is the rock's weight. So
T₁ = m₁g = 37.8 N
When immersed in water, the tension reduces to T₂ = 32.0 N. The net force on the rock is then
∑ F₂ = T₂ + B₂ - m₁g = 0
where B₂ is the magnitude of the buoyant force. Then
B₂ = m₁g - T₂ = 37.8 N - 32.0 N = 5.8 N
B₂ is also the weight of the water that was displaced by submerging the rock. Let m₂ be the mass of the displaced water; then
5.8 N = m₂g ==> m₂ ≈ 0.592 kg
If one takes the density of water to be 1.00 g/cm³ = 1.00 × 10³ kg/m³, then the volume of water V that was displaced was
1.00 × 10³ kg/m³ = m₂/V ==> V ≈ 0.000592 m³ = 592 cm³
and this is also the volume of the rock.
When immersed in the unknown liquid, the tension reduces further to T₃ = 20.2 N, and so the net force on the rock is
∑ F₃ = T₃ + B₃ - m₁g = 0
which means the buoyant force is
B₃ = m₁g - T₃ = 37.8 N - 20.2 N = 17.6 N
The mass m₃ of the liquid displaced is then
17.6 N = m₃g ==> m₃ ≈ 1.80 kg
Then the density ρ of the unknown liquid is
ρ = m₃/V ≈ (1.80 kg)/(0.000592 m³) ≈ 3040 kg/m³ = 3.04 g/cm³