Answer:
5√3 miles per hour
Explanation:
Throughout their journey, the component of velocity of rowboat B to the east is the same as that of rowboat A:
(10 mph)(cos(π/3)) = 5 mph
Meanwhile, the component of velocity of rowboat B to the north is ...
(10 mph)(sin(π/3)) = 5√3 mph
Since rowboat A is always traveling in a direction that is at right angles to the direction between the boats, it contributes nothing to their relative speed. Since rowboat B is always directly north of rowboat A, its speed to the north is their relative speed.
The distance between the rowboats is increasing at 5√3 mph ≈ 8.66 mph.