Let s be the side length of the square. The dimensions of the rectangle are three times the side of the square (i.e. 3s), and two less than the side of the square (i.e. s-2).
So, the area of this rectangle is

The area of the square is
, and we know that the two areas are the same, so we have

The solution s=0 would lead to the extreme case where the rectangle and the square are actually a point, so we accept the solution s=3.