All three of your options are correct equations that can be used to solve for y, the length of the room:
(y - 5) * y = 750
y^2 - 5y - 750 = 0
(y + 25)(y - 30) = 0
1st equation is in the form of area of rectangle = L*W, where L is the length of room and W is the width of the room.
2nd equation is factored form of the quadratic equation obtained from the first equation
3rd equation is the factored form of 2nd equation, it gives two possible solutions for y which are y = 25 and y = -30
Note that y = -30 is an extraneous solution as the length of a room can't be negative.