I think there are two main reason for scientists to study the past climate of the earth. First, the climate is a very complex system and extremely difficult to model; having data of past climate events could be very useful to better understand, make more accurate predictions and refine the models. Second, to stablish parallelisms between past and current climate trends like the levels of CO2 in the cretaceous and our current levels and how similar are the effects of such levels of CO2 in both periods.
I think that even though the data obtained by paleo climatologists is not based on direct observations, it provides a lot of information like levels of oxygen, carbon dioxide, and hydrogen that could be very useful to reconstruct the climate conditions on a given period. Also, with that data they can run simulations to better understand our past and current climate.