It describes what the narrator knows from his past. The narrator is visiting his old friend, and in this excerpt, he is reminiscing about the past - he says that they were close associates when they were younger, but that his friend was always distant and reserved. So, he is thinking about his own past in which he spent time with his friend.
I think.