The prologue makes us understand he had been part of the crusades when it says "Then longen folk to go on pilgrimages, and palmers for to seeken strange strands"
The first reference to the crusades we found is the word "pilgrimage" because it means a journey to a secred place and Jerusalem was called the holy land and believe to be a sacred place because Jesus tomb is there. It makes another reference to the crusades when it talks about strange shores, as it was the first time they Christians travel to Jerusalem.