Answer: By the time they came, we had finished our dinner.
Came is past tense, so I would assume it would be had finished
4.7m questions
6.1m answers