Answer: By the time they came, we had finished our dinner.
Came is past tense, so I would assume it would be had finished
9.5m questions
12.2m answers