Answer: By the time they came, we had finished our dinner.
Came is past tense, so I would assume it would be had finished
1.6m questions
2.0m answers