They said he was unfit to be King because of the distance it was from the colonies to Britain. 3,000 miles over-seas is quite a long way. The colonist also thought 'How can the King know how life is over here, and therefor, how can he make these laws and taxes without consulting us. Letting us have part of the say'.
(Also, make sure your question is clear. Such as the time in which the colonists were to say what the King is unfit to be.)