This is constantly brought up between the Israel and Palestine conflict and it has some consequence.
Firstly, the new inhabitants of said land might recieve persecution because that land didn't belong to them.
Secondly, the original owners of the land would feel upset if they themselves did not agree to have their land taken away and given to another group of persons.
All in all, these differences would cause persecution, rascism, and, in the case of Israel, war.