Reason #1:
The stone blocks are so big and heavy that it's hard to believe that they could have been moved and stacked by the people of 2,000 years ago, who are believed to have been ignorant savages.
Reason #2:
The people making such a claim don't bother, or aren't capable, to read reliable sources of information and research. They're satisfied to just latch onto the nearest conspiracy theory that makes them feel as warm and comfortable as wetting their pants, and they're just one baby step short of becoming flat earthers.