This makes more sense when you are talking about a variable being greater than/less than or equal to a number.
For example, x ≥ 5.
x can be any of the numbers from 5 to infinity, including 5. You can see that 6 is greater than or equal to 5. A trillion is greater than of equal to 5. And 5 itself is greater than or equal to 5. All of these are solutions to x. In interval notation, we could write [5,∞). The solution set of this inequality is all real numbers x such that x ≥ 5. (For just a plain number being greater than/less than or equal to another number, this seems a little unnecessary, so I understand your confusion.)
You can see it graphically in the attached image.