In addition, the expression can have bounds, such as:
@variable(model, x >= 0)
@variable(model, x <= 0)
@variable(model, x == 0)
@variable(model, 0 <= x <= 1)
It is perhaps easier to understand if one considers the alternative way using keyword arguments
lower_bound::Float64: an alternative to x >= lb, sets the value of the variable lower bound.
upper_bound::Float64: an alternative to x <= ub, sets the value of the variable upper bound.
This is not a JuMP issue, this is a solver issue, solvers often do not accept strict inequality because it does not make sense in context. Solvers often work with 64 bits floating-point numbers. In this representation, the difference between x and the first value smaller than x is so small that it will basically always be inside your tolerances. You can set your tolerance to zero, but this probably will not do what you want (correct solutions may be disregarded because it is impossible for the solver to reach them without getting the value slightly off).