The answer is that Julia uses hash to determine whether the value has changed, and the default hash doesn’t recurse into your object’s fields, it just uses objectid(x). I think there’s some discussion of this in various threads, but I agree, its mildly surprising.
In any case, you can fix it by defining your own hash which does involve the contents, e.g. something like:
Base.hash(x::X, h::UInt) = hash(X, hash(x.a, h))
After that, your last example works as expected.