Whenever you multiply by the matrix or its inverse (or equivalent, e.g. A \ b), you should generically expect to lose about 9 digits to roundoff errors, which is very close to what you are seeing:
julia> eps() * 1e9
2.220446049250313e-7
Not necessarily, but solving an ill-conditioning problem generally means going back to an earlier step, understanding where the ill-conditioning comes from, and re-arranging your computation to avoid it. (That is, you wouldn’t construct your matrix at all, but instead do some equivalent process from earlier inputs. A classic example of this is solving least-squares by QR factorization rather than the normal equations.)