To be clear, decimal floats do not need to be stored in strings. There is an IEEE decimal-floating point format using fixed-width binary-coded decimal, available in Julia via the DecFP.jl package. It is implemented only in software on the hardware supported by Julia, so it is slower than binary floating-point, but it’s still useful e.g. in financial applications where exact representation of human decimal inputs is required.
Note also that, contrary to the popular superstition that 1.0 + 1.0 == 2.0 ± ε, ordinary binary floats ala Float64 are perfectly exact for working with integers up to 2^53 == maxintfloat(Float64). 3.0 * 7.0 and 9.0 + 5.0 are doing exact integer arithmetic, which happens to be stored as Float64.
That being said, I don’t think we seriously want to make all integers default to floating-point storage (ala Javascript, Matlab, or R). Putting aside questions of range of exactness (53 vs 64 bits), computer hardware is optimized to perform indexing operations (e.g. memory access) using integers, and using floats everywhere would lead to a massive slowdown. People only don’t care about this in languages where loops are slow anyway.
Parsing decimal strings to Float64 and converting back to strings is actually lossless in Julia — as long as your decimal has < 14 digits, which should be true for any decimal that is worth representing exactly — because the float-printing algorithm (Ryu, formerly Grisu) prints the shortest decimal that rounds to the same binary-float value. See this discussion.