I think doing semantically clever things is very hard, because a persistent cache needs to store different versions of foo(x::Int) depending on how dependencies like Base.:+ changed, in a possibly load-order dependent way, different sessions and package versions (developers churn out a lot of versions in a single sesssion!).
That seems like a fairly nice summary of some of the challenges someone might encounter if they wanted to resolve https://github.com/JuliaLang/issues/265 for example. But unfortunately, as you mentioned above, this “caching between sessions [is] impossible”.