(I made the previous MWE slightly more concise).
I found this 2017 JuliaCon talk by Jameson Nash that addresses memoization, showing a more general example to memoize any input, and also a macro that looks like what Memoize.jl is essentially doing.
Unless at some point the discussion will show a different majority consensus, I’d summarize the solution for other users stumbling upon this thread:
For general memoization it is recommended to check out Memoize.jl.
For lazy construction of an object, this thread illustrates different approaches.