That makes sense (I have a similar approach for my OS software) but for Julia it’s not a good idea: the LTS is definitely not the best code, probably not even the most tested since the majority of user upgrade quickly to new versions I think.
I agree it’s confusing because you would expect an LTS to get all the bugfixes, and it’s not the case for some reason. But even if it did get all the fixes, you should use a more recent version: with LTS you’re stuck with old, inferior versions of third-party packages because the developers wanted to use important features introduced in Julia 1.3 for example.
Here’s a recent take from a core developer:
In summary, the language and ecosystem are moving too fast right now for LTS to be the best environment.
Fortunately, Julia has you covered when it comes to running old code 
It’s easy to install the old Julia version that was used for an old project (just a zip file to uncompress somewhere). And assuming you have made a Julia project for your publication (and you really should have) then you have a Project.toml and Manifest.toml that describe the versions of packages used in that project. Years later you can get the correct package versions by running ]instantiate. And starting with Julia 1.7, the Julia version itself will be recorded in the manifest.