I finally managed to resolve this issue after a week of trying to learn how package installation in Julia works. It seems that due to Proxy issue I did not manage to properly download the library. What I did was indeed very trivial.
I opened .julia/packages/Zstd_jll/UffgW/Artifacts.toml and there looked for the artifact link that matches architecture of my machine. I downloaded it on my machine (manually and not with Pkg) and since the Artifacts are immutable, I could safely put the downloaded file under .julia/artifacts/.
Then I needed to tell to my machine where to look for the file! I had to update relevant part of .julia/packages/Zstd_jll/UffgW/Artifacts.toml as follwing:
[[Zstd]]
arch = "x86_64"
git-tree-sha1 = "e6ce8cd51fdf23fc9c9e8abde61b8a05d2502ac9"
libc = "glibc"
os = "linux"
[[Zstd.download]]
sha256 = "f81d29d10ca06a7b37494a7bad4d3395f5491aecdf3c483f08b868642b7fdf73"
url = "file:/home/myUserName/.julia/artifacts/Zstd.v1.5.0.x86_64-linux-gnu.tar.gz"
Now back to Julia REPEl, I could install my main library like Pkg.add("XXX") where it will requires Zstd to be already installed.
It may not be in agreement with philosophy of Julia, but due to my limitations, I had to go this way and indeed it resolved my problem. Happy that I did not give up.