@peteole, You’re simply missing libhipamd64_.so. Try installing AMDGPU@0.2.10 or above, it should ship with HIP_jll. I have AMD Ryzen 7 PRO 4750U with Radeon Graphics and I can run basic examples just fine!
Many thanks to @jpsamaroo for singlehandedly making this happen on AMD GPUs!