My guess is that you installed from one of those package managers that do not create a symbolic link to the library as is common use on unix and that is why it doesn’t find the libgmt.so
If you run, in a unix shell, gmt --show-bindir it will show you the location of the GMT executable. Then use ldd to find its dependencies. Something like
ldd /usr/bin/gmt | grep libgmt
you will see references to (in my linux box)
libgmt.so.6 => /lib64/libgmt.so.6 (0x00007f45f97ee000)
and
[jluis@fct-gmt lib64]$ ls /usr/lib64/libgmt*
/usr/lib64/libgmt.so /usr/lib64/libgmt.so.6 /usr/lib64/libgmt.so.6.1.0
The problem in your case (again, my guess) is that your installation lacks the libgmt.so, which is a symbolic link (that should be there). The solution is to create one yourself (in the directory of the shared libs, /usr/lib64/ in this example) do
ln -s libgmt.so libgmt.so.6