Thanks for this @GunnarFarneback. As you point out, apparently using the PEM SSH key format works, but not the default OPENSSH format (which I was using) because (quoting you)
LibGit2gets its ssh functionality fromlibssh2, which indeed is less capable of taking the~/.sshconfigurations into account than thesshcommand line interface used by thegitcommand line interface.
But it seems like @StefanKarpinski is working on improving this.
Apparently the above solution involving ssh-agent somehow makes incompatible keys available to libssh2 via an intermediary.
Just thought that summarizing the situation might be helpful for the next confused googler.