Okay, I figured out the cause of this error. Once again, the error message had nothing to do with the actual error. But such is life :).
The problem here is that in the R environment, I needed to install the Curl package. So all of the curl and libcurl dependencies on my computer were fine, there were no issues with Openssl versions as referenced in the error message, CURL_OPENSSL_4 not found.
So if you see this message pop up, then check the R environment and see if you are missing the Curl package.