# Julia 1.6 libcurl firewall download issue: Windows Schannel certificate revocation check failure

**URL:** <https://discourse.julialang.org/t/julia-1-6-libcurl-firewall-download-issue-windows-schannel-certificate-revocation-check-failure/58021>\
**Category:** General Usage\
**Tags:** libcurl\
**Created:** [March 26, 2021, 2:43pm UTC](https://discourse.julialang.org/t/julia-1-6-libcurl-firewall-download-issue-windows-schannel-certificate-revocation-check-failure/58021 "2021-03-26T14:43:26Z")\
**Posts on this page:** 14\
**Page:** 2

<div class="post-metadata">

**Author:** ![lrnv](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lrnv/32/19373_2.png) [@lrnv](https://discourse.julialang.org/u/lrnv)\
**Post date:** [March 30, 2021, 1:50pm UTC](https://discourse.julialang.org/t/julia-1-6-libcurl-firewall-download-issue-windows-schannel-certificate-revocation-check-failure/58021/21 "2021-03-30T13:50:39Z")

</div>

> [@giordano](#):
>
> Saying that setting the insecure option makes it work perfectly sounds like an oxymoron though 😉

Indeed, but it is not my problem anymore : this is now the problem of my adminsys that did not provide a proxy that respects the standards, nor tried to do something when I told him 😉

---

<div class="post-metadata">

**Author:** ![StefanKarpinski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefankarpinski/32/24_2.png) [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)\
**Post date:** [April 20, 2021, 8:48pm UTC](https://discourse.julialang.org/t/julia-1-6-libcurl-firewall-download-issue-windows-schannel-certificate-revocation-check-failure/58021/22 "2021-04-20T20:48:04Z")

</div>

If you are behind a MITM proxy, it is not actually insecure to turn host verification off since the certificate you’d be verifying is fake anyway. If the proxy is not verifying the server’s identity for you by checking the actual certificate, that would be insecure but presumably the proxy is configured securely. Whether that is the case or not, the client has no control over that anyway, so there’s nothing we can do about it.

There are two things required for a client connection behind a MITM proxy to “just work”:

1. The client must have a CA root installed that allows it to verify fake certificates created by the MITM proxy. That seems to be the case here, since otherwise we’d get a different error.

2. The proxy should be configured to handle certificate revocation checks from Windows machines. That seems not to be the case here, as the error indicates that revocation checking failed.

The first requirement is necessary for all operating systems. As of Julia 1.6, we use system TLS engines on macOS and Windows, so if the MITM CA root has been added to the system certificate stores, then that step should be fine. On Linux, we look for a PEM file in common places and use the first one we find, so even though there’s no “system TLS engine” we ought to pick up a MITM CA root if one has been installed.

The second requirement only affects Windows because each OS does certificate revocation checking differently. Linux doesn’t do certificate revocation checks at all, which is obviously insecure, but also not our problem to solve. If there’s ever a standard way to handle certificate revocation checks on Linux, we can hook into it. MacOS does offline updates to its certificate revocation list, so this kind of error cannot happen: the system TLS engine checks a certificate against the revocation list that it already has; it doesn’t try to update that list during the host verification process. It may try (and fail) to update the CRL list at some other time, but that doesn’t block individual TLS requests.

Windows, on the other hand, does a synchronous certificate revocation checks while verifying each host’s identity. That means that if it hasn’t recently checked whether the certificate for a given host has been revoked, it will contact a Microsoft server to check that _while_ it is in the process of verifying the validity of the certificate for the host you are trying to connect to. This is where people are hitting problems: they seem to have a MITM CA root installed, but when Windows tries to see if the certificate has been revoked, that check is being blocked or failing.

We could add an option to not do CRL checks while still doing certificate verification, but I’m not sure if this is actually necessary or useful. This problem only seems to occur when behind a MITM firewall, in which case it’s just as secure to skip verification altogether since the certificate you’d be verifying isn’t real anyway. You might as well turn host verification off altogether when you’re behind a MITM firewall.

It’s possible that I’m misunderstanding something here because this stuff is complicated and documentation is both poor and spread all across the Internet.

---

<div class="post-metadata">

**Author:** ![StefanKarpinski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefankarpinski/32/24_2.png) [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)\
**Post date:** [April 20, 2021, 9:18pm UTC](https://discourse.julialang.org/t/julia-1-6-libcurl-firewall-download-issue-windows-schannel-certificate-revocation-check-failure/58021/23 "2021-04-20T21:18:03Z")

</div>

This post gives details on how Windows does CRL checks and how to configure things so that they will work correctly: [Troubleshooting network retrieval of CRLs - Browsers | Microsoft Docs](https://docs.microsoft.com/en-us/troubleshoot/browsers/description-of-cryptography-api-proxy-from-crl).

---

<div class="post-metadata">

**Author:** ![Keno](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/keno/32/285_2.png) [@Keno](https://discourse.julialang.org/u/Keno)\
**Post date:** [April 20, 2021, 9:23pm UTC](https://discourse.julialang.org/t/julia-1-6-libcurl-firewall-download-issue-windows-schannel-certificate-revocation-check-failure/58021/24 "2021-04-20T21:23:47Z")

</div>

> [@StefanKarpinski](#):
>
> You might as well turn host verification off altogether when you’re behind a MITM firewall.

Well, unless someone inside your MITM firewall is doing yet another MITM attack with a certificate that was not installed on your machine.

---

<div class="post-metadata">

**Author:** ![StefanKarpinski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefankarpinski/32/24_2.png) [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)\
**Post date:** [April 20, 2021, 9:24pm UTC](https://discourse.julialang.org/t/julia-1-6-libcurl-firewall-download-issue-windows-schannel-certificate-revocation-check-failure/58021/25 "2021-04-20T21:24:20Z")

</div>

Sure, but that’s also out of our control.

---

<div class="post-metadata">

**Author:** ![Keno](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/keno/32/285_2.png) [@Keno](https://discourse.julialang.org/u/Keno)\
**Post date:** [April 20, 2021, 9:26pm UTC](https://discourse.julialang.org/t/julia-1-6-libcurl-firewall-download-issue-windows-schannel-certificate-revocation-check-failure/58021/26 "2021-04-20T21:26:51Z")

</div>

Right, I’m saying that’s a case where verifying certificates (presumably installed by your snooping IT department) without verifying CRLs makes sense (because it still blocks MITM inside the firewall)

---

<div class="post-metadata">

**Author:** ![StefanKarpinski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefankarpinski/32/24_2.png) [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)\
**Post date:** [April 20, 2021, 9:38pm UTC](https://discourse.julialang.org/t/julia-1-6-libcurl-firewall-download-issue-windows-schannel-certificate-revocation-check-failure/58021/27 "2021-04-20T21:38:48Z")

</div>

Oh right, I see what you mean. Someone could be on the local network doing a MITM attack.

---

<div class="post-metadata">

**Author:** ![StefanKarpinski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefankarpinski/32/24_2.png) [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)\
**Post date:** [April 21, 2021, 2:17pm UTC](https://discourse.julialang.org/t/julia-1-6-libcurl-firewall-download-issue-windows-schannel-certificate-revocation-check-failure/58021/28 "2021-04-21T14:17:03Z")

</div>

9 posts were split to a new topic: [Julia 1.6 other libcurl download issues](https://discourse.julialang.org/t/julia-1-6-other-libcurl-download-issues/59738)

---

<div class="post-metadata">

**Author:** ![StefanKarpinski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefankarpinski/32/24_2.png) [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)\
**Post date:** [April 21, 2021, 2:26pm UTC](https://discourse.julialang.org/t/julia-1-6-libcurl-firewall-download-issue-windows-schannel-certificate-revocation-check-failure/58021/29 "2021-04-21T14:26:48Z")

</div>

In an effort to keep this discussion about one problem, I have moved the posts that are about unrelated download problems to a different topic. Let’s keep this discussion specifically about the Windows Schannel certificate revocation check failure that was originally posted. There does seem to be a libcurl option called `CURLSSLOPT_REVOKE_BEST_EFFORT` that we could set for this (see [CURLOPT\_SSL\_OPTIONS](https://curl.se/libcurl/c/CURLOPT_SSL_OPTIONS.html)). According to the docs:

> Tells libcurl to ignore certificate revocation checks in case of missing or offline distribution points for those SSL backends where such behavior is present. This option is only supported for Schannel (the native Windows SSL library).

So basically specifically for this problem. Setting this by default seems like the best option to me: it will work and check for revoked certificate to the extent possible but not fail on Windows if the CRL server cannot be reached. This is not perfect since an attacker could theoretically block your access to the CRL server and use a very recently revoked certificate, but that seems pretty improbable. Even with this option set, the behavior on Windows is as secure as the default macOS behavior and both are more secure than Linux where certificates are never checked for revocation at all.

---

<div class="post-metadata">

**Author:** ![StefanKarpinski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefankarpinski/32/24_2.png) [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)\
**Post date:** [April 21, 2021, 3:16pm UTC](https://discourse.julialang.org/t/julia-1-6-libcurl-firewall-download-issue-windows-schannel-certificate-revocation-check-failure/58021/30 "2021-04-21T15:16:05Z")

</div>

Pull request to set this option: [set TLS option CURLSSLOPT\_REVOKE\_BEST\_EFFORT by StefanKarpinski · Pull Request #115 · JuliaLang/Downloads.jl · GitHub](https://github.com/JuliaLang/Downloads.jl/pull/115).

---

<div class="post-metadata">

**Author:** ![StefanKarpinski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefankarpinski/32/24_2.png) [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)\
**Post date:** [April 21, 2021, 4:47pm UTC](https://discourse.julialang.org/t/julia-1-6-libcurl-firewall-download-issue-windows-schannel-certificate-revocation-check-failure/58021/31 "2021-04-21T16:47:54Z")

</div>

It would be great if someone who is seeing this problem could test this. It’s a little tricky to test, but here are some relatively simple steps that don’t require recompiling Julia:

1. Use `pkg>` mode to checkout a dev copy of `Downloads.jl`:

```julia
pkg> dev Downloads

```

2. Go into `~/.julia/dev/Downloads` and checkout the PR branch:

```julia
git fetch origin
git checkout -b sk/tls-revoke-best-effort origin/sk/tls-revoke-best-effort

```

3. Edit the `~/.julia/dev/Downloads/Project.toml` file and modify the UUID somehow, e.g. by changing the last digit to some other valid hex digit.
4. Change directory into the `~/.julia/dev/Downloads`
5. Start Julia 1.6 in that directory with the `--project` option
6. Try the problematic download from the Julia REPL:

```julia
using Downloads: download
url = "https://github.com/JuliaBinaryWrappers/GLFW_jll.jl/releases/download/GLFW-v3.3.3%2B0/GLFW.v3.3.3.x86_64-linux-gnu.tar.gz"
file = download(url)
@assert open(Base._crc32c, file) == 0x40e5ef54

```

If that works behind a firewall without error, downloading the file and getting the right CRC checksum, then the problem is fixed by setting this option.

---

<div class="post-metadata">

**Author:** ![StefanKarpinski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefankarpinski/32/24_2.png) [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)\
**Post date:** [April 23, 2021, 6:11am UTC](https://discourse.julialang.org/t/julia-1-6-libcurl-firewall-download-issue-windows-schannel-certificate-revocation-check-failure/58021/32 "2021-04-23T06:11:11Z")

</div>

Anyone?

---

<div class="post-metadata">

**Author:** ![johnh](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/johnh/32/3615_2.png) [@johnh](https://discourse.julialang.org/u/johnh)\
**Post date:** [April 23, 2021, 6:36am UTC](https://discourse.julialang.org/t/julia-1-6-libcurl-firewall-download-issue-windows-schannel-certificate-revocation-check-failure/58021/33 "2021-04-23T06:36:22Z")

</div>

Have a Windows 10 laptop. but am not behind a firewall, sorry.

---

<div class="post-metadata">

**Author:** ![HerAdri](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/heradri/32/5816_2.png) [@HerAdri](https://discourse.julialang.org/u/HerAdri)\
**Post date:** [April 23, 2021, 7:56am UTC](https://discourse.julialang.org/t/julia-1-6-libcurl-firewall-download-issue-windows-schannel-certificate-revocation-check-failure/58021/34 "2021-04-23T07:56:42Z")

</div>

[quote=“StefanKarpinski, post:31, topic:58021”]

```julia
(@v1.6) pkg> dev Downloads
     Cloning git-repo `https://github.com/JuliaLang/Downloads.jl.git`
ERROR: failed to clone from https://github.com/JuliaLang/Downloads.jl.git, error: GitError(Code:ERROR, Class:OS, failed to send request: The connection with the server was terminated abnormally
)

julia> using Downloads

(@v1.6) pkg> dev Downloads
     Cloning git-repo `https://github.com/JuliaLang/Downloads.jl.git`
ERROR: failed to clone from https://github.com/JuliaLang/Downloads.jl.git, error: GitError(Code:ERROR, Class:OS, failed to send request: The connection with the server was terminated abnormally
)
julia> names(Downloads)
6-element Vector{Symbol}:
 :Downloader
 :Downloads
 :RequestError
 :Response
 :download
 :request

```

[Julia 1.6.0 packages installation](https://discourse.julialang.org/t/julia-1-6-0-packages-installation/58552/10)

[Previous page](https://discourse.julialang.org/t/julia-1-6-libcurl-firewall-download-issue-windows-schannel-certificate-revocation-check-failure/58021.md?page=1)