PSA: backdoor in xz-utils and relevance for the Julia ecosystem

Statement of the problem

A thread in oss-security mailing list reported that the XZ Utils project was affected by a backdoor: one of the current maintainers of the project appeared to have injected malicious code, included in versions v5.6.0 and v5.6.1 of the xz-utils package, that under certain circumstances could potentially compromise an OpenSSH client and obtain login credentials to remote systems.

Affected systems and conditions for the backdoor to work

XZ Utils is somewhat popular in Linux distributions. Quoting from the XZ Utils Wikipedia article:

The malicious mechanism consists of:

  1. Two test files that contain the malicious binary code. These files are available in the git repository, but remains dormant unless extracted and injected into the program.[16] The code uses the glibc IFUNC mechanism to replace an existing function in OpenSSH called RSA_public_decrypt with a malicious version. OpenSSH normally does not load liblzma, but a common third-party patch used by several Linux distributions cause it to load libsystemd, which in turn loads lzma.[16]
  2. A modified version of build-to-host.m4, which extracts a script that performs the actual injection. The modified m4 file is not present in the git reponsitory; it is only available from tar files released by the maintainer separate from git.[16]
  3. A script that extracts the malicious code from “test case” files and injects them into liblzma. The file appears to only perform the injection when the system being built on (1) is an x86-64 Linux system (2) uses glibc and GCC (3) is being built via dpkg or rpm.[16]

It is unknown whether this backdoor was intentionally placed by a maintainer or whether a maintainer was compromised.[17]

The list of affected Linux distributions includes Debian unstable[18], Arch Linux[19], Fedora Rawhide[20], Kali Linux[21], OpenSUSE Tumbleweed[22]. Confirmed to not be affected are Red Hat Enterprise Linux[23], SUSE Linux Enterprise[22], Amazon Linux[24].

If you use one of the affected systems, you may want to update your system as soon as possible, to make sure you get your version of xz-utils downgraded to a version not currently known to be affected by any backdoors.

As detailed in the oss-security thread, the backdoor is installed in the liblzma library only when several conditions are met:

  1. the target system is x86_64-linux-gnu (Linux kernel, on x86_64 architecture, using glibc as standard C library)
  2. the compiler is GCC
  3. the build system detects dpkg or rpm are being used

Then, when the liblzma library contains the malicious payload, the exploit works only when all the following conditions are met:

  1. the TERM environment variable is not set
  2. the name of the executable (the argv[0] argument inside the program) needs to be /usr/sbin/sshd
  3. the environment variables LD_DEBUG, LD_PROFILE are not set
  4. the environment variable LANG needs to be set

Relevance for the Julia ecosystem

You may know that in the Julia ecosystem we ship pre-built binaries for several packages written in C/C++, Fortran, Rust, Go, by using the BinaryBuilder framework. We also happen to provide builds of xz-utils, in the form of the XZ_jll package. We have builds for both the affected versions, v5.6.0 and v5.6.1. However, BinaryBuilder does not meet condition 3 (building through dpkg or rpm) to activate the backdoor during compilation. The announcement of the security exploit in the oss-security mailing list includes a script to detect whether a build of liblzma is vulnerable, by checking a certain pattern is present in the hexadecimal dump of the library. I personally verified our builds of liblzma.so in XZ_jll v5.6.0 and v5.6.1 do not contain the known pattern. I know other people have independently carried out the same test, I encourage you to do the same if you’re still suspicious.

Furthermore, it is highly unlikely any user meets condition 2 for actually being vulnerable to the exploit: the executable /usr/sbin/sshd in user’s systems will most likely not link to our builds of liblzma.so (which, as stated in the previous paragraph, does not appear to be affected by the security threat), but rather to a build of liblzma.so provided by the operating system, if linking to liblzma.so at all.

To summarise, to the best of our knowledge we do not believe our builds of the XZ Utils package in the form of the XZ_jll package are subject to any disclosed security vulnerability.

Measures taken in the Julia ecosystem to reduce the impact of the vulnerability

That said, out of an abundance of caution, we immediately yanked versions 5.6.0 and 5.6.1 of XZ_jll from the General registry, as soon as we were made aware of the security threat by @semarie on Slack. If you now try to install or update XZ_jll from the General registry you will only get versions up to v5.4.6, which at the moment are not known to actively carry security vulnerability. And again, even if you happen to have installed XZ_jll v5.6.0 or v5.6.1 during last month we do not expect this to have concretely posed a security threat for your system. But remember that if you use one of the affected systems mentioned above, you may be vulnerable through the packages provided by your operating system.

We will continue to monitor the situation, if it will become apparent that also previous versions of XZ Utils may have included security vulnerabilities we will yank also them from the General registry.

83 Likes

Looking at the Debian and Gentoo discussions, there is some thought to reverting further back to eliminate or minimize the exposure to the author, Jia Tan, whose account was used introduce the backdoor.

XZ uses even minor versions for stable releases and odd minor versions for development releases.

XZ version 5.2.5 has no commits from the offending account:
https://git.tukaani.org/?p=xz.git;a=shortlog;h=refs/tags/v5.2.5

XZ version 5.2.6 is the first version where the offending account begins “contributing”:
https://git.tukaani.org/?p=xz.git;a=shortlog;h=refs/tags/v5.2.6

Version 5.2.5 does have a security advisory though: ZDI-22-619 / CVE-2022-1271.

To that end I have created a pull request for another xz version 5.2.5 with the above CVE patched:

If merged, I think it will create XZ_jll.jl v5.2.5+3.

9 Likes

Specify version 5.2.5 to obtain a “Jia Tan” free version of XZ

XZ_jll.jl 5.2.5+3 has been released. This version contains no commits from Jia Tan and includes a patch for CVE-2022-1271.

The v5.2.5 tag and the patch have been checked against Lasse Collin’s public key.

To install, I recommend the following procedure.

] add XZ_jll@5.2.5
pin XZ_jll

Equivalently,

using Pkg
Pkg.add(name="XZ_jll", version=v"5.2.5+3")
Pkg.pin(name="XZ_jll", version=v"5.2.5+3")

The default XZ_jll version is 5.4.6, signed by Jia Tan

Installing XZ_jll without specifying a version will install version 5.4.6 of XZ. The v5.4.6 tag was signed by Jia Tan, the apparent author of the backdoor, on January 26th, 2024. At the time of this writing, there are no known vulnerabilities in version 5.4.6.

(@v1.10) pkg> activate --temp
  Activating new project at `/tmp/jl_pcjWcT`

(jl_pcjWcT) pkg> add XZ_jll
    Updating registry at `~/.julia/registries/General.toml`
   Resolving package versions...
    Updating `/tmp/jl_pcjWcT/Project.toml`
  [ffd25f8a] + XZ_jll v5.4.6+0

To obtain version 5.2.5 not signed by Jia Tan, you must specify version 5.2.5:

(jl_YD86jd) pkg> add XZ_jll@5.2.5
   Resolving package versions...
   Installed XZ_jll ─ v5.2.5+3
    Updating `/tmp/jl_YD86jd/Project.toml`
⌃ [ffd25f8a] + XZ_jll v5.2.5+3
10 Likes