Install from source: no such instruction

I’m installing from source on an Intel Skylake. Installation ends/fails on:

 /usr/bin/install -c -m 644  /work2/00434/eijkhout/stampede3/julia/julia-git/deps/srccache/mpfr-4.2.1/AUTHORS /work2/00434/eijkhout/stampede3/julia/julia-git/deps/srccache/mpfr-4.2.1/BUGS /work2/00434/eijkhout/stampede3/julia/julia-git/deps/srccache/mpfr-4.2.1/COPYING /work2/00434/eijkhout/stampede3/julia/julia-git/deps/srccache/mpfr-4.2.1/COPYING.LESSER /work2/00434/eijkhout/stampede3/julia/julia-git/deps/srccache/mpfr-4.2.1/NEWS /work2/00434/eijkhout/stampede3/julia/julia-git/deps/srccache/mpfr-4.2.1/TODO '/work2/00434/eijkhout/stampede3/julia/build-git-stampede3-intel24.0/usr-staging/mpfr-4.2.1/work2/00434/eijkhout/stampede3/julia/build-git-stampede3-intel24.0/usr/share/doc/mpfr/.'
 /usr/bin/mkdir -p '/work2/00434/eijkhout/stampede3/julia/build-git-stampede3-intel24.0/usr-staging/mpfr-4.2.1/work2/00434/eijkhout/stampede3/julia/build-git-stampede3-intel24.0/usr/lib/pkgconfig'
 /usr/bin/install -c -m 644 mpfr.pc '/work2/00434/eijkhout/stampede3/julia/build-git-stampede3-intel24.0/usr-staging/mpfr-4.2.1/work2/00434/eijkhout/stampede3/julia/build-git-stampede3-intel24.0/usr/lib/pkgconfig'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   477  100   477    0     0    991      0 --:--:-- --:--:-- --:--:--   991
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (22) The requested URL returned error: 404
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   260  100   260    0     0   1656      0 --:--:-- --:--:-- --:--:--  1645
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  188k    0  188k    0     0   339k      0 --:--:-- --:--:-- --:--:--  339k
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   327  100   327    0     0   2018      0 --:--:-- --:--:-- --:--:--  2018
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 6564k    0 6564k    0     0  5516k      0 --:--:--  0:00:01 --:--:-- 12.5M
{standard input}: Assembler messages:
{standard input}:495: Error: no such instruction: `vmovw (%rsi),%xmm0'
{standard input}:1043: Error: no such instruction: `vmovw (%rsi),%xmm0'
{standard input}:11153: Error: no such instruction: `vmovw (%rsi),%xmm0'
{standard input}:11805: Error: no such instruction: `vmovw (%rsi),%xmm0'
{standard input}:12167: Error: no such instruction: `vmovw (%rdx),%xmm0'
{standard input}:12786: Error: no such instruction: `vmovw (%rcx),%xmm0'
{standard input}:13539: Error: no such instruction: `vmovw (%rdx),%xmm0'
{standard input}:14486: Error: no such instruction: `vmovw (%rax),%xmm0'
{standard input}:15122: Error: no such instruction: `vmovw (%rax),%xmm0'

What version of binutils are you using? Sounds like it may be too old? Also, this is during compilation of MPFR (one of Julia dependencies), so you may want to direct your question to its developers.

1 Like

How do I find the version of binutils? What is the minimum you accept?
Can I install MPFR separately and link it in? What are the options for that?

Something like as --version or ld --version.

That’s again a question for MPFR for this specific problem.

Pass USE_SYSTEM_MPFR=1 to make. I’m assuming you’re already doing something like USE_BINARYBUILDER=0 or USE_BINARYBUILDER_MPFR=0, since you’re compiling MPFR in the first place. Note that we do expect a minimum version for MPFR (if anything that’s the version that has been tested), for example for Julia v1.11 that’s MPFR v4.2.1:

Interesting. With my own MPFR 4.2.1 it builds just fine.

Anyway. After an hour of compilation it bombs on 7zip it looks like.

-- Installing: /work2/00434/eijkhout/stampede3/julia/build-git-stampede3-gcc13.2/usr-staging/libgit2-a418d9d4ab87bae16b87d8f37143a4687ae0e4b2/work2/00434/eijkhout/stampede3/julia/build-git-stampede3-gcc13.2/usr/include/git2.h
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   439  100   439    0     0   1134      0 --:--:-- --:--:-- --:--:--  1131
100  221k  100  221k    0     0   267k      0 --:--:-- --:--:-- --:--:--  267k
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   477  100   477    0     0    813      0 --:--:-- --:--:-- --:--:--   812
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (22) The requested URL returned error: 404
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   260  100   260    0     0   1468      0 --:--:-- --:--:-- --:--:--  1468
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  188k    0  188k    0     0   345k      0 --:--:-- --:--:-- --:--:-- 81.6M
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   327  100   327    0     0   2043      0 --:--:-- --:--:-- --:--:--  2043
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 6564k    0 6564k    0     0  4917k      0 --:--:--  0:00:01 --:--:-- 20.0M
{standard input}: Assembler messages:
{standard input}:495: Error: no such instruction: `vmovw (%rsi),%xmm0'
{standard input}:1043: Error: no such instruction: `vmovw (%rsi),%xmm0'
{standard input}:11153: Error: no such instruction: `vmovw (%rsi),%xmm0'
{standard input}:11805: Error: no such instruction: `vmovw (%rsi),%xmm0'
{standard input}:12167: Error: no such instruction: `vmovw (%rdx),%xmm0'
{standard input}:12786: Error: no such instruction: `vmovw (%rcx),%xmm0'
{standard input}:13539: Error: no such instruction: `vmovw (%rdx),%xmm0'
{standard input}:14486: Error: no such instruction: `vmovw (%rax),%xmm0'
{standard input}:15122: Error: no such instruction: `vmovw (%rax),%xmm0'
make[4]: *** [makefile.list:469: lz4.o] Error 1
make[3]: *** [makefile:17: 7za] Error 2
make[2]: *** [/work2/00434/eijkhout/stampede3/julia/julia-git/deps/p7zip.mk:20: p7zip-17.04/build-compiled] Error 2

that was that software with major security holes, right? Have they been patched?

Anyway, can I skip 7zip or install my own, in which latter case what flags?

We use GitHub - p7zip-project/p7zip: A new p7zip fork with additional codecs and improvements (forked from https://sourceforge.net/projects/sevenzip/ AND https://sourceforge.net/projects/p7zip/). (as homebrew also does), which is a patched version of p7zip download | SourceForge.net, which is probably what you’re thinking of.

USE_SYSTEM_P7ZIP=1

1 Like

But I’m still not making it through a whole build.

    CC /work2/00434/eijkhout/stampede3/julia/build-git-stampede3-gcc13.2/src/runtime_intrinsics.o
{standard input}: Assembler messages:
{standard input}:7548: Error: no such instruction: `vmovw %xmm0,%edx'
{standard input}:7833: Error: no such instruction: `vmovw %eax,%xmm0'
{standard input}:8046: Error: no such instruction: `vmovw %eax,%xmm0'
{standard input}:8218: Error: no such instruction: `vmovw %eax,%xmm0'
{standard input}:8229: Error: no such instruction: `vmovw .LC9(%rip),%xmm0'
{standard input}:8333: Error: no such instruction: `vmovw %eax,%xmm0'
{standard input}:8346: Error: no such instruction: `vmovw .LC9(%rip),%xmm0'
make[2]: *** [/work2/00434/eijkhout/stampede3/julia/julia-git/src/Makefile:237: runtime_intrinsics.o] Error 1

can you show uname -a? The only thing I can think of is that your CPU is weird and doesn’t support AVX instructions somehow…

[staff ~:4] uname -a
Linux staff.stampede3.tacc.utexas.edu 5.14.0-362.24.1.el9_3.0.1.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Apr 4 22:31:43 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

This is the login node for a cluster. When I install on a compute node it does seem to get beyond this point.

Can you also give lscpu?

is at version “17.5.0+1” from Sept (not marked as security update, and LTS has a year old version), but I do see for 7-zip, though not p7zip: 7-Zip - Wikipedia

Versions prior to 23.0 also contained an arbitrary code execution vulnerability, fixed on 7 May 2023. A year later, a severe Remote code execution vulnerability was found on versions prior to 24.07 (CVE-2024-11477), it was fixed on 20 November 2024.

You might have XZ vulnerability in mind. XZ Utils backdoor - Wikipedia

a malicious backdoor was introduced to the Linux build of the xz utility within the liblzma library in versions 5.6.0 and 5.6.1 by an account using the name “Jia Tan”

What matters is what Julia decoding methods Julia supports, XZ/liblzma or whatever. What does Julia need and use for itself? It likely should disable everything else if not done already.

Good advice for those building. For those using precompiled Julia, I think the compression method can actually be chosen at runtime, I’m not sure if it applies in all cases, but I think I saw something like that.

Nothing wrong with doing that if you need to (or do not trust Julia builds…) but why? Do you need to, assuming Julia’s build is secure?

A few years ago a user reported that the precompiled binary was several times slower than a build from source. And at the time the build from source was a press of the button. For some reason this time it isn’t.

Another reason for wanting to build it from source is that want to integrate it with my custom MPI. But that’s the next step.

That’s a bit surprising. Julia is a compiler, compiling it from source instead of using pre-built binaries won’t change the code it generates. Also, official binaries use function multiversioning for different ISAs, to provide efficient cached code for different CPUs (a from-source build by default would target only the native host CPU).

Compiling Julia from source can affect the performance of the runtime (e.g. garbage collector), but that’s shouldn’t in principle be a bottleneck, or if it is that’s a problem on its own: ideally you’d like an application to spend most of its time in doing number crunching.

I don’t see the way how you compile Julia affects in any way the integration with MPI.

replace 7z with gzip · Issue #3824 · JuliaLang/Pkg.jl

Making install in script
 /usr/bin/mkdir -p '/work2/00434/eijkhout/stampede3/julia/julia-git/usr-staging/nghttp2-1.59.0/work2/00434/eijkhout/stampede3/julia/julia-git/usr/share/nghttp2'
 /usr/bin/install -c /work2/00434/eijkhout/stampede3/julia/julia-git/deps/srccache/nghttp2-1.59.0/script/fetch-ocsp-response '/work2/00434/eijkhout/stampede3/julia/julia-git/usr-staging/nghttp2-1.59.0/work2/00434/eijkhout/stampede3/julia/julia-git/usr/share/nghttp2'
 /usr/bin/mkdir -p '/work2/00434/eijkhout/stampede3/julia/julia-git/usr-staging/nghttp2-1.59.0/work2/00434/eijkhout/stampede3/julia/julia-git/usr/share/doc/nghttp2'
 /usr/bin/install -c -m 644 /work2/00434/eijkhout/stampede3/julia/julia-git/deps/srccache/nghttp2-1.59.0/README.rst '/work2/00434/eijkhout/stampede3/julia/julia-git/usr-staging/nghttp2-1.59.0/work2/00434/eijkhout/stampede3/julia/julia-git/usr/share/doc/nghttp2'
patching file lib/md4.c
Hunk #1 FAILED at 28.
1 out of 1 hunk FAILED -- saving rejects to file lib/md4.c.rej
make[3]: *** [/work2/00434/eijkhout/stampede3/julia/julia-git/deps/curl.mk:62: scratch/curl-8.6.0/source-extracted/curl-memdup.patch-applied] Error 1
make[2]: *** [Makefile:79: julia-deps] Error 2
make[1]: *** [Makefile:66: /work2/00434/eijkhout/stampede3/julia/julia-git/doc/_build/html/en/index.html] Error 2

Suggestions?