Why does `code_native` output 32-bit assembly in 64-bit Julia?

General Usage
1

I’m running Julia 0.6.3 (downloaded from the website) on macOS 10.13.5. In the REPL, I can use code_llvm to see how Julia lowers the + operator for integers:

julia> code_llvm(+, (Int, Int))

define i64 @"jlsys_+_60775"(i64, i64) #0 !dbg !5 {
top:
  %2 = add i64 %1, %0
  ret i64 %2
}

All good: it’s generating an LLVM function that adds two unboxed 64-bit integers, and returns an unboxed 64-bit integer as the result. Let’s see what the machine code for that looks like:

julia> code_native(+, (Int, Int), :intel)
        .section        __TEXT,__text,regular,pure_instructions
Filename: int.jl
        push    ebp
        dec     eax
        mov     ebp, esp
Source line: 32
        dec     eax
        lea     eax, [edi + esi]
        pop     ebp
        ret
Source line: 32
        nop
        nop
        nop
        nop
        nop
        nop

What in the world is going on here? First off, eax gets decremented twice, needlessly. Second, since we’re doing 64-bit addition, shouldn’t it be lea rax, [rdi + rsi]?

This is just limited to the assembly dump, though. Actual 64-bit addition works fine, which means that weirdly, the results of code_native are not representative of the code Julia is actually executing. So what gives? Am I using code_native incorrectly? Is this a bug? Should I just pretend every exx is an rxx? And even if I did that substitution, what’s with the clearly needless dec instructions? Are they some kind of padding for alignment?

I should note that this issue applies to every code_native dump I’ve looked at; I isolated the + example because it was so simple.

2

My machine gets proper 64-bit assembly, so I’d guess this is specific to either mac or your binaries. However, I’d suggest defining f(x,y)=x+y, i.e. wrapping intrinsics, for stuff like code_native(f, (Int,Int), :intel). Intrinsics are somewhat finicky; in fact introspection does not work on them at all, for me. I am surprised that you’re not getting

julia> code_native(+, (Int,Int), :intel)
WARNING: Could not determine size of symbol
1 Like
3

I tried this on a downloaded 0.7.0-beta.0 on Sierra and on a locally built binary from today on High Sierra. In the first case, I got a similar result as above: eax instead of rax etc. (It makes no difference if I define my own f(x,y)=x+y or look at + directly.)

In the second case (locally built 0.7.0-beta.214 on High Sierra) the machine code is correct, so either this has recently been fixed, or it has to do with Sierra vs High Sierra, or there is a problem with the build process for the downloaded binaries.

4

Strange. I’m just trying it on a Fedora VM, and I reproduce what you’re seeing, though I’m not sure it’s for the reason that you give? I was under the impression that + was just a normal function, implemented for (::Int, ::Int) in terms of an intrinsic.

Actually, this works identically for me on both macOS and Fedora:

julia> code_typed(+, (Int,Int))
1-element Array{Any,1}:
 CodeInfo(:(begin 
        return (Base.add_int)(x, y)::Int64
    end))=>Int64

Base.add_int would appear to be the actual intrinsic. It’s seeming to me, then, that

  1. Linux and macOS use different code for displaying assembly.
  2. Neither platform’s assembly displayer works quite right.

Given that the assembly printouts on both platforms shows nops past the end of the function body (normally used for inserting padding between pieces of code), I’m guessing that code_native is actually giving a dis-assembly of whatever machine code LLVM has in RAM, and I wouldn’t find it particularly surprising to see a different disassembler used for different platforms.

I still have no idea why the assembly dump of + would fail on Linux, though since there’s no Julia error, I’m wondering if the WARNING is actually just the return text of an external library function code_native hooks into; “Could not determine size of symbol” sounds much more like a disassembly issue than a Julia-level problem (I’m not even sure what a symbol’s size would be in a Julia context).

Also, I’m starting to wonder if the extra dec instructions I’m seeing are due to a mistaken decoding of the actual instructions, though I’d have to get out an ISA reference manual to check that theory.

5

Ok, this appears to be two bugs, then. So, let’s summarize the seen behavior, in order to open an issue:

1-a @code_native +(3,4) gives assembly output.
1-b @code_native +(3,4) gives WARNING: Could not determine size of symbol

2-a f(x,y)=x+y; @code_native f(3,4) gives the expected 64-bit result [*].
2-b f(x,y)=x+y; @code_native f(3,4) gives an unexpected 32-bit result.

I am running archlinux on x86_64. I observed (1-b,2-a) for both the binary packaged 0.6.3 and source-built 0.7-master, and observed this for a long time (didn’t even occur to me that this might be an unexpected bug instead of a known wart). @Per reports (1-a, 2-b) on official binaries of 0.7.0-beta.0 on Sierra and (1-a,2-a) on source-built 0.7.0-beta.214 on High Sierra. @lcmylin , what did you experience on which machines?

[*] Expected:

julia> f(x,y)=x+y; @code_native f(3,4)
	.text
Filename: REPL[4]
	pushq	%rbp
	movq	%rsp, %rbp
Source line: 1
	leaq	(%rdi,%rsi), %rax
	popq	%rbp
	retq
	nopw	(%rax,%rax)
6

I experienced (1-a,2-b) with official 0.6.3 binaries on macOS High Sierra (10.13.5). I experienced (1-b,2-a) with official 0.6.3 binaries on Fedora 28 (Docker container, fedora:latest after interactively running dnf update and dnf install julia). I haven’t tested source-built or 0.7 binaries on either OS.

7

I tried 0.6.3 on Sierra and got 1-a 2-b before building the system image. Then built the system image with

include(joinpath(JULIA_HOME, Base.DATAROOTDIR, "julia", "build_sysimg.jl"))
build_sysimg(force=true)

After restarting Julia, I got 1-b 2-b.

8

How strange. I guess code_native is seriously unstable then. Probably worth filing a bug report for, though I’d find it hard to believe that these aren’t known issues already…

9

I don’t see anything related to this on the issues page right now, so I’ll probably post an issue later today.

10

I’ve opened an issue on Github.
https://github.com/JuliaLang/julia/issues/28046

1 Like