What's the accepted way to ensure same integer behavior between 32-bit and 64-bit Julia?

So far I’ve been writing things assuming 64-bit Julia. How do I ensure that integers work the same for 32-bit Julia, and is there a quick way to edit integer literals in existing code to do that?

As an aside, why is there still a difference? Int32 and Int64 exists for both, why not use the same default Int?

Probably unnecessary, but a brief example:

# example.jl
const x = 2133133133
const y = 2*x
println("x::", typeof(x), " === ", x)
println("y::", typeof(y), " === ", y)

#=
32-bit
x::Int32 === 2133133133
y::Int32 === -28701030

64-bit
x::Int64 === 2133133133
y::Int64 === 4266266266
=#

I think the issue is that literal integers on 32 bit systems will generally be interpreted as 32 bit numbers, unless they’re too large for that. Perhaps you can convert it to Int64 before multiplying?

4 Likes

Yeah that’s what I’m referring to. I’m more asking this because I want to know if there’s an accepted way to do this besides manually changing all the integer literals e.g. 5 to Int64(5).

You can use a macro for that. For example via the SwapLiterals.jl package:

julia> using SwapLiterals

julia> @swapliterals [Int => Int32] begin
           println(typeof(1))
           2147483647 + 1 == typemin(Int32)
       end
Int32
true

julia> @swapliterals Int=>Int32 2147483648
ERROR: LoadError: InexactError: trunc(Int32, 2147483648)
[...]
7 Likes

Editing example.jl to be inside a SwapLiterals.@swapliterals block worked like a charm. I actually first tried @swapliterals include("example.jl") in the REPL because I was hoping I didn’t have to edit the code itself and forgot that macros worked on code. I wonder if there IS a way to invoke a macro in the REPL on a script’s code, sounds like something somebody would have come up with for some other reason by now.

I also tried SafeREPL.swapliterals!, and it’s a little strange. Editing example.jl to start with the line using SafeREPL; swapliterals!(Int => Int64) then running include("example.jl") printed the 32-bit output. However, it DID change the default integer literal to Int64 for lines entered in the REPL. In fact, pasting lines from example.jl afterward resulted in the expected 64-bit output, even if running include("example.jl") yet after that still printed the 32-bit output.a

I guess it possible to hack something up to achieve that, but I don’t know of an already existing such tool.

Yes, SafeREPL has an effect only on literals input at the REPL, so using SafeREPL is usually not something you will use in scripts (except in the “startup.jl” file).

Yeah I’ll ask about that in a separate post, and I doubt there is a quick answer. For now, adding a @swapliterals block to each file seem to be the best option, definitely beats adding Float64() to each and every integer literal.

Edit: Read just enough of the docs to make my first macro ever, but I’m not a fan of the dangerous eval in there. There’s probably a very simple way to improve this, but I’m still too confused by the code parsing process.

# need to import SwapLiterals
macro script64(scriptPath)
  script_string = read( eval(scriptPath), String)
  wrapped_string = "SwapLiterals.@swapliterals Int => Int64 begin " * script_string * " end"
  Meta.parse(wrapped_string)
end

Maybe include(expr -> :(@somemacro $expr), "/path/to/script")?

3 Likes

Kinda kicking myself for not checking the docs of include first because applying @swapliterals to every expression in a script sounds exactly what I’m looking for.

However, it’s throwing a

LoadError: syntax: invalid syntax (outerref @swapliterals) # REPL[26], line 1 # path\to\example.jl, line 1

I think include is reading linecount comments between each line of my script, and that the error happens because a macro can’t be applied to a comment. It’s easy to see all of them when I run include(println, "path/to/example.jl"):

#= path/to/example.jl:1 =#
x = 2133133133
#= path/to/example.jl:2 =#
y = 2x
#= path/to/example.jl:3 =#
println("x::", typeof(x), " === ", x)
#= path/to/example.jl:4 =#
println("y::", typeof(y), " === ", y)

I guess you could do expr -> expr isa LineNumberNode ? expr : :(@somemacro $expr) or expr -> expr isa Union{Integer,Expr} ? :(@somemacro $expr) : expr

1 Like

include(expr -> expr isa LineNumberNode ? expr : :(@swapliterals Int => Int64 $expr), "path/to/example.jl")
is throwing a
LoadError: syntax: invalid syntax (outerref @swapliterals) # REPL[16], line 1 =>(Int, Int64) x = 2133133133

Now that it’s throwing an error at the first line of script, I guess it wasn’t because of the macro being applied to the LineNumberNode (not a comment as I first thought).

Try

include(expr -> macroexpand(Main, :(@swapliterals $expr)), "path/to/example.jl")
5 Likes

I also didn’t know this method of include. Steven’s solution (using macroexpand) works for me. Actually, you could do it in this alternate way using a currently non-public SwapLiterals function (I should make a public API for this eventually):

swapper = SwapLiterals.literals_swapper([Int => Int64])
include(swapper, "path/to/example.jl")

This seems to be faster than invoking directly @swapliterals on each expression (@swapliterals has to do some work to transform its arguments into the swapper object above for each expression of the file).

3 Likes

Yep, that works. I’m still unfamiliar with macros, so I don’t understand why adding the macroexpand made the difference. I tried it out with a throwaway LineNumberNode in the REPL and it seems macroexpand is needed to prevent some extra error-throwing fluff from being stuffed into the expression:

julia> expr = LineNumberNode(0)
:(#= line 0 =#)

julia> macroexpand(Main, :(@swapliterals $expr)
:(#= line 0 =#)

julia> :(@swapliterals $expr)
:(#= REPL[16]:1 =# @swapliterals #= line 0 =#)

That non-public function works too and looks exactly how I’d like to do the Int => Int64 swap, to boot. I look forward to how it turns out in SwapLiterals!

Oh, there is one thing I feel I should let you know, there was this weird discrepancy when I tried the line include(expr -> macroexpand(Main, :(@swapliterals $expr)), "path/to/example.jl"). Since that line didn’t specify any integer types, it expectedly made both x and y BigInt. I made a slight edit to the script x = 3133133133…and the line made x Int64 (the default behavior of 32-bit Julia) instead of BigInt. This also happens when I try swaps = SwapLiterals.literals_swapper([Int => BigInt]).

1 Like

Do include(expr -> macroexpand(Main, :(@swapliterals Int=>Int64 $expr)), "path/to/example.jl")?

2 Likes

Yes, I did add the Int => Int64 argument soon after I copy-pasted the line and found it made x and y BigInt by default. I’m mentioning it because when I accidentally re-ran the line after I edited the x value to force 32-bit Julia to use Int64, it appears that @swapliterals failed to make x a BigInt (y was still BigInt).

Oh right, thanks for notifying me about that. As I never use 32-bits Julia, I forgot there can be Int64 literals which are not Int. The documention was not lying, but this is certainly unexpected, I will fix that.

2 Likes

I think I fixed the Int64 literals problem on a branch, but I can’t test much as I’m not able to use a 32-bits Julia. If @Benny or someone else would like to try this out to check that the problem above is solved, you can update the packages with

add SwapLiterals#int64-literals
add SafeREPL#int64-literals

in Pkg mode. Also feel free to comment directly on the PR.

2 Likes

Not much of a test but I used a different example script:

# example2.jl
x32 = 2147483647
x64 = 9223372036854775807
x128 = 170141183460469231731687303715884105727
println( "x32::", typeof(x32), "\n",
         "x64::", typeof(x64), "\n",
         "x128::", typeof(x128), "\n",
       )

And the outcomes:

julia> include( "example2.jl" )
x32::Int32
x64::Int64
x128::Int128

julia> include(expr -> macroexpand(Main, :(@swapliterals Int => Int64 $expr)), "example2.jl" )
x32::Int64
x64::Int64
x128::Int128

julia> include( SwapLiterals.literals_swapper([Int => Int64]) , "example2.jl" )
x32::Int64
x64::Int64
x128::Int128

julia> include(expr -> macroexpand(Main, :(@swapliterals $expr)), "example2.jl" )
x32::BigInt
x64::BigInt
x128::BigInt

julia> include( SwapLiterals.literals_swapper([Int => BigInt]) , "example2.jl" )
x32::BigInt
x64::Int64
x128::Int128

So it seems that @swapliterals works as expected (still, hardly a rigorous test), but if you want to make literals_swapper public for the same purpose, then I’d find it convenient to have the option to give no argument to get @swapliterals’s default BigInt behavior.

2 Likes