BitIntegers: creation in loop; how to avoid allocations?

General Usage
1

I asked this question in a GitHub issue but got no replies:

I’d like to create all BitIntegers of byte length up to 1024. I don’t want to write 128 lines of code for that, so I’m trying to use a loop. Running the following code gives an error:

for k=8:8:1024  k∈(8,16,32,64,128) && continue;  
    @eval BitIntegers.@define_integers $k Int$k UInt$k end

ERROR: syntax: invalid type signature around /home/user/.julia/packages/BitIntegers/VJRl1/src/BitIntegers.jl:62
Stacktrace:
 [1] top-level scope
   @ none:1
 [2] eval(m::Module, e::Any)
   @ Core ./boot.jl:370
 [3] top-level scope
   @ REPL[11]:2

Running the code below works but does not create my UInt’s in the global scope:

for k=8:8:1024  k∈(8,16,32,64,128) && continue; 
    eval("BitIntegers.@define_integers $k Int$k UInt$k;") end

julia> UInt24
ERROR: UndefVarError: `UInt24` not defined

Is there a workaround for doing this?

Also, an unrelated question: If I create Matrix{Int128} and do row and column operations on it (like Smith normal form), that will cause no additional allocations. However, on Matrix{Int1024} there are many additional allocations. Is there a way to avoid this? Are BitIntegers still converting internally to BigInts? If so, then what is the purpose of BitIntegers, what advantage do they offer compared to just using BigInts from the start?

Theoretically, would it be possible to refactor this package in a way that BitIntegers would behave more like Int64 and Int128 with regard to allocations? Or is the inherent nature of CPUs and RAMs such that this is impossible? (I noticed there are no integers of custom byte sizes in rust, go, …, so I was wondering…)

2

The following works for me:

julia> for k=8:8:1024  k∈(8,16,32,64,128,512,1024) && continue;
           eval(Meta.parse("BitIntegers.@define_integers $k Int$k UInt$k;")); end

Also, an unrelated question: If I create Matrix{Int128} and do row and column operations on it (like Smith normal form), that will cause no additional allocations. However, on Matrix{Int1024} there are many additional allocations. Is there a way to avoid this? Are BitInteger s still converting internally to BigInt s?

Are you doing divisions? Then see point 1) of the README GitHub - rfourquet/BitIntegers.jl: Fixed-width integers similar to builtin ones.

Or is the inherent nature of CPUs and RAMs such that this is impossible? (I noticed there are no integers of custom byte sizes in rust, go, …, so I was wondering…) (I noticed there are no integers of custom byte sizes in rust, go, …, so I was wondering…)

They don’t exist because there aren’t that many application. In my experience, either you want machine integers (aka Int64) for control flow and counting things, or you want to do some exact computations, in which case you want arbitrary large integers. An application that falls in the middle of these two is working efficiently modulo m (m an arbitrary integer). Then there exist special purpose libraries that implement fast operations modulo m by representing everything with a fixed number of bits.

2 Likes
3

Thank you, it works for me too now. I just needed to add 256 to the tuple of ignored bitwidths.

Regarding allocations, so only div is causing allocations? Operations +, *, mod, <<, >> do not, even for 1024-width integers?

Yeah, BitIntegers can be also used to represent sets, as an alternative to BitVectors.

4

I did a bit of benchmarking in the REPL and as far as I can tell, the answer is yes.

(@rfourquet: please correct me if I am wrong.)

1 Like
5

If you don’t want to use Meta.parse, you can say

@eval BitIntegers.@define_integers $k $(Symbol("Int", k)) $(Symbol("UInt", k))
1 Like
6

If sets is your application, there is also the GitHub - matthias314/SmallCollections.jl: A Julia package providing variable-length set and vector types that don't allocate.

1 Like