Convert a number to a byte array

how to do it? I’ve been trying to find information for hours. and I don’t understand how to work with bytes ( I’m a C/C++ programmer).

NOT WORKING!
an example explaining the essence:
a = 0x1234
b = Array{UInt8}(undef,2)
b = copy(a)

you should get the following:
b[1] = 0x34
b[2] = 0x12
BUT:
b = 0x1234 (not array)

1 Like

I also want to know the answer.

You want reinterpret:

julia> a = 0x1234
0x1234

julia> b = reinterpret(UInt8, [a])
2-element reinterpret(UInt8, ::Vector{UInt16}):
 0x34
 0x12

You are confusing assignment with mutation.

4 Likes

Try this:

a = 0x1234
b = reinterpret(NTuple{2,UInt8}, a)
(0x34, 0x12)
4 Likes

thank you very much

how do I do the same with BigInt?

You can’t use reinterpret because a BigInt is stored as a pointer and some lengths, but you can get at the raw bytes in a variety of ways, from low-level (exploit the raw pointer) to high-level. The best-documented way is to use the digits! function:

julia> a = big(11)^100
137806123398222701841183371720896367762643312000384664331464775521549852095523076769401159497458526446001

jjulia> digits!(Vector{UInt8}(undef, ndigits(a; base=256)), a, base=256)
44-element Vector{UInt8}:
 0xb1
 0xe1
 0x4f
 0x64
 0xe2
 0x52
 0xe1
    ⋮
 0xcc
 0xf1
 0x0e
 0x71
 0xd8
 0x03

See also the discussion in BigInt to bytes — since that discussion, digits! got optimized using mpz_export as explained in the thread.

What are you trying to accomplish?

4 Likes

I am currently studying encryption algorithms. I was looking for solutions that would help me work at a high level (I was thinking about python). but I came across Julia. and now I’m trying to use it to do tests. and then write an optimized version in C

can you also suggest reverse operations?)

int ← array
BigInt ← array

If you are reading an Int from a binary file or a stream io, you can do read(io, Int). You can even do this with an array bytes via read(io, IOBuffer(bytes)), or you can alternatively do reinterpret(Int, bytes)[1].

With BigInt, it depends on how the data is encoded in the array. If it’s an array bytes of base-256 digits (bytes) in little-endian order (least significant first) like what is returned from digits, you can treat it as a polynomial and use evalpoly(BigInt(256), bytes).

(There are also lower-level functions Base.GMP.MPZ.export! and Base.GMP.MPZ.import! that interface the low-level GMP export/import functions, but these are currently undocumented. It seems like it would be worthwhile to have a documented fast path, analogous to Python’s int.from_bytes function, though a need for this doesn’t seem to have come up much.)

What are you trying to accomplish? Converting BigInt values to and from arrays of base-256 digits doesn’t seem to be a very common operation. You can use parse and string to convert from/to strings in various bases, typically for serialization (storage) purposes, and you can also use serialize/deserialize.

4 Likes

If you need a larger fixed-width integer, you might be interested in BitIntegers.jl:

julia> using BitIntegers

julia> reinterpret(UInt8, [UInt1024(0xFFFFFF)])
128-element reinterpret(UInt8, ::Vector{UInt1024}):
 0xff
 0xff
 0xff
 0x00
 0x00
 0x00
 0x00
 0x00
...

The package provides 1024 bit integers and allows for the definition of other fixed width integers.

julia> using BitIntegers

julia> BitIntegers.@define_integers 2048
@uint2048_str (macro with 1 method)

julia> UInt2048
UInt2048

julia> sizeof(UInt2048)
256
5 Likes

I wrote above that I am working on encryption algorithms. not only do I need to work with large numbers, but I also need to work with their bits

You can do bitwise operations directly on BigInt values. Can you give an example of an operation that you would need to perform that requires conversion to/from a byte array?

Or is it more that you want to encrypt/decrypt an arbitrary byte stream by treating chunks of it as bignums?

RFC 8032 - Edwards-Curve Digital Signature Algorithm (EdDSA) example it was necessary to change the highest bit of the most significant byte (a 32-byte number). algorithm eddsa

Can’t you use ndigits(n, base=256) to get the number of bytes (= base-256 digits), and then xor with a 0x80 (= 10000000 in binary) in that byte?

xor(n, big(0x80) << ((ndigits(n, base=256) - 1) * 8))
1 Like

a good option :smiley:

In principle there is an even more efficient option: call the low-level mpz_combit function to flip a single bit of a bignum. Currently, Julia doesn’t export any high-level interface to this, but you can “easily” write one:

function combit!(n::BigInt, bit::Integer)
    @ccall "libgmp".__gmpz_combit(n::Ref{BigInt}, bit::Culong)::Cvoid
    return n
end
combit(n::BigInt, bit::Integer) = combit!(deepcopy(n), bit)

Then you can do combit(n, ndigits(n, base=256) * 8 - 1) to flip the highest bit of the most significant byte, or even call combit! to change n in-place.

It may not be worth the trouble, since I doubt that bit flipping is a performance-critical step in the cryptographic computation, but it would be nice to expose some more of these functions: expose GMP bit-flipping operations (tstbit, combit, etcetera) · Issue #53793 · JuliaLang/julia · GitHub

1 Like

Thanks again! I’m not chasing optimization here, the usual unoptimized code will be enough for me.

You may very well have additional reasons for writing it in C but if we’re only discussing speed,chances are that you can get about the same performance in Julia as in C.

3 Likes

Note that this requires additional care when doing bignum operations, because when writing high-level operations each bignum operation like x + y heap-allocates a new bignum for the result. When writing low-level code you really want to work in-place as much as possible, reusing one of the operands for the result.

Of course, in Julia you do have access to the low-level in-place GMP functions if you really want, and there have been some attempts (e.g. Inplace.jl) to provide a higher-level interface to this. But it is important to point out that highly optimized Julia bignum code may look quite different from high-level bignum code.

(But it’s still easier to optimize the Julia code, where you only need to make localized incremental changes, than it is to rewrite from scratch in C/C++.)

5 Likes