Converting string of bytes to integer

I am working on interfacing Arduino with Julia. There is a particular experiment of reading the values from a sensor connected to Arduino. For this, I have written a function in Julia 1.6.0. This function reads the values coming from the serial port. These values are strings like @\x02. So, I need to convert this string (let us say, s) into an integer value. For this, I am using the parse function, as given below:

num = parse(Int, s)                    # Convert String to integer
println(num) 

While executing this, I get an error as shown below:

LoadError: ArgumentError: invalid base 10 digit ‘@’ in “@\x02”

While finding its solution on the Internet, I came across the built-in functions in Python, as given on Bytes To Int Jsong - Learn.co. By following the functions given on this website, I could get the desired value from the string in Python, as shown below:

>>> lit = b'@\x02'
>>> n = int.from_bytes(lit, 'little')
>>> n
576

Could you please help me with implementing the same in Julia? Is there any built-in function in Julia that can help in converting from string (of bytes) to integer?

Thanking you in anticipation,

Regards,
Sudhakar

Strings in both Python (str) and Julia (String) encode text in Unicode. It’s not the same as the byte sequences you’re working with. The b'@\x02' in your Python example makes a bytes object, not a string. Julia also has bytes literals (docs here).
I did a bit of reading and I can get 576 by going through an IOBuffer:

lit = b"@\x02"
iolit = IOBuffer(lit)
n = read(iolit, Int16) # 576
# cannot read anymore from iolit, already at end
close(iolit)

I don’t think making and closing an IOBuffer for every couple of bytes like this is the right answer, but at least this shows it’s feasible. I also don’t know how to take the byte order into account (the argument 'little' in the Python example).

5 Likes

Thanks, @Benny, for your response. I have been able to get the value now. However, I have a function that receives the strings and then returns the equivalent integer value. This function is as given below.

function analogRead(file_des::SerialPorts.SerialPort , pin_no::Int64)
  str = "A"*string(Char(48+pin_no))   #"An" for analog value on pin n
  write(file_des,str)
  sleep(0.1)                          # Delay next step by 100 milliseconds
  n = bytesavailable(file_des)          # Get number of bytes in input buffer
  s = read(file_des, n)                # Read n bytes from SerialPort
  #######################################
  # Need to add the code to get the integer num from s 
  #######################################
 return num                            # Return the equivalent integer
end

In the above function, when I get the string in s, I need to make it a byte literal and then convert it into an integer value. In other words, I need to add a command in this function, which takes s (assuming that s = “@\x02”) and returns the byte literal of this s (b"@\x02"). This means I need to know how I can add b in this string. I have tried the star(*) operator to concatenate b (referring to byte) and the string s. However, it creates another string “b@\x02”, whereas we need b"@\x02".

Could you please help me with how to obtain a byte literal from the string?

Thanking you in anticipation,

Regards,
Sudhakar

I got this working by rewriting the function as given below:

function analogRead(file_des::SerialPorts.SerialPort , pin_no::Int64)
  str = "A"*string(Char(48+pin_no))   #"An" for analog value on pin n
  write(file_des,str)
  sleep(0.1)                          # Delay next step by 100 milliseconds
  n = bytesavailable(file_des)          # Get number of bytes in input buffer
  s = read(file_des, n)                # Read n bytes from SerialPort
  iolit = IOBuffer(s) 
  num = read(iolit, Int16)
  # println(sizeof(num))
  # println(num[1])
  # @printf("value = %d", Int(num))
  close(iolit)
  
  # typeof(s) 
  # k = parse(Int,s)                    # Convert String to integer
  return num                           # Return the integer
end

Thank you, @Benny, for your response.

Using reinterpret should be faster than going through IOBuffers:

# Generate random `Int16`s
julia> nums = rand(Int16, 2)
2-element Vector{Int16}:
  10792
 -18596

# Convert to bytes, this is your actual input
julia> bytes = collect(reinterpret(UInt8, nums))
4-element Vector{UInt8}:
 0x28
 0x2a
 0x5c
 0xb7

# Reintepret as `Int16`, you'll get back the same numbers we generated above
julia> reinterpret(Int16, bytes)
2-element reinterpret(Int16, ::Vector{UInt8}):
  10792
 -18596

Or, if you want to start from a byte string:

# your original example
julia> reinterpret(Int16, b"@\x02")
1-element reinterpret(Int16, ::Base.CodeUnits{UInt8, String}):
 576

# Generate a new random byte string
julia> input = String(rand(UInt8, 8))
"\xb2\Ub2aab\xca3 "

# Reinterpret as `Int16`
julia> reinterpret(Int16, codeunits(input))
4-element reinterpret(Int16, ::Base.CodeUnits{UInt8, String}):
  -3406
 -21838
 -13653
   8243

Benchmarks:

julia> using BenchmarkTools

julia> f(s) = reinterpret(Int16, s)
f (generic function with 1 method)

julia> function g(s)
           io = IOBuffer(s)
           n = read(io, Int16)
           close(io)
           return n
       end
g (generic function with 1 method)

julia> @btime f($(b"@\x02"))
  3.542 ns (0 allocations: 0 bytes)
1-element reinterpret(Int16, ::Base.CodeUnits{UInt8, String}):
 576

julia> @btime g($(b"@\x02"))
  10.278 ns (1 allocation: 64 bytes)
576

Note that reinterpret is completely non-allocating and you don’t need to read numbers one-by-one

7 Likes
  1. I suggest removing the Solution mark from my earlier response because, as I pointed out (and demonstrated by @giordano) , making and closing IOBuffers like that every time you read bytes from SerialPort wastes memory and hurts performance. It’s better if you leave this question open so that people who have worked on similar projects know to give you better answers.
  2. @giordano answer actually looks like a great replacement for Python’s int.from_bytes, can even be used to switch between other types.
  3. It actually looks like SerialPort is already some sort of IO object because it has read and write, so making IOBuffer seems redundant.
  4. Literals syntax like b"" is just a convenient way to write a type of object in code, it’s not a b concatenated to a String object. It’s like how Int8[1,2,3] makes an Array{Int8, 1}; it’s not an Int8 concatenated to [1,2,3].
1 Like

Why not just num = read(file_des, Int16)?

4 Likes