Difference between `take!(IOBuffer(test))` and `Vector{UInt8}(test)`

Impressium · May 11, 2023, 7:25am

Hi all

there are often more than one way to do one thing in programming. My question is, why is the on thing faster than the other. Should we change it?

julia> length(test)
1000000

julia> @btime take!(IOBuffer(test));
  27.200 μs (4 allocations: 976.80 KiB)

julia> @btime Vector{UInt8}(test);
  333.500 μs (2 allocations: 976.67 KiB)

julia> length(test)
1

julia> @btime take!(IOBuffer(test));
  32.998 ns (3 allocations: 192 bytes)

julia> @btime Vector{UInt8}(test);
  124.556 ns (1 allocation: 64 bytes)

refs:

github.com

JuliaLang/julia/blob/0b5ec1f57eb38102235568de0a5f6a8a3ae92ace/base/strings/io.jl#L308


      
          ```jldoctest
          julia> io = IOBuffer("Haho");
          
          
julia> String(take!(io))
          "Haho"
          
          
julia> String(take!(io))
          "Haho"
          ```
          """
          IOBuffer(str::String) = IOBuffer(unsafe_wrap(Vector{UInt8}, str))
          IOBuffer(s::SubString{String}) = IOBuffer(view(unsafe_wrap(Vector{UInt8}, s.string), s.offset + 1 : s.offset + sizeof(s)))
          
          
# join is implemented using IO
          
          
"""
              join([io::IO,] iterator [, delim [, last]])
          
          
Join any `iterator` into a single string, inserting the given delimiter (if any) between
          adjacent items.  If `last` is given, it will be used instead of `delim` between the last
          two items.  Each item of `iterator` is converted to a string via `print(io::IOBuffer, x)`.

github.com

JuliaLang/julia/blob/0b5ec1f57eb38102235568de0a5f6a8a3ae92ace/base/strings/string.jl#L104


      
          
          
Create a new `String` from an existing `AbstractString`.
          """
          String(s::AbstractString) = print_to_string(s)
          @assume_effects :total String(s::Symbol) = unsafe_string(unsafe_convert(Ptr{UInt8}, s))
          
          
unsafe_wrap(::Type{Vector{UInt8}}, s::String) = ccall(:jl_string_to_array, Ref{Vector{UInt8}}, (Any,), s)
          unsafe_wrap(::Type{Vector{UInt8}}, s::FastContiguousSubArray{UInt8,1,Vector{UInt8}}) = unsafe_wrap(Vector{UInt8}, pointer(s), size(s))
          
          
Vector{UInt8}(s::CodeUnits{UInt8,String}) = copyto!(Vector{UInt8}(undef, length(s)), s)
          Vector{UInt8}(s::String) = Vector{UInt8}(codeunits(s))
          Array{UInt8}(s::String)  = Vector{UInt8}(codeunits(s))
          
          
String(s::CodeUnits{UInt8,String}) = s.s
          
          
## low-level functions ##
          
          
pointer(s::String) = unsafe_convert(Ptr{UInt8}, s)
          pointer(s::String, i::Integer) = pointer(s) + Int(i)::Int - 1
          
          
ncodeunits(s::String) = Core.sizeof(s)

tested on:

julia> versioninfo()
Julia Version 1.8.5
Commit 17cfb8e65e (2023-01-08 06:45 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: 8 × 11th Gen Intel(R) Core(TM) i7-11800H @ 2.30GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-13.0.1 (ORCJIT, tigerlake)
  Threads: 1 on 8 virtual cores
Environment:
  JULIA_PKG_USE_CLI_GIT = true

Cheers

greatpet · May 12, 2023, 9:06pm

How did you define your test variable? I got similar timings for the two (using Julia 1.9.0):

julia> using BenchmarkTools

julia> const test = fill(UInt8(1), 1000000);

julia> @btime take!(IOBuffer(test));
  32.839 μs (3 allocations: 976.73 KiB)

julia> @btime Vector{UInt8}(test);
  31.311 μs (2 allocations: 976.67 KiB)