### Author:  [@sivakon](https://discourse.julialang.org/u/sivakon)
#### Post date: [September 11, 2024, 10:56pm UTC](https://discourse.julialang.org/t/string-optimisation-in-julia/119301/1 "2024-09-11T22:56:05Z")
Came across this benchmark that tries to create large string and save to file.
> **[GitHub - shagrouni/langs\_string\_build\_test: langs\_string\_build\_test](https://github.com/shagrouni/langs_string_build_test/)**
>
> langs\_string\_build\_test
The difference between Julia and Java is very high. (I’m assuming Java version the string is being saved and reused without allocating?). Does Julia have that optimization?
---
### Author:  [@matthias314](https://discourse.julialang.org/u/matthias314)
#### Post date: [September 12, 2024, 12:05am UTC](https://discourse.julialang.org/t/string-optimisation-in-julia/119301/2 "2024-09-12T00:05:50Z")
The `IOBuffer` in the Julia code is created without `sizehint`:
>
With, say,
```julia
buf = IOBuffer(sizehint = num*(ndigits(num) + 3))
```
it runs faster. In the rather small examples I tried it took ~~1/3~~ 40% less time.
EDIT: For Python at least there is nothing like `sizehint`, either. I don’t know if adding it defies the purpose of the benchmark.
---
### Author:  [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)
#### Post date: [September 12, 2024, 12:24am UTC](https://discourse.julialang.org/t/string-optimisation-in-julia/119301/3 "2024-09-12T00:24:21Z")
1/3rd less time still isn’t great.
---
### Author:  [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)
#### Post date: [September 12, 2024, 12:29am UTC](https://discourse.julialang.org/t/string-optimisation-in-julia/119301/4 "2024-09-12T00:29:28Z")
Also changing it to
```julia
for i in 1:num
print(buf, " J ")
print(buf, i)
end
```
makes it 38% faster.
---
### Author:  [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)
#### Post date: [September 12, 2024, 12:41am UTC](https://discourse.julialang.org/t/string-optimisation-in-julia/119301/5 "2024-09-12T00:41:53Z")
And another 40% from
```julia
for i in 1:num
write(buf, " J ")
write(buf, string(i))
end
```
---
### Author:  [@tecosaur](https://discourse.julialang.org/u/tecosaur)
#### Post date: [September 12, 2024, 6:45am UTC](https://discourse.julialang.org/t/string-optimisation-in-julia/119301/6 "2024-09-12T06:45:00Z")
I’m surprised to see that
```julia
print(buf, " J ")
print(buf, i)
```
performs differently to
```julia
print(buf, " J ", i)
```
and that
```julia
write(buf, string(i))
```
is better than
```julia
print(buf, i)
```
---
### Author:  [@sivakon](https://discourse.julialang.org/u/sivakon)
#### Post date: [September 12, 2024, 10:20am UTC](https://discourse.julialang.org/t/string-optimisation-in-julia/119301/7 "2024-09-12T10:20:07Z")
I moved J above the loop, and it sped up a little.
---
### Author:  [@Tortar](https://discourse.julialang.org/u/Tortar)
#### Post date: [September 12, 2024, 10:29am UTC](https://discourse.julialang.org/t/string-optimisation-in-julia/119301/8 "2024-09-12T10:29:53Z")
and maybe also surprisingly considering the context I tested (with `sizehint` on the buffer)
```julia
write(buf, " J ", string(i))
```
and it performs as
```julia
write(buf, " J ")
write(buf, string(i))
```
seems like the automatic conversion is slow somehow, and slower with more than one argument?
---
### Author:  [@greatpet](https://discourse.julialang.org/u/greatpet)
#### Post date: [September 12, 2024, 11:00am UTC](https://discourse.julialang.org/t/string-optimisation-in-julia/119301/9 "2024-09-12T11:00:27Z")
[Vararg functions](https://docs.julialang.org/en/v1/manual/functions/#Varargs-Functions) are often type-unstable. For example,
```julia
buf = IOBuffer()
@code_warntype print(buf, " J ", 1)
```
prints out warnings for type instabilities. This doesn’t happen with
```julia
buf = IOBuffer()
@code_warntype print(buf, " J ")
@code_warntype print(buf, 1)
```
Is there some Julia package which adopts an API similar to C++'s iostream, which is type stable when printing more than one argument?
---
### Author:  [@stevengj](https://discourse.julialang.org/u/stevengj)
#### Post date: [September 12, 2024, 12:21pm UTC](https://discourse.julialang.org/t/string-optimisation-in-julia/119301/10 "2024-09-12T12:21:15Z")
> [@greatpet](#):
>
> [Vararg functions](https://docs.julialang.org/en/v1/manual/functions/#Varargs-Functions) are often type-unstable. For example,
>
> ```julia
> buf = IOBuffer()
> @code_warntype print(buf, " J ", 1)
>
> ```
This is because `for` loops over a heterogeneous tuple are type-unstable. As discussed in [For statement type instability - #7 by stevengj](https://discourse.julialang.org/t/for-statement-type-instability/118817/7), however, it’s possible to fix this by unrolling, and should be a simple 1-line patch to change [the `print` definition](https://github.com/JuliaLang/julia/blob/945517ba4e15f7470b8790a696ba5404ef047f2f/base/strings/io.jl#L42-L52) to something like:
```julia
function myprint(io::IO, xs...)
lock(io)
try
foreach(xs) do x
print(io, x)
end
finally
unlock(io)
end
return nothing
end
```
which should be type-stable and hopefully should perform better. (Similarly for [`Base.print_to_string`](https://github.com/JuliaLang/julia/blob/945517ba4e15f7470b8790a696ba5404ef047f2f/base/strings/io.jl#L137-L151) and `Base.string_with_env`.) Anyone want to submit a PR?
As for `write(io, string(n))` vs. `print(io, n)`, I’m not sure what is going on (if there is really a difference?), because `print(io, n)` calls [`show(io, n)` which calls `write(io, string(n))`](https://github.com/JuliaLang/julia/blob/master/base/show.jl#L1244) already?
(However, the fact that writing an integer to a stream requires allocation of a string, for `string(n)`, is certainly hurting us in writing to an `IOBuffer` where writes are fast. At least in the `IOBuffer` case, we should in principle be able to use a view into the `IOBuffer` itself as the necessary buffer. The downside of this would be needing a long list of specialized `IOBuffer` methods for `show`, unless we do some clever refactoring.)
---
### Author:  [@gdalle](https://discourse.julialang.org/u/gdalle)
#### Post date: [September 12, 2024, 12:59pm UTC](https://discourse.julialang.org/t/string-optimisation-in-julia/119301/11 "2024-09-12T12:59:30Z")
>
>
> Following a discussion on \[Discourse\](https://discourse.julialang.org/t/string-o…ptimisation-in-julia/119301/10?u=gdalle), this PR tries to improve \`print\` (and variants) for more than one argument.
> The idea is that \`for\` is type-unstable over the tuple \`args\`, while \`foreach\` unrolls. I have no idea how to properly evaluate the potential performance gains though.
---
### Author:  [@nhz2](https://discourse.julialang.org/u/nhz2)
#### Post date: [September 12, 2024, 1:08pm UTC](https://discourse.julialang.org/t/string-optimisation-in-julia/119301/12 "2024-09-12T13:08:23Z")
This is about 10x faster than the original for me:
```julia
function do_thing(num)
s = Vector{UInt8}(undef, num*(ndigits(num) + 3))
digits = UInt8[0x30]
p = 1
for i in 1:num
s[p] = UInt8(' ')
s[p+1] = UInt8('J')
s[p+2] = UInt8(' ')
p += 3
n = 0
while true
digits[end-n] += 0x01
if digits[end-n] == 0x3A
digits[end-n] = 0x30
n += 1
if length(digits) == n
pushfirst!(digits, 0x31)
break
end
else
break
end
end
copyto!(s, p, digits, 1, length(digits))
p += length(digits)
end
resize!(s, p-1)
s
end
```
---
### Author:  [@Palli](https://discourse.julialang.org/u/Palli)
#### Post date: [September 12, 2024, 1:22pm UTC](https://discourse.julialang.org/t/string-optimisation-in-julia/119301/13 "2024-09-12T13:22:31Z")
Intriguingly all the languages fail at this eventually, Rust is the last to fail, and Julia and Perl otherwise survive the longest, and Perl and Python can be faster than Julia from the start…
Java and C# are however the first to “Failed to save”, then C++ and Nim. Then C also before Julia, so is this an important benchmark at any iteration count?
Also is this improved in later (nightly) Julia? Or at least has a good workaround in any Julia?
---
### Author:  [@PeterSimon](https://discourse.julialang.org/u/PeterSimon)
#### Post date: [September 12, 2024, 3:46pm UTC](https://discourse.julialang.org/t/string-optimisation-in-julia/119301/14 "2024-09-12T15:46:18Z")
> [@Palli](#):
>
> Intriguingly all the languages fail at this eventually
The tests were performed on a machine with only [16 GBytes RAM](https://github.com/shagrouni/langs_string_build_test/#test-environment). So I’m not sure what is being tested for the larger cases. Use of paging by Windows?
---
### Author:  [@sivakon](https://discourse.julialang.org/u/sivakon)
#### Post date: [September 13, 2024, 6:47am UTC](https://discourse.julialang.org/t/string-optimisation-in-julia/119301/15 "2024-09-13T06:47:40Z")
why does printing an integer require string conversion? This can be improved.
---
### Author:  [@tecosaur](https://discourse.julialang.org/u/tecosaur)
#### Post date: [September 13, 2024, 9:50am UTC](https://discourse.julialang.org/t/string-optimisation-in-julia/119301/16 "2024-09-13T09:50:54Z")
I also wonder if small string optimization (SSO) could help by allowing `" J "` to be fully stack allocated with no need for a pointer lookup.
---
### Author:  [@Palli](https://discourse.julialang.org/u/Palli)
#### Post date: [September 13, 2024, 11:48am UTC](https://discourse.julialang.org/t/string-optimisation-in-julia/119301/17 "2024-09-13T11:48:23Z")
Yes, it can, is on my TODO list… (a new string type that only live on the stack when small; there’s already a package for such, without my idea of a pointer for a to the heap, only used when longer), but until then not using `String`s rather `Char`s should help:
```julia
print(buf, ' ', 'J', ' ', 1)`
```
---
### Author:  [@greatpet](https://discourse.julialang.org/u/greatpet)
#### Post date: [September 13, 2024, 12:52pm UTC](https://discourse.julialang.org/t/string-optimisation-in-julia/119301/18 "2024-09-13T12:52:49Z")
Will small-string-optimization enable const-folding, i.e compile-time computation of strings?
---
### Author:  [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)
#### Post date: [September 13, 2024, 12:54pm UTC](https://discourse.julialang.org/t/string-optimisation-in-julia/119301/19 "2024-09-13T12:54:51Z")
Strings are already be somewhat consant foldable (although the compiler understandably has some difficulty reasoning about some of the very illegal things we do in Base to construct strings)
---
### Author:  [@tecosaur](https://discourse.julialang.org/u/tecosaur)
#### Post date: [September 13, 2024, 1:29pm UTC](https://discourse.julialang.org/t/string-optimisation-in-julia/119301/20 "2024-09-13T13:29:03Z")
Some of the basic testing I’ve done around a potential SSO (latest: [Slack](https://julialang.slack.com/archives/C688QKS7Q/p1726211201109249)) has me hopeful about the potential benefits in replacing the internal structure of `String` with a short/long form (under the hood). Actually doing so is currently beyond me, but we’ll see what happens 🙂
[Next page](https://discourse.julialang.org/t/string-optimisation-in-julia/119301.md?page=2)