Julia simple String process and file I/O is slow

General Usage Performance
,
4

String is “whatever” – you can’t do anything much better in any language. But I guess you can at least reduce allocation a bit.

Not to mention we’re also spending like 30% of time writing to IO line by line, who knows how Python / Julia do buffer differently.

(reminds me of a test of printing 10^5 lines, pythons is significantly faster than a lot of languages because it prints multiple lines together instead of “flush” the IO every single time

 function func1(strd)
           URLS = split(replace(strd, "\n" => ""), "✖")
           filter!(occursin("."), URLS)
           trackers = open("trackers.txt", "w");
           write(trackers, join(URLS,"\n"))
           #print("Successfully completed\n");
           nothing
       end
func (generic function with 1 method)


julia> @btime func1($bigstring)
  209.628 μs (32 allocations: 122.61 KiB)

julia> function func2(strd)
           a = String[];
           for url in split(replace(strd, "\n" => ""), "✖")
               if occursin(".",url)
                   push!(a, url);
               end
           end
           trackers = open("trackers.txt", "w");
           for url in a
               write(trackers, url*"\n");
           end
           close(trackers);
           #print("Successfully completed\n");
       end
func (generic function with 1 method)

julia> @btime func2($bigstring)
  434.133 μs (3122 allocations: 294.19 KiB)
3 Likes