# Funny Benchmark with Julia (no longer) at the bottom

**URL:** <https://discourse.julialang.org/t/funny-benchmark-with-julia-no-longer-at-the-bottom/104611>\
**Category:** Performance\
**Tags:** benchmark\
**Created:** [October 5, 2023, 1:12pm UTC](https://discourse.julialang.org/t/funny-benchmark-with-julia-no-longer-at-the-bottom/104611 "2023-10-05T13:12:30Z")\
**Posts on this page:** 20\
**Page:** 4

<div class="post-metadata">

**Author:** ![dlakelan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlakelan/32/8491_2.png) [@dlakelan](https://discourse.julialang.org/u/dlakelan)\
**Post date:** [October 9, 2023, 4:32pm UTC](https://discourse.julialang.org/t/funny-benchmark-with-julia-no-longer-at-the-bottom/104611/61 "2023-10-09T16:32:27Z")

</div>

Gotcha so you can’t use any library that disables bounds checking _except_ stuff included as part of the standard library. It seems kinda arbitrary, particularly for a language like Julia where we’ve seen stuff pushed out of stdlib into packages, but it’s at least a reasonably well defined criterion. Thanks.

---

<div class="post-metadata">

**Author:** ![jling](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jling/32/212909_2.png) [@jling](https://discourse.julialang.org/u/jling)\
**Post date:** [October 9, 2023, 4:53pm UTC](https://discourse.julialang.org/t/funny-benchmark-with-julia-no-longer-at-the-bottom/104611/62 "2023-10-09T16:53:03Z")

</div>

> [@dlakelan](#):
>
> Gotcha so you can’t use any library that disables bounds checking _except_ stuff included as part of the standard library

yeah I would say for example, if Julia compiler figures out you don’t need bound check for

```julia
for i in eachindex(ary)
    ary[i]
end

```

it’s totally fine. But in this case, we have pattern of using value of one vector to index another, I don’t see a way to tell compiler it can ignore bounds

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [October 9, 2023, 5:45pm UTC](https://discourse.julialang.org/t/funny-benchmark-with-julia-no-longer-at-the-bottom/104611/63 "2023-10-09T17:45:07Z")

</div>

I seem to remember some recent posts by core devs (cannot find them now) saying that people should stop using `@inbounds`, not just for safety, but also because it could _prevent_ certain types of analysis and optimization.

Can anyone recall or explain what that was all about?

---

<div class="post-metadata">

**Author:** ![mikmoore](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mikmoore/32/31109_2.png) [@mikmoore](https://discourse.julialang.org/u/mikmoore)\
**Post date:** [October 9, 2023, 6:50pm UTC](https://discourse.julialang.org/t/funny-benchmark-with-julia-no-longer-at-the-bottom/104611/64 "2023-10-09T18:50:47Z")

</div>

> [@DNF](#):
>
> I seem to remember some recent posts by core devs (cannot find them now) saying that people should stop using `@inbounds`, not just for safety, but also because it could _prevent_ certain types of analysis and optimization.

I’m not finding a load of specific examples, but one was offered in [this comment on #48245](https://github.com/JuliaLang/julia/issues/48245#issuecomment-1398833632). Some other topics worth a glance might be [#50107](https://github.com/JuliaLang/julia/pull/50107) [#50641](https://github.com/JuliaLang/julia/pull/50641).

---

<div class="post-metadata">

**Author:** ![Syx\_Pek](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/syx_pek/32/6364_2.png) [@Syx\_Pek](https://discourse.julialang.org/u/Syx_Pek)\
**Post date:** [October 10, 2023, 3:25am UTC](https://discourse.julialang.org/t/funny-benchmark-with-julia-no-longer-at-the-bottom/104611/65 "2023-10-10T03:25:59Z")

</div>

It’s unlikely that a priority-queue approach will beat the manual top five selection as per above. (That is if you just throw the entire array into a priority-queue.)

```julia
function fastmaxindex!(xs::Vector{Int}, topn, maxn, maxv)
    ...
end

function special_sort!(xs::Vector{T}) where T
    # Only sorts the first 32 elements of a vector in > order.
    Lft, Rgt = 15, 18
    if xs[16] < xs[17]
        xs[16], xs[17] = xs[17], xs[16]
    end

    for _ in 1:15
        if xs[Lft] < xs[Rgt]
            xs[Lft], xs[Rgt] = xs[Rgt], xs[Lft]
        end
        Tmp = Rgt
        Val = xs[Tmp]
        while Val > xs[Tmp - 1]
            xs[Tmp] = xs[Tmp - 1]
            Tmp -= 1
        end
        xs[Tmp] = Val

        Tmp = Lft
        Val = Xs[Tmp]
        while Val < xs[Tmp + 1]
            xs[Tmp] = xs[Tmp + 1]
            Tmp += 1
        end
        xs[Tmp] = Val

        Rgt += 1
        Lft -= 1
    end
    return
end

function special_heapify!(xs::Vector{T}) where T
    N = length(xs) - 1
    Parent = N ÷ 2
    Child = 2Parent
    while Parent > 0
        CurrParent, CurrChild = Parent, Child
        Val = xs[CurrParent]
        for _ in 1:5
            BiggerChild = CurrChild + (xs[CurrChild + 1] > xs[CurrChild])
            (xs[BiggerChild] <= Val) && break
            xs[CurrParent] = xs[BiggerChild]
            CurrParent = BiggerChild
            CurrChild = 2BiggerChild
            (CurrChild > N) && break
        end
        xs[CurrParent] = Val
        Parent -= 1
        Child -= 2
    end
    return
end

function top_five!(xs::Vector{T}) where T
    special_heapify!(xs)
    xs[32] = xs[end] # Note that we destroy the array here
    special_sort!(xs)
    return
end

"""
julia> Arr = rand(Int, 1_000_000);

julia> top_five!(Arr)

julia> fastmaxindex!(Arr, 5, zeros(Int, 5), zeros(Int, 5))
5-element Vector{Int64}:
 1
 2
 3
 4
 5
"""

```

Results

```julia
julia> U = zeros(Int, 5); V = zeros(Int, 5);

julia> @btime fastmaxindex!(Arr, 5, $U, $V) setup = (Arr = rand(Int, 1_000_000))     
  552.900 μs (0 allocations: 0 bytes)

julia> @btime top_five!(Arr) setup = (Arr = rand(Int, 1_000_000))
  3.377 ms (0 allocations: 0 bytes)

```

---

<div class="post-metadata">

**Author:** ![Syx\_Pek](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/syx_pek/32/6364_2.png) [@Syx\_Pek](https://discourse.julialang.org/u/Syx_Pek)\
**Post date:** [October 10, 2023, 4:01am UTC](https://discourse.julialang.org/t/funny-benchmark-with-julia-no-longer-at-the-bottom/104611/66 "2023-10-10T04:01:42Z")

</div>

I can make `fastmaxindex!` faster though. Putting a sentinel (i.e. `topn += 1; maxv[begin] = typemax(Int)`) to remove the `idx == 1 && break` is not faster due to the bounds checking forced.

```julia
function fastermaxindex!(xs::Vector{Int}, topn, maxv, maxn)
    maxn .= 1
    maxv .= 0
    req = 0
    for (i, x) in enumerate(xs)
        x > req || continue
        idx = topn 
        u = maxv[idx - 1]
        req = min(u, x)
        while x > u
            maxv[idx] = u
            maxn[idx] = maxn[idx - 1]
            idx -= 1
            idx == 1 && break
            u = maxv[idx - 1]
        end

        maxv[idx] = x
        maxn[idx] = i
    end
    
    return maxn
end

```

Results

```julia
julia> U = zeros(Int, 5); V = zeros(Int, 5);

julia> @btime fastmaxindex!(Arr, 5, $U, $V) setup = (Arr = rand(Int, 1_000_000))    
  558.600 μs (0 allocations: 0 bytes)

julia> @btime fastermaxindex!(Arr, 5, $U, $V) setup = (Arr = rand(Int, 1_000_000))  
  395.900 μs (0 allocations: 0 bytes)

```

Note that there is some variance in the speed of the algorithms base on the input. For example, if the array is sorted smallest to largest, the fastest algorithm is actually a variant of the priority-queue algorithm I have above (the priority-queue has low variance in times and can be made to about 2ms, while on that sorted smallest to largest scenario, `fastmaxindex!` takes 15ms and `fastermaxindex!` takes around 3ms).

---

<div class="post-metadata">

**Author:** ![Syx\_Pek](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/syx_pek/32/6364_2.png) [@Syx\_Pek](https://discourse.julialang.org/u/Syx_Pek)\
**Post date:** [October 10, 2023, 7:01am UTC](https://discourse.julialang.org/t/funny-benchmark-with-julia-no-longer-at-the-bottom/104611/67 "2023-10-10T07:01:39Z")

</div>

Another thing I tried is to unroll the loop using `@generated`. But this yielded no improvement on the random case.

```julia
@generated function unrollmaxindex!(xs::Vector{Int}, topn::Val{T}, maxv, maxn) where T
    quote
        maxn .= 1
        maxv .= 0
        req = 0
        for (i, x) in enumerate(xs)
            x > req || continue
            $(
                if T == 1
                    quote
                        maxv[1] = req = x
                        maxn[1] = i
                    end
                else # T > 1
                    Exprs = Expr[]
                    push!(Exprs, :(u = maxv[$(T - 1)]))
                    push!(Exprs, :(req = min(u, x)))
                    for j in T:-1:2
                        push!(Exprs, :(x ≤ u && begin
                                                    maxv[$j] = x
                                                    maxn[$j] = i
                                                    continue
                                                end))                    
                        push!(Exprs, :(maxv[$j] = u))
                        push!(Exprs, :(maxn[$j] = maxn[$(j - 1)]))
                        if j != 2
                            push!(Exprs, :(u = maxv[$(j - 2)]))
                        else
                            push!(Exprs, :(maxv[$(j - 1)] = x))
                            push!(Exprs, :(maxn[$(j - 1)] = i))
                        end
                    end
                    Expr(:block, Exprs...)
                end
            )
        end
    
        return maxn
    end
end

```

However, this does yield an improvement on the worst case.

```julia
julia> @btime fastermaxindex!(Arr, 5, $U, $V) setup = (Arr = collect(1:1_000_000))
  3.271 ms (0 allocations: 0 bytes)

julia> @btime unrollmaxindex!(Arr, Val{5}(), $U, $V) setup = (Arr = collect(1:1_000_000))
  2.237 ms (0 allocations: 0 bytes)

```

This is the fastest version of the function I have so far, even beating my optimised priority-queue methods. Though for the benchmark, I don’t know how much compilation time this would add.

---

<div class="post-metadata">

**Author:** ![jling](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jling/32/212909_2.png) [@jling](https://discourse.julialang.org/u/jling)\
**Post date:** [October 10, 2023, 8:42am UTC](https://discourse.julialang.org/t/funny-benchmark-with-julia-no-longer-at-the-bottom/104611/68 "2023-10-10T08:42:54Z")

</div>

~~if you test this you may find it to be slower than the current one. (slower when I tested it locally on the real problem)~~

It’s about same speed as the old one

Notice the current `U` and `V` are `MVector`

---

<div class="post-metadata">

**Author:** ![Palli](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/palli/32/3380_2.png) [@Palli](https://discourse.julialang.org/u/Palli)\
**Post date:** [October 10, 2023, 10:29am UTC](https://discourse.julialang.org/t/funny-benchmark-with-julia-no-longer-at-the-bottom/104611/69 "2023-10-10T10:29:25Z")

</div>

Your statement on Julia fastest is now outdated (and maybe some of my text from below, which is from a DM, that was suggested I posted to the thread):

Julia is actually now 6% slower than Go, but what is worse, it scales worse, is 18% slower than Go, when scaling to “15k posts” (the new column).

The startup-cost of Julia is known, while after it’s fixed known cost (and trivial compilation time) I thought we would be on equal footing, so this is bad. It’s understandable Zig is fastest in the first column, now, and we could likely match/beat it, but likely only with StaticCompiler.jl. It’s sort of interesting to me that Zig loses this edge, should scale well, as well or better than Go?

It’s very intriguing that Java OOMs (the only language to do so, while its Graal implementation doesn’t).

> [@jling](#):
>
> we don’t have much to optimize, all I can see is:
> 
> - bound check (can’t remove)
> - `Dict` being slow

Would a LittleDict help? Or a hybrid of such (fixed size) and a regular Dict? There are also other Dict implementations available.

I believe you can avoid bounds checks, not just by `inbounds` that is directly disallowed (supposedly since it’s unsafe), but also always by asserts (and/or clever use of eachindex; and slicing). Arguable using asserts would be bending the rules, and unsafe, just a different for of saying `@inbounds`.

I’m not sure though, I think with asserts in right places could be safe in all cases (otherwise abort), and you move bounds checking out of loops, where they are only a problem. And you could say the compiler did it…

> [@algunion](#):
>
> @Palli, I would say that the original (the 600ms one) was just a straightforward implementation without any kind of type-instability. Maybe there was some unnecessary allocation

I assumed that to be the reason. I haven’t (yet) looked at the code in much detail. If there are unnecessary allocations then maybe Bumper.jl helps. It sort of gets you a Mojo advantage, and I think it should be added to Julia as a stdlib, but also to make its use transparent by the compiler. I see by now its changed and allocated 1/8th of physical memory by default (per Task), which sounds bad, but is actually great when you understand why (it’s only virtual memory and Linux overcommits, I would though be worried about Windows that doesn’t). It’s very likely it used `@inbounds`, but I think it could be avoided as explained.

---

<div class="post-metadata">

**Author:** ![jling](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jling/32/212909_2.png) [@jling](https://discourse.julialang.org/u/jling)\
**Post date:** [October 10, 2023, 10:30am UTC](https://discourse.julialang.org/t/funny-benchmark-with-julia-no-longer-at-the-bottom/104611/70 "2023-10-10T10:30:15Z")

</div>

> [@Palli](#):
>
> The startup-cost of Julia is known, while after it’s fixed known cost (and trivial compilation time) I thought we would be on equal footing, so this is bad. It’s understandable Zig is fastest in the first column, now, and we could likely match/beat it, but likely only with StaticCompiler.jl.

this benchmark does not contain startup or precompile time

> [@Palli](#):
>
> Would a LlittleDict help? Or a hybrid of such (fixed size) and a regular Dict? There are also other Dict implementations available.

Lilith has pointed out Dict-related accounts for 1% of run time, so it’s not just that

* * *

I assert it’s impossible to remove the bound check, because we’re using value from one thing to index another thing. In fact, I have confirmed the Golang implementation doesn’t elide bound check:

```go
> go run -gcflags="-d=ssa/check_bce" main.go
# command-line-arguments
./main.go:59:28: Found IsInBounds
./main.go:61:20: Found IsInBounds <--------- this is the hotspot
./main.go:64:18: Found IsInBounds
./main.go:79:15: Found IsSliceInBounds
./main.go:79:29: Found IsSliceInBounds
./main.go:82:9: Found IsInBounds
./main.go:89:26: Found IsInBounds
./main.go:93:18: Found IsInBounds
./main.go:94:18: Found IsInBounds

```

---

<div class="post-metadata">

**Author:** ![Syx\_Pek](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/syx_pek/32/6364_2.png) [@Syx\_Pek](https://discourse.julialang.org/u/Syx_Pek)\
**Post date:** [October 10, 2023, 10:42am UTC](https://discourse.julialang.org/t/funny-benchmark-with-julia-no-longer-at-the-bottom/104611/71 "2023-10-10T10:42:07Z")

</div>

Here is a completely different implementation, with fairly comparable speed. This might be “cheating” with regards to the rules, but at least it is of some interest.

```julia
function fastermaxindex!(tagmask::Vector{T}, i, topn, maxn, maxv) where T
    maxn .= 1
    maxv .= 0
    req = 0

    A = tagmask[i]

    for (j, mask) in enumerate(tagmask)
        i == j && continue
        x = count_ones(A & mask)
        x > req || continue
        idx = topn 
        u = maxv[idx - 1]
        req = min(u, x)
        while x > u
            maxv[idx] = u
            maxn[idx] = maxn[idx - 1]
            idx -= 1
            idx == 1 && break
            u = maxv[idx - 1]
        end

        maxv[idx] = x
        maxn[idx] = j
    end

    maxn    
end

for Type in [UInt128, Int]
    @eval function getrelatedposts!(tagmap, posts, topn, maxn, maxv, relatedposts, ::Val{$Type})
        tagmask = zeros($Type, length(posts))
        for (idx, post) in enumerate(posts)
            for tag in post.tags
                tagmask[idx] |= one($Type) << tagmap[tag]
            end
        end

        for (i, post) in enumerate(posts)
            fastermaxindex!(tagmask, i, topn, maxn, maxv)
            relatedpost = RelatedPost(post._id, post.tags, SVector{topn}(@view posts[maxn]))
            relatedposts[i] = relatedpost
        end

        return
    end
end

function fasterrelated(posts)
    tagmap = Dict{String, Int}()

    Ct = 0
    for (idx, post) in enumerate(posts)
        for tag in post.tags
            get!(() -> (Ct += 1), tagmap, tag)
        end
    end
    
    topn = 5
    maxn = MVector{topn, Int}(undef)
    maxv = MVector{topn, Int}(undef)
    
    relatedposts = Vector{RelatedPost}(undef, length(posts))
    if Ct < 64
        getrelatedposts!(tagmap, posts, topn, maxn, maxv, relatedposts, Val{Int}())
    else
        getrelatedposts!(tagmap, posts, topn, maxn, maxv, relatedposts, Val{UInt128}())
    end
    
    return relatedposts
end

```

It is marginally faster on my machine with the small test data. I’m not sure how well my solution would scale comparatively.

```julia
julia> @benchmark fasterrelated($posts)
BenchmarkTools.Trial: 334 samples with 1 evaluation.
 Range (min … max): 14.020 ms … 19.219 ms ┊ GC (min … max): 0.00% … 17.58%
 Time (median): 14.696 ms ┊ GC (median): 0.00%
 Time (mean ± σ): 14.994 ms ± 985.883 μs ┊ GC (mean ± σ): 1.76% ± 4.95%

        ▇█▃▃▃▄
  ▂▅▅▇▇███████▆▅▇▃▅▃▂▄▃▂▃▂▁▁▁▂▁▁▁▁▂▂▁▁▁▁▁▁▁▁▂▁▁▂▃▂▂▃▃▂▃▁▃▂▃▃▁▃ ▃
  14 ms Histogram: frequency by time 18.5 ms <

 Memory estimate: 3.66 MiB, allocs estimate: 55014.

julia> @benchmark related($posts)
BenchmarkTools.Trial: 303 samples with 1 evaluation.
 Range (min … max): 15.764 ms … 18.739 ms ┊ GC (min … max): 0.00% … 9.99%
 Time (median): 16.419 ms ┊ GC (median): 0.00%
 Time (mean ± σ): 16.472 ms ± 399.564 μs ┊ GC (mean ± σ): 0.13% ± 1.00%

          ▃▁▂▅▅▄▁▃▁ █▃▂▁
  ▃▁▄▃▃▃▅▇█████████▆█████▇▅▅▄▇▆█▅▆▃▆▃▄▇▅▃▃▃▃▁▁▃▁▁▁▃▁▁▁▁▃▁▁▁▁▁▃ ▄
  15.8 ms Histogram: frequency by time 17.9 ms <

 Memory estimate: 1.29 MiB, allocs estimate: 181.

```

---

<div class="post-metadata">

**Author:** ![algunion](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/algunion/32/51630_2.png) [@algunion](https://discourse.julialang.org/u/algunion)\
**Post date:** [October 10, 2023, 6:25pm UTC](https://discourse.julialang.org/t/funny-benchmark-with-julia-no-longer-at-the-bottom/104611/72 "2023-10-10T18:25:53Z")

</div>

Let’s go concurrent.

I did the most naive thing possible: add a parallel loop with a degree of parallelism equal to `nthreads()`. Nothing fancy.

If you notice flagrantly stupid, please slap me.

On my local machine with `--threads 4` (because now the testing server is 4 vCPUs):

**21ms → 8ms**

[Concurrent Julia - testing the waters by algunion · Pull Request #178 · jinyus/related\_post\_gen (github.com)](https://github.com/jinyus/related_post_gen/pull/178)

---

<div class="post-metadata">

**Author:** ![algunion](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/algunion/32/51630_2.png) [@algunion](https://discourse.julialang.org/u/algunion)\
**Post date:** [October 10, 2023, 6:39pm UTC](https://discourse.julialang.org/t/funny-benchmark-with-julia-no-longer-at-the-bottom/104611/73 "2023-10-10T18:39:19Z")

</div>

> [@giordano](#):
>
> Someone should tell them that running benchmarks on GitHub-hosted CI machines is going to be very noisy.

They updated the machine:

> NB: The benchmark runs on the Digital Ocean Droplet (CPU-Optimized).
> 
> - CPU: 4 vCPUs
> - RAM: 8GB
> - OS: Ubuntu 22.04

---

<div class="post-metadata">

**Author:** ![algunion](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/algunion/32/51630_2.png) [@algunion](https://discourse.julialang.org/u/algunion)\
**Post date:** [October 10, 2023, 9:19pm UTC](https://discourse.julialang.org/t/funny-benchmark-with-julia-no-longer-at-the-bottom/104611/74 "2023-10-10T21:19:06Z")

</div>

@jling ,

I _revived_ your [PR](https://github.com/jinyus/related_post_gen/pull/163) on my machine and added `PrecompileTools`.

I use this workflow to ensure the precompilation of the `related` function and its methods:

```julia
@setup_workload begin
    json_string = read("../posts.json", String)
    posts = JSON3.read(json_string, Vector{PostData})

    @compile_workload begin
        related(posts)
    end
end

```

However, when first-calling the `related` using the precompiled project, I get:

```julia
allocations: 3.928 MiB, 47.20% compilation time

```

If I remove the `PrecompileTools` workflow I get:

```julia
1.29 M allocations: 83.222 MiB, 986.72% compilation time

```

So it is clear that `PrecompileTools` is doing a great amount of compilation there - but not enough to just be able to call the function right away without some additional compilation?

I didn’t care too much about precompilation until now (always sufficed to just do a preemptive function call).

Any suggestions?

---

<div class="post-metadata">

**Author:** ![algunion](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/algunion/32/51630_2.png) [@algunion](https://discourse.julialang.org/u/algunion)\
**Post date:** [October 10, 2023, 9:25pm UTC](https://discourse.julialang.org/t/funny-benchmark-with-julia-no-longer-at-the-bottom/104611/75 "2023-10-10T21:25:07Z")

</div>

Let’s hope that this benchmark becomes more popular 🙂

 ![standard](https://global.discourse-cdn.com/julialang/original/3X/9/8/981e035d020732914ad0b84cd49f3692b8c4d879.png)

And the first shot at the parallel implementation seems decent, too:

 ![multicore](https://global.discourse-cdn.com/julialang/original/3X/4/c/4c2fd7bb8d31de7a4af6be983b09a662c7bf07c7.png)

So that everybody knows - the last jump is due to @Lilith improvement. My work for the parallel implementation was just adding a naive parallel loop over the standard implementation.

---

<div class="post-metadata">

**Author:** ![giordano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giordano/32/2166_2.png) [@giordano](https://discourse.julialang.org/u/giordano)\
**Post date:** [October 10, 2023, 10:17pm UTC](https://discourse.julialang.org/t/funny-benchmark-with-julia-no-longer-at-the-bottom/104611/76 "2023-10-10T22:17:51Z")

</div>

Exercise for the reader: do a plot speed vs lines of code (excluding comments, docstrings and blank lines) for the various languages, like

> <https://twitter.com/ChapelLanguage/status/1623389242822111232>

---

<div class="post-metadata">

**Author:** ![korbinian](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/korbinian/32/11270_2.png) [@korbinian](https://discourse.julialang.org/u/korbinian)\
**Post date:** [October 11, 2023, 12:35am UTC](https://discourse.julialang.org/t/funny-benchmark-with-julia-no-longer-at-the-bottom/104611/77 "2023-10-11T00:35:31Z")

</div>

There is the [bucket queue algorithm](https://en.wikipedia.org/wiki/Bucket_queue) for priority queues, which is fast for a small number of values/buckets. Here is a Julia implementation (26 lines): [https://github.com/korbinian90/ROMEO.jl/blob/master/src/priorityqueue.jl](https://github.com/korbinian90/ROMEO.jl/blob/master/src/priorityqueue.jl)

Not sure if this can improve anything, since Julia is already fastest. I might try today

---

<div class="post-metadata">

**Author:** ![algunion](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/algunion/32/51630_2.png) [@algunion](https://discourse.julialang.org/u/algunion)\
**Post date:** [October 11, 2023, 1:53am UTC](https://discourse.julialang.org/t/funny-benchmark-with-julia-no-longer-at-the-bottom/104611/78 "2023-10-11T01:53:29Z")

</div>

![jlfast](https://global.discourse-cdn.com/julialang/original/3X/0/e/0e1af1c92b4599425785bc432e5da2a711541546.png)

No more comment.

Thanks @Lilith for spotting and fixing this:

> <https://github.com/jinyus/related_post_gen/pull/182>
>
> Julia is worried that \`topn\` might be mutated and so deoptimizes. The Julia peop…le should really figure this out because there is literally only one assignment to \`topn\`, but whatever, moving the definition into the body of the loop speeds things up.
> 
> I also removed trailing whitespace.

---

<div class="post-metadata">

**Author:** ![algunion](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/algunion/32/51630_2.png) [@algunion](https://discourse.julialang.org/u/algunion)\
**Post date:** [October 11, 2023, 2:06am UTC](https://discourse.julialang.org/t/funny-benchmark-with-julia-no-longer-at-the-bottom/104611/79 "2023-10-11T02:06:34Z")

</div>

> [@Syx\_Pek](#):
>
> Though for the benchmark, I don’t know how much compilation time this would add.

Compilation time is not relevant for this benchmark. So, if you can confirm the improvement on the dataset, feel free to do a PR.

> **[GitHub - jinyus/related\_post\_gen: Data Processing benchmark featuring Rust,...](https://github.com/jinyus/related_post_gen)**
>
> Data Processing benchmark featuring Rust, Go, Swift, Zig, Julia etc. - GitHub - jinyus/related\_post\_gen: Data Processing benchmark featuring Rust, Go, Swift, Zig, Julia etc.

Also, if you want to test it on multiple random datasets, you can generate new datasets of various sizes using (the script is found in the root directory of the repo):

```julia
python3 gen_fake_posts.py 30000

```

---

<div class="post-metadata">

**Author:** ![korbinian](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/korbinian/32/11270_2.png) [@korbinian](https://discourse.julialang.org/u/korbinian)\
**Post date:** [October 11, 2023, 3:38am UTC](https://discourse.julialang.org/t/funny-benchmark-with-julia-no-longer-at-the-bottom/104611/80 "2023-10-11T03:38:36Z")

</div>

Oh, I [tried it](https://github.com/korbinian90/related_post_gen/blob/main/julia/related.jl), but the bucket queue was 10x slower for retrieving the largest 5 elements than fastmaxindex!  
2 reasons I think

1. It’s hard to preallocate and reuse the memory with a bucket queue
2. It solves a slightly different problem well, iteratively enqueue and dequeue for a long run

fastmaxindex! looks super efficient for the given problem

[Previous page](https://discourse.julialang.org/t/funny-benchmark-with-julia-no-longer-at-the-bottom/104611.md?page=3)

[Next page](https://discourse.julialang.org/t/funny-benchmark-with-julia-no-longer-at-the-bottom/104611.md?page=5)