Funny Benchmark with Julia (no longer) at the bottom

General Usage Performance
101

What makes this boilerplate is not how you write/space it, but the fact that you need to write it at all, i.e., in Scala you just define

class Petness(prive: String, val readme: String, var writeme: String)

making

  • prive private, i.e., no getter and setter
  • readme read only, i.e., getter, but no setter
  • writeme read and writable, i.e., getter and setter

Only when you need non-standard behaviour, you have to spell out getters and setters explicitly.

1 Like
102

Exactly! and Scala’s version will not compress anywhere near as much as the Java version, which is basically reflecting the fact that Scala probably doesn’t have a lot of useless fluff that you have to write or read when working with it. So smaller compression ratios == more human friendly code.

Ideally we’d look at a 3D plot with three dimensionless axes:

  1. Runtime / Minimum Runtime
  2. Compressed Size / Minimum Compressed Size
  3. Original Size / Compressed Size

(all of them arranged to be O(1) for good implementations, and smaller is better)

103

Totally tangential and my information theory is quite rusty, but I’d say the interesting limit is the Kolmogorov complexity rather than the Shannon entropy. (Which is equally information theoretic flavored.)

2 Likes
104

Related work

1 Like
105

What do you guys think about this?

I agree that it is essential to be fair in the context of the rules - but for somebody who does not understand Julia at a deep enough level, I think it is understandable why one would consider that unsafe means unsafe and that is the end of it.

Besides what @jling already presented, are there some good arguments?

Or should we agree that, indeed, we are breaking the rules by using the StaticArrays package?

106

Noice. That article basically says the same thing I did, which is that compression ratio is a dimensionless measure of verbosity. Cobol compresses about 20:1 and R compresses about 4:1. I wonder how much Julia compresses… I’d guess maybe even less than R.

107

Here’s my current replacement for StaticArrays.jl:

abstract type Vector5{T} <: AbstractVector{T} end
Base.size(::Vector5) = (5,)
Base.IndexStyle(::Type{<: Vector5}) = IndexLinear()
Base.getindex(A::Vector5, i::Int) = 
    i == 1 ? A.a :
    i == 2 ? A.b :
    i == 3 ? A.c :
    i == 4 ? A.d :
    i == 5 ? A.e :
    throw(BoundsError(A, i))

mutable struct MVector5{T} <: Vector5{T}
    a::T
    b::T
    c::T
    d::T
    e::T
end
MVector5{T}() where T = MVector5{T}(zero(T), zero(T), zero(T), zero(T), zero(T))
Base.setindex!(A::MVector5, v, i::Int) =
    i == 1 ? A.a = v :
    i == 2 ? A.b = v :
    i == 3 ? A.c = v :
    i == 4 ? A.d = v :
    i == 5 ? A.e = v :
    throw(BoundsError(A, i))

struct SVector5{T} <: Vector5{T}
    a::T
    b::T
    c::T
    d::T
    e::T
end
SVector5{T}(nt::NTuple{5,T}) where T  =
    SVector5{T}(nt...)
SVector5(av::AbstractVector{T}) where T  =
    SVector5{T}(av...)

edit:

Some thoughts for improvement:

  1. Would it be better to just use getfield and setfield! to implement getindex and setindex!?
2 Likes
108

They’re free to choose their rules. It’s not clear to me if C would be allowed as a language, I guess since it doesn’t have any checks they aren’t “disabled”? Seems arbitrary to me.

I think I’d personally prefer a rule something like no use of unsafe in code written by the submitter. They’ve excluded “standard library” already, So there’s some arbitrary boundary set by whether the language maintainers want it in the “standard” library or in some “external package”. Julia chooses to push stuff into external packages for good reasons. But StaticVector is still a “standard package” and the code isn’t unsafe (ie. it does bounds check).

109

New pull request submitted, removing StaticArrays.jl:

1 Like
110

My simple static vector implementation may need more optimization, but my current computer is not quite up to the task.

I also created a version that uses a package so that the overall benchmarking time is reduced due to pkgimages:

111

this is not going to be fast. The reason MVector was fast was due to its use of NTuple, and thus the need for unsafe_*.

What we need here is a built-in fixed-size array backed by stack allocation when it’s small

112

Not really, no. The reason MVector needed to do those pointer calls is that we don’t have a way of mutating tuples. i.e. it’s a difference between

mutable struct V1{N, T}
    v::NTuple{N, T}
end

and

mutable struct V2{N, T}
    v1::T
    v2::T
    v3::T
    ...
    vN::T
end

With V1 we can only replace the entire Tuple at a time, not the individual elements of tuple without resorting to pointer tricks. We unfortunately can’t make the second one for arbitrary sizes, but if we could that’s how MVector would be implemented.

113

Then why is it slow?

114

Because there’s a ton of other infrastructure built around StaticArrays.jl, and for the most part Mark’s implementation is just hitting default defined methods that aren’t taking advantage of the fixed size. Also the getindex and setindex! methods were using if-else branching which is not ideal, and also wasn’t eligible remove the error branch.

115

One thing that interests me here is that unlike StaticArrays.jl we do not have to solve the general problem. We know n = 5.

My current suspicion is that some of the important missing interface is the broadcasting.

You can see that StaticArrays.jl has a lot of machinery here:

116

try manually override fill!() and use fill!()

function fill!(ary::MVector5, val)
     for i in eachindex(ary)
          ary[i] = val
     end
end
117

I don’t think the unsafe related rules are sustainable anymore.

1 Like
118

tbh, Go is safe and fast somehow, I would be happy to learn what Golang is doing that’s so good

2 Likes
119

image
image1487×780 56.8 KB

This is from this Go commit :slight_smile: OMG

Some cleanup and a more generalized approach with Insertion sort by DrBlury · Pull Request #204 · jinyus/related_post_gen (github.com)

2 Likes
120

It’s their benchmark, their rules. If they don’t want to see the actual fastest code for the problem, it’s their lost.