Interesting NASA "Language-Comparison" repository

#1

I came across this GitHub repository the other day, discovered it within the troves of NASA’s software catalog – the repository’s entry on the catalog is here. I thought this might be useful for Julia novices.

Basic Comparison of Various Computing Languages
Python, Julia, Matlab, IDL, R, Java, Scala, C, Fortran
We write codes to test the performance of the several computer languages. The codes solve simple tasks to test how the different languages handle loops, vectorization, multiprocessing, array manipulation, reading collections of files, etc. The goal is not to determine which language is better than the other but instead to highlight the strengths and weaknesses of each language. We also want stress the importance of having languages that can inter-operate with others to solve complex applications. The results (updated at least once a year) from this study are presented (including source codes) through a NASA website accessible to practitioners who need such information to determine which language is appropriate to accomplish a specific task.

7 Likes
#2

I just cloned the repo and tried to further optimise the laplace_jacobi benchmark. Much of the speed of C in this benchmark comes from the optimisation flag -Ofast. In fact, executing the julia script with julia -O3 --math-mode=fast give pretty much the same times, if not slightly better.

10 Likes
#3

You might consider making an issue on their repository to bring this to their attention.

3 Likes
#4

There is no license file (that I can find).

Is it automatically in the public domain since it’s the work of US government employees?

#5
#6

That’s not a software license though.

2 Likes
#7

On the catalog entry there’s this:

image
image736×686 34.8 KB

#8

Just because US government employees do work doesn’t automatically put something in the public domain - e.g. the designs of nuclear weapons, launch-codes, spy-planes, and encryption software are not public domain!

#9

I am benchmarking with more care the laplace_jacobi test, but the julia code is a bit slower than the C version. This is the MWE

using BenchmarkTools

const lib = tempname()
source = """
double _csolve(int N, double *u[]) {
    double tmp, diff;
    int i, j, r;
    double err;
    err = 0.0;
    for (r = 1; r != 1000; r++) {
        for (i = 1; i < N-1; i++) {
            for (j = 1; j < N-1; j++) {
                tmp = u[i][j];
                u[i][j] = ( 4.0*(u[i-1][j  ] + u[i+1][j  ]  +
                                 u[i  ][j-1] + u[i  ][j+1]) +
                                 u[i-1][j-1] + u[i+1][j+1]  +
                                 u[i+1][j-1] + u[i-1][j+1])/20.0;
                diff = u[i][j] - tmp;
                err += diff*diff;
            }
        }
    }
    return err;
}

"""
open(`gcc -fPIC -Ofast -march=native -xc -shared -o $(lib).dylib -`, "w") do pipe
    print(pipe, source)
end

function csolve(u::Vector{Vector{Float64}})
    ccall(("_csolve", lib), Float64, (Cint, Ptr{Ptr{Float64}}), length(u), u)
end

function solve(u::Matrix)
    n, m = size(u)
    error = zero(eltype(u))
    @inbounds begin
        for r = 1:1000
            for i in 2:n-1
                @simd for j in 2:m-1
                    temp = u[i, j]
                    u[i, j] = ( 4.0*(u[i-1, j]   + u[i+1, j]    +
                                     u[i,   j-1] + u[i,   j+1]) +
                                     u[i-1, j-1] + u[i+1, j+1]  +
                                     u[i+1, j-1] + u[i-1, j+1])/20.0
                    diff = u[i, j] - temp
                    error += diff*diff
                end
            end
        end
    end
    return error
end


N = 100
 u = rand(N, N)
cu = [rand(N) for i = 1:N]
@btime  solve($u)
@btime csolve($cu)

Running this with

julia --math-mode=fast --check-bounds=no -O3 test_laplace_jacobi_4.jl

gives

  72.297 ms (0 allocations: 0 bytes)
  56.904 ms (3 allocations: 1.78 KiB)

Any idea how to get to C speed?

#10

I will say this, the README states:

We implement the test cases from an angle of a novice programmer who is not familiar with the optimization techniques available in the languages.

I think it would be good to develop optimized versions of these routines, but perhaps as a forked project.

1 Like
#11

Those are probably classified

4 Likes
#12

Those are probably classified

They most certainly are, which supports my point. Works by US government employees are not necessarily public domain. From the Wikipedia page:

Publication of an otherwise protected work by the U.S. government does not put that work in the public domain. For example, government publications may include works copyrighted by a contractor or grantee; copyrighted material assigned to the U.S. Government; or copyrighted information from other sources.

#13

Just guessing: what happens if you swap the loops over i and j? Does simd do much here?

1 Like
#14

This is a stencil operation, so exchange i,j won’t help.(You still need to access its 8-neighborhood)

1 Like
#15

BTW, I did a similar comparison (not as comprehensive) for ESA a few years ago.

Here is the repo:

And the paper I wrote about it:

7 Likes
#16

I did that. What’s shown is the fastest ordering of looping on my machine.

#18

If you publish to github, people are automatically allowed to fork it, even without a license, I believe, as part of minimum terms of service.

However, without a license, the default copyright laws apply, meaning that you retain all rights to your source code and no one may reproduce, distribute, or create derivative works from your work.

Note: If you publish your source code in a public repository on GitHub, according to the Terms of Service, other GitHub users have the right to view and fork your repository within the GitHub site. If you have already created a public repository and no longer want users to have access to it, you can make your repository private. When you convert a public repository to a private repository, existing forks or local copies created by other users will still exist. For more information, see “Making a public repository private.”

2 Likes
#19

Those are protected by plain old secrecy—as in not telling people things you don’t want them to know—not copyright. If a government employee creates a work and posts it on the internet, then it is neither secret nor protected by copyright, therefore public domain.

6 Likes
#20

As a US government employee a few months ago, we were encouraged to put our work in the public domain as much as possible. Obviously we have secret and proprietary stuff that can’t be shared, but if we put for public consumption, it’s in the public domain.

6 Likes
#21

hmmm, I may be wrong but in America things are not protected by copyright unless they are explicitly stated not to be protected?