Julia Philosophical Direction

Offtopic
1

On the Julia homepage, the goals of Julia are stated as:

  • Fast
  • Composable (multiple dispatch)
  • Dynamic
  • General
  • Reproducible
  • Open Source

Other informal goals would be “batteries included” (base has much of what you need), eliminating the two language problem, cross-platform, and support for parallel computation (either on 1 machine or a network; Julia having been designed in the era of parallelism).

Let me pose a question on the goal of “Fast”:

  • What direction are the developers and major contributors setting for “fast” in Julia – is the direction to incorporate more SIMD, more multiprocessing, in base and libraries?
  • Is there a goal to be “best in class” fast?

Base is mostly single-threaded, which is understandable, as just getting dozens of statements and thousands of functions created and working is a major undertaking. “Fast” comes from JIT compilation and multiple dispatch. Success!

Today, however, we have several JuliaSIMD packages, which vectorize and thread many functions (or your own). There is Floops for parallel execution. Package NumericExtensions.jl provides high performance computational support, with some implementations having been incorporated into base (sum, maximum, minimum). IntelVectorMath.jl ties into the Intel Vector Math Library for many base mathematical functions, achieving significant speedups.

The Development Process

Usually what I observe in code development for library functions is roughly the following:

  1. get it working (single threaded)
  2. get it working better (code flow, algorithm improvements; single threaded)
  3. get it working faster (use SIMD/AVX if possible)
  4. get it working faster (use multiprocessing, if possible)

Related to my two questions above, it could be restated – would the “philosophy” or “goal” be to get Julia base and libraries to steps 3 & 4?

I posted this in offtopic to encourage bloviating from anyone with an interest in the topic.

3 Likes
2

3 and 4 are somewhat within scope of Base. That said, there will likely continue to be lots of work on these in packages since packages are a much better place for experimental APIs.

1 Like
3

I doubt that it is a goal, informal or not. Quite the contrary, general philosophy is that most interesting things should live in external packages, as far as I know.

5 Likes
4

Early on, “batteries included” was a theme. At some point, a lot was moved out of base into packages – as it was realized that having it in base fossilized it and made them hard to change.

Fossilization, though, is a process that takes place over time. It’s possible that as external packages fossilize, they could be brought into base.

There is also an interface aspect to this. For example, sin(x) is a well defined function with well know return values. The implementation of sin(x) could change long as the return values didn’t change. As microprocessors support more parallelism, sin(x) could keep up with that, as long as the return values don’t change. Right now we have AVX512. What’s next, AVX1024? Who knows?

2 Likes
5
  1. get it working based on a recursively self-improving set of algorithms.
  2. ~ self-improvement without human control
  3. …

The main topic of JuliaCon2030:
Can a “machine with JuliaAI code” have a mind, consciousness, and mental states?

2 Likes
6

This is why we all require brain implants – to keep up with the robots we are otherwise creating.

2 Likes
7

One could say those are goals. I would call them values.

I think the goals are in the section ’Ecosystem’. The goal of Julia is to enable the ecosystem to thrive.

2 Likes