# Julia's applicable context is getting narrower over time?

**URL:** <https://discourse.julialang.org/t/julias-applicable-context-is-getting-narrower-over-time/55042>\
**Category:** Offtopic\
**Created:** [February 11, 2021, 4:52am UTC](https://discourse.julialang.org/t/julias-applicable-context-is-getting-narrower-over-time/55042 "2021-02-11T04:52:50Z")\
**Posts on this page:** 20\
**Page:** 4

<div class="post-metadata">

**Author:** ![pbayer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pbayer/32/11675_2.png) [@pbayer](https://discourse.julialang.org/u/pbayer)\
**Post date:** [February 15, 2021, 8:56am UTC](https://discourse.julialang.org/t/julias-applicable-context-is-getting-narrower-over-time/55042/68 "2021-02-15T08:56:13Z")

</div>

> [@Yifan\_Liu](#):
>
> I heard some Python guys said that Julia’s niche will get narrower over time. The argument is that …

I think those people are getting too much of our attention. There are always people grumbling about new things. Time spent in arguing with them is almost entirely wasted. Instead we should improve our stuff and encourage those who are working on it. Thus we will convince increasingly those who are watching and willing to try something new.

See [Fukuda’s parable](https://books.google.de/books?id=ZGxSr7WlB2UC&pg=PA79&lpg=PA79&dq=fukuda+parable&source=bl&ots=uFbcNM42D3&sig=ACfU3U3nnkvokRKez1iCoU7WbiTgIBvHGw&hl=de&sa=X&ved=2ahUKEwjhqo_9wOvuAhVJa8AKHeybBac4ChDoATAIegQICxAC#v=onepage&q=fukuda%20parable&f=false), e.g. that [explanation here](https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=&ved=2ahUKEwiJyIylvevuAhV0nVwKHXVXBEAQFjASegQIFRAC&url=https%3A%2F%2Fwww.linkedin.com%2Fpulse%2Fwhere-should-you-spend-your-attention-paul-hill&usg=AOvVaw3oVxFnJlkUtQR_xn2dKB1l)

---

<div class="post-metadata">

**Author:** ![carstenbauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carstenbauer/32/4981_2.png) [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)\
**Post date:** [February 15, 2021, 9:33am UTC](https://discourse.julialang.org/t/julias-applicable-context-is-getting-narrower-over-time/55042/69 "2021-02-15T09:33:03Z")

</div>

I guess these people getting too much attention is an instance of [Cunningham's Law - Meta](https://meta.wikimedia.org/wiki/Cunningham%27s_Law).

---

<div class="post-metadata">

**Author:** ![SamBrand](https://avatars.discourse-cdn.com/v4/letter/s/c37758/32.png) [@SamBrand](https://discourse.julialang.org/u/SamBrand)\
**Post date:** [February 15, 2021, 1:09pm UTC](https://discourse.julialang.org/t/julias-applicable-context-is-getting-narrower-over-time/55042/70 "2021-02-15T13:09:49Z")

</div>

I was just trying to make a similar point on quora about Julia.

Python users tend to swap very rapidly between “look its nice that other languages have some nice features, but everyone understands python its easy.” and “You can do that in python, here are steps: 1,2,3,4,5,6,7,8…” The concrete example was that python was fine for metaprogramming because you have meta-classes… Virtually everyone who uses julia does metaprogramming because its easy. Hardly anyone in python uses meta-classes as far as I can tell, its deep magic for an inner circle of python hardcore types.

The main point of Julia is it _works_. I’ve been under extreme time pressure to parse some new epidemic data, develop a new model, fit it, make forecasts.

Dataframes+Differential equations + DynamicHMC + Plots = job done on the first pass.

In python I’d still be fiddling with torchdiffeq, because the log-likelihood calculation required a convolution operation on top of solving the incidence rate over time, and as soon as you deviate from the beaten path in python it all grinds rapidly to a halt. In Julia its just more julia functions for the differentiable stack.

Its not _speed_ per se, because hardcore pythonistas eventually get fast code, its the sheer ease of use for solving complex problems.

---

<div class="post-metadata">

**Author:** ![oheil](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oheil/32/220745_2.png) [@oheil](https://discourse.julialang.org/u/oheil)\
**Post date:** [February 15, 2021, 1:40pm UTC](https://discourse.julialang.org/t/julias-applicable-context-is-getting-narrower-over-time/55042/71 "2021-02-15T13:40:39Z")

</div>

> [@SamBrand](#):
>
> as soon as you deviate from the beaten path in python it all grinds rapidly to a halt.

Same holds for R.

---

<div class="post-metadata">

**Author:** ![Henrique\_Becker](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/henrique_becker/32/15443_2.png) [@Henrique\_Becker](https://discourse.julialang.org/u/Henrique_Becker)\
**Post date:** [February 15, 2021, 1:53pm UTC](https://discourse.julialang.org/t/julias-applicable-context-is-getting-narrower-over-time/55042/72 "2021-02-15T13:53:36Z")

</div>

> [@SamBrand](#):
>
> Virtually everyone who uses julia does metaprogramming because its easy

Sorry, but I have to heavily disagree here. If you are talking about being an “user” of metaprogramming, ok, but if you are talking about doing metaprogramming themselves, I would just guess the vast majority of the Julia users never wrote a macro. And certainly is not “easy” in the _absolute_ sense, maybe _relatively_ when compared to Python meta-classes which I do not know anything about.

---

<div class="post-metadata">

**Author:** ![SamBrand](https://avatars.discourse-cdn.com/v4/letter/s/c37758/32.png) [@SamBrand](https://discourse.julialang.org/u/SamBrand)\
**Post date:** [February 15, 2021, 1:58pm UTC](https://discourse.julialang.org/t/julias-applicable-context-is-getting-narrower-over-time/55042/73 "2021-02-15T13:58:12Z")

</div>

Really?

One of the things I like about Julia is I can fiddle with models using f(expr) → expr or macros, saves C&Ping, speeds up development.

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [February 15, 2021, 1:59pm UTC](https://discourse.julialang.org/t/julias-applicable-context-is-getting-narrower-over-time/55042/74 "2021-02-15T13:59:34Z")

</div>

> [@tim.holy](#):
>
> There are cases where Julia is slower than the competition and others where it’s the fastest implementation known.

Part of that view comes from many Julia-sites where more or less one reads that Julia gives “within a factor of two” performance of C. For example, the micro-benchmarks here:

> **[Julia Micro-Benchmarks](https://julialang.org/benchmarks/)**
>
> The official website for the Julia Language. Julia is a language that is fast, dynamic, easy to use, and open source. Click here to learn more.

The description of the benchmarks is clear in that they are not the most efficient implementation in every language, they are mostly “equivalent” implementations. Why is C more frequently faster than any other language?

That question does not apply only to Julia. Could well be asked for the Fortran implementations, which there look in average more time than the Julia ones (and nobody discusses if Fortran is a high-performance language).

---

<div class="post-metadata">

**Author:** ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)\
**Post date:** [February 15, 2021, 2:36pm UTC](https://discourse.julialang.org/t/julias-applicable-context-is-getting-narrower-over-time/55042/75 "2021-02-15T14:36:25Z")

</div>

Mandatory link at this point:

[![](https://global.discourse-cdn.com/julialang/original/3X/7/5/759062729d830f12cdfa7374e0be51cc40e42a54.jpeg "JuliaCon 2019 | Keynote: Professor Steven G. Johnson") ](https://www.youtube.com/watch?v=mSgXWpvQEHE&t=580)

---

<div class="post-metadata">

**Author:** ![tim.holy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tim.holy/32/52_2.png) [@tim.holy](https://discourse.julialang.org/u/tim.holy)\
**Post date:** [February 15, 2021, 2:45pm UTC](https://discourse.julialang.org/t/julias-applicable-context-is-getting-narrower-over-time/55042/76 "2021-02-15T14:45:49Z")

</div>

Nice to have a good example. I can’t build the Microbenchmarks currently (see [Build fails on Arch Linux: deps/scratch/OpenBLAS.v0.3.10-0.x86\_64-linux-gnu-libgfortran5 missing · Issue #43 · JuliaLang/Microbenchmarks · GitHub](https://github.com/JuliaLang/Microbenchmarks/issues/43#issuecomment-779252954)), but from the graph it looks like `parseint` is the case where Julia lags C the most. If I take out a `@static` version check, the Julia benchmark is

```julia
julia> function parseintperf(t)
           local n, m
           for i=1:t
               n = rand(UInt32)
               s = string(n, base = 16)
               m = UInt32(parse(Int64, s, base = 16))
               @assert m == n
           end
           return n
       end

```

and the C one is

```julia
long parse_int(const char *s, long base) {
    long n = 0;
    for (; *s; ++s) {
        char c = *s;
        long d = 0;
        if (c >= '0' && c <= '9') d = c-'0';
        else if (c >= 'A' && c <= 'Z') d = c-'A' + (int) 10;
        else if (c >= 'a' && c <= 'z') d = c-'a' + (int) 10;
        else exit(-1);

        if (base <= d) exit(-1);
        n = n*base + d;
    }
    return n;
}

    tmin = 10.0;
    for (int i=0; i<NITER; ++i) {
        t = clock_now();
        char s[11];
        for (int k=0; k<1000 * 100; ++k) {
            uint32_t n = dsfmt_gv_genrand_uint32();
            sprintf(s, "%x", n);
            uint32_t m = (uint32_t)parse_int(s, 16);
            assert(m == n);
        }
        t = clock_now()-t;
        if (t < tmin) tmin = t;
    }

```

(just snipping out the relevant bits). On my machine:

```julia
julia> @benchmark parseintperf(1000)
BenchmarkTools.Trial: 
  memory estimate: 93.75 KiB
  allocs estimate: 2000
  --------------
  minimum time: 94.564 μs (0.00% GC)
  median time: 103.344 μs (0.00% GC)
  mean time: 111.886 μs (4.97% GC)
  maximum time: 3.408 ms (96.54% GC)
  --------------
  samples: 10000
  evals/sample: 1

```

and if I compile just the minimum I need to run `parseint`

```julia
$ gcc perfint.c -o perfint
tim@diva:~/src/Microbenchmarks$ ./perfint 
c,parse_integers,0.118811

```

Note the C version returns the minimum time; for the Julia version, the “minimum”, “median”, and “mean” times are all faster than this. So despite what the benchmarks say, the Julia version is slightly faster. Moreover, if you look carefully, the C version parses immediately to `long` which is the same as `Int32`. If we define

```julia
julia> function parseintperf2(t)
           local n, m
           for i=1:t
               n = rand(UInt32)
               s = string(n, base = 16)
               m = parse(UInt32, s, base = 16)
               @assert m == n
           end
           return n
       end

```

then I get

```julia
julia> @benchmark parseintperf2(1000)
BenchmarkTools.Trial: 
  memory estimate: 93.75 KiB
  allocs estimate: 2000
  --------------
  minimum time: 87.846 μs (0.00% GC)
  median time: 92.150 μs (0.00% GC)
  mean time: 100.223 μs (5.38% GC)
  maximum time: 3.157 ms (96.51% GC)
  --------------
  samples: 10000
  evals/sample: 1

```

which is a considerable advantage for the Julia version.

This just illustrates my point: Julia is _capable_ of matching C, it just comes down to how things are written. You can write great C code and lousy Julia code, or lousy C code and great Julia code, and when the quality difference is large the victor is predictable based on quality and not language. Fundamentally Julia is capable of everything C can do, and it can do oh-so-much-more besides.

EDIT: just remembered I should add optimization flags for C.

```julia
tim@diva:~/src/Microbenchmarks$ gcc -O3 perfint.c -o perfint
tim@diva:~/src/Microbenchmarks$ ./perfint 
c,parse_integers,0.089710

```

So C is capable of tying Julia, but not beating it.

---

<div class="post-metadata">

**Author:** ![tim.holy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tim.holy/32/52_2.png) [@tim.holy](https://discourse.julialang.org/u/tim.holy)\
**Post date:** [February 15, 2021, 2:54pm UTC](https://discourse.julialang.org/t/julias-applicable-context-is-getting-narrower-over-time/55042/77 "2021-02-15T14:54:04Z")

</div>

I’d like to add one other point: defending these points takes a lot of time that I’d rather be using for other activities. I just hope everyone gets notions of magic out of their heads and realizes that compilers are compilers are compilers, and when you’re comparing the same compiler in different settings (LLVM on Julia vs C), your general expectation is that they should be equivalent. The exception comes when something gets in the way of the compiler being able to “understand” the code and generate proper optimizations, and Julia (unlike Python/Matlab/etc) was explicitly designed not to get in the way.

That’s not to say that there isn’t room for more compiler optimization, but a lot of microbenchmarks are comparing settings where Julia’s compiler long ago reached parity.

---

<div class="post-metadata">

**Author:** ![oxinabox](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oxinabox/32/206603_2.png) [@oxinabox](https://discourse.julialang.org/u/oxinabox)\
**Post date:** [February 15, 2021, 2:55pm UTC](https://discourse.julialang.org/t/julias-applicable-context-is-getting-narrower-over-time/55042/78 "2021-02-15T14:55:53Z")

</div>

Seconding (3rding) this.  
Once very rarely needs to define new macros, and almost never need them for performance reasons.  
(One does need to use existing macros like `@inbounds` sometimes, but rarely definine new ones)

I just checked and Invenia’s whole internal ~400k LOC code-base, defines only 3 macros.  
2 of them are used only in tests,  
and the last just generates a good error message for dimension mismatches.  
I know whe have a handful more in our open source code, like Mocking.jl’s `@mock`, NamedDIms.jl’s `declare_matmul`.  
But still, I think that makes the point that one can write a whole ton of julia code without ever needing to write a macro.

> saves C&Ping,

might i recommend structuring your code such that a function can do this?  
It is less powerful, and as such easier to reason able.  
(e.g. you know it isn’t just going to assign to a local variable or somehting)

---

<div class="post-metadata">

**Author:** ![GunnarFarneback](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gunnarfarneback/32/1827_2.png) [@GunnarFarneback](https://discourse.julialang.org/u/GunnarFarneback)\
**Post date:** [February 15, 2021, 3:05pm UTC](https://discourse.julialang.org/t/julias-applicable-context-is-getting-narrower-over-time/55042/79 "2021-02-15T15:05:22Z")

</div>

> [@Henrique\_Becker](#):
>
> And certainly is not “easy” in the _absolute_ sense, maybe _relatively_ when compared to Python meta-classes which I do not know anything about.

I have never used Python metaclasses either but from a quick read they seem to be about modifying aspects of the workings and construction of classes. Although that can be a big deal in Python it’s far from the generality of Julia’s metaprogramming. A closer correspondence is the Python `ast` module, which I have used once to port a piece of Julia metaprogramming. It worked out okay, partly because the necessary code transformations perfectly matched the available methods in the `ast` object and partly because I only needed to mimic the existing Julia code. Had this not been the case I would have been stuck in the nightmares of opaque ast objects.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [February 15, 2021, 3:21pm UTC](https://discourse.julialang.org/t/julias-applicable-context-is-getting-narrower-over-time/55042/80 "2021-02-15T15:21:57Z")

</div>

> [@tim.holy](#):
>
> which is a considerable advantage for the Julia version.

This is especially surprising since `string(n, base=16)` allocates a new object on the heap for every iteration, whereas the C version writes over and over again to the same stack-allocated static array. Presumably the Julia version would be even faster if you rewrote it to use pre-allocated string output. (_Update_: I tried it, and working in-place is 20% faster than `parseintperf2` on my machine, but required forking [`Base.hex`](https://github.com/JuliaLang/julia/blob/686bc62fe544e6a69bc67c590a3bd07e095948d4/base/intfuncs.jl#L697-L716) to replace `string(n, base=16)` with an in-place version, combined with [StringViews.jl](https://github.com/JuliaStrings/StringViews.jl) to call `parse` on the overwritten buffer without making a copy. But this is hardly “magic”—just pure Julia code.)

(On the other hand, the Julia version slows down if you use `s = @sprintf "%x" n` instead of `string(n, base=16)`, reflecting the decades of extraordinary optimization effort that has gone into C `printf` implementations.)

Apples-to-apples cross-language benchmarking is hard!

---

<div class="post-metadata">

**Author:** ![paulmelis](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/paulmelis/32/35063_2.png) [@paulmelis](https://discourse.julialang.org/u/paulmelis)\
**Post date:** [February 15, 2021, 3:39pm UTC](https://discourse.julialang.org/t/julias-applicable-context-is-getting-narrower-over-time/55042/81 "2021-02-15T15:39:32Z")

</div>

> [@stevengj](#):
>
> This is especially surprising since `string(n, base=16)` allocates a new object on the heap for every iteration, whereas the C version writes over and over again to the same stack-allocated static array. Presumably the Julia version would be even faster if you rewrote it to use pre-allocated string output.

Also, every time I see benchmarks based on code from different languages using some kind of `rand()` function it makes me wonder if those calls are comparable in performance, as their generation time is part of the overall benchmark time.

---

<div class="post-metadata">

**Author:** ![Jeff\_Emanuel](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jeff_emanuel/32/15440_2.png) [@Jeff\_Emanuel](https://discourse.julialang.org/u/Jeff_Emanuel)\
**Post date:** [February 15, 2021, 3:42pm UTC](https://discourse.julialang.org/t/julias-applicable-context-is-getting-narrower-over-time/55042/82 "2021-02-15T15:42:07Z")

</div>

> [@pixel27](#):
>
> In C do you have a pointer to a structure but want to treat it as a pointer to an array of floats, no problem. Have a pointer to something but know that 57 bytes ahead is a structure of type Foo, not an issue. C trusts that you know what that pointer points at, just tell it and it will let you do it.

Until the next release of the compiler and struct layout changes, or you try to port your code to a different computer or compiler. This type of unsafe micro-optimization causes more time in fixing maintenance problems than it is likely to save in run time.

---

<div class="post-metadata">

**Author:** ![paulmelis](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/paulmelis/32/35063_2.png) [@paulmelis](https://discourse.julialang.org/u/paulmelis)\
**Post date:** [February 15, 2021, 3:45pm UTC](https://discourse.julialang.org/t/julias-applicable-context-is-getting-narrower-over-time/55042/83 "2021-02-15T15:45:33Z")

</div>

> [@tim.holy](#):
>
> So C is capable of tying Julia, but not beating it.

I might be misreading, but the C code is doing the timed loop 100.000 times, while you’re only using a 1.000 loops with Julia? I.e. `for (int k=0; k<1000 * 100; ++k) {` versus `@benchmark parseintperf(1000)`?

---

<div class="post-metadata">

**Author:** ![Marcell\_Havlik](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marcell_havlik/32/37424_2.png) [@Marcell\_Havlik](https://discourse.julialang.org/u/Marcell_Havlik)\
**Post date:** [February 15, 2021, 3:50pm UTC](https://discourse.julialang.org/t/julias-applicable-context-is-getting-narrower-over-time/55042/84 "2021-02-15T15:50:45Z")

</div>

I think on JuliaCon 2021 someone should create a theme with “The fastest coding ever in the world, achieved with julia”.  
Or I hope I will have time to show my optimised workflow. I think this is an other dimension of programming and after this video everything else will look like stone age. :o

I think the simplicity that comes with Julia is key to success in future. A pretty good analogy is electric and petrol cars, the cars got simpler with a magnitude and it is easier to develop in each part of the car. 🙂

---

<div class="post-metadata">

**Author:** ![tim.holy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tim.holy/32/52_2.png) [@tim.holy](https://discourse.julialang.org/u/tim.holy)\
**Post date:** [February 15, 2021, 3:52pm UTC](https://discourse.julialang.org/t/julias-applicable-context-is-getting-narrower-over-time/55042/85 "2021-02-15T15:52:31Z")

</div>

Sorry, in my abbreviated snippet I omitted a key line in the C code:

```julia
print_perf("parse_integers", tmin / 100);

```

Full details are at [https://github.com/JuliaLang/Microbenchmarks](https://github.com/JuliaLang/Microbenchmarks)

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [February 15, 2021, 3:58pm UTC](https://discourse.julialang.org/t/julias-applicable-context-is-getting-narrower-over-time/55042/86 "2021-02-15T15:58:47Z")

</div>

> [@SamBrand](#):
>
> The concrete example was that python was fine for metaprogramming because you have meta-classes

It never occurred to me to consider using meta-classes as meta-programming. It’s been a while, but I seem to remember that you use meta-classes to inspect (and perhaps manipulate) meta-information about classes; you’re not really using code to transform code.

Am I misremembering this?

---

<div class="post-metadata">

**Author:** ![TheCedarPrince](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/thecedarprince/32/17323_2.png) [@TheCedarPrince](https://discourse.julialang.org/u/TheCedarPrince)\
**Post date:** [February 15, 2021, 4:08pm UTC](https://discourse.julialang.org/t/julias-applicable-context-is-getting-narrower-over-time/55042/87 "2021-02-15T16:08:56Z")

</div>

Hey @tim.holy - I agree that your time could certainly be better spent elsewhere, but I personally have learned and benefited from these posts you are putting here. I am reminded of a recent Slack discussion about people needing to defend Julia in the workplace. Each one of your statements have been solid and serve as great evidence to support usage. I am certainly bookmarking this post and will use it as a point of reference for whenever discussions like this arise.

So personally, thank you so much for taking the time to go in-depth on the questions in the discussion. ~ tcp 🌳

[Previous page](https://discourse.julialang.org/t/julias-applicable-context-is-getting-narrower-over-time/55042.md?page=3)

[Next page](https://discourse.julialang.org/t/julias-applicable-context-is-getting-narrower-over-time/55042.md?page=5)