Yeah haha. I can’t believe people still talk about the expression problem like it’s still unsolved. It’s old news now: multiple dispatch and trait handling in Julia does it. Add new types and functions to the system and you don’t have to modify previous code. Been doing this for years and keep finding people going “man, don’t you wish you could extend core functions and types a package supports without changing the package?” 
maybe you need to make a video with a more provocative title.
8 Likes
The true problem is that, adding union types (not sum types) to those type system might break them.( I am talking about the expression problem baused by sum and union types)
So people keep complaining about that because they think, we can add arbitary things that we like into the type system, just like C++ has gotten so many magical things into the language (They have some kinds of genetic types and some kinds of multiple overload,which might looks like those in Julia ast a first view). However types systems are not like syntax, they are not quite extendable.
The core problem is subtyping. In most OO languages, we use inheritance( one kinds of subtyping) to create ordering relations. Adding genetics will be fine, because genetics are designed to be invariant. Adding sum types will be fine, because you need to wrap them into a new type, so new sum type has no ordering relations to the types it used (then expression problem occurs, because old codes are not available at all).
In order to solve this, we need to introduce ordering relations between old types and union types, that is, for example, A<:Union{A,B}, this is then first problem. What’s more, we need to consider the ordering relations between Union{A,B} and A,B’s parent types, for example, Union{Float64,Float128,Float32...}>:AbstractFloat?The second problem is that, union type is a purely type construction, while vaules of sum types can be constructed. In many languages, they don’t distinguish this (thet have no subtyping), and even many OO languages have inheritance, but all the types are actually constructable, that is not the same in Julia (while in Nim, they both have sum types and union types) .They have only one set of types for compilation and run time.Adding union types will cause the systems split.
These problems are both easy and difficult to solve. The hardest part is that, programmer are required to learn these differences and thus, type systems. Considering much resistance from old-fashioned programmers to the these new ideas in type systems (“genetics are evil”,“don’t use lambda function”),it would not easy to add union types in these type systems (however, I think if you appeal to them that, union types is some magical static compilation optimization implemented by powerful templates which can reuse your old codes without creating new sum types , they might adopt that…)
1 Like
I think we get widely off-topic.(part of my mistakes)
Though I do not completely agree the design of Nim. But I find they also have some amazing libraries of numerical calculations. For example,some of mratsim’s amazing works, Arraymancer, and there is a HPC toolbox.
Though I’am not sure whether performance of these packages are comparable to Julia’s packages, documents of these packages are quite good and detailed. And it seems Nim also have a good GPU and HPC support.
What is most impressive is the package Arraymancer, (previously there is also a developer providing some useful links to these new deep learning compiler design, might be useful for some Julian.
And Nim can provide compiler as a library,(I think rust can also do this too), so this partly solves the dynamic runtime problem (use the runtime that we need).However Julia depends on LLVM, this dependency is not quite easy to remove.
And they also have macros, but I wonder how to have macro in a Python-like language(by forcing indentation?）It seems that their macros don’t need @ place ahead, which looks much more simple. I think this is really great(just like in Lisp)
4 Likes
I agree, mratsim’s Nim work is really impressive!
If static AOT compilation was really important to me, I’d definitely be considering using Nim more and I’m kinda surprised it’s not more popular in that space. Personally, I quite like the dynamism and interactivity of Julia and also really like that it doesn’t do any class based OO stuff so I feel little reason to use Nim currently.
Hopefully one day, we can be friendly competitors with Nim in the static AOT world, but I think for right now we’re relatively similar languages that target different use scenarios.
We could also have macros without a special sigil (@) in front of them, but it was decided that it should be required so that it’s always clear when you’re looking at a macro. I think this was a wise choice because it makes it much easier to look at a large body of code and pick out areas where particularly weird things might happen. It’s purely to help with reasoning about code.
That also happens to be why we have the convention of putting ! in the name of mutating functions, though that is impractical to enforce at the compiler / parser level so it’s just a convention instead of mandatory.
12 Likes
Rust developers agree, which is why Rust macros end with !.
For the reasons laid out in the video, I think this is a better case for something like Clojure’s spec than Some.
1 Like
as of Juliacon 2019, static compilation still seems to be on the radar JuliaCon 2019 | What's Bad About Julia | Jeff Bezanson - YouTube
1 Like
There is big difference.
Compile could tell you that there is problem with “fermat fish” also if your test input doesn’t include one (with overflow there are many) and test runtime doesn’t produce any error.
I think that this list is just a laundry list of user demands (it says so in the title), not necessarily implying any specific focus on them.
I am under the impression that current priorities are the new parallel paradigm (mostly in 1.3), then “time to first X”. Also, I think that making compilation faster (#33326) and the repl more responsive (#33333) would reduce the need for precompiling large sets of code. In this sense, this may be a surrogate target: users just want faster startup times, and think that this is the way to achieve it because other languages do it. But it may not be the solution for Julia. We shall see.
1 Like
speaking only for myself, the main reasons I want AOT do not have to do with startup time, but with distribution, linkalbe libraries and compile-time verification of correctness.
I mean, better startup time would also be good, but it’s just one out of four good reasons, and not the most important (to me).
8 Likes
Same here. Actually, I made the switch from Julia to Nim for these exact same reasons. As much as I love Julia’s syntax more, I simply need the possibility of compiling self-contained binaries.
5 Likes
Just a nitpick, but I really hate that term.
@label 1 Unless your entire program runs in the type domain, there’s no way that your type checker can tell you that your program is ‘correct’ unless you’re redefining correct to mean something completely different from what people usually mean by that word.
If your program does run at compile time, well then your compile time has become your runtime and you’re now using a dynamic language. If you then want to type check your compile time computation for ‘correctness’ then @goto 1.
5 Likes
First, love your recursive response from a stylistic perspective.
I’m being a little semantically sloppy here, it’s true (Julians should appreciate that… hrmph). I understand that no type system can verify that a program produces the correct results.
I guess I more mean (as you well know) that static, strict, strong type checking can prevent some runtime errors, and if you’re like me, you can use all the help you can get! If you have a better term than “correctness”, I’ll be happy to use it in the future. Maybe “type-correctness”?
As you can see, even my spoken language is a compromise between correctness and expressiveness. Probably the reason I like Julia…
5 Likes
As far as i can tell, Nim does not support multi-dimensional arrays or any of the array manipulation syntax and features that are very (very) handy for scientific computing.
You are able to solve your problem in Nim, so I’m interested to know if you should have been using Julia to solve your problem in the first place.
Or do you just like julia, and if you had your choice you would just rather use it (aside from it’s application to numerical analysis) ?
Yeah, sorry I just felt the need to rant about that term even though I knew what you actually meant 
.
As has been mentioned multiple times in this thread, mratsim’s Arraymancer library is pretty kickass for multi-dimensional arrays in Nim.
If your complaint is that it’s not builtin then I’d say julia has a similar situation where many packages people think should be builtin are instead in libraries. Personally, I’ve grown to like that situation. It lets the libraries breathe instead of calcifying in a stdlib or heaven forbid, Base.
7 Likes
It’s the never-ending compromise that’s hard to balance, right? You either put it in the standard library and it calcifies, or you end up like JavaScript with the npm ecosystem and you got literally millions of lines of “anonymous” JavaScript in your dependencies folder. As someone who is primarily a consumer of the standard library, I like knowing that the API I’m programming against is going to be maintained until the language disappears, but I can understand if the language maintainers don’t feel the same!
3 Likes
As Mason said, Arraymancer is an amazing library for a lot of scientific computing related stuff.
That’s pretty much it. I love coding in Julia and I would love to use it not just to prototype stuff, but to build full static binaries too. And for me this is where Nim comes into place.
1 Like
That’s actually a pretty easy thing to have optionally. I think the problem is best phrased like this:
Suppose I have
function f(x)
sleep(100) # This represents some big important calculation that takes a while
x + 1
end
It’s pretty frustrating if you have to wait the full 100 seconds just to get an error from f("hi").
In that case, if I want to demand from the compiler that it knows what the return type of f(::String) is before running f (and accepting that there are possible cases where methods can be dynamically overwritten but we’re willing to ignore those), I can write a macro like this:
macro check_inferred(fcall, isstrict=:(true))
quote
local T_inferred = @code_typed($fcall)[2]
if isconcretetype(T_inferred)
$fcall
else
if $isstrict
error("Function call did not infer.")
else
@warn "Function call did not infer. You may want to interrupt."
$fcall
end
end
end |> esc
end
such that
julia> @check_inferred f("hi")
ERROR: Did not infer concrete type!
Stacktrace:
[1] error(::String) at /Users/mason/julia/usr/lib/julia/sys.dylib:?
[2] top-level scope at REPL[43]:7
without having to wait 100 seconds. If you just want a warning so you can decide for yourself if you want to wait and see what happens, just do
julia> @check_inferred f("hi") false
┌ Warning: Function call did not infer. You may want to interrupt.
└ @ Main REPL[49]:10
so that you get a warning and then you can choose to interrupt if you wish.
This works with a macro annotation at the callsite but it’s actually possible to do something similar at the function definition site too in a way such that you only pay the cost of type checking once at the first compile time instead of each time you invoke the function (though that has even more potential to become pathological) and I’ve written up a method to do it here, caveats and all.
4 Likes
Not complaining, just curious as to the situation that allowed the project(s) to move to Nim. After all if you were doing heavy duty numerical programming in Julia and then all of a sudden said, I can move this to Nim ! I would be quite surprised.
multiple times + 1 
And yes, it does look to be quite nice.
However, yes, it would be my preference that something like that would be in the language as a built-in. it’s hardly a requirement, it’s not built-in to python and so numpy was the way i got things done for a long time, and i was happy to have it.
We regularly have this debate in Nim for:
- streams
- serialization
- async
- multithreading
- crypto
- bigint
- …
Ideally the standard library sets a consistent API/interface and libraries can compete on implementation but the interface should be something that users would actually used so it should be implemented as a library first. And then you risk ecosystem fragmentation, especially in where compatibility matters.
Another side is that some entreprise users might allow using the core language + standard library but using third-party packages might be subject to approval so having more features in the standard library is beneficial.
Basically, a macro gives you the AST nodes and you can manipulate them at will.
If I take my FizzBuzz exemple (be sure to read the hilarious inspiration)
It is defined like this:
const
NumDigits = 10
NumHidden = 100
let ctx = newContext Tensor[float32]
network ctx, FizzBuzzNet:
layers:
hidden: Linear(NumDigits, NumHidden)
output: Linear(NumHidden, 4)
forward x:
x.hidden.relu.output
let model = ctx.init(FizzBuzzNet)
let optim = model.optimizerSGD(0.05'f32)
# ....
echo answer
# @["1", "2", "fizz", "4", "buzz", "6", "7", "8", "fizz", "10",
# "11", "12", "13", "14", "15", "16", "17", "fizz", "19", "buzz",
# "fizz", "22", "23", "24", "buzz", "26", "fizz", "28", "29", "30",
# "31", "32", "fizz", "34", "buzz", "36", "37", "38", "39", "40",
# "41", "fizz", "43", "44", "fizzbuzz", "46", "47", "fizz", "49", "50",
# "fizz", "52","53", "54", "buzz", "56", "fizz", "58", "59", "fizzbuzz",
# "61", "62", "63", "64", "buzz", "fizz", "67", "68", "fizz", "buzz",
# "71", "fizz", "73", "74", "75", "76", "77","fizz", "79", "buzz",
# "fizz", "82", "83", "fizz", "buzz", "86", "fizz", "88", "89", "90",
# "91", "92", "fizz", "94", "buzz", "fizz", "97", "98", "fizz", "buzz"]
This is what the raw data I work with in the network macro:
- View rawdata
import macros
macro network(ctx: typed, modelName: untyped, body: untyped): untyped =
echo "\n----------------"
echo "\"network\" macro invocation\n"
echo ctx.repr
echo "--"
echo modelName.repr
echo "--"
echo "Parsed"
echo body.repr
echo "--"
echo "AST tree representation"
echo body.treeRepr
echo "--"
echo "Lisp representation"
echo body.lispRepr
var myContext = 0 # placeholder
network myContext, FizzBuzzNet:
layers:
hidden: Linear(NumDigits, NumHidden)
output: Linear(NumHidden, 4)
forward x:
x.hidden.relu.output
- Output
----------------
"network" macro invocation
myContext
--
FizzBuzzNet
--
Parsed
layers:
hidden:
Linear(NumDigits, NumHidden)
output:
Linear(NumHidden, 4)
forward x,
x.hidden.relu.output
--
AST tree representation
StmtList
Call
Ident "layers"
StmtList
Call
Ident "hidden"
StmtList
Call
Ident "Linear"
Ident "NumDigits"
Ident "NumHidden"
Call
Ident "output"
StmtList
Call
Ident "Linear"
Ident "NumHidden"
IntLit 4
Command
Ident "forward"
Ident "x"
StmtList
DotExpr
DotExpr
DotExpr
Ident "x"
Ident "hidden"
Ident "relu"
Ident "output"
--
Lisp representation
StmtList(Call(Ident("layers"), StmtList(Call(Ident("hidden"), StmtList(Call(Ident("Linear"), Ident("NumDigits"), Ident("NumHidden")))), Call(Ident("output"), StmtList(Call(Ident("Linear"), Ident("NumHidden"), IntLit(4)))))), Command(Ident("forward"), Ident("x"), StmtList(DotExpr(DotExpr(DotExpr(Ident("x"), Ident("hidden")), Ident("relu")), Ident("output")))))
So in my macros I process this declarative AST tree to transform it into a valid AST tree that the compiler can compile into code. It’s not different from creating code from a json, yaml, protobuf file except that I can do it during Nim compile-time without an additional compilation step.
Also it’s type-checked and has good error reporting. For example JIT Assembler like Xbyak (used in Intel MKL-DNN) or ASMJit (used in Facebook’s PyTorch), requires a 2-step codegen via C++ and Javascript respectively that parses a table of opcodes. In Nim I can easily have a friendly declarative syntax for those opcodes.
Due to those codegen capabilities, I really think Nim is the best language to create domain-specific languages (though I never tried OCaml so obviously I’m biaised). The npeg parser generator is probably the best example but there are other like this declarative opengl shader generator.
My main bottleneck is the allocator, I didn’t write a caching or pool allocator backend yet to avoid deallocating/reallocating memory in a loop (very important in deep learning mini-batch training).
However, I am in general very fast. For example I added randomized SVD last weekend and this is my performance versus Scikit-learn and Facebook’s fbpca on an overclocked i9-9980XE:
| Problem size | fbpca (MKL) | scikit-learn (MKL) | Arraymancer (MKL) | Arraymancer (OpenBLAS) |
|---|---|---|---|---|
| 20000x4000 → 40 components | 2.12s | 3.58s | 0.16s | 0.30s |
| 4000x20000 → 40 components | 0.40s | 0.21s | 0.09s | 0.12s |
Memory usage is also twice less (but do SVDs in a row and Nim GC might get lazy in collecting/reusing the unused memory until pressured to do so, hence my need of a custom allocator)
6 Likes