Is Julia a general purpose language?

[yurivish]

@jeff.bezanson

I encourage this. A programming language is there to provide building blocks, not to tell you exactly how to write everything. You should define the notation you need. That is especially true in the lisp ethos, where user-defined and built-in forms have exactly the same syntax.

Yes, agreed! The only complaint I have about that solution is that the notation not as concise as it would be with language support, which adds up particularly with nested Dicts . This is also often the problem with using macros to define custom notation — if a notation is used in small amounts in many places, the syntactic tax starts to add up.

But the lisp ethos is exactly the reason I was very excited about an idea that came up on one of the triage calls for allowing lexically-scoped definitions to customize the behavior of standard syntactic forms, like [ ], which lower to function calls. Maybe that can be how the { } customization works?

Here’s one place where it was discussed.

[yurivish]

Would d.a be equivalent to d["a"] or d[:a] ?

Good question. I think it should lower to d[:a] by default, and perhaps there can be a JSONDict type for which it calls d["a"]. I’ll also note the little-known feature of dot syntax that d."a" is valid — I use this little trick in my HTML DSL, where dot overloading adds CSS classes.

[pfitzseb]

Browsing through nested dicts can also be made much more pleasant:

[Sukera]

That’s Juno specific though, isn’t it?

[pfitzseb]

No, that’s a regular REPL.

[wookyoung]

I recommend to use JSON2.jl which could parse into a NamedTuple, better and simple representation.
https://github.com/quinnj/JSON2.jl

julia> JSON2.read(String(r.body))
(slideshow = (author = "Yours Truly", date = "date of publication", slides = Any[(title = "Wake up to WonderWidgets!", type = "all"), (items = Any["Why <em>WonderWidgets</em> are great", "Who <em>buys</em> WonderWidgets"], title = "Overview", type = "all")], title = "Sample Slide Show"),)

[Liso]

where is macro ishow defined?

I tried here which pointed here and I am curious now where I did mistake.

[pfitzseb]

I’ve moved the REPL functionality to REPLTreeViews.jl (unregistered) so that TreeViews.jl can stay a minimal package.

[StefanKarpinski]

The K language also has ordered dicts which are heavily used. Around the time Jeff made that PR, we talked to some long-time K users who were switching to Julia and asked them about whether it was a useful property that K’s dicts were ordered. The answer we got from several people was that it could be useful but it was also a common source of problems because one part of the code would construct a dict with a very specific ordering and some other person would fail to realize that the ordering is significant and then trash it, not realizing that order was important in this particular dict.

Back in Julia land, if someone cares about order they can use an OrderedDict and not only get order but also signal explicitly by the type that the order matters. I’m not against making Dict ordered but this is a real consideration.

[johnh]

@StefanKarpinski so lets get this straight - there is already an OrderedDict structure in DataStructures.jl
This would satisfy what @sirex is asking for
I would also say to @sirex that Julia IS a general purpose language - that is why there are lots of packages existing to do various things. Like the OrderedDicts that you like, or like JSON.
If everything were to be put in Base then the whole thing would become unwieldy and not general purpose.
If you want to use Julia, get used to dealing with packages and you will be happy.

[yuyichao]

I certainly agree there’s good reason to use ordered dict. I was just talking based on my own experience since the key-value pair data structure I’ve used the most outside julia and python is actually std::map, which is not “ordered” but “sorted”. (I know it’s not a hash table but it has the same API as one and good enough for pre-c++11 code). In doing that, I also find that I occationally do need the ordering guarantee from it.

So from a usefulness standpoint, I don’t think it’s super clear that one ordering is actually better than another since It’s strongly based on people’s past experience. That makes the bold claim that not having orderred dict is a mistake unjustified and IMO as long as both datastructures are provided and well supported it should be good enough.

[StefanKarpinski]

Sorted dicts are not as general purpose however since the keys must be sortable which general Julia objects are not. Ordered dicts don’t have such a limitation.

[sirex]

Thank you all for really good explanations. I learned a lot! I will try to summarize what I learned.

What I understood, that JSON data model (nested dicts and lists) is not first class citizen and will not become one because this is not considered as a fundamental data model worth including into Julia as language feature with dedicated syntax. It is possible to have nested dicts and lists in Julia without any third party packages, but the whole thing is far less usable compared with arrays. The only available syntactic sugar is =>. But I don’t really understand why => is needed, because Dict(a = 1) is even better.

Also I feel that name JSON data model is confusing, because people think, that I’m talking about JSON data serialization format. Maybe nested dicts and lists would be better name? I don’t know how to properly call it, maybe JON (Julia Object Notation)?

Those who work with JON (most web developers) should use third party packages and one of many options available to work with JSON data model.

Here are some options for writing JON:

Dict("a" => Dict("b" => [Dict("c" => "d")]))

Dict(:a => Dict(:b => [Dict(:c => "d")]))

Dict([(:a, Dict([(:b, "c")]))])

(a = (b = [(c = "d",)],),)

using DataStructures
OrderedDict("a" => OrderedDict("b" => [OrderedDict("c" => "d")]))

using MacroTools
@json {"a": {"b": [{"c": "d"}]}}

using MacroTools
@json {a: {b: [c: "d"]}}

using ?
json"""{"a": {"b": "c"}}"""

using ?
D(a = D(b = [D(c = "D")]))

Options for printing JON:

julia> data
Dict{String,Dict{String,Array{Dict{String,String},1}}} with 1 entry:
  "a" => Dict("b"=>[Dict("c"=>"d")])

julia> repr(data)
"Dict(\"a\"=>Dict(\"b\"=>[Dict(\"c\"=>\"d\")]))"

julia> print(repr(data))
Dict("a"=>Dict("b"=>[Dict("c"=>"d")]))

julia> using JSON
julia> JSON.print(json, 2)
{
  "a": {
    "b": [
      {
        "c": "d"
      }
    ]
  }
}

julia> using ?
julia> @ishow json
▾ a
  ▾ b
    ▾ 0
      ▾ c
        "d"

Personally I would prefer to have only one best option built into Julia in a similar fashion as with arrays, but at least now I know that using one of many third party packages is the way to work with JON.

For me as a web developer this is a bit disappointing, but I think one of third party packages can fix it, not sure which one yet.

And here is how I would like it implemented:

julia> data = {"a": "b": [{"c": "d"}]}
julia> {data..., "x": [1, 2, 3]}
Dict{Any: Any}
a:
  b:
    - c: d
x: [1, 2, 3]
julia> @json data
{
  "a": {
    "b": [
      {
        "c": "d"
      }
    ]
  }
}
julia> @json data > filename.json
julia> {i: i^2 for i=1:9 if isodd(i)}

[jeff.bezanson]

@sirex

Dict{Any: Any}

I find it hard to believe it could be that important to write types this way instead of as Dict{Any, Any}. JSON doesn’t even have typed dicts. There are many types other than Dict (julia is general purpose, remember?), and the colon notation only makes sense for dicts.

However, the rest of the Dict printing could in fact be changed to look like that. REPL output is supposed to look nice and doesn’t necessarily match input syntax (though we don’t want to confuse people either).

The JSON input syntax you want is (or could be) available with @json, so your entire complaint of dicts being unusable and not first class basically comes down to not wanting to write @json. That seems like a bit of an exaggeration to me. And, as I said above, { } is available and could be made customizable, so that using JSON could actually provide similar syntax.

You could also follow @yurivish and use

D(;kw...) = Dict(kw)

to write dicts with symbol keys as D(a=1, b=2). The reason => is needed is that in a = 1 the a is not evaluated but is a literal symbol, while in a => 1 the a is evaluated.

You should also be aware that other use cases also want { } syntax. One example is tabular data. Is JSON more important than tabular data? I don’t know; databases and dataframes are a pretty big economy. The point is, a language needs to be extensible and put functionality in libraries because one shouldn’t have to decide which use cases are more important. We need to support all of them.

I will also repeat the other important take-away here, which is please don’t title a thread “Is Julia a general purpose language?” when in fact what you want to do is suggest different dict syntax.

[yurivish]

@jeff.bezanson

You should also be aware that other use cases also want { } syntax. One example is tabular data.

That’s interesting, I don’t think I’ve seen this mentioned anywhere. Can you explain how tabular data would be different from a 2D matrix using the array literal syntax?

[a b c
 d e f]

[jeff.bezanson]

There hasn’t been a detailed proposal, but one difference is that it would produce a dataframe-like structure, a collection of columns as vectors rather than a single 2d array. Column names might also be part of it. Another possibility is alternate named tuple syntax, so that you could write e.g. {a.x, b.y} instead of (x = a.x, y = b.y). Something about missing data could also play a role there.

[StefanKarpinski]

Related: JavaScript only allows strings as dict keys, whereas in Julia you can use any object at all as a dictionary key. This makes it much harder to design a completely general dictionary syntax in Julia since you can’t assume that the keys are always going to be strings.

[kevbonham]

@sirex

What I understood, that JSON data model (nested dicts and lists) is not first class citizen and will not become one because this is not considered as a fundamental data model worth including into Julia as language feature with dedicated syntax

This first class citizen thing is a bit spurious. As @ChrisRackauckas and others mentioned, Julia is mostly Julia, so stuff in packages are not second class citizens, they’re first class too!

@sirex

Personally I would prefer to have only one best option built into Julia in a similar fashion as with arrays

I invite you to browse JuliaArrays · GitHub if you think there’s only one best option for arrays

This diversity is a strength, not a weakness.

@sirex

And here is how I would like it implemented:

Great! Go write a package that does it, I’m sure you’re not alone! The good news is, the types in your package will be first class citizens

[johnh]

@sirex

What I understood, that JSON data model (nested dicts and lists) is not first class citizen and will not become one

As others have pointed out, defined types and functions from packages are first class citizens in Julia.

Think of Julia as a pack of greyhounds pulling a sled. Sure, there is a lead greyhound in the pack. But that dog is lean, mean and stripped down for speed. There are other dogs running along with it at the same speed. You can change dogs if one is not doing a good job.

Think of multiple dispatch as the time taken to harness up all the dogs in the correct order.

My analogy falls down when the dogs stop for the night - huskies have lots of fur for a good reason!

[Tamas_Papp]

@johnh

My analogy falls down when the dogs stop for the night

That’s compilation. They are building their stamina to be fast the following day.