Julia is an interesting language because the standard library is just a bunch of live Julia code, so whatever is done in Base can be done in a package. Over time, it has been recognized that in order to be a general purpose language, you cannot have the whole “kitchen sink” included with every installation, and so things have moved towards packages. There is absolutely no advantage to moving JSON.jl into Julia’s Base, but there are many advantages for keeping it separate. Julia’s package manager makes it one nice line to get any package, so there it stays to do its job.
16 Likes
Install MacroTools and paste the following:
using MacroTools
function prockey(key)
if @capture(key, (a_ : b_) | (a_ => b_) | (a_ = b_))
return :($(string(a))=>$b)
end
error("Invalid json key syntax $key")
end
function procmap(d)
if @capture(d, f_(xs__))
return :($f($(map(procmap, xs)...)))
elseif !@capture(d, {xs__})
return d
else
return :($(Dict{String, Any})($(map(prockey, xs)...)))
end
end
macro json(ex)
esc(MacroTools.prewalk(procmap, ex))
end
and now you have the syntax you want:
julia> @json {"a": {"b": "c"}}
Dict{String,Any} with 1 entry:
"a" => Dict{String,Any}("b"=>"c")
julia> @json {a: "foo", b: [1,2,3], c: {10 => 3}}
Dict{String,Any} with 3 entries:
"c" => Dict{String,Any}("10"=>3)
"b" => [1, 2, 3]
"a" => "foo"
Lets create a type for the dict
julia> d = @json {"a": "foo", b:3, c:[1,2,3]}
Dict{String,Any} with 3 entries:
"c" => [1, 2, 3]
"b" => 3
"a" => "foo"
julia> struct MyStruct
a
b
c
end
julia> s = MyStruct([d[string(k)] for k in fieldnames(MyStruct)]...)
MyStruct("foo", 3, [1, 2, 3])
julia> s.b
3
17 Likes
Here, what I wanted to say, that Python has possibility to serialize almost any object and persist in as pickle format. Of course pickle format is not a very good thing, but if json-like data model would get same amount of attention like vectors, then probably there would a way to serialize any object to a json-like data model and serialize it to any json-like data format.
Like python has __dict__, Julia probably should have something like Dict for any type of object with possibility to add new types for custom objects with custom serialization needs.
But as I say, this would be only see that dicts are no less important that arrays.
“Very poor” is rather strong given that your only real complaints seem to be about syntax and printing. The array is the fundamental data structure of computing. It’s hard to do anything efficiently without arrays. In order to implement an efficient dictionary data structure, you need fast arrays. You can implement an array in terms of a dictionary by using integers as keys, but that’s going to be one slow array. The claim that dictionaries are as important as arrays is just not true. They are very useful, but they are not as fundamental at all.
The : in that syntax already means something in Julia. We used to use { k => v } as a literal syntax for constructing dictionaries, but it turned out that people wanted to use lots of other types of associative data structures in Julia, not just the built-in Dict type. This is unlike JavaScript or Python where there is one true dict type. Julia is much more like Java in this than JavaScript or Python. The Dict(...) syntax lets you easily choose a type and you can do things like this:
julia> Dict("*"^k => k for k = 1:10)
Dict{String,Int64} with 10 entries:
"**********" => 10
"***" => 3
"*******" => 7
"********" => 8
"*" => 1
"**" => 2
"****" => 4
"*********" => 9
"*****" => 5
"******" => 6
Which simply falls out of generator syntax and associative collection construction taking any iterable of pairs. It’s slightly more verbose but far more general and elegant. If you’re writing enough dictionary literals that it matters, consider actually using JSON syntax instead of Julia code for the data. For more details, see the “Constructing dictionaries” section in Julia 0.5 Highlights.
JSON.jl, YAML.jl, … I’m sure you can spot the pattern.
Everything in Python is based on dicts, so this is not too surprising. It’s also not too good for performance. And I assume that better performance is one of the reasons you’re interested in Julia…
5 Likes
Ok, thanks for explaining that, After seeing things like linear algebra built-in, I got wrong impression that Julia has plans to have rich standard library like python.
It used to be completely built in, but then it was moved to the standard library and you have to using LinearAlgebra to get it. Things like SuiteSparse are looking to be removed entirely.
1 Like
I thought, “they have a point about dict support being disappointing”. Then I opened the repl and tried a few things out. That changed my mind. Aside from Dict(:key => “value”) being awkward to type compared to other languages. Hmmm… I better go back to the repl before I make that statement.
d = Dict(:a => 97, :b => 98)
this works
d = Dict([(:a,97),(:b,98])
or
d = Dict()
d[:a] = 97
d[:b] = 98
Still => is awkward to type ( top row, then shift-bottom row ) or [(:key,“value”)] compared to for example clojure’s {:key “value”}
I think it’s more that Julia has excellent Dictionary support, but it isn’t promoted as strongly as the numerics story.
2 Likes
This is available in the Serialization package in the stdlib.
IIUC, your other suggestion is to have a standard function for getting a dict-like representation of an object? So for example the JSON package could call dict(x) on an arbitrary x to get an object that can be saved as a JSON dict. I suppose we could add something like that. Actually the JSON package already has a function JSON.lower that can be defined to customize how any type is converted to something JSON-compatible.
You can serialize anything in Julia with the… wait for it… serialize function. You can deserialize anything with the corresponding deserialize function.
https://docs.julialang.org/en/v1/stdlib/Serialization/
JSON is a horrible serialization format. It’s incredibly inefficient to read and write. It’s only good by comparison to truly abysmal formats like XML. Which is why people keep inventing binary formats like BSON as alternatives. You can read and write BSON using—you guessed it!—BSON.jl. And of course JSON only lets you serialize a very small set of types, which is fine if all you want to do is communicate with a REST end point, but doesn’t get you very far if you want to save arbitrary Julia data structures. In general, JSON is of pretty limited value in Julia and in the situations like web work where it’s necessary you need to use an external package like HTTP anyway so where’s the harm in also loading JSON?
Objects in Python are just dicts with some methods. That is not how data structures in Julia work, so insisting that Julia data types work like dicts and Python objects is not going to go very well. Again, do you like performance? Then basing your entire data model on untyped dictionaries isn’t a great idea.
But as I say, this would be only see that dicts are no less important that arrays.
You’ve said this a lot but it’s very much false. Dicts are far less important than arrays in computing in general. The only context in which you can argue that a dict is of equivalent importance to an array is one in which you don’t care about performance at all, since the cost of implementing an array in terms of a dict is that your arrays will be very slow.
4 Likes
I thought there already was a package providing such a macro. If not I think that could be added to JSON.jl.
4 Likes
Taking a step back, I would like to make a couple of observations…
This could have been a perfectly reasonable conversation about dictionary syntax and printing and possible API improvements to the JSON package. But no, it has to be a referendum on an existential question like whether Julia is “general purpose”. Great, there goes any chance for a pleasant discussion.
Your understanding of what makes a language “general purpose” also seems to be somewhat backwards. Having more things built in does not make a language more general purpose. A general purpose language is one in which anything can be built. Therefore a truly general purpose language needs hardly anything built in because you can build anything in it. In fact, having lots of things built in—or implemented in another language through some extension mechanism—is a really major sign that maybe your language is not actually general purpose since it’s apparently not good enough to build those things in.
25 Likes
Thank you all for near real time responses! I think I understood what I’m asking myself 
Since I’m coming from web development background, JSON is really important there, both as syntax and as data model.
As @StefanKarpinski well said:
My main frustration was about syntax and printing.
If Julia would have built-in and well promoted support for a macro that lets you easily write large and deeply nested JSON structures, then I would not be scared of Dict("a", Dict("a", ... syntax, knowing there is another way.
And printing deeply nested Dicts should also be readable same way as large multi dimensional arrays. And it would be perfect if I could paste Dict output into macro without changing anything.
Your prompt responses gave me some hope for web developers, even if in all these years you forgot about nice JSON input and output. At least in what I do, working with JSON-like data structures is really important and it helps if I can move data snippets between code, output and API without spending too much time refactoring it.
1 Like
In general the default printing in the REPL is not parseable — it isn’t for arrays either. You can get parseable output with show or repr:
julia> repr(Dict( 1=>2, 2=>Dict(0=>1) ))
"Dict{Int64,Any}(2=>Dict(0=>1),1=>2)"
I understand that Dict( ) is more verbose than { }, but at least this is parseable. I think the best way forward is to add more utilities to JSON.jl for entering and printing JSON data.
Making more things built-in is not the answer — it ultimately makes little difference in functionality, and leads to slower development. There also isn’t enough syntax to go around: we can’t give { } to JSON without taking it from something else. “Modern general purpose” languages do not make everything built-in; they use libraries.
3 Likes
It is an “improvement” not a “fix”, according to Python documentation What’s New In Python 3.7 — Python 3.12.0 documentation
Python has __slots__ mechanism to disable __dict__ in a custom object. “Even” Python doesn’t use dictionaries when it’s performance critical.
I know it’s not a Julia-Python comparison thread, but Julia’s show system is way more sophisticated than Python.
2 Likes
Because it prevents you from using that syntax for anything else. ASCII only provides a limited set of operators and paired delimiters, and the decision was made that there were better uses for : and {}.
One option which hasn’t been mentioned would be to define a nonstandard string literal so that you could use json directly:
dict = json"""
{
"slideshow": {
"slides": [
{
"title": "Wake up to WonderWidgets!",
"type": "all"
},
{
"items": [
"Why <em>WonderWidgets</em> are great",
"Who <em>buys</em> WonderWidgets"
],
"title": "Overview",
"type": "all"
}
],
"author": "Yours Truly",
"title": "Sample Slide Show",
"date": "date of publication"
}
}
"""
11 Likes
FWIW, most dict/map usage I’ve seen doesn’t care about the ordering. For the ones (I won’t say it’s rare but it’s for sure much less than half) that do, it’s also split between ordered and sorted, making any argument for a particular order being the right default a bit unconvincing.
The julia PR and the pypy change inspiring it has a large focus on the performance and I don’t think either argued much about the usefulness of the change more than making the ordered dict implementation trivial.
1 Like
As someone who does a lot of web stuff and has used Julia quite heavily, I will say that coming from the web/javascript world, both Julia and Python do feel more awkward in dealing with dicts compared to Javascript, where they are indeed a common data structure — I’d say dicts are used almost as often as arrays in that world. For common data structures, friendly syntax matters.
Julia has convenient syntax for tuples, named tuples, and arrays, but not dicts. I sometimes define a helper function to use D(a=1, b=2) as Dict creation syntax, which is almost as concise as Javascript object literals.
It would also be very nice if Base supported concise syntax for getting and setting dict entries via getfield overloading on AbstractDict, which would also help to close the friendliness gap for dictionaries compared to JS.
5 Likes
Would d.a be equivalent to d["a"] or d[:a]?
I encourage this. A programming language is there to provide building blocks, not to tell you exactly how to write everything. You should define the notation you need. That is especially true in the lisp ethos, where user-defined and built-in forms have exactly the same syntax. With fancy syntax custom notation is harder, but we still strive for it where possible.
In fact maybe we should make { } customizable somehow. Of course that could be very confusing, but at this point no single use for the syntax seems compelling enough to steal it.
16 Likes