Looking beyond Unicode

[pitao]

Unicode is amazing, don’t get me wrong.

typealias ℂ Complex{Float64}
function Base.angle(z₁::ℂ, z₂::ℂ)::Float64
  α = abs( angle(z₂) - angle(z₁) )
  return min(α, 2π-α)
end

yum yum yum. A scrumptious math munch and no mistake! Tangible benefits. Massively easier to read code. Code mirrors formulae and notation conventions in academic papers. Also with so much choice, we can use glyphs artfully to indicate purpose. A matrix can use different glyph from a vector. I can use 𝒳, 𝒴 = ... as containers, for 𝓍 in 𝒳 etc, and I’m still free to use x,y,z. I don’t run out of letters, so I don’t need long variables. Long variables can’t be parsed by our visual cortex as efficiently (the fovea can only handle 3-4 letters at a go, so you will notice you can instantly spot 𝓍 above whereas you’ll have to hunt to find the word “choice”.

It’s a huge Win!

But are we not simply jumping from the frying pan of ASCII to the fire of Unicode Font Hell™?

e.g. \vec fails to render with the default Atom font. \cdot fails to render with Roboto Mono. 𝑳 fails to render here (that should be a capital L lookalike, works in Atom, fails in GitHub).

This is going to be a nightmare, if I can’t display a random .jl I find on GitHub without knowing which font it was built with. It’s going to really grate.

I would still rather Unicode Hell over ASCII. But maybe we can get the best of both worlds? We could start collectively agreeing on a Julia font. I like the idea of that. That would get us out of the immediate danger zone.

But what if we look further ahead?

How about having actual math formulae rendering in code?

So something like:

    x = \frac{a}{b}

would render as a proper formula whilst simultaneously invoking frac(a,b) for some frac function. So you can edit the code in the same way we use MathJax in documents, and you can additionally render the code.

The experience might be similar to this Discourse editor, where you manually edit the markdown file in the left pane, and it dynamically renders in the right pane.

This might be doable with a reasonably light layer over Julia, although I think it might rupture its co-evolution with JuPyteR notebooks.

It would have a huge advantage in that we would return to ASCII.

Also by displaying formulae correctly it would allow a whole new dimension of expressiveness. And if the coder can wire up the corresponding MathJax function names manually, there is pretty much no end to the potential ingenuity.

We seem to be moving into a paradigm where people are realising that we spend much more time reading code than writing it, so by shifting the burden of effort towards writing it’s actually creating an overall win.

I think that in years to come we will look back upon this Unicode explosion as a short period of chaos.

Is anyone interested in exploring this avenue?

[stevengj]

Getting Unicode display working relies just on installing the proper fonts, and greater coverage will happen inevitably over time without us doing anything. (e.g. GNU Unifont has complete Unicode coverage and can be configured as a fallback font.) The documents display and can be edited in any text editor. You don’t need to “know what font it was built with.”

Doing display of things like \frac{a}{b} requires us to write our own editor, and would mean that Julia programmers could not use their editor of choice. This seems like a nonstarter to me.

A halfway measure would be to write your own editor (or plugins for editors like Atom) that takes a standard Julia notation like a // b and renders it as \frac{a}{b}. (The Fortress language tried something like this — look into it!). This is reasonably practical and might be interesting to do, although I don’t know how popular it would be.

[oxinabox]

@stevengj

A halfway measure would be to write your own editor (or plugins for editors like Atom) that takes a standard Julia notation like a // b and renders it as \frac{a}{b}. (The Fortress language tried something like this — look into it!). This is reasonably practical and might be interesting to do, although I don’t know how popular it would be.

@ChrisRackauckas, @fengyang.wang and I had a discussion about this a while back.

Which prompted the creation of GitHub - oxinabox/Expr2LaTeX.jl: Take a julia `Expr` and render it as LaTeX. Deprecated in favour of https://github.com/korsbo/Latexify.jl
which demos that we can render julia expressions.
to finish it off nicely requires making a CST parser to map between the render and the code.
writing all the code to glue it into an IDE plugin.

I’m not sold on the idea. but it is interesting.

[ChrisRackauckas]

@oxinabox

Which prompted the creation of GitHub - oxinabox/Expr2LaTeX.jl: Take a julia `Expr` and render it as LaTeX. Deprecated in favour of https://github.com/korsbo/Latexify.jl
which demos that we can render julia expressions.
to finish it off nicely requires making a CST parser to map between the render and the code.
writing all the code to glue it into an IDE plugin.

Basically the idea is, Mathematica’s notebook is really awesome for some applications (symbolic computing), and if we could get something similar in Julia then I would be very happy.

[pitao]

Imagine if you could code like this:

(Image from here which relates to GitHub - kasperpeulen/discourse-mathjax: Mathjax plugin for discourse – it might be quite nice to install this plug-in for Julia discourse so that e.g. x^2 renders correctly).

Your text editor is on the left, it renders in real-time to the right.

And the final document renders as a TeX (or TeX-enabled markdown) document. So any existing technology that can render such documents will render our source code without modification. This would mean that an academic could publish a live paper that is just a few kilobytes of code, prints to paper, displays electronically, is Google searchable, and executes. It would also mean that code files could be completely self-documenting.

It would require a lightweight TeX → Julia wrapper. Something that converts the TeX source to an AST, which is then considered as a Julia AST and compiled. Each TeX function would require a Julia implementation.

[pitao]

PS: Throw in something like deTeXify and you have a really smooth workflow.

(New users are only allowed 2 links per post otherwise I would have stuck this in with the previous post).

[ScottPJones]

This is pretty interesting.
I think a string macro could be possibly be written that takes an expression written in LaTex, and produces a Julia AST.
I’m not so sure about having the entire file being convertible into a Julia program.
Personally, I kind of prefer the idea going the other way, having something (a macro?) that takes a Julia expression, and converts it to a beautifully rendered equation (we’d been chatting about that not too long ago on JuliaLang/julia - Gitter, and @oxinabox coded something up in an impressively short time)

[JeffreySarnoff]

@stevengj If you know, is there some structural or behavioral reason not to ship a good, symbol-covering freely redistributable Unicode font with the REPL and Juno and the other very heavily used and unencumbered ways to work with Julia? It seems that would be a reasonable first step; and people using a non-free editor/environment would be able to get that font for their use too.

[bramtayl]

I used to use Lyx to write LaTEX and it was very helpful. Unfortunately, RStudio never had a visual LaTEX editor, so making rmarkdown documents with LaTEX was much more difficult.

I’ve recently been doing all symbolic math (even just for display purposes) in jupyter using sympy and init_printing(). Unfortunately, getting profession reports out of jupyter requires quite a bit of messing around with templates, which I don’t have the patience for.