According to the CDC, there are one million blind people in the US. Are there any ways to make Julia more accessible to the blind community? … sort of like how LaTeX style variables made Julia more accessible to the science communities.
I ask because I am in the very early stages of considering/designing a low cost braille interface (https://olarth.medium.com/surf-ad7d4726f438) and I am curious about what (if any) early design decisions could promote the use of Julia in the blind community.
Julia’s syntax isn’t so different from other languages, so I expect blind programmers read and write Julia as well as any other language. Something that came to mind is how colors in @code_warntype are used to visually represent levels of type stability, and I expect (could be easily wrong) that there would be a better representation than color.
Fortunately, Julia can be written in ASCII only. All the operators except ⊻ for xor, a rare operator in most code, and ÷ for integer division (both available in ASCII function form), possibly somewhat less rare, not to be confused with the very common / for division, resulting in Float64 result.
The operators “in”, and “not in”, can also be written as ∈ and ∉ and at least the former Unicode symbol for “in” is somewhat popular. There are a ton more Unicode operators that can be defined, but I don’t see regularly used.
You are maybe thinking of Braille for English only, but you got me intrigued and I looked at the Icelandic (since my native lanuage), and the:
and the square brackets [ ] are different in Danish, there more verbose, and the special Icelandic quotes are written the same as the regular ones (probably a good thing).Finish has a different braille for ! which is a very important symbol in Julia (for end of mutating functions).
All the tab completion sequences are ASCII, right? If so, non-ASCII characters can be reverted to ASCII before being output as synthesized speech or braille (for reading the rare bits of non-ASCII code). There would have to be some indication of a tab-completed character, though, 1 xor 0 isn’t equivalent to 1 ⊻ 0.
1 ⊻ 0 can be written as xor(1, 0) but I doubt a screen reader should do that transformation. I think if Julia programmers knew of this issue, it would be ok to read the former as 1 xor 0, in most cases… as typing that in would give an error that you would just need to be aware of, and know how to fix. Or at least ou get an error in most cases…:
I very much doubt you would see xor e.g. in this context (absurd, but valid code):
julia> [1 xor 0]
1×3 Matrix{Any}:
1 xor 0
while legal, not meaning the same as [1 ⊻ 0] which is also legal, and while more plausible, I also very much doubt you would see that done.
Perhaps different colors could be represented by different heights of braille dots? The following mechanical drawing animation shows how a display pin ratchet mechanism can hold a display pin (i.e., a braille dot) at any of 11 levels in increments of 0.1 from 0 to 1. In the mechanical drawing, the ratchet mechanism is shown as an unrolled cylinder, but in the physical device raising and lowering the display pin cylinder causes it to rotate, which can be felt due to a rotating groove in the top of the display pin. In the current rough draft design, the highest level (level 1) corresponds to a height of 3/4 inch (= 1.905 cm).
(It would be great if the physical display pin ratchet mechanisms end up working as reliably as the Julia code animating their mechanical drawing.)
Is it possible for Julia coding to not rely on a screen reader? In this braille literacy report, the section “The Paradox of Technology” describes problems with solely relying on screen readers.
Well in you mean Braille, then you can use ASCII-only, at least in your own code, as people do e.g. in Python (and a lot of people to in Julia code that isn’t math-heavy), which does allow for Unicode/UTF-8, but there not in variable names, thus there you would write out e.g. alpha, beta, etc. But in Julia all of UTF-8 is also allowed for variable and for function names, so you can have this problem:
julia> b = 1; # regular (Latin) b, not to be confused with:
julia> б = 2; # equivalent Cyrillic letter, or:
julia> β = 3; # Greek beta, also equivalent, not to be confused with:
julia> ß = 4; # German letter
You’re not likely to see all these variables in the same program (probably just Greek in addition to ASCII), while it’s not disallowed. And the first three map to the same Braille, i.e. they are only equivalent for transliteration purposes and for Braille.
If you’re reading code from others you would have a problem. It’s basically the same problem blind people would have as with discussing alpha, beta, omicron COVID variants, if it weren’t spelled out.
In math it’s very common to use Greek with Latin letters (and some people to that in code, because it’s possible, so avoid that), and even many more, Fraktur etc. (I recall a professor using 4 or 5 alphabets/“fonts”, teaching the logic part of the otherwise Prolog course (other teacher taught the language), and I found it slightly mad).
A screen-reader could read out “beta” for you, so in a sense slightly better for you, except, I suppose you wouldn’t know if it were beta, or β…
It may be that being able to shorten symbols (e.g., LaTeX style variables) and being able to access more than one line at a time (e.g., hard-copy braille instead of a screen reader or a single line refreshable braille display) both are preferred for the same reason: they both make better use of working memory.
Concerning shorter symbols, it is interesting to hear Susan Osterhaus talk about her three decades of experience using the Nemeth code as a math teacher for blind students at the Texas School for the Blind and Visually Impaired (now the Perkins School for the Blind):
And so a man named Dr. Abraham Nemeth decided to create this special code, and he was a professor of mathematics himself and he wanted to be able to read and write, you know, in all these symbols in a code. So he invented the Nemeth code, and I ended up teaching myself, my students and the rest of the staff the Nemeth code, and as I was teaching myself and learning, I saw how beautifully it was done, how logical it was. … Dr. Nemeth has passed away now, but I want to say “Thank you, Dr. Nemeth” because I just don’t know how I really could have done what I did without the Nemeth code. I’m not saying that I feel like I’m a good teacher, but having that Nemeth code, that ability to give these students the higher mathematics using these higher-level math symbols was just a real necessity. And so I think that’s the main thing that has really expanded this world of mathematics to, I’m going to just say, the average student. I’m not saying I didn’t have some very brilliant students, but the average student can now take mathematics and enjoy it and, i hope, have as much fun with it as I have over the years.
Concerning accessing more than one line at a time, this is what she has to say:
Again, when you get to the braille reader, some of them [online test administrators] have said to me, ‘What about refreshable braille?’ And that’s … and basically a lot students are using refreshable braille now. But I’ve heard not everyone. But, you know, just about all of the students that I know, they have some type of refreshable braille. However, at the present time, they get one line of refreshable braille. Well, when I do math, yes, I may do one line at a time, but I look back at the line before. I want kind of this bigger picture. And there are some things that you can create in math that require your looking at more than one line at a time. For instance, a number line graph can be created, and in fact this is the standard way. Now it is considered the standard way in the United States and Canada that we make number line graphs. And they require three lines. You can’t do that on a one-line refreshable. But again, I’m going to tell you right now, teachers are still saying, “We still need the hard-copy braille, the hard-copy tactile graphics for now.” They don’t feel that the technology is there yet.
So I’m taking a whack at building a cheap (e.g., the display pins are bamboo) braille display capable of simultaneously displaying nine lines of braille.
I wonder if Julia can somehow support shorter symbols, perhaps the Nemeth code, for the blind community.
Since Nemeth Braille code is available, it seems it could support most uses of Julia. But Braille likely can’t support all of Unicode (i.e. not all alphabets at the same time, including Arabic/Hebrew, the latter sometimes used in math), or the full subset allowed in Julia, and note it’s always expanding, e.g. in Julia 1.8:
∀, ∃, and ∄ are now allowed as identifier characters (#42314).
Of the important symbols I see supported, @ (for Julia macros, or email…), √, ÷, ≤, ≠, ≈, ±, ∓, ∈ (I didn’t see ∉ which you might also expect in Julia), ∋, ⊕
Note, Julia has √ but also ∛, while not anything for higher-level roots, which is more or less an alias for the cbrt function. Those are meant for what you would expect, not for variables. However, I tested and see it possible:
julia> √ = 4; # while using as a variable isn't an error, it's a rather bad idea, for some reason possible but not:
julia> ∛ = 8;
ERROR: cannot assign a value to variable Base.∛ from module Main
Stacktrace:
[1] top-level scope
@ REPL[34]:1
Might be a bug in Julia allowing one not the other…
Technically speaking, since Julia supports full Unicode, including:
contains all 256 possible patterns of an 8-dot braille cell, thereby including the complete 6-dot cell range
this would be a problem:
julia> ⠗ = 1;
julia> R = 2; # R maps to the above Braille
I’ve never seen Braille used for names of variables (or function) names… I doubt there’s much use for that, just an oversight to allow (no pun intended…). If/since not used, I would support making Braille symbols disallowed for that (but not e.g. in strings), if that helps. I or you could open an issue.
FYI some trivia: Having said that, braille IS used in some sense in Julia, for a different purpose. To show sparse matrices, just so you know.
I was initially imagining that a tab-completed character would be represented in Braille via its ASCII sequence like \beta] with ] filling in for TAB, but it really doesn’t cover much of Unicode. The docs for tab-completed sequences don’t cover Cyrillic e.g. б. It is such a pain to input Unicode characters besides ASCII it is almost unheard of, and even tab-completed characters are rare (e.g. I prefer pi to π so text editing doesn’t rely on tab completion), but what to do in the rare cases where source code contains these characters? Technically, the 8 dots in Braille Patterns can represent a byte, so it could directly represent UTF-8, but I’m certain no human reads that way.
I don’t understand the problem enough yet to feel comfortable opening an issue about it. In contrast, I am reasonably comfortable with the display’s design concerning its ability to easily reconfigure (e.g., from 6 dot braille cells to 8 dot braille cells to some coordinate system for entering graph data). As shown below, the movable strings (green) are configured to partition the display pins into 12 rows of 32 columns of 6 dot braille cells.
Ideally, we would be able to observe the problems encountered by a person with expertise in both Julia and Nemeth code. My guess is that no such person exists. I suspect I will recruit a math teacher for the blind and just ask what would be needed in the prototype device to easily read and write Julia code.
There are a couple seemingly unexplored dimensions to this problem:
Is this device intended to be Julia-specific? I had been assuming the Julia-Braille translation would be done with intermediate Julia software running on the computer (I suppose a device could do it too if it’s powerful enough).
I’m not sure that’s good, since \ is already an operator in Julia (as in MATLAB). E.g.
julia> a = [α\β] # would read out load as .. \ \beta] ]
Yes, I do not see it here, while could be added (I guess since not too useful; for math, but you can just type in or copy-paste): Unicode Input · The Julia Language
You will have a problem if you want to cover all of Unicode (which is supported), or even if you just want to support what the REPL supports with tab-completion (a partial list, NOT all of this is defined to be valid for variables):
ℎ “Planck Constant”
ℍ “Double-Struck Capital H / Double-Struck H”
ₕ “Latin Subscript Small Letter H”
∎ \QED “End Of Proof”
∭ “Triple Integral”
I don’t even know what much of this is:
∐ “N-Ary Coproduct”
≋ “Triple Tilde”
≎̸ “Geometrically Equivalent To + Combining Long Solidus Overlay / Non-Spacing Long Slash Overlay”
₧ (that one displays differently, a font issue I guess, from where I copy-pasted from) “Peseta Sign”
Latin Small Letter Turned T
“Potted Plant”, “People Hugging”, “Latin Letter Glottal Stop”
You will have trouble with e.g. from my Icelandic alphabet:
Ð “Latin Capital Letter Eth”, not to be confused with:
Đ “Latin Capital Letter D With Stroke / Latin Capital Letter D Bar”
I opened an issue: https://github.com/JuliaLang/julia/issues/44418
If we regret it (I don’t think we will) then we could undo it later (then wouldn’t be a breaking change).
No, but Julia currently seems like the best fit for a math education setting (for the sighted and the blind). Concerning computing power, the user’s smartphone will overlook the grid of display pins … primarily because image processing is cheaper than 2304 display pin position sensors.
Maybe instead of the Braille supporting more of Unicode or mixing multiple alphabets, we just need something to rewrite Julia source code (or really any language that allows all Unicode characters) to match any target Braille alphabet. The source code would be analyzed, special characters replaced, infix operators changed to ASCII method calls (actually, I don’t know off the top of my head if all non-ASCII operators/constants/methods have an ASCII equivalent, it just feels like it should be true). All the changes would be stored in a reversible cipher. The Braille user can read and write normally, and if they’re editing someone else’s code, the cipher can be used to revert the style of code to be familiar to that someone. Hell, I’d use such a package to match ASCII if I run across source code with a lot of characters I have learn tab-completion or copy-and-paste in order to edit.
The details are mostly over my head. However, if there is an open project that demonstrates it kind of works, I would be glad to make code contributions to try to improve it.
There seems to be a couple of packages for character-wise conversion of Unicode text to ASCII TextUnidecode.jl and Unidecode.jl, but just from the space insertions and : inclusions, those alone are apparently unsuitable for rewriting Julia source files to valid code.
You should get your project up and running for ASCII, it’ll cover most of Julia, and maybe try to at least support the more common non-ASCII symbols with special Braille characters and put a dummy character for the rest. Increasing Braille support for non-ASCII, converting source files and printouts to valid code using only ASCII (or any other set of characters), expanding Julia beyond Unicode, these are not trivial projects. Absolutely keep these sort of discussions active as you work, it’s very important to improve the language’s accessibility.
But it can’t be done, in general (in most cases, yes, for operators), because it’s not true. You can’t change to ASCII operator form, always, but you can change Unicode operators to method calls, aka functions (likely except to exotic “custom infix operators”), when that’s not possible. Just changing to operators would be an easier transformation, and it would practical if you allow some exceptions.
For users own functions, they could define confusingly different functions, and with your proposal even more confusing for the blind, since then a totally invisible difference:
julia> α(x) = ...
julia> alpha(x) = ...
It’s not advised, and I rarely if ever see Unicode for function names, at least much more for variables. It’s very plausible this could happen with a Unicode version being defined in one package dependency and the ASCII version, for different purposes, in another package, so it would be dangerous to allow blind people to see the same, when they are not (could be a non-default option to do that…, as this should likely never happen in practice?).
There are two exceptions for the built-in operators:
a ⊻ b, doesn’t have an equivalent, only a function form xor(a, b) and a ÷ b has div(a, b) (not to be confused with a / b, the most common division form, giving a floating point number, as opposed to an integer for the former).
Plus most problematic (but I see rarely if ever used) Julia allows: https://github.com/JuliaLang/julia/blob/master/HISTORY.md
Custom infix operators can now be defined by appending Unicode combining marks, primes, and sub/superscripts to other operators. For example, +̂ₐ″ is parsed as an infix operator with the same precedence as + (#22089).
some more to be aware of (potentially difficult for Bralle, didn’t look into if ᵀ, as opposed to T, is well supported with Braille):
The postfix conjugate transpose operator ’ now accepts Unicode modifiers as suffixes, so e.g. a’ᵀ is parsed as var"‘ᵀ"(a), which can be defined by the user. a’ᵀ parsed as a’ * ᵀ before, so this is a minor breaking change (#37247).
I’m not sure this is super helpful, even for the seeing, for Braille purposes could be changed to regular numbers it seems:
Support for unicode bold digits and double-struck digits 0 through 9 as valid identifiers (#32838).
And that’s the current use of Braille patterns in Julia:
Display large sparse matrices with a Unicode “spy” plot of their nonzero patterns, and display small sparse matrices by an Matrix-like 2d layout of their contents (#33821).