Julia and braille

Yeah the more I think about it, the more the Julia-to-ASCII-Julia conversion seems untenable.

I did think that some analysis of the source code, as opposed to character-wise replacement, was necessary, but you’re right, I didn’t realize how much. My idea was feasible for reading a single source file: if alpha is not in the source code, then α → alpha, or if alpha and alpha1 are present, then α → alpha2. However, include or importing can implicitly introduce alpha and alpha1 from other files into the namespace, in which case a cipher bounded to a single source file cannot possibly know to rename α → alpha2. Moreover, typing alpha2 to mean α in other scripts or the REPL can only work if the reverse cipher alpha2 → α acts beyond that single source file.

I was thinking that any non-ASCII infix operator would be changed to an ASCII function form e.g. a ⊻ b to xor(a,b). However, there would be an issue with operators that don’t already have ASCII-named methods. Even if there’s a consistent cipher like ↓ → downarrow, defining a ↓(x, y) method for 3 ↓ 5 doesn’t automatically make downarrow(3, 5) valid code, and 3 downarrow 5 certainly isn’t valid code. Either the parser needs to incorporate ASCII equivalents or the reverse cipher needs to operate everywhere or, again, you can’t type what you read from one script into another script or the REPL.

The modifying marks are also a problem. I would’ve imagined a cipher for the modifying marks e.g. 3 +̂ₐ″ 5 to +hatsubasuperquote(3, 5), but right off the bat you can see that +hatsubasuperquote is parsed with + as a unary operator, not +hatsubasuperquote as a single variable.

I think it’s probably best to not try to translate anything, going for exact valid code, e.g. not a ⊻ b to xor(a, b) but rather to a xor b even though invalid code. Blind people are I guess good with reading between the lines.

I would try to support ASCII, plus the standard Unicode operators Julia has, in some way, but not all of Unicode, and additionally Greek. Then you have really good Julia support, almost as good as seemingly needed/possible.

What I found (two online translators do not support e.g. Greek with): Unicode Braille Converter

`, {, |, and } are not used and their Braille ASCII rendition is not defined.

That would be a serious limitation (also in ASCII-only Python or C), if there weren’t a workaround. There is but most may not know it, at least programmers using braille will.

It’s intriguing that seemingly every language is supported with a braille (through different sets), including e.g. Chinese, both “Mandarin (contracted), Cantonese (uncontracted and no space between words)” (but alphabets only) and even “China Braille Musical Signs”. But there’s no “braille Unicode”, it seems, to support mixing all languages together as with Unicode, only possible to mix the most important alphabets for math (with English).

Nemeth braille encodes some math alphabets not in Unicode, e.g., Greek Script and Russian Script. Meanwhile, Unicode has math double-struck and monospace English alphanumerics, which don’t exist in Nemeth braille. Unicode also has six alphabets that aren’t mentioned in the Nemeth specification but that can be defined unambiguously with Nemeth indicators, namely bold Fraktur (Nemeth calls Fraktur “German”), bold Script, and Sans Serif bold and/or italic.

It’s good to know Hebrew and Fraktur letters (and Russian) also supported for math with English, plus Greek and which Unicode does actually support…

Off-topic:
Hindi is large, but “the Khmer Language has the largest alphabet in the world, with a total of 74 letters, consisting of 33 consonants, 23 vowels and 12 independent vowels.” Khmer is supported, even though the 6-dot braille patterns only support 64 combination, so it’s variable-length, like UTF-8. The 8-dot braille seems not very common, exotic.

“Rotokas, an indegeounous language to Papua New Guinea is the smallest known language in the world with a total of 12 letters!” (it’s a subset of English alphabet so seems supported that way).

I was intrigued to know W is 豆贝尔维 (dòu bèi ěr wéi) in Chinese (doesn’t seem fitting with the video there: Mandarin Chinese Pinyin Pronunciation (Better Chinese) - YouTube):
Chinese Alphabet and Pronunciation

“Mongolian has one of the longest word lengths, by letter, in the world, with an average of 13 letters per word!”

Here is a video of Susan Osterhaus (the same previously quoted math teacher with three decades of experience teaching blind students) describing some of her Nemeth code reference sheets: Nemeth Code Reference Sheets: Algebra I, Algebra II Reference Sheets by TSBVI - 8 - YouTube

Oddly enough, the Nemeth code in ASCII for the “is greater than or equal to” symbol is not >= (as we have come to expect in programming languages). It is instead .1:

She says

… but if you look at the ASCII, you might go, “What?” And it just depends. If you’re an ASCII person, that makes all the sense in the world. I’ll tell you, to me, that drives me crazy.

Perhaps that is near the root of the Julia and braille problem. Some Nemeth braille users prefer to edit braille cells and others prefer to edit ASCII even though they are both trying to create the same print-ready math equations. A solution allowing them to collaborate easily (i.e., without driving each other crazy) on a lengthy math derivation involving a sequence of dozens of math equations I think would be the same solution that would allow a person editing Nemeth braille cells and a person editing Nemeth ASCII and a person editing with a Unicode capable editor and a person editing with a Unicode incapable editor all to collaborate easily (i.e., without driving each other crazy) on a Julia source code file.

As a first pass guess at a solution, how about a no-frills lingua franca section of code that Julia can execute followed by a comment block that defines the specific substitutions necessary to convert the lingua franca section of code to the user’s preferred editing format.

Making version 0 more manageable, I’ve reduced the number of display pins by a factor of four. Now, the design has a total of 576 display pins in a grid of 18 rows and 32 columns (or 6 rows and 16 columns of 6 dot braille cells) as shown in the following mechanical drawings of the top and bottom surface.

I’m intrigued by the 8-dot Luxembourgish braille.

  • According to its Wikipedia page, “It is the only eight-dot alphabet listed in UNESCO.”
  • The bottom row’s two dots are used as shift and ctrl keys that operate on the well-known 6-dot braille cell. Being able to shift to upper case letters in a single braille cell seems like it could significantly shorten braille source code that uses a lot of capital letters.
  • The numeric characters are separate from the alphabet characters. Not requiring a leading cell to differentiate between a string of numbers and a string of letters seems like it would be easier to read (not having to remember the state of a prior switch to number mode).
  • The bottom row ctrl dot is currently used to encode the accented characters Ä, Ë, and É but for Julia code I think it would be more appropriate for that ctrl dot to cause a switch to the Greek alphabet.
  • I have not yet tried to fit all of Julia’s symbol operators into this 8-dot cell, but certainly it could cover a lot of them. For example, perhaps ctrl-’ = ` (i.e., ctrl apostrophe = backward apostrophe).
1 Like

It seems as if a couple types of translations (implemented in the display device) would be very helpful:

  1. contractions of long strings of julia text that have little information, and
  2. translation from other braille encoding schemes (e.g., when collaborating with another person who prefers to use a different type of braille).

Concerning 1, naively displaying the entire REPL prompt and indenting would waste a lot of the display’s very limited braille cells. (With 8 dot braille cells, the proposed display has just 4.5 rows of 16 columns of braille cells.) For example in this code, all four braille cell rows are needed to display “Hello, World!” because of the lengthy prompt and indentation.

# column counts:
#        1111111
#234567890123456
julia> println(
       "Hello,"*
       " World"*
       "!")

Contracting the REPL’s prompt and indentation would save a lot of display space. On a related note, does anyone know of a reasonable estimate of the frequency of use of Julia’s keywords? It would save display space to define contractions for the longest and most frequently used Julia keywords. (I Googled julia keyword frequency of use and got plenty of links to text analysis Julia applications, but I didn’t see any that were self-analyzing Julia’s use of keywords in a reasonably large corpus of Julia source code.)

My idea to reduce the width of the proposed display from 64 display pins to 32 display pins (= 16 braille cells) is a failed idea. After seeing how complicated it would be to write the simple

println("Hello, World!")

in lines that are only 16 characters wde, the width of the proposed display is back to 64 display pins (= 32 braille cells), which is enough to hold println(“Hello, World!”) in one line … bringing the total number of display pins up to 1152 (= 64*18) but significantly improving the proposed display’s Time To Hello World (TTHW) metric.The resulting 32:9 aspect ratio also might improve literary accessibility by being able to simultaneously display a tactile graphic along side its braille caption. Here is the updated mechanical drawing of the display surface, with strings partitioning the display into 8 dot cells.

Here’s an early attempt at a Luxembourgish/QWERTY/Julia (LQJ) table: an approach to encouraging succinct Julia code when it is written in braille (because refreshable braille cells are scarce) by mapping multi-byte Unicode characters to single-byte Unicode offsets. The left side of the table representing English and Greek alphabets looks good. (To see details, zoom in.) The right side of the table, representing the numbers and operators and miscellaneous letters from other languages, still needs a lot of work. The Julia code that generates the table is in the function lqj() in the appendix of this essay about the proposed tactile display.

2 Likes

Using only lower case characters as the index into the above LQJ table leaves 17 columns of the table unused. I’m wondering about using Julia’s string interpolation syntax to access those 17 columns. For example, perhaps $i could represent a single byte contraction of "if " and $I a single byte contraction of "elseif ".

Any thoughts on the pros or cons of using Julia’s string interpolation this way?

It doesn’t sound like you want string interpolation. It seems like you want a completely different abbreviated surface syntax that maps to a Julia backend. Which is totally fine, but you’ll need to implement your own parser.

But I would get significant input from visually impaired programmers first, and up to now that seems to be absent from this discussion?

This stackoverflow thread is particularly relevant here — it includes comments from several professional programmers who are visually impaired, and a summary quote is:

The majority of blind computer users and programmers use a screen reader of some sort.

i.e. an audio interface, sometimes supplemented by Braille.

There have also been a number of past works on programming for the visually impaired, and it seems that virtually none of them focus on Braille:

In consequence, I’m concerned that this whole discussion has become unmoored from the actual needs of visually impaired programmers.

4 Likes

Yes, it’s time to get a sort-of-working (SOW) prototype in front of people.

When I started thinking about this project a couple months ago, what intrigued me then (and still does) is that it is a cheap rickety collection of wood, bamboo and string that opens the gate to a lot of sensing and processing capabilities because it is overlooked by a smartphone.

I think it has a pretty good chance of helping with the problem of creating tactile graphics and maybe even making them interactive. It might also be able to help scale the work of braille teachers who are in short supply as described in “The Braille Literacy Crisis in America”.

My guess is that whether blind programmers will prefer braille or a screen reader will primarily be a matter of speed, whichever allows them to move their focus to different lines fastest will be preferred. And I don’t think they’ve yet had a fast braille option. As described here by the American Foundation for the Blind, there are 18 different refreshable braille products made by 10 different companies but they all are limited to displaying just a single line of braille at a time. (How did that cluster happen?)

I’ll post to this discussion when I have an early prototype in hand. Meanwhile, I plan to continue posting notes about progress to this essay … mainly as an organizational aid for myself.

It is relevant but a little dated. A more recent view about assistive technologies for the blind is in the Jan 2022 research report “Workplace Technology” by the American Foundation for the Blind. On page 33,

In an open-ended question, participants were asked what additional accommodations
they believed would allow them to perform their work responsibilities more efficiently.
The most mentioned accommodations were:
• multiline braille displays or other improvements to the smooth functioning of braille
displays (n=24)
• artificial intelligence (n=18)
• some form of smart glasses (n=17)
• an indoor GPS (n=9)
• some form of visual interpreting service (n=7)

And on page 50 of the same research report,

When asked about technology they would like to see developed that would increase
their productivity, participants responded that they would like companies to create
content and technology that is accessible from the onset. They would like to see
the enforcement of laws related to accessibility and improvements to products,
such as better OCR software, more enhanced voice controls, and affordable multiline
braille displays.

2 Likes

So basically, blind programmers are stuck with an interface like ed but somewhat worse? That’s impressively terrible.

One of the ways it is somewhat worse is price. According to the previously linked American Foundation for the Blind review of refreshable braille devices,

The price of braille displays range from $3,500 to $15,000, depending on the number of characters displayed.

So an 80 character line editor costs $15,000.

A quick update:

  • The highest risk design seems to be the inch long ratchet mechanism (shown in the animated gif).
  • To quickly iterate on that ratchet mechanism design and make a few prototype braille displays, I ordered a high speed 3D printer 10 weeks ago and it is scheduled to be delivered in November.
  • I’m starting to think that much of the braille display’s display pin layer can be made out of recycled materials.
  • In February, I ordered a raspberry pi to control the pushing of the display pins in a prototype braille display. The part was back ordered and was finally delivered last week. The packing list had the following notice: “These items are controlled by the U.S. Government and authorized for export only to the country of ultimate destination for use by the ultimate consignee or end-user(s) herein identified. They may not be resold, transferred, or otherwise disposed of, to any other country or any person other than the ultimate consignee or end-user(s), either in their original form or after being incorporated into other items, without first obtaining approval from the U.S. Government or as authorized by U.S. law and regulations.” I guess the export controls make sense but I was not expecting a part from the UK to be subject to US export controls.

ratchet