Strange Lispy warning on master

Sorry, I copied the wrong version - my code is a bit in transition - I’m testing a new version where it is Str{cse"UTF-8"} instead of Str{:UTF8} (and the cse macro returns a CSE{cs"Unicode", enc"UTF8"} type.)

Frequently the 3 separate concepts of character sets, code points, and encodings get confused and conflated, and I wanted to avoid that (and allow for things to be as generic and extensible as possible).
For example, the official Chinese standard GB18030 has a number of characters added, that are not yet part of the Unicode character set, but they represented using Unicode code points in one of the Private Use Areas.
Because UTF-8 uses so much extra space for Chinese characters, they encode GB18030 with a 1,2,4 byte multibyte scheme, with ASCII and control characters in 1 byte, 99% of the used Chinese characters in 2 byte forms, and anything else in a 4 byte form that can encode any Unicode code point as well.

When I get around to adding full support for encodings to my package, I wish to be able to easily handle situations like that, which would end up with something like: CSE{cs"Unicode", enc"GB18030"} to get that 1,2,4 byte encoding scheme.

PR https://github.com/JuliaLang/julia/pull/25336 fixes this by removing the 0.6 deprecation.

1 Like