I’ve been getting failures loading quite a few packages because of the move of these (at least the exports) to Unicode. For a number of these incredibly frequently used names, wouldn’t it be better to have the name exported from Base, so that instead of a deprecation warning, you’d get a method err…

[image] ScottPJones: The new state means that if you want to define a pure ASCII, ISO-8859-1, GB2132 or EUC lowercase method, you must have a using Unicode, even if you are doing nothing at all with Unicode, which does not make much sense to me. It doesn’t seem like that much of an imposition…

I am jumping into this conversation without much knowledge in string processing, but even without sufficient knowledge on this topic, I sympathize with @ScottPJones argument. If something is applicable to any kind of string that is not necessarily Unicode, it doesn’t make sense to have it defined th…

The definition of what it means to uppercase a character is (by our definition) inherently a matter of what the Unicode standard defines it to be. That said, the meaning of uppercase and lowercase mappings are not going to change for several of these encodings, so it does seem reasonable to allow th…

We can move these functions to Base at any time in 1.x releases if we want. But really, what’s the big deal with importing the Unicode module? Presumably, even if your string is pure ASCII or ISO-8859-1, you’ll want your implementation to be consistent with Unicode, so that’s not a misnomer either. …

GB2132, JIS, EUC are not consistent with Unicode, for example, and I’ll be able to support those. There are other character sets that are not even consistent with ASCII (such as EBCDIC).

[image] ScottPJones: GB2132, JIS, EUC are not consistent with Unicode In what way? Do they define their own uppercase/lowercase mappings that are incompatible with Unicode?

Yes. For example, even ANSI Latin1 is not really fully compatible with Unicode when it comes to uppercase, even though the characters present in ANSI Latin1 are a pure subset of Unicode, because there are 2 characters whose uppercase versions are not present in Latin 1. julia> uppercase("µ")[1] E…

What should the result of upper casing a Latin-1 string containing either of these letters be?

[image] ScottPJones: GB2132, JIS, EUC are not consistent with Unicode, for example, and I’ll be able to support those. There are other character sets that are not even consistent with ASCII (such as EBCDIC). While I have never encountered these character sets, I can imagine scenarios (eg dat…

[image] StefanKarpinski: What should the result of upper casing a Latin-1 string containing either of these letters be? I think in C, you would just get the lower case character back, but in Julia it would feel more natural to throw an InexactError or similar. For example one could imagine t…

Problems with deprecations of islower, lowercase, isupper, uppercase

Internals & Design

StefanKarpinski December 16, 2017, 7:38pm 13

In what way? Do they define their own uppercase/lowercase mappings that are incompatible with Unicode?

Topic		Replies	Views
Changes to the representation of Char Internals & Design	14	3050	December 12, 2017
String indices : byte indexing feels wrong New to Julia strings , unicode	18	1700	December 5, 2023
Is Julia well-suited for string manipulation? General Usage strings	24	3327	March 24, 2023
Performance of length(::String) Performance	24	4179	July 28, 2018
Problem processing non utf8 string New to Julia	16	2365	June 1, 2018

Problems with deprecations of islower, lowercase, isupper, uppercase

Related topics