# StringIndex idea (Julia 2.0)

**URL:** <https://discourse.julialang.org/t/stringindex-idea-julia-2-0/107185>\
**Category:** Internals & Design\
**Tags:** strings, indexing\
**Created:** [December 5, 2023, 8:23pm UTC](https://discourse.julialang.org/t/stringindex-idea-julia-2-0/107185 "2023-12-05T20:23:43Z")\
**Posts on this page:** 13\
**Page:** 4

<div class="post-metadata">

**Author:** ![sijo](https://avatars.discourse-cdn.com/v4/letter/s/da6949/32.png) [@sijo](https://discourse.julialang.org/u/sijo)\
**Post date:** [February 9, 2024, 8:44am UTC](https://discourse.julialang.org/t/stringindex-idea-julia-2-0/107185/64 "2024-02-09T08:44:26Z")

</div>

Interesting but `my_string[end-9th:end-5th]` is disturbing, it goes against the semantics of `th`…

---

<div class="post-metadata">

**Author:** ![JM\_Beckers](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jm_beckers/32/22482_2.png) [@JM\_Beckers](https://discourse.julialang.org/u/JM_Beckers)\
**Post date:** [February 9, 2024, 9:00am UTC](https://discourse.julialang.org/t/stringindex-idea-julia-2-0/107185/65 "2024-02-09T09:00:48Z")

</div>

> [@sijo](#):
>
> `my_string[end-9th:end-5th]` is disturbing,

maybe

`my_string[9thfromend:5thfromend]` or something similar

---

<div class="post-metadata">

**Author:** ![sijo](https://avatars.discourse-cdn.com/v4/letter/s/da6949/32.png) [@sijo](https://discourse.julialang.org/u/sijo)\
**Post date:** [February 9, 2024, 10:31am UTC](https://discourse.julialang.org/t/stringindex-idea-julia-2-0/107185/66 "2024-02-09T10:31:59Z")

</div>

I’d say `my_string[end-9char:end-5char]` would make most sense. To echo @goerz’s idea above we could have both `9char` and `9grapheme`…

---

<div class="post-metadata">

**Author:** ![uniment](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/uniment/32/24532_2.png) [@uniment](https://discourse.julialang.org/u/uniment)\
**Post date:** [February 9, 2024, 2:11pm UTC](https://discourse.julialang.org/t/stringindex-idea-julia-2-0/107185/67 "2024-02-09T14:11:46Z")

</div>

> [@sijo](#):
>
> it goes against the semantics of `th`…

Can you explain what gives you this impression? I read `end-9th` as “the 9th from the end,” which describes the exact semantics that is hoped for.

I think the tricky part is to remember that `end` describes the _positional index_ of the last element, rather than the _ordinal index_ of the last element.

---

<div class="post-metadata">

**Author:** ![mnemnion](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mnemnion/32/206596_2.png) [@mnemnion](https://discourse.julialang.org/u/mnemnion)\
**Post date:** [February 9, 2024, 3:51pm UTC](https://discourse.julialang.org/t/stringindex-idea-julia-2-0/107185/68 "2024-02-09T15:51:12Z")

</div>

I wouldn’t want to conflate ordinal syntax with `StringIndex`, although I think you’re on to something with using juxtaposition for `StringIndex` types.

Two reasons. The first is that indexing by codepoint isn’t an ordinal vs. cardinal distinction, so “If you use `4`, it’s a byte index, if you use `4th` it’s a codepoint index” would just be something to remember, it’s not at all obvious that `4th` means the fourth codepoint, not the fourth codeunit.

The other one is that there are at minimum four ways to want to index a string: by its codeunits, codepoints, graphemes, and textwidth. I would argue that _most times_ a developer is tempted to use codepoints, what they actually want is graphemes. Codepoints will prevent throwing an invalid index error, but they don’t prevent splitting up 👍🏼 or 🇦🇺 or  (I don’t have a good way to type Latin characters with combining codepoints, just pretend that one is composed).

But a syntax such as `string[1ch:5ch]`, `string[1gr:5gr]`, `string[1wc:5wc]` is a nicely-compact way to generate those sorts of index. `wc` because of the `wcwidth` function which never quite made it into the C standard, it could be `1tw` to remind users that it’s using the `textwidth` function under the hood. I think this is fitting because the various approaches to indexing are effectively units, and this looks like a unit, and units are a place where highly-abbreviated names are considered acceptable, no one insists on `1m` being spelled `1meter`.

If this system also had `CharString` `GraphemeString` etc., the set should include `1cu` (maybe just `1u`?), so those can be addressed by a codeunit, since each would have a default interpretation of an `Int`. In fact, maybe _that_ is the place to unify with ordinal indexing, since it carries the intended meaning of “index this in the normal way, whether or not that differs from whatever weird expectation the indexable type has for a number”.

---

<div class="post-metadata">

**Author:** ![uniment](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/uniment/32/24532_2.png) [@uniment](https://discourse.julialang.org/u/uniment)\
**Post date:** [February 9, 2024, 4:56pm UTC](https://discourse.julialang.org/t/stringindex-idea-julia-2-0/107185/69 "2024-02-09T16:56:42Z")

</div>

> [@mnemnion](#):
>
> if you use `4th` it’s a codepoint index” would just be something to remember, it’s not at all obvious that `4th` means the fourth codepoint, not the fourth codeunit.

I don’t know what a codepoint is; I’m just a dumb user. All I know is that when I iterate over a `String` I get a sequence of `Char`s, and I want `my_string[4th] === (my_string...,)[4]` and `my_string[6th:10th] === join((my_string...,)[6:10])` and `my_string[end] === my_string[length(my_string)th]`.

I would think whoever cares about special string-specific indexing by codepoint, codeunit, grapheme or whatever should be okay with using a library for that.

---

<div class="post-metadata">

**Author:** ![mnemnion](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mnemnion/32/206596_2.png) [@mnemnion](https://discourse.julialang.org/u/mnemnion)\
**Post date:** [February 9, 2024, 5:06pm UTC](https://discourse.julialang.org/t/stringindex-idea-julia-2-0/107185/70 "2024-02-09T17:06:15Z")

</div>

> [@uniment](#):
>
> I don’t know what a codepoint is; I’m just a dumb user.

A codepoint is simply a `Char`, in every sense that matters.

Languages shouldn’t be optimizing for “dumb users” to write less-buggy code which still has bugs in it. Strings come with essential complexity and should have the tools they need to work with all of that complexity, not just some of it.

You haven’t explained why a `Char` is the correct “ordinal” way to count a string. In every other proposed use of `OrdinalIndexing`, `indexable[5]` and `indexable[5th]` are the same value, if the indexable type starts with an index of `1`, which `String` does. Reusing that syntax for Strings breaks that contract.

I suspect you’re just reaching for it out of convenience: there’s a proposal on the table for a second syntax for indexing, why not reuse it here. Why not: because it doesn’t fit the application.

Let me put it this way: do you think your dumb user would expect the “fifth” element of “hey! 👋🏻” to be the first half of the emoji?

---

<div class="post-metadata">

**Author:** ![uniment](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/uniment/32/24532_2.png) [@uniment](https://discourse.julialang.org/u/uniment)\
**Post date:** [February 9, 2024, 5:29pm UTC](https://discourse.julialang.org/t/stringindex-idea-julia-2-0/107185/71 "2024-02-09T17:29:40Z")

</div>

> [@mnemnion](#):
>
> You haven’t explained why a `Char` is the correct “ordinal” way to count a string.

I think I have:

> [@uniment](#):
>
> when I iterate over a `String` I get a sequence of `Char`s

In other words, I would expect this to work:

```julia
for (i,c) = enumerate(my_string)
    @assert c === my_string[(i)th]
end

```

Iteration is the common thread tying all sorts of collections together, and ordinal indexing can be considered a [reliable, but sometimes inefficient] way to index them.

And indexing by both positional and ordinal index can get the best of both worlds when that’s an option.

> [@mnemnion](#):
>
> In every other proposed use of `OrdinalIndexing`, `indexable[5]` and `indexable[5th]` are the same value, if the indexable type starts with an index of `1`, which `String` does. Reusing that syntax for Strings breaks that contract.

This is because the `OrdinalIndexing` package was originally conceived only for dense arrays. I’m not proposing to implement it as-is; I propose a new contract (see `for` loop above) that makes it actually useful for strings and ordered dictionaries and whatever else, while retaining its behavior for dense arrays.

> [@](#):
>
> do you think your dumb user would expect the “fifth” element of “hey! 👋🏻” to be the first half of the emoji?

I would expect `my_string[5th]` to be `(my_string...,)[5]`, whatever that may be.

If you disagree that `(my_string...,)[5]` should be whatever it is, then that’s an issue that should be taken up with `String`’s iteration interface.

---

<div class="post-metadata">

**Author:** ![mnemnion](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mnemnion/32/206596_2.png) [@mnemnion](https://discourse.julialang.org/u/mnemnion)\
**Post date:** [February 9, 2024, 5:37pm UTC](https://discourse.julialang.org/t/stringindex-idea-julia-2-0/107185/72 "2024-02-09T17:37:25Z")

</div>

> [@uniment](#):
>
> In other words, I would expect this to work:
> 
> ```julia
> for (i,c) = enumerate(my_string)
> @assert c === my_string[(i)th]
> end
> 
> ```
> 
> Iteration is the common thread tying all sorts of collections together, and ordinal indexing can be considered a [reliable, but sometimes inefficient] way to index them.

Is that the actual contract of `OrdinalIndexing`? Does this apply to sparse arrays as well, for example? If so, ok, this is reasonable. If that isn’t in fact a correct generalization of the idiom, then it shouldn’t be used.

I still think it’s important to have index types which aren’t based on Chars, though. There are scripts like Devanagari, and many like it, where codepoints are practically useless.

---

<div class="post-metadata">

**Author:** ![uniment](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/uniment/32/24532_2.png) [@uniment](https://discourse.julialang.org/u/uniment)\
**Post date:** [February 9, 2024, 6:36pm UTC](https://discourse.julialang.org/t/stringindex-idea-julia-2-0/107185/73 "2024-02-09T18:36:51Z")

</div>

> [@](#):
>
> Is that the actual contract of `OrdinalIndexing`? … Does this apply to sparse arrays as well, for example?

Yes my idea is strictly a generalization of that package’s contract to work with non-array types, with the exception of changing the semantics such that whereas `4+5th` would previously create a `9th` index, by my proposal it would remain `4+5th` (a `PositionalOrdinalIndex(4,5)` representing the combination of a positional index of 4 _and_ an ordinal index of 5; to get just the `9th` ordinal index one would write `(4+5)th` or `4th+5th`).

> [@](#):
>
> I still think it’s important to have index types which aren’t based on Chars, though.

Ok! I think there’s room for both the ordinal indexing `th` concept _and_ the string-specific `ch`, `gr`, `wc` ideas, although the latter would be better in a package imo (maybe wrong)

---

<div class="post-metadata">

**Author:** ![mnemnion](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mnemnion/32/206596_2.png) [@mnemnion](https://discourse.julialang.org/u/mnemnion)\
**Post date:** [February 9, 2024, 6:38pm UTC](https://discourse.julialang.org/t/stringindex-idea-julia-2-0/107185/74 "2024-02-09T18:38:21Z")

</div>

> [@uniment](#):
>
> Yes my idea is strictly a generalization of that package’s contract to work with non-array types, with the exception of changing the semantics such that whereas `4+5th` would previously create a `9th` index, by my proposal it would remain `4+5th` (a `PositionalOrdinalIndex(4,5)` representing the combination of a positional index of 4 _and_ an ordinal index of 5; to get just the `9th` ordinal index one would write `(4+5)th` or `4th+5th`).

Sounds like a reasonable PR to submit to that package then.

---

<div class="post-metadata">

**Author:** ![uniment](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/uniment/32/24532_2.png) [@uniment](https://discourse.julialang.org/u/uniment)\
**Post date:** [February 9, 2024, 6:40pm UTC](https://discourse.julialang.org/t/stringindex-idea-julia-2-0/107185/75 "2024-02-09T18:40:11Z")

</div>

Sure! More importantly, a language feature to consider.

---

<div class="post-metadata">

**Author:** ![mnemnion](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mnemnion/32/206596_2.png) [@mnemnion](https://discourse.julialang.org/u/mnemnion)\
**Post date:** [March 27, 2024, 10:11pm UTC](https://discourse.julialang.org/t/stringindex-idea-julia-2-0/107185/76 "2024-03-27T22:11:11Z")

</div>

> [@mnemnion](#):
>
> But a syntax such as `string[1ch:5ch]`, `string[1gr:5gr]`, `string[1wc:5wc]` is a nicely-compact way to generate those sorts of index. `wc` because of the `wcwidth` function which never quite made it into the C standard, it could be `1tw` to remind users that it’s using the `textwidth` function under the hood. I think this is fitting because the various approaches to indexing are effectively units, and this looks like a unit, and units are a place where highly-abbreviated names are considered acceptable, no one insists on `1m` being spelled `1meter`.

So I went ahead and [implemented this](https://discourse.julialang.org/t/ann-stringunits-jl/112202).

[Previous page](https://discourse.julialang.org/t/stringindex-idea-julia-2-0/107185.md?page=3)