# StringIndex idea (Julia 2.0)

**URL:** <https://discourse.julialang.org/t/stringindex-idea-julia-2-0/107185>\
**Category:** Internals & Design\
**Tags:** strings, indexing\
**Created:** [December 5, 2023, 8:23pm UTC](https://discourse.julialang.org/t/stringindex-idea-julia-2-0/107185 "2023-12-05T20:23:43Z")\
**Posts on this page:** 20\
**Page:** 3

<div class="post-metadata">

**Author:** ![goerz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/goerz/32/3269_2.png) [@goerz](https://discourse.julialang.org/u/goerz)\
**Post date:** [February 5, 2024, 6:01pm UTC](https://discourse.julialang.org/t/stringindex-idea-julia-2-0/107185/41 "2024-02-05T18:01:53Z")

</div>

> [@stevengj](#):
>
> You really want to do string processing based on [`textwidth`](https://docs.julialang.org/en/v1/base/strings/#Base.Unicode.textwidth)

> Fortunately, Julia already has just such a function, done correctly, the [`lpad`](https://docs.julialang.org/en/v1/base/strings/#Base.lpad) function

And this is why I love Julia, and I’m more than happy to have switched from Fortran/Python to Julia. It also demonstrates why Julia’s string handling is in good hands, and really doesn’t need to go back to the drawing board. 😉

(Thanks for pointing out the bug in the Python code. You’re right, although it’s probably good enough for practical use)

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [February 15, 2024, 3:28pm UTC](https://discourse.julialang.org/t/stringindex-idea-julia-2-0/107185/42 "2024-02-15T15:28:36Z")

</div>

8 posts were split to a new topic: [Truncating padded text](https://discourse.julialang.org/t/truncating-padded-text/110256)

---

<div class="post-metadata">

**Author:** ![mnemnion](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mnemnion/32/206596_2.png) [@mnemnion](https://discourse.julialang.org/u/mnemnion)\
**Post date:** [February 5, 2024, 6:56pm UTC](https://discourse.julialang.org/t/stringindex-idea-julia-2-0/107185/46 "2024-02-05T18:56:15Z")

</div>

> [@stevengj](#):
>
> `s[1:5]` is buggy code for an arbitrary `s`, certainly. But it seems more like a programmer/coding problem than a language one.
> 
> “Give me the first five codepoints of a string” is possible to do properly in Julia right now, via `s[1:nextind(s,1,4)]`. Admittedly, that syntax is a little awkward, and if this were an important/common operation for arbitrary Unicode strings then it would be good to have a nicer syntax. But is it an important or common operation?

The idea here is to make it easier to write less buggy code, surely. Julia doesn’t have to provide `nextind` or `prevind` either, or return a full character when indexed. It could do what Lua does: treat a string as an opaque byte array, and leave encoding matters up to userspace. I’m glad it doesn’t!

I’m also glad it doesn’t use UTF-32 so as to get O(1) codepoint indexing, I consider all these choices good. I wouldn’t want indexing into a string to do anything else, certainly I wouldn’t want to turn indexing with an `Int` into a `String` to turn into a codepoint operation. One of the core advantages of the view proposal is that it introduces no breaking changes, everything now in the language functions identically.

I think a collection of view types which treat indexing differently would be useful, and a collection of index types to go with them is an essential consequence of having them. It’s a good fit for Julia’s approach to genericity and dispatch, and it would greatly simplify code which works with several aspects of a string simultaneously. It exposes the complexity of strings and Unicode to the user, while also managing that complexity. I’ve written some grapheme-respecting code, and it’s a hassle, it would have been much easier to write correctly if a `GraphemeView` were available.

Others have also answered this, but yes, truncation and padding are very common operations on strings, and really should be done using graphemes or textwidth (bugs notwithstanding), depending on the medium. Codepoints aren’t as commonly useful, in fact, but it would be odd to omit them on that basis, they’re a valid and important way to handle a string.

I think all this would be a good addition to the core system, but maybe it should start life as a package to prove out the concept. There are some real unsolved questions about how all this code would work together, all of which would need to be answered before this could become a permanent addition to the Julia 1.x API.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [February 5, 2024, 8:14pm UTC](https://discourse.julialang.org/t/stringindex-idea-julia-2-0/107185/47 "2024-02-05T20:14:14Z")

</div>

> [@mnemnion](#):
>
> The idea here is to make it easier to write less buggy code, surely.

Yes, but that’s the argument for an opaque `StringIndex` type, ala [Restrict indexing into strings to a special `ByteIndex` or `StringIndex` type · Issue #9297 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/issues/9297) as mentioned above.

I don’t think it’s an argument for O(1) character indexing (e.g. via a `CharString` with cached indices as you were proposing).

> [@mnemnion](#):
>
> Codepoints aren’t as commonly useful, in fact, but it would be odd to omit them on that basis, they’re a valid and important way to handle a string.

We don’t “omit” codepoints — you can certainly access them, and unlike Python we have a whole type (`Char`) to represent them — we just don’t support O(1) indexing of the `n`-th codepoint (at least, not with `String` / UTF-8 … `UTF32String` in LegacyStrings.jl does this, of course).

To be clear, I’m not necessarily opposed to opaque string indices to catch bugs, though these introduce their own complications as noted above, and would be a breaking change (as return values from `findnext` etc). What I’m arguing against is the insistence that first-class string support requires a type with O(1) codepoint indexing, which so far is still apparently lacking any common practical application, and is not supported in the default string type of any mainstream language other than Python3.

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [February 5, 2024, 8:20pm UTC](https://discourse.julialang.org/t/stringindex-idea-julia-2-0/107185/48 "2024-02-05T20:20:27Z")

</div>

I think the problem here might be that `String` is index-able at all. I think we would likely see fewer complaints if we instead had `nth_char(::AbstractString, ::Integer)` and `nth_byte(::AbstractString, Integer)`, and `getindex(::AbstractString, Integer)` errored with a helpful error message.

---

<div class="post-metadata">

**Author:** ![mnemnion](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mnemnion/32/206596_2.png) [@mnemnion](https://discourse.julialang.org/u/mnemnion)\
**Post date:** [February 5, 2024, 9:12pm UTC](https://discourse.julialang.org/t/stringindex-idea-julia-2-0/107185/49 "2024-02-05T21:12:38Z")

</div>

I would expect a lot of complaints if you were unable to index a string, like you can in every single other programming language I’ve ever used.

The issue is that there’s more than one sensible way to index into a string, and a String type has to support one of those, rather than several.

There’s an obvious analogy here to reshaping a multidimensional Array as a Vector. A `GraphemeString` (or `GraphemeView`, not sure which emphasis is better) would reshape a String as graphemes. That’s basically the whole idea. A `GraphemeIndex` is its dual, an index into (any) string in terms of its graphemes.

---

<div class="post-metadata">

**Author:** ![goerz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/goerz/32/3269_2.png) [@goerz](https://discourse.julialang.org/u/goerz)\
**Post date:** [February 5, 2024, 10:03pm UTC](https://discourse.julialang.org/t/stringindex-idea-julia-2-0/107185/50 "2024-02-05T22:03:55Z")

</div>

I don’t think anyone would object to a `StringViews` package with `CodepointsView`, `GraphemesView`, and `TermColumnsView`, each with their own indexing and iteration, and each defining `v[i]` as the appropriate `SubString` of the `String` they wrap.

That sounds like a super useful thing!

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [February 5, 2024, 10:06pm UTC](https://discourse.julialang.org/t/stringindex-idea-julia-2-0/107185/51 "2024-02-05T22:06:57Z")

</div>

> [@goerz](#):
>
> I don’t think anyone would object to a `StringViews` package

There is already a StringViews.jl package for a totally different purpose. But definitely someone would be welcome write a package that provide alternate indexing views of strings, if they feel so inclined.

> [@goerz](#):
>
> That sounds like a super useful thing!

(So far, no one has pointed to an actual practical use?)

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [February 5, 2024, 10:13pm UTC](https://discourse.julialang.org/t/stringindex-idea-julia-2-0/107185/52 "2024-02-05T22:13:17Z")

</div>

> [@mnemnion](#):
>
> I would expect a lot of complaints if you were unable to index a string, like you can in every single other programming language I’ve ever used.

Julia allows you to index into a string, too. It provides n-th codeunit for an arbitrary n in O(1), and allows you to save the location of arbitrary characters and substrings (e.g. during searching/iteration) for later O(1) access.

What it doesn’t provide is O(1) (or similar) access to the n-th codepoint. But neither does C, C++, Go, Swift, Rust, Java, Javascript, C#, or any other mainstream language other than Python3 AFAIK. Mostly they provide random access into codeunits (either UTF-8 or UTF-16) in their default string types, not Unicode characters. So I’m not sure what programming languages you’re referring to.

---

<div class="post-metadata">

**Author:** ![mnemnion](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mnemnion/32/206596_2.png) [@mnemnion](https://discourse.julialang.org/u/mnemnion)\
**Post date:** [February 5, 2024, 10:16pm UTC](https://discourse.julialang.org/t/stringindex-idea-julia-2-0/107185/53 "2024-02-05T22:16:29Z")

</div>

> [@stevengj](#):
>
> Julia allows you to index into a string, too.

I quoted the entire post I was replying to, which is above the one you’re objecting to, causing discourse to remove the quote, which may have something to do with the severe confusion you’re exhibiting here.

Specifically, there was a proposal (perhaps a joke?) to disalllow `getindex` on strings entirely.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [February 5, 2024, 10:23pm UTC](https://discourse.julialang.org/t/stringindex-idea-julia-2-0/107185/54 "2024-02-05T22:23:07Z")

</div>

> [@mnemnion](#):
>
> Specifically, there was a proposal (perhaps a joke?) to disalllow `getindex` on strings entirely.

@Oscar_Smith suggested `getindex(::AbstractString, Integer)`, not `getindex` entirely — I suspect that he was referring to the proposal to support `getindex` only for opaque `StringIndex` types (which was the genesis of this whole thread). I agree that we should have a `getindex` of some kind.

---

<div class="post-metadata">

**Author:** ![mnemnion](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mnemnion/32/206596_2.png) [@mnemnion](https://discourse.julialang.org/u/mnemnion)\
**Post date:** [February 5, 2024, 10:23pm UTC](https://discourse.julialang.org/t/stringindex-idea-julia-2-0/107185/55 "2024-02-05T22:23:56Z")

</div>

And I was saying that, in every programming language I’ve ever worked with, you can index a string with an integer.

Do you know of exceptions?

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [February 5, 2024, 10:27pm UTC](https://discourse.julialang.org/t/stringindex-idea-julia-2-0/107185/56 "2024-02-05T22:27:54Z")

</div>

> [@mnemnion](#):
>
> And I was saying that, in every programming language I’ve ever worked with, you can index a string with an integer. Do you know of exceptions?

Swift has [`String.Index`](https://developer.apple.com/documentation/swift/string/index). [Rust gives an error](https://web.mit.edu/rust-lang_v1.25/arch/amd64_ubuntu1404/share/doc/rust/html/book/second-edition/ch08-02-strings.html) if you try to index a string directly (it allows integer indices for string _slices_, but very similar to Julia these are byte indices and will cause a runtime error if they don’t fall at character boundaries). ~~It looks like [Kotlin requires `string.get(n)` rather than supporting `string[n]`](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin/-string/)~~ Kotlin `s[n]` is a synonym for `s.get(n)`, apparently. There are probably other examples, but likely only from relatively new languages where UTF-8 is fully entrenched. (Many other languages support consecutive integer indices which return codeunits rather than codepoints, of course.)

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [February 5, 2024, 10:36pm UTC](https://discourse.julialang.org/t/stringindex-idea-julia-2-0/107185/57 "2024-02-05T22:36:09Z")

</div>

No I was referring to getting rid of getting rid of `getindex` on Strings entirely and replacing it with a function lookup (since there are multiple possible things `"hi"[1]` could mean).

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [February 5, 2024, 10:37pm UTC](https://discourse.julialang.org/t/stringindex-idea-julia-2-0/107185/58 "2024-02-05T22:37:56Z")

</div>

> [@Oscar\_Smith](#):
>
> No I was referring to getting rid of getting rid of `getindex` on Strings entirely and replacing it with a function lookup (since there are multiple possible things `"hi"[1]` could mean).

That seems too breaking even for a hypothetical Julia 2.0.

What’s the problem with `"hi"[i]` where `i` is an opaque type returned by, e.g. `i = findfirst(somepredicate, "hi")`?

---

<div class="post-metadata">

**Author:** ![mnemnion](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mnemnion/32/206596_2.png) [@mnemnion](https://discourse.julialang.org/u/mnemnion)\
**Post date:** [February 5, 2024, 10:41pm UTC](https://discourse.julialang.org/t/stringindex-idea-julia-2-0/107185/59 "2024-02-05T22:41:20Z")

</div>

> [@Oscar\_Smith](#):
>
> No I was referring to getting rid of getting rid of `getindex` on Strings entirely

Julia is a language where you can do this:

```julia
julia> i = 1
1

julia> i[1]
1

```

Disallowing indexing on Strings would be a very weird thing to do.

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [February 5, 2024, 10:58pm UTC](https://discourse.julialang.org/t/stringindex-idea-julia-2-0/107185/60 "2024-02-05T22:58:00Z")

</div>

The downside of opaque type indexing is that you then have to figure out what the promotion rules are (i.e. can you add 1?)

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [February 5, 2024, 10:58pm UTC](https://discourse.julialang.org/t/stringindex-idea-julia-2-0/107185/61 "2024-02-05T22:58:56Z")

</div>

IMO, the fact that Number behaves like a 0 dimensional collection sometimes was a pretty big mistake that was only not removed because by 0.7 too many things used it.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [February 6, 2024, 12:25am UTC](https://discourse.julialang.org/t/stringindex-idea-julia-2-0/107185/62 "2024-02-06T00:25:06Z")

</div>

> [@Oscar\_Smith](#):
>
> The downside of opaque type indexing is that you then have to figure out what the promotion rules are (i.e. can you add 1?)

See the beginning of this thread and the linked issue for discussion of precisely that.

---

<div class="post-metadata">

**Author:** ![uniment](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/uniment/32/24532_2.png) [@uniment](https://discourse.julialang.org/u/uniment)\
**Post date:** [February 8, 2024, 9:57pm UTC](https://discourse.julialang.org/t/stringindex-idea-julia-2-0/107185/63 "2024-02-08T21:57:15Z")

</div>

This idea can be made more ergonomic by merging it with another idea discussed [previously](https://discourse.julialang.org/t/ordinal-indexing-as-a-language-feature/91970):

Something similar to @jishnub’s [`OrdinalIndexing`](https://github.com/jishnub/OrdinalIndexing.jl/) could be added to the language—this is almost identical to this thread’s OP, but with a nicer user interface that can generalize to other indexable types. The purpose of `OrdinalIndexing` is to guarantee 1-based indexing even for `OffsetArray`s. For example:

```julia
my_array = OffsetArray(1:10, -10)
my_array[4th] == 4

```

Applying this concept here, per-character string indexing could go like:

```julia
my_string[6th:10th]

```

(This is reminiscent of the `my_string.nth(i)` function in Rust)

The `4th` object (let’s call it `OrdinalIndex(4)`) would store a single integer representing an _ordinal_ index instead of a _positional_ index. Arithmetic and ranges of `OrdinalIndex` would work as intuitively expected.

When an `OrdinalIndex` is added to an `Integer`, it’s promoted to a `PositionOrdinalIndex`, which could contain both `position_offset` and `ordinal_index` fields like the OP of this thread. For example, `0+6th` would construct a `PositionOrdinalIndex(0,6)` and `6:7th` could be `PositionOrdinalIndex(6,0):PositionOrdinalIndex(0,7)`. (for 6th to 7th, use `(6:7)th` or `6th:7th`). Adding `PositionOrdinalIndex` objects would add both fields, similar to a `Complex`.

Then, indexing from the end should work like this:

```julia
my_string[end-9th:end-5th]

```

It would also be really awesome for ordered dictionaries, wherein it could be imposed that `::OrdinalIndex` and `::PositionOrdinalIndex` hashkeys are not permitted, so that the dictionary entries can be accessed in insertion order by `::OrdinalIndex` and `::PositionOrdinalIndex`-typed indices:

```julia
my_ordered_dict[1th:100th] # first 100 entries

```

Maybe it could also be interesting for accessing sparse matrices—I’m not sure.

I’m agnostic on names btw, and as @StefanKarpinski ponders above it might not be worthwhile to maintain a `OrdinalIndex` type separate from `PositionOrdinalIndex` since the position will often be const-propped anyway: using only the latter would make the implementation of this idea analogous to the representation of complex numbers (wherein all imaginary numbers are represented as `Complex` even with zero real component; similar to `im ≡ Complex(false,true)` we would define `const th = PositionOrdinalIndex(false,true)`).

[Previous page](https://discourse.julialang.org/t/stringindex-idea-julia-2-0/107185.md?page=2)

[Next page](https://discourse.julialang.org/t/stringindex-idea-julia-2-0/107185.md?page=4)