# StringDocument() of the TextAnalysis package

**URL:** <https://discourse.julialang.org/t/stringdocument-of-the-textanalysis-package/62589>\
**Category:** New to Julia\
**Tags:** question, strings\
**Created:** [June 8, 2021, 4:56pm UTC](https://discourse.julialang.org/t/stringdocument-of-the-textanalysis-package/62589 "2021-06-08T16:56:13Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Alex\_Tantos](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/alex_tantos/32/10636_2.png) [@Alex\_Tantos](https://discourse.julialang.org/u/Alex_Tantos)\
**Post date:** [June 8, 2021, 4:56pm UTC](https://discourse.julialang.org/t/stringdocument-of-the-textanalysis-package/62589/1 "2021-06-08T16:56:14Z")

</div>

Hi there!

Has anybody been using the StringDocument() function of the TextAnalysis package for unicode encoded language strings?  
However, if I use StringDocument() with a readin (greek) string from a file with the read() function, everything works:

> mydata = open(“/Users/atantos/Desktop/test.txt”) do file  
> read(file, String)  
> end

> StringDocument(mydata) # works fine

Here is the code line with the error message:

> StringDocument(“είμαι οκ”)  
> Error showing value of type StringDocument{String}:  
> ERROR: StringIndexError: invalid index [8], valid nearby indices [7]=\>‘α’, [9]=\>‘ι’  
> Stacktrace:  
> [1] string\_index\_err(s::String, i::Int64)  
> @ Base ./strings/string.jl:12  
> [2] getindex  
> @ ./strings/string.jl:263 [inlined]  
> [3] summary(d::StringDocument{String})  
> @ TextAnalysis ~/.julia/packages/TextAnalysis/B0QxG/src/show.jl:16  
> [4] show(io::IOContext{Base.TTY}, #unused#::MIME{Symbol(“text/plain”)}, d::StringDocument{String})  
> @ TextAnalysis ~/.julia/packages/TextAnalysis/B0QxG/src/show.jl:45  
> [5] (::REPL.var"#38#39"{REPL.REPLDisplay{REPL.LineEditREPL}, MIME{Symbol(“text/plain”)}, Base.RefValue{Any}})(io::Any)  
> @ REPL /Users/julia/buildbot/worker/package\_macos64/build/usr/share/julia/stdlib/v1.6/REPL/src/REPL.jl:220  
> [6] with\_repl\_linfo(f::Any, repl::REPL.LineEditREPL)  
> @ REPL /Users/julia/buildbot/worker/package\_macos64/build/usr/share/julia/stdlib/v1.6/REPL/src/REPL.jl:462  
> [7] display(d::REPL.REPLDisplay, mime::MIME{Symbol(“text/plain”)}, x::Any)  
> @ REPL /Users/julia/buildbot/worker/package\_macos64/build/usr/share/julia/stdlib/v1.6/REPL/src/REPL.jl:213  
> [8] display(d::REPL.REPLDisplay, x::Any)  
> @ REPL /Users/julia/buildbot/worker/package\_macos64/build/usr/share/julia/stdlib/v1.6/REPL/src/REPL.jl:225  
> [9] display(x::Any)  
> @ Base.Multimedia ./multimedia.jl:328  
> [10] #invokelatest#2  
> @ ./essentials.jl:708 [inlined]  
> [11] invokelatest  
> @ ./essentials.jl:706 [inlined]  
> [12] print\_response(errio::IO, response::Any, show\_value::Bool, have\_color::Bool, specialdisplay::Union{Nothing, AbstractDisplay})  
> @ REPL /Users/julia/buildbot/worker/package\_macos64/build/usr/share/julia/stdlib/v1.6/REPL/src/REPL.jl:247  
> [13] (::REPL.var"#40#41"{REPL.LineEditREPL, Pair{Any, Bool}, Bool, Bool})(io::Any)  
> @ REPL /Users/julia/buildbot/worker/package\_macos64/build/usr/share/julia/stdlib/v1.6/REPL/src/REPL.jl:231  
> [14] with\_repl\_linfo(f::Any, repl::REPL.LineEditREPL)  
> @ REPL /Users/julia/buildbot/worker/package\_macos64/build/usr/share/julia/stdlib/v1.6/REPL/src/REPL.jl:462  
> [15] print\_response(repl::REPL.AbstractREPL, response::Any, show\_value::Bool, have\_color::Bool)  
> @ REPL /Users/julia/buildbot/worker/package\_macos64/build/usr/share/julia/stdlib/v1.6/REPL/src/REPL.jl:229  
> [16] (::REPL.var"#do\_respond#61"{Bool, Bool, REPL.var"#72#82"{REPL.LineEditREPL, REPL.REPLHistoryProvider}, REPL.LineEditREPL, REPL.LineEdit.Prompt})(s::REPL.LineEdit.MIState, buf::Any, ok::Bool)  
> @ REPL /Users/julia/buildbot/worker/package\_macos64/build/usr/share/julia/stdlib/v1.6/REPL/src/REPL.jl:798  
> [17] #invokelatest#2  
> @ ./essentials.jl:708 [inlined]  
> [18] invokelatest  
> @ ./essentials.jl:706 [inlined]  
> [19] run\_interface(terminal::REPL.Terminals.TextTerminal, m::REPL.LineEdit.ModalInterface, s::REPL.LineEdit.MIState)  
> @ REPL.LineEdit /Users/julia/buildbot/worker/package\_macos64/build/usr/share/julia/stdlib/v1.6/REPL/src/LineEdit.jl:2441  
> [20] run\_frontend(repl::REPL.LineEditREPL, backend::REPL.REPLBackendRef)  
> @ REPL /Users/julia/buildbot/worker/package\_macos64/build/usr/share/julia/stdlib/v1.6/REPL/src/REPL.jl:1126  
> [21] (::REPL.var"#44#49"{REPL.LineEditREPL, REPL.REPLBackendRef})()  
> @ REPL ./task.jl:411

---

<div class="post-metadata">

**Author:** ![ericphanson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ericphanson/32/215186_2.png) [@ericphanson](https://discourse.julialang.org/u/ericphanson)\
**Post date:** [June 8, 2021, 7:39pm UTC](https://discourse.julialang.org/t/stringdocument-of-the-textanalysis-package/62589/2 "2021-06-08T19:39:22Z")

</div>

It looks like that’s occuring in the `show` method, i.e. when it’s trying to print a summary. I filed [fix string indexing in `summary` by ericphanson · Pull Request #257 · JuliaText/TextAnalysis.jl · GitHub](https://github.com/JuliaText/TextAnalysis.jl/pull/257) to try to help fix it.

---

<div class="post-metadata">

**Author:** ![pitsianis](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pitsianis/32/26588_2.png) [@pitsianis](https://discourse.julialang.org/u/pitsianis)\
**Post date:** [October 24, 2021, 11:07am UTC](https://discourse.julialang.org/t/stringdocument-of-the-textanalysis-package/62589/3 "2021-10-24T11:07:30Z")

</div>

@ericphanson

```julia
using TextAnalysis
StringDocument(“είμαι οκ”)

```

still not working. I suspect it has to do with the fact that the string encoding is multi-byte. Function `nextind()` has to be used to move to the correct character starting byte position and not land in-between.

---

<div class="post-metadata">

**Author:** ![ericphanson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ericphanson/32/215186_2.png) [@ericphanson](https://discourse.julialang.org/u/ericphanson)\
**Post date:** [October 24, 2021, 5:52pm UTC](https://discourse.julialang.org/t/stringdocument-of-the-textanalysis-package/62589/4 "2021-10-24T17:52:50Z")

</div>

It looks like a new release wasn’t tagged since my fix was merged, cc @avik. (It’s also possible there are other unrelated Unicode-handling bugs.)