# Way to make SharedArray over fixed length Strings?

**URL:** <https://discourse.julialang.org/t/way-to-make-sharedarray-over-fixed-length-strings/7082>\
**Category:** General Usage\
**Tags:** question\
**Created:** [November 15, 2017, 4:29am UTC](https://discourse.julialang.org/t/way-to-make-sharedarray-over-fixed-length-strings/7082 "2017-11-15T04:29:24Z")\
**Posts on this page:** 16\
**Page:** 1

<div class="post-metadata">

**Author:** ![djsegal](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/djsegal/32/13752_2.png) [@djsegal](https://discourse.julialang.org/u/djsegal)\
**Post date:** [November 15, 2017, 4:29am UTC](https://discourse.julialang.org/t/way-to-make-sharedarray-over-fixed-length-strings/7082/1 "2017-11-15T04:29:24Z")

</div>

if you do `SharedArray{AbstractString}(10, 1)`

you get the following error:

`ERROR: ArgumentError: type of SharedArray elements must be bits types, got AbstractString`

Is there a package for making fixed length strings to allow their use in SharedArrays?

Thanks!

* * *

edit: is there a way to make it work with [LegacyStrings.jl](https://github.com/JuliaArchive/LegacyStrings.jl)?

---

<div class="post-metadata">

**Author:** ![rdeits](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rdeits/32/286_2.png) [@rdeits](https://discourse.julialang.org/u/rdeits)\
**Post date:** [November 15, 2017, 6:31am UTC](https://discourse.julialang.org/t/way-to-make-sharedarray-over-fixed-length-strings/7082/2 "2017-11-15T06:31:52Z")

</div>

What about an `SVector{N, Char}` from StaticArrays.jl?

```julia
julia> using StaticArrays

julia> isbits(SVector{10, Char})
true

```

---

<div class="post-metadata">

**Author:** ![djsegal](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/djsegal/32/13752_2.png) [@djsegal](https://discourse.julialang.org/u/djsegal)\
**Post date:** [November 15, 2017, 7:11am UTC](https://discourse.julialang.org/t/way-to-make-sharedarray-over-fixed-length-strings/7082/3 "2017-11-15T07:11:54Z")

</div>

How do you treat that as a string though?

And is there a way to do build in `rpad()`'ing?

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [November 15, 2017, 7:38am UTC](https://discourse.julialang.org/t/way-to-make-sharedarray-over-fixed-length-strings/7082/4 "2017-11-15T07:38:12Z")

</div>

```julia
julia> s = SVector{4,Char}(b"abcd")
4-element SVector{4,Char}:
 'a'
 'b'
 'c'
 'd'

julia> convert(String, s)
"abcd"

```

As for padding, just fill the remaining characters with spaces?

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [November 15, 2017, 1:43pm UTC](https://discourse.julialang.org/t/way-to-make-sharedarray-over-fixed-length-strings/7082/5 "2017-11-15T13:43:03Z")

</div>

Might be nice to have a package implementing a fixed-length `SString{n,T} <: AbstractString` type on top of `SVector{n,T}` (supporting both `T = Char` for UTF-32 and `T = UInt8` for ASCII). If you don’t need to support strings containing NUL, then you could also use NUL-padding to give a string of length `≤ n` with fixed-length storage.

---

<div class="post-metadata">

**Author:** ![djsegal](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/djsegal/32/13752_2.png) [@djsegal](https://discourse.julialang.org/u/djsegal)\
**Post date:** [November 16, 2017, 7:21am UTC](https://discourse.julialang.org/t/way-to-make-sharedarray-over-fixed-length-strings/7082/6 "2017-11-16T07:21:23Z")

</div>

Can you guys take a look at this package I wrote up?

[https://github.com/djsegal/FixedLengthStrings.jl](https://github.com/djsegal/FixedLengthStrings.jl)

It’s probably more of a proof of concept than an efficient design at this point.

// I basically just started with @stevengj’s [LatexStrings.jl](https://github.com/stevengj/LaTeXStrings.jl) and worked my way backward

---

<div class="post-metadata">

**Author:** ![sdanisch](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sdanisch/32/1406_2.png) [@sdanisch](https://discourse.julialang.org/u/sdanisch)\
**Post date:** [November 16, 2017, 5:31pm UTC](https://discourse.julialang.org/t/way-to-make-sharedarray-over-fixed-length-strings/7082/7 "2017-11-16T17:31:02Z")

</div>

There is also: [https://github.com/JuliaComputing/FixedSizeStrings.jl](https://github.com/JuliaComputing/FixedSizeStrings.jl)

---

<div class="post-metadata">

**Author:** ![djsegal](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/djsegal/32/13752_2.png) [@djsegal](https://discourse.julialang.org/u/djsegal)\
**Post date:** [November 16, 2017, 10:25pm UTC](https://discourse.julialang.org/t/way-to-make-sharedarray-over-fixed-length-strings/7082/8 "2017-11-16T22:25:49Z")

</div>

Well damn… Can anyone discuss the pros & cons of the two implementations?

I’m cool with letting the package I wrote die, I just want to know it’s for the right reasons.

---

<div class="post-metadata">

**Author:** ![sdanisch](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sdanisch/32/1406_2.png) [@sdanisch](https://discourse.julialang.org/u/sdanisch)\
**Post date:** [November 16, 2017, 10:54pm UTC](https://discourse.julialang.org/t/way-to-make-sharedarray-over-fixed-length-strings/7082/9 "2017-11-16T22:54:48Z")

</div>

Well, yours doesn’t really seem to be optimized for performance since you don’t properly propagate the fixed size.  
Also, this is a pretty crazy use of parse and eval:

[https://github.com/djsegal/FixedLengthStrings.jl/blob/663bdea5d05bf88988c99013461f8497cc0846f9/src/FixedLengthStrings.jl#L25](https://github.com/djsegal/FixedLengthStrings.jl/blob/663bdea5d05bf88988c99013461f8497cc0846f9/src/FixedLengthStrings.jl#L25)

You do realize, that you can just do `SVector{length(string), UInt8}(Vector{UInt8}(string))` ? 🙂

In general, I always found it useful to have a fixed size string, that looks more like this:

```julia
struct FString{N}
length::Int 
data::NTuple{N, UInt8}
end

```

So that one can have a `Vector{FString{32}}` with variable sized strings - which seems to be a shortcoming of FixedSizeStrings.

I made a prototype for such a type for my [GPU blog post](https://medium.com/@sdanisch/writing-extendable-and-hardware-agnostic-gpu-libraries-b21c145a8dad): [fixed\_strings.jl](https://gist.github.com/SimonDanisch/02e74622e0577f199c1b1a8a65390c24#file-fixedstring-jl)

All packages don’t support UTF8, so a package that would offer that would be nice.

In general, I recommend to just extend the existing package with more functionality - especially when both packages are still that small!

---

<div class="post-metadata">

**Author:** ![djsegal](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/djsegal/32/13752_2.png) [@djsegal](https://discourse.julialang.org/u/djsegal)\
**Post date:** [November 17, 2017, 12:48am UTC](https://discourse.julialang.org/t/way-to-make-sharedarray-over-fixed-length-strings/7082/10 "2017-11-17T00:48:20Z")

</div>

> [@sdanisch](#):
>
> In general, I recommend to just extend the existing package with more functionality - especially when both packages are still that small!

I wholeheartedly agree with this. There was just no way to find your code?

It seems like you only came forward because of Cunningham’s Law:

> “The best way to get the right answer on the Internet is not to ask a question, it’s to post the wrong answer.”

* * *

What do you think the best route is moving forward?

I just don’t really like writing pointless code and wasting time.

// and from [ANN: HigherPrecision](https://discourse.julialang.org/t/ann-higherprecision/6956) and [Pruning and quality control for the package ecosystem](https://discourse.julialang.org/t/pruning-and-quality-control-for-the-package-ecosystem/7021), it seems like this kind of thing is happening all the time to Julians ☹

---

<div class="post-metadata">

**Author:** ![sdanisch](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sdanisch/32/1406_2.png) [@sdanisch](https://discourse.julialang.org/u/sdanisch)\
**Post date:** [November 17, 2017, 1:50am UTC](https://discourse.julialang.org/t/way-to-make-sharedarray-over-fixed-length-strings/7082/11 "2017-11-17T01:50:34Z")

</div>

> There was just no way to find your code?

Sorry 😉  
I meant extending FixedSizeStrings, which isn’t my package. I should probably contribute my code to that package as well 🙂

---

<div class="post-metadata">

**Author:** ![sdanisch](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sdanisch/32/1406_2.png) [@sdanisch](https://discourse.julialang.org/u/sdanisch)\
**Post date:** [November 17, 2017, 2:05am UTC](https://discourse.julialang.org/t/way-to-make-sharedarray-over-fixed-length-strings/7082/12 "2017-11-17T02:05:02Z")

</div>

you could figure out whats missing for your use case in FixedSizeStrings and make a PR - feel free to ping me for guidance 🙂 i should also figure out what i could add…

---

<div class="post-metadata">

**Author:** ![djsegal](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/djsegal/32/13752_2.png) [@djsegal](https://discourse.julialang.org/u/djsegal)\
**Post date:** [September 1, 2018, 10:14pm UTC](https://discourse.julialang.org/t/way-to-make-sharedarray-over-fixed-length-strings/7082/13 "2018-09-01T22:14:36Z")

</div>

Is there a definitive solution to having SharedArray’s over fixed length strings?

Ran into a use case for symbols recently too

---

<div class="post-metadata">

**Author:** ![Luapulu](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/luapulu/32/17669_2.png) [@Luapulu](https://discourse.julialang.org/u/Luapulu)\
**Post date:** [November 22, 2020, 1:08pm UTC](https://discourse.julialang.org/t/way-to-make-sharedarray-over-fixed-length-strings/7082/14 "2020-11-22T13:08:34Z")

</div>

This may be of interest: [https://github.com/JuliaComputing/FixedSizeStrings.jl/pull/7](https://github.com/JuliaComputing/FixedSizeStrings.jl/pull/7)

---

<div class="post-metadata">

**Author:** ![oxinabox](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oxinabox/32/206603_2.png) [@oxinabox](https://discourse.julialang.org/u/oxinabox)\
**Post date:** [November 22, 2020, 1:33pm UTC](https://discourse.julialang.org/t/way-to-make-sharedarray-over-fixed-length-strings/7082/15 "2020-11-22T13:33:18Z")

</div>

ShortStrings.jl should work.  
If your strings are relatively short.  
It is super fast for short strings but becomes much slower the longer the string is. Would not use it it over 255 bytes at most  
[https://github.com/JuliaString/ShortStrings.jl](https://github.com/JuliaString/ShortStrings.jl)

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [November 22, 2020, 1:56pm UTC](https://discourse.julialang.org/t/way-to-make-sharedarray-over-fixed-length-strings/7082/16 "2020-11-22T13:56:20Z")

</div>

Note that if you want a fixed number of _bytes_, then you can use a [StringView](https://github.com/JuliaStrings/StringViews.jl) of an [SVector](https://github.com/JuliaArrays/StaticArrays.jl):

```julia
julia> using StringViews, StaticArrays

julia> s = StringView(SVector{4,UInt8}(b"abcd"))
"abcd"

```

It is probably more useful to have a fixed _upper bound_ on the number of bytes, in which case you can use a `StringView` of a `SubArray` of an `SVector`.

```julia
julia> s = StringView(@view SVector{8,UInt8}(b"abcd ")[1:4])
"abcd"

```

For example, given an array `a` of strings, you can convert it to an array of strings with fixed-length inline (`isbits`) storage via:

```julia
julia> function tofixedstr(s::String, nbytes)
           n = ncodeunits(s)
           StringView(@view SVector{nbytes}(codeunits(s * ' '^(nbytes-n)))[Base.OneTo(n)])
       end
tofixedstr (generic function with 3 methods)

julia> function tofixedstr(a::AbstractVector{String})
           npad = maximum(ncodeunits, a)
           tofixedstr.(a, npad)
       end
tofixedstr (generic function with 3 methods)

julia> a = ["foo", "blärg", "l♡ve"]
3-element Array{String,1}:
 "foo"
 "blärg"
 "l♡ve"

julia> b = tofixedstr(a)
3-element Array{StringView{SubArray{UInt8,1,SArray{Tuple{6},UInt8,1,6},Tuple{Base.OneTo{Int64}},true}},1}:
 "foo"
 "blärg"
 "l♡ve"

```

Note that this is a bits type, so all of the data is stored inline, as needed for `SharedArray`:

```julia
julia> isbits(b[1])
true

julia> println(reinterpret(UInt8, b))
UInt8[0x66, 0x6f, 0x6f, 0x20, 0x20, 0x20, 0x00, 0x00, 0x03, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x62, 0x6c, 0xc3, 0xa4, 0x72, 0x67, 0x00, 0x00, 0x06, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x6c, 0xe2, 0x99, 0xa1, 0x76, 0x65, 0x00, 0x00, 0x06, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00]

```

(Note that `0x20` is the `' '` padding byte in each underlying `SVector`.)

All the usual string operations should work, e.g.

```julia
julia> sort(b)
3-element Array{StringView{SubArray{UInt8,1,SArray{Tuple{6},UInt8,1,6},Tuple{Base.OneTo{Int64}},true}},1}:
 "blärg"
 "foo"
 "l♡ve"

```