# Sections of strings or arrays and concrete types

**URL:** <https://discourse.julialang.org/t/sections-of-strings-or-arrays-and-concrete-types/53041>\
**Category:** Performance\
**Created:** [January 8, 2021, 1:37pm UTC](https://discourse.julialang.org/t/sections-of-strings-or-arrays-and-concrete-types/53041 "2021-01-08T13:37:13Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![tlienart](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tlienart/32/7640_2.png) [@tlienart](https://discourse.julialang.org/u/tlienart)\
**Post date:** [January 8, 2021, 1:37pm UTC](https://discourse.julialang.org/t/sections-of-strings-or-arrays-and-concrete-types/53041/1 "2021-01-08T13:37:13Z")

</div>

I’m starting to wade into the wonderful world of SnoopCompile and the many insights it gives (and realising along the way that my understanding of types, inference etc, is shallow to say the least). One thing I got from reading the docs (and getting excellent comments by @tim.holy) was that it’s better, where possible, to use concrete types to reduce the time to first call.

In my code, I often have to repeatedly work on sections of arrays: either a subset of a String or a subset of a Vector say to simplify a `Vector{Int}`. Intuitively, it seemed to me to be a good idea to not create new objects for these, so I often basically do:

```julia
subset = SubString(original_string, range)
do_thing(subset)

```

and the same with `@view` of vectors. This is often in a recursive function so the signature of `do_thing` in my code is typically something like:

```julia
do_thing(obj::Union{SubString{String}, String})

```

or

```julia
do_thing(obj::AbstractVector{Int})

```

but these are not concrete types and from a discussion with @tim.holy ; I believe that this is not ideal.

My question is then, am I better off making the original objects into a sub-version of themselves before processing them? i.e.:

```julia
processed_original_string = SubString(original_string)
do_thing(processed_original_string)

```

and

```julia
processed_original_array = @view original_array[eachindex(original_array)]
do_thing(processed_original_array)

```

?

A subsidiary question is about returned types, does concreteness matter here as well? For instance let’s say I have a function which returns a vector with objects of type either `A` or `B` or both, does something like:

```julia
return_vector(args)::Vector{Union{A,B}}

```

cause issues? Thanks a lot!

---

<div class="post-metadata">

**Author:** ![tim.holy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tim.holy/32/52_2.png) [@tim.holy](https://discourse.julialang.org/u/tim.holy)\
**Post date:** [January 8, 2021, 1:55pm UTC](https://discourse.julialang.org/t/sections-of-strings-or-arrays-and-concrete-types/53041/2 "2021-01-08T13:55:02Z")

</div>

A lot of it comes down to the tradeoff among costs. As you say, creating a `String` from a `SubString{String}` does have a cost:

```julia
julia> str = SubString("hello", 2:4)
"ell"

julia> using BenchmarkTools

julia> @btime String($str)
  11.030 ns (1 allocation: 32 bytes)
"ell"

```

If that’s in the inner loop of your program, you might come to hate that 11ns—you’d be better off if you didn’t need to convert.

In contrast, suppose you have

```julia
function somebigfunction(filename::AbstractString)
    # this takes 300ms to compile and 1ms to execute
end

```

In terms of runtime performance, if `somebigfunction` takes 1ms to execute, you’ll never notice the 11ns it would take to convert the `filename` to a String. So from a runtime performance standpoint, it’s completely, utterly irrelevant whether you convert to a `String` before calling it. But you might notice paying 300ms each time you compile it for:

- `filename::String`
- `filename::SubString{String}`
- `filename::AbstractString` (for those cases where you’re calling this with poor inferrability of `filename`)

If you have to compile it for all 3 types, you might have to wait a full second, whereas it’s only a third of a second if you could get away with compiling it only once.

A way to standardize on `filename::String` and ensure that you never pay that 300ms for any other type is to define your methods like this:

```julia
function somebigfunction(filename::String)
    # this takes 300ms to compile and 1ms to execute
end
somebigfunction(filename::AbstractString) = somebigfunction(String(filename)::String)

```

Now, you can’t always do this: part of Julia’s power is the ability to write generic functions that can handle “anything.” But you don’t always need that power, and that power comes with the cost of pretty bad latencies. Fortunately, Julia is flexible enough that you don’t have to use that power if you don’t want to.

As is hopefully clear, these are not black vs white issues: it requires understanding where the costs of a package are coming from, and you might do one thing in one circumstance and the exact opposite in a different circumstance. It comes down to understanding the tradeoffs in each case.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [January 8, 2021, 2:01pm UTC](https://discourse.julialang.org/t/sections-of-strings-or-arrays-and-concrete-types/53041/3 "2021-01-08T14:01:12Z")

</div>

> [@tlienart](#):
>
> This is often in a recursive function so the signature of `do_thing` in my code is typically something like:
> 
> ```julia
> do_thing(obj::Union{SubString{String}, String})
> 
> ```
> 
> or
> 
> ```julia
> do_thing(obj::AbstractVector{Int})
> 
> ```
> 
> but these are not concrete types and from a discussion with @tim.holy ; I believe that this is not ideal.

There is **no runtime** performance problem with using abstract types in function signatures. You don’t even need to declare a type at all — `do_thing(obj) = ...` is perfectly fine with [rare exceptions](https://docs.julialang.org/en/v1/manual/performance-tips/#Be-aware-of-when-Julia-avoids-specializing) for `Type` and `Function` arguments.

The compiler will generate specialized versions of your functions based on the concrete types inferred _wherever it is called_. For example, if you define `f(x)=x+1` and then write `f(3) + f(2.5)`, the compiler will generate (and inline, in this case) specialized versions of `f` for `Int` and `Float64` overhead.

The only reasons to declare argument types are usually dispatch (to have different methods of the function for different types), correctness (to prevent an invalid argument from being passed by accident), or clarity (to indicate to the caller what type is expected). You should generally make your argument declarations as **abstract as possible** for correctness, so that your code is more generic/extensible.

e.g. use `obj::AbstractString` or `obj::AbstractVector{<:Integer}` rather than `Union{SubString{String}, String}` or `obj::AbstractVector{Int}`.

- This is an entirely separate from the question what argument types you should _pass_, e.g. passing a view vs. making a copy.

- As @timholy points out, _compile-time_ performance may be worse if you have to compile more specializations of a function. But that’s more about what argument types you _pass_ than what you _declare_ in the function definition.

---

<div class="post-metadata">

**Author:** ![tim.holy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tim.holy/32/52_2.png) [@tim.holy](https://discourse.julialang.org/u/tim.holy)\
**Post date:** [January 8, 2021, 2:07pm UTC](https://discourse.julialang.org/t/sections-of-strings-or-arrays-and-concrete-types/53041/4 "2021-01-08T14:07:14Z")

</div>

All that’s true, but anyone playing with SnoopCompile has a goal to reduce latency. In such cases, there are reasons to limit signatures. This is just a glorified version of [this style guide tip](https://docs.julialang.org/en/v1/manual/style-guide/#Handle-excess-argument-diversity-in-the-caller) and one of many tricks for reducing latency.

When you _need_ Julia’s specialization, the last thing you want to do is get in its way. But when you don’t need the specialization (e.g., there’s nothing in `somebigfunction` that cares one whit about how `filename` is actually represented), then there’s every reason to try to eliminate unhelpful and latency-costly type-diversity.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [January 8, 2021, 2:10pm UTC](https://discourse.julialang.org/t/sections-of-strings-or-arrays-and-concrete-types/53041/5 "2021-01-08T14:10:50Z")

</div>

> [@tim.holy](#):
>
> All that’s true, but anyone playing with SnoopCompile has a goal to reduce latency. In such cases, there are reasons to limit signatures. This is just a glorified version of [this style guide tip](https://docs.julialang.org/en/v1/manual/style-guide/#Handle-excess-argument-diversity-in-the-caller) and one of many tricks for reducing latency.

But as you point out in the style guide, this is a function of what arguments you pass, not of what arguments you declare in the method definition, though narrowing the definition temporarily could help you catch such call sites.

I generally consider overly narrow _declarations_ to be an anti-pattern in Julia that most new Julia programmers fall prey to instinctively; writing generic code is hard but extremely useful.

---

<div class="post-metadata">

**Author:** ![tlienart](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tlienart/32/7640_2.png) [@tlienart](https://discourse.julialang.org/u/tlienart)\
**Post date:** [January 8, 2021, 2:16pm UTC](https://discourse.julialang.org/t/sections-of-strings-or-arrays-and-concrete-types/53041/6 "2021-01-08T14:16:21Z")

</div>

Thanks both, I think in my context the answer is pretty clear in that the functions I’m talking about are not exposed to the user (“backend functions”) and are exactly of the type that Tim Holy describes. So in that respect I could “afford” to make them pretty specific if it helps fight noticeable latency.

For a package providing functions that I could expect users to extend, then I think @stevengj 's advice holds and it’s not a good idea to over-specialise.

PS: if I understood both your answers correctly, specifying the return type has no impact on inference latency?

---

<div class="post-metadata">

**Author:** ![aviatesk](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aviatesk/32/7610_2.png) [@aviatesk](https://discourse.julialang.org/u/aviatesk)\
**Post date:** [January 8, 2021, 2:21pm UTC](https://discourse.julialang.org/t/sections-of-strings-or-arrays-and-concrete-types/53041/7 "2021-01-08T14:21:53Z")

</div>

> [@tlienart](#):
>
> PS: if I understood both your answers correctly, specifying the return type has no impact on inference latency?

The return type declaration of a function wouldn’t affect its compilation latency, but can affect the compilation of its _callers_. If your function’s return value is used within a callee but its type is hard to infer, then declaring it would help the compiler a lot. We can’t tell whether it’s best to do that without its usage context.

---

<div class="post-metadata">

**Author:** ![tim.holy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tim.holy/32/52_2.png) [@tim.holy](https://discourse.julialang.org/u/tim.holy)\
**Post date:** [January 8, 2021, 2:35pm UTC](https://discourse.julialang.org/t/sections-of-strings-or-arrays-and-concrete-types/53041/8 "2021-01-08T14:35:15Z")

</div>

> [@stevengj](#):
>
> I generally consider overly narrow _declarations_ to be an anti-pattern in Julia that most new Julia programmers fall prey to instinctively; writing generic code is hard but extremely useful.

Learning the virtues of generic code should indeed be the first step. But once that lesson is absorbed, ambitious developers can go on to understand how their design choices affect latency. Good tools for identifying the source of latency have been a long time coming, but now that we’re mostly there we should encourage experimentation and innovation in this space, just as we coach people through how to improve the runtime performance of their code. For better or for worse, declaring argument types is sometimes an effective way to reduce latency.

---

<div class="post-metadata">

**Author:** ![Henrique\_Becker](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/henrique_becker/32/15443_2.png) [@Henrique\_Becker](https://discourse.julialang.org/u/Henrique_Becker)\
**Post date:** [January 8, 2021, 6:16pm UTC](https://discourse.julialang.org/t/sections-of-strings-or-arrays-and-concrete-types/53041/9 "2021-01-08T18:16:30Z")

</div>

Sorry if I missed something, but what about:

```julia
function somebigfunction(@nospecialize filename_ :: AbstractString)
    filename = String(filename_)::String
    # ... rest of your code here
end

```

as a compromise?

---

<div class="post-metadata">

**Author:** ![tim.holy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tim.holy/32/52_2.png) [@tim.holy](https://discourse.julialang.org/u/tim.holy)\
**Post date:** [January 8, 2021, 8:42pm UTC](https://discourse.julialang.org/t/sections-of-strings-or-arrays-and-concrete-types/53041/10 "2021-01-08T20:42:12Z")

</div>

That also reduces compile-cost, but there are tradeoffs:

- `@nospecialize` doesn’t prevent specialization in inference, only codegen (the logic being that it’s useful for callers who know the type concretely to know what kind of type will be returned)
- if you trigger non-inferrability in the call graph from `somebigfunction`, it might hurt runtime performance or make you more vulnerable to invalidation.

From the standpoint of latency the ideal situation, when you can achieve it, is concrete inference with low diversity of argument types.

Again, all these considerations are for when it doesn’t cost you in some other way that matters more to you. If you need Julia’s specialization wizardry, by all means one should embrace it.