Is there a way to test or detect whether a value (let’s say an Int) is constant (i.e. known to the compiler) at runtime? This has been asked before, but not often and not recently. I’m just wondering if this is now possible.
Why would I want this? Let’s say I have some code that operates on vectors. Sometimes the length of these vectors might be small, and through the power of constant propagation, constant at runtime. In this case it would (let’s assume) be advantageous, from a performance perspective, to use an SVector. However, in the case that the vector length is either large, or not constant at runtime, using a Vector for the computation would (let’s assume) be more efficient. In this scenario, I would want to branch to one of two different code paths depending on whether the length is constant at runtime (and small), rather than dynamic dispatch the creation of an SVector.
There may be a better way to do this, but @code_llvm can show compile-time constants. It’s just really hard to see it when the compiler rearranges a struct. Cheap calls with constant inputs are often computed at compile-time, but if necessary I compare @code_llvm to see if a call was replaced by a constant. Argument type parameters and constant global variables are guaranteed compile-time constants. Technically they were computed at runtime, but before the compile-time of the function that uses them.
It’s something I have wanted, but currently that is not available.
The closest you can get is to use Val and branch on the runtime value being small or large.
function g(x)
if x < 3
f(Val(x))
else
f(x)
end
end
f(::Val{x}) where x = # do something for small
f(x) = # do fallback
It might be feasible to add a query like: is_known_constant but that’s a bit iffy and we don’t like to leak those kinds of compiler implementation details.
Technically g would be dynamically dispatching f(Val(x)) because x is not a constant when compiled. However, x is a compile-time constant for f(::Val{x}) where x, so that method body can be optimized. The way I see it, it’s less an issue of knowing when x is a constant than making surex a constant.
On top of the issues mentioned above, SVector is not universally faster than Vector, especially when it comes to compile time, due to the way they work internally. I think something like this and constant propagating the length for that is going to be better than detecting & manually branching on whether something is a constant or not; we generally don’t want to leak whether some code was interpreted or constant folded.
That looks cool and seems to address the issue(?) of reshaped Matrixes allocating a Vector despite sharing the same buffer. Cropped up a couple times on discourse in the last few days, both involving a view that is a linear slice of a Matrix. The new Array definition there looks similar to ReshapedArray now, which makes sense.
This code and result demonstrates why being able to branch on whether something is known to be a constant is of such value:
Code
using StaticArrays, BenchmarkTools, Plots
function myfunc(::Val{n}) where n
m = 2 ^ n
V = rand(SVector{m})
return V' * V
end
function myfunc(n)
m = 2 ^ n
V = rand(m)
return V' * V
end
const topend = 9
function timeit()
times = zeros(topend, 3)
times[1, 1] = @belapsed myfunc(Val(1))
times[2, 1] = @belapsed myfunc(Val(2))
times[3, 1] = @belapsed myfunc(Val(3))
times[4, 1] = @belapsed myfunc(Val(4))
times[5, 1] = @belapsed myfunc(Val(5))
times[6, 1] = @belapsed myfunc(Val(6))
times[7, 1] = @belapsed myfunc(Val(7))
times[8, 1] = @belapsed myfunc(Val(8))
times[9, 1] = @belapsed myfunc(Val(9))
for a = 1:topend
times[a, 2] = @belapsed myfunc(Val($a))
times[a, 3] = @belapsed myfunc($a)
end
return times
end
times = timeit()
plot(2 .^ (1:topend), times .* 10e6, xlabel="Length of vector", ylabel="Function time (ms)", label=["SVector w. static dispatch" "SVector w. dynamic dispatch" "Vector"], xscale=:log10, yscale=:log10)
What it shows is that if I know that vector length is constant, I can get significant speedups by using SVector instead of Vector, up to a certain length. However, if the vector length is not constant, then the penalty incurred by dynamic dispatch negates all speed advantages of SVector, making it slower than Vector for all lengths.
It is not currently possible to exploit this fact to speed up some code without also risking slowing down the same code under different conditions, because, from what has been said here, I understand it isn’t possible to detect if a value is constant at runtime. This looks like it would be a useful feature.
Your feature request sounds an awful lot like Create staticint.jl by Tokazama · Pull Request #44538 · JuliaLang/julia · GitHub and the discussion within. In particular, the context and discussion surrounding compile time implications (not to mention “is a constant” not being very well defined/exposed at all in the type system) were contentious topics on triage, with the conclusion that the feature doesn’t seem worth the added maintenance cost, compared to branching (instead of dispatch) on the length.
A dispatch time of 10ms sounds extremely good to me - how often do you expect to call the dispatch on an unknown input size?
You fundamentally cannot specify a constant integer in a method compiled for ::Int, and your use of a type parameter like ::Val{N} is as good a function barrier as you can get. Unless you straight up write an integer literal (or branch to 3 or 4 of them, on account of small Union-splitting) inside a function, you cannot get around the need for 1 runtime dispatch of that function barrier. In your test case, your function was just so cheap it got overshadowed by that 1 dispatch. In much more expensive functions, this technique can save a lot of time and allocations.
Type inference in julia is all about deciding exactly that - and ours being incredibly complicated is a well known issue. A value being constant just isn’t part of the type system at all - hence, it’s not exposed. It’s only an optimization.
I’m not entirely sure I understand what you’re saying here, but…
I agree with this. The point is that the dispatch can happen deep inside a library, while the value on which the dispatch happens is defined outside the library. It could be an integer literal, propagated through as a constant. Or it could be a dynamic value.
It’s the package authors’ responsibility to keep their functions type stable so they can be put together without dynamic dispatch happening outside the end-user’s control. In fact I’ve questioned some oddly restrictive function signatures in StaticArrays before only to realize it’s for that very purpose.
As I understand it, I could wrap an integer literal in a StaticInt type, and branch on this type within my library, instead of using the isconstant function or macro. I think this is fundamentally different from what I’m proposing, but can be used to achieve the same end.
Absolutely. I want my library to be performant and type stable. That’s why I need the isconstant functionality. Otherwise it can be type stable (using Vector), but not performant, or performant in some cases (using SVector, when the length is small and a constant) but not type stable (and therefore not performant) when the length is small but not constant.
There’s no such thing. Given a runtime integer value, you cannot know whether it was a hard-coded constant from anywhere, it just doesn’t carry that information. There is a isconst function but you have to give it a module and a Symbol to check if a global variable was declared const.
You already showed the way to make a compile-time constant, that’s how you do it. StaticInt wouldn’t help elide a dynamic dispatch, it would be the same thing as a Val except it has the extra feature of arithmetic like StaticInt{1}() + StaticInt{2}(), which is really not intended for runtime.
It isn’t, because you’re not only interested in whether the value was a constant, you also want to specialize your code based on the value of that constant. That’s how StaticArrays can be faster here in the first place; by knowing how large the input & output matrices are (during compilation!), it’s possible to generate a kernel function for exactly that operation. Using Val just makes that also visible to the compiler, by lifting the runtime value to the type domain (incurring dynamic overhead, as you’ve noticed).
The reason we’re saying that constant propagation is usually better here is because of the compile time overhead associated with large StaticArrays (which use large tuples under the hood, which ultimately causes the overhead). There are some reasons why constant propagation and regular Array don’t play nice and why generating specialized kernels for particular sizes is hard, but that’s exactly why I linked the discussion on a buffer type above, which aims to solve that/make it easier on the compiler.
I understand there is no such thing. That’s why I have created a feature request for it.
Perhaps “runtime” is an unhelpful distinction, and there is a linguistic misunderstanding here. It seems to me that julia doesn’t have a clear-cut compile time - there seem to be several stages of compilation inference. Some constants are propagated at a particular stage of compilation. If, by the time the code is run, a value has been determined to be constant, such that a call to somefunc(Val(v)) will be statically, and not dynamically dispatched, then that value would be what I meant by a “runtime constant”.
I presume the compiler can determine whether it will need to do a dynamic or static dispatch. That’s the information I’m asking for. It would enable a compile-time branch, not a runtime branch. The unused branch would be compiled away.
