ANN: ToggleableAsserts.jl

Package Announcements
1

Following two back to back questions on slack on the topic of “how can I turn off assertions in my code?” I decided to make a small package insipired by this clever trick: @assert alternatives? - #14 by WschW

Here’s a little demonstration showing how we can have assertions that we can turn on or off without any runtime penalty by recompiling functions depending on the assertion:

using ToggleableAsserts

function foo(u, v)
    @toggled_assert length(u) == length(v)
    1
end

julia> foo([1, 2], [1])
ERROR: AssertionError: length(u) == length(v)
Stacktrace:
 [1] foo(::Array{Int64,1}, ::Array{Int64,1}) at ./REPL[1]:2
 [2] top-level scope at REPL[2]:1 

julia> toggle(false)
[ Info: Toggleable asserts turned off.

julia> foo([1, 2], [1])
1

julia> @code_llvm foo([1,2], [1])
;  @ REPL[1]:2 within `foo'
define i64 @julia_foo_16854(%jl_value_t addrspace(10)* nonnull align 16 dereferenceable(40), %jl_value_t addrspace(10)* nonnull align 16 dereferenceable(40)) {
top:
  ret i64 1
}

Edit 1: The @toggle macro has been changed to a function: toggle following this comment: ANN: ToggleableAsserts.jl - #11 by rdeits

Edit 2: ToggleableAsserts.jl is now a registered package. Install it by simply running

pkg> add ToggleableAsserts
10 Likes
2

Er? Is this thing Thread safe? What if multiple threads called “@toggle false” and “@toggle true”?

3

Good question!

I’m not actually sure what guarantees julia gives in that regard, so I threw @toggle in a lock. If anyone reading this knows about thread safety, let me know if what I did here was sufficient and / or overkill: https://github.com/MasonProtter/ToggleableAsserts.jl/commit/3029fa0565115ea6dad07468061e389d70ae0db7.

I feel like entering and exiting a debugging mode like this is a pretty weird thing to do in a multithreaded context though anyways…

4

Okay, I’ve made it so that the only valid inputs to the @toggle macro are literally writing true or false because I realized that writing things like rand(Bool) in there would cause all sorts of problems with my implementation and I think that is a reasonable restriction.

I’ve also submitted the package to the general registry and am now in the mandatory waiting period for new packages.

1 Like
5

You need to hit top level scope between changing or you will get world age errors.

2 Likes
6

Here is an improvement to your module. Use a Reference to store your global state

module ToggleableAsserts

const ASSERT_TOGGLE = Ref(true)

macro toggled_assert(cond, text=nothing)
    if text==nothing
        assert_stmt = esc(:(@assert $cond))
    else
        assert_stmt = esc(:(@assert $cond $text))
    end
    :(ToggleableAsserts.ASSERT_TOGGLE[] ? $assert_stmt  : nothing)
end

const toggle_lock = ReentrantLock()

macro toggle(bool)
    bool ∈ [:true, :false] || error("Toggle only takes true or false literals.")
    quote
        lock(ToggleableAsserts.toggle_lock) do
            @assert $bool isa Bool
            ToggleableAsserts.ASSERT_TOGGLE[] = $bool
            on_or_off = $bool ? "on." : "off."
            @info "Toggleable asserts turned "*on_or_off
        end
    end
end

export @toggled_assert, @toggle

end # module
7

@kristoffer.carlsson Hm, you’re right! I think in that case I’ll try to make the @toggle macro error unless it’s called from the global scope.

@StevenSiew That could be useful for some purposes, but it imposes a runtime overhead even if the assertion is turned off, so it’s not the direction I’d want to move this package.

8

oh yeah! I forgot that macros occurs at compile time.

9

Sometimes we get the hero we do not deserve, XD.

I am one of the people that started these threads asking for alternatives in the past. However, because my PhD, I never had the time to make the solution a package. I thank you for this. I will make use of it.

1 Like
10

I couldn’t find a way to detect if @toggle was invoked in the global scope or not, so I just ended up adding a warning to the README.

11

As far as I can tell, there’s no need for @toggle to be a macro–it can just be a function:

function toggle(enable::Bool)
    lock(toggle_lock) do
        @eval ToggleableAsserts assert_toggle() = $enable
        on_or_off = enable ? "on." : "off."
        @info "Toggleable asserts turned "*on_or_off
    end
end

There’s nothing wrong with using a macro, but it’s an unnecessary layer of complexity.

3 Likes
12

Good point! I originally made a macro because I intended to modify assert_toggle() directly with the output code and then realized that wouldn’t work so I switched to doing an eval and never realized I could get rid of the macro.

13

I love this idea (had a thought of implementing such a utility myself)!

I am reminded of Java asserts that are toggled at classloading time. Thread safety of loading a class is ensured by the JVM. As a result, thread safety does not incur any runtime overhead post classloading because the assert-triggered bytecode blocks are guarded by am immutable “static final” boolean flag field (the JIT is within its right to and will in fact elide all of them).

I wonder if something similar could be done in Julia – I think it requires some kind of a hook during include() so that assertion status could be set at some intermediate code representation level. Checking locks at runtime will almost surely result in overhead that’s too high for truly liberal assert usage (defensive programming, etc).

14

If I’m interpreting you correctly here, it seems you’re saying that ToggleableAsserts incurs a runtime penalty due to checking locks. That is not the case. The only time a lock is checked is when the assertions are toggled on or off. Once the assertions are turned off, they are completely removed from the function body by the compiler:

using ToggleableAsserts, BenchmarkTools

function f()
    @toggled_assert (sleep(1); true)
    1
end

g() = 1

julia> @btime f();
  1.002 s (7 allocations: 160 bytes)

julia> @btime g();
  0.019 ns (0 allocations: 0 bytes)

julia> toggle(false)
[ Info: Toggleable asserts turned off.

julia> @btime f();
  0.019 ns (0 allocations: 0 bytes)

julia> @code_llvm f()
define i64 @julia_f_17080() {
top:
  ret i64 1
}

julia> @code_llvm g()
define i64 @julia_g_17114() {
top:
  ret i64 1
}

As you can see, once I run toggle(off), the assertions are completely gone.

1 Like
15

In other news, ToggleableAsserts.jl is now registered, so you can now install it by simply running

pkg> add ToggleableAsserts
16

The only time a lock is checked is when the assertions are turned on or off. When the assertions are turned off, they are completely removed from the function body from the compiler…

If so, this is exactly the desired outcome.

I think I was reacting to earlier comments that advocated against using macros here – my understanding of Julia facilities is that some language construct is required that would work at an intermediate representation level. My expectation would have been for that to be macros, but perhaps that’s not the only way… I have not looked at your code to understand the details of thread safety in the latest design. I surmised from comments that there is a boolean flag that’s being read and written under some lock protection (the @toggle false, @toggle true thing). When Julia JIT decides to compile a function, does it read the same lock? Or is it read at parsing time? I am not educated about Julia “class loading” process (compared with, say, Java spec of the process Chapter 5. Loading, Linking, and Initializing. Note an explicit mention of when assertions are enabled).

Also note that in Java the assertions can be enabled in one class and disabled in another, etc. It’s not a matter of “entering and leaving debug mode all the time”, it’s actually very useful for isolating bugs in some suspect areas while avoiding slowdowns in the rest of runtime. Java assertion facility is also convenient because “packages” are organized in a tree and there is a notion of “subpackages” with associated ability to set assertion status at subtree levels. Quoting https://docs.oracle.com/cd/E19683-01/806-7930/assert-4/index.html:

With one argument ending in “…”, assertions are enabled in the specified package and any subpackages by default. If the argument is simply “…”, assertions are enabled in the unnamed package in the current working directory. With one argument not ending in “…”, assertions are enabled in the specified class.

(Java assertions can be enabled at various levels of granularity via JVM command line arguments but also at runtime, via the ClassLoader API. ClassLoader (Java Platform SE 7 ) This is all very nice for “componentization”.)

I wish Julia would adopt a Java-like naming convention for packages/modules and avoid R/python “wild west” future in that regard … :nerd_face:

17

It’s incredibly simple and actually shorter than your above post. Here’s literally all the package code:

assert_toggle() = true

macro toggled_assert(cond, text=nothing)
    if text==nothing
        assert_stmt = esc(:(@assert $cond))
    else
        assert_stmt = esc(:(@assert $cond $text))
    end
    :(assert_toggle() ? $assert_stmt  : nothing)
end

const toggle_lock = ReentrantLock()

function toggle(enable::Bool)
    lock(toggle_lock) do
        @eval ToggleableAsserts assert_toggle() = $enable
        on_or_off = enable ? "on." : "off."
        @info "Toggleable asserts turned "*on_or_off
    end
end

So if I write @toggled_assert cond somewhere, that will be macroexpanded to

if assert_toggle()
   @assert cond
end

And then writing toggle(false) will overwrite the definition of assert_toggle to

assert_toggle() = false

Due to julia’s code invalidation machinery, this makes any method relying on assert_toggle to get recompiled. Since assert_toggle is a constant function, code saying
if assert_toggle() is equivalent to if false and so those blocks are completely skipped by the compiler.

Something similar is definitely possible (and easy!) here too, but I haven’t gotten around to it yet.

1 Like
18

Great. I need to learn more about Julia’s code loading process. I was not aware that it could track and recompile dependencies. (could get expensive in some scenarios, I’d imagine).

19

You only pay for what you use. If you have a package loaded that uses @toggled_assert a thousand times and you do toggle(false), those thousand methods only get recompiled if you (or a function you called) actually call them. The cost isn’t up front.

I don’t really see any way that one could lessen this cost. Since we want the assertions to be elided, the only way to do that is recompilation.

2 Likes