# \[ANN\] BootlegCassette.jl

**URL:** <https://discourse.julialang.org/t/ann-bootlegcassette-jl/52909>\
**Category:** Package Announcements\
**Created:** [January 6, 2021, 12:01am UTC](https://discourse.julialang.org/t/ann-bootlegcassette-jl/52909 "2021-01-06T00:01:05Z")\
**Posts on this page:** 19\
**Page:** 1

<div class="post-metadata">

**Author:** ![Mason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mason/32/2423_2.png) [@Mason](https://discourse.julialang.org/u/Mason)\
**Post date:** [January 6, 2021, 12:01am UTC](https://discourse.julialang.org/t/ann-bootlegcassette-jl/52909/1 "2021-01-06T00:01:06Z")

</div>

[Cassette.jl](https://github.com/JuliaLabs/Cassette.jl) is an incredibly cool package that allows one to dictate how functions are dispatched in special execution contexts, but it’s currently not fully functional on Julia 1.6 / master, so I made a minimal package that replicates it’s main feature and API: contextual dispatch. I call it [BootlegCassette.jl](https://github.com/MasonProtter/BootlegCassette.jl). If you don’t need Cassette’s tagging capabilities or some of the more advanced usages of certain APIs, this package _may_ be a drop in replacement.

I mostly made this for fun and learning, but if anyone wants to try and get more things working or improve performance, I’ll gladly collaborate.

Happy overdubbing!

```julia
using BootlegCassette
const Cassette = BootlegCassette

Cassette.@context Ctx 
Cassette.prehook(::Ctx, f, args...) = println(f, args)
Cassette.overdub(Ctx(), /, 1, 2)

#+RESULTS
float(1,)
AbstractFloat(1,)
Float64(1,)
sitofp(Float64, 1)
float(2,)
AbstractFloat(2,)
Float64(2,)
sitofp(Float64, 2)
/(1.0, 2.0)
div_float(1.0, 2.0)
0.5

```

---

<div class="post-metadata">

**Author:** ![JeffreySarnoff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jeffreysarnoff/32/1980_2.png) [@JeffreySarnoff](https://discourse.julialang.org/u/JeffreySarnoff)\
**Post date:** [January 6, 2021, 12:37am UTC](https://discourse.julialang.org/t/ann-bootlegcassette-jl/52909/2 "2021-01-06T00:37:54Z")

</div>

nice name

---

<div class="post-metadata">

**Author:** ![Elrod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/elrod/32/22461_2.png) [@Elrod](https://discourse.julialang.org/u/Elrod)\
**Post date:** [January 6, 2021, 12:56am UTC](https://discourse.julialang.org/t/ann-bootlegcassette-jl/52909/3 "2021-01-06T00:56:40Z")

</div>

Does this have any overhead? I’ve stayed away from Casette for that reason.  
If not, can it be used to replace calls to `Base.ifelse` with `IfElse.ifelse`, for example?

I’m wondering if it’d be useful for trying to run generic code with SIMD, in SPMD style.

---

<div class="post-metadata">

**Author:** ![Mason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mason/32/2423_2.png) [@Mason](https://discourse.julialang.org/u/Mason)\
**Post date:** [January 6, 2021, 1:16am UTC](https://discourse.julialang.org/t/ann-bootlegcassette-jl/52909/4 "2021-01-06T01:16:52Z")

</div>

Yes, this actually has greater overhead than Cassette in my experiments. However, there are tricks one can use to avoid that for some use cases. For instance, if I understand correctly CUDA.jl will basically build up a whole new function by tracing through a function call and inlining _everything_ into one big new function IR where all the dynamic behaviour and such has been excised.

I think such an approach _may_ be feasible for LoopVectorization.jl, but I’m not sure. You’d probably want to use IRTools.jl for this directly, or if you can, wait and see what the AbstractInterpreter interface ends up being like.

---

<div class="post-metadata">

**Author:** ![Mason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mason/32/2423_2.png) [@Mason](https://discourse.julialang.org/u/Mason)\
**Post date:** [January 6, 2021, 1:24am UTC](https://discourse.julialang.org/t/ann-bootlegcassette-jl/52909/5 "2021-01-06T01:24:07Z")

</div>

> [@Elrod](#):
>
> can it be used to replace calls to `Base.ifelse` with `IfElse.ifelse` , for example?

Not only can approaches like this do that, but they can also even replace `if ... else` if you’re devious enough.

---

<div class="post-metadata">

**Author:** ![Elrod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/elrod/32/22461_2.png) [@Elrod](https://discourse.julialang.org/u/Elrod)\
**Post date:** [January 6, 2021, 1:58am UTC](https://discourse.julialang.org/t/ann-bootlegcassette-jl/52909/6 "2021-01-06T01:58:26Z")

</div>

> [@Mason](#):
>
> I think such an approach _may_ be feasible for LoopVectorization.jl, but I’m not sure. You’d probably want to use IRTools.jl for this directly, or if you can, wait and see what the AbstractInterpreter interface ends up being like.

Okay, I’ll probably wait for AbstractInterpreter, but replacing `if ... else ...` statements is definitely something I’d like to be able to do.  
It’d be tricky even if you have the infrastructure. You’d have to walk both sides and see all assignments to combine them.  
Ideally, you also don’t just walk both sides, because maybe you can combine them:

```julia
if cond
    y = 3x
    z = exp(y)
else
    z = 3x
    y = exp(z)
end

```

---

<div class="post-metadata">

**Author:** ![Mason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mason/32/2423_2.png) [@Mason](https://discourse.julialang.org/u/Mason)\
**Post date:** [January 6, 2021, 2:36am UTC](https://discourse.julialang.org/t/ann-bootlegcassette-jl/52909/7 "2021-01-06T02:36:51Z")

</div>

> [@Elrod](#):
>
> Ideally, you also don’t just walk both sides, because maybe you can combine them:
> 
> ```julia
> if cond
> y = 3x
> z = exp(y)
> else
> z = 3x
> y = exp(z)
> end
> 
> ```

This could be done in a common sub expression elimination pass I think.

---

<div class="post-metadata">

**Author:** ![Elrod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/elrod/32/22461_2.png) [@Elrod](https://discourse.julialang.org/u/Elrod)\
**Post date:** [January 6, 2021, 2:39am UTC](https://discourse.julialang.org/t/ann-bootlegcassette-jl/52909/8 "2021-01-06T02:39:38Z")

</div>

Yeah, you’re right. Relying on the naive approach:

```julia
y1 = 3*x
z1 = exp(y1)
z2 = 3*x
y2 = exp(z2)
y = ifelse(cond, y1, y2)
z = ifelse(cond, z1, z2)
y, z

```

and relying on CSE is probably the best way to do this. If any problematic/suboptimal examples turn up, we can worry about finding out why and how to fix them then.

---

<div class="post-metadata">

**Author:** ![baggepinnen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/baggepinnen/32/693_2.png) [@baggepinnen](https://discourse.julialang.org/u/baggepinnen)\
**Post date:** [January 6, 2021, 6:14am UTC](https://discourse.julialang.org/t/ann-bootlegcassette-jl/52909/9 "2021-01-06T06:14:12Z")

</div>

Have you guys seen  
[https://github.com/FluxML/Hydra.jl](https://github.com/FluxML/Hydra.jl)  
The goal of this package is exactly to enable SPMD for arbitrary programs. Unfortunately, it appears to have been abandoned.

There was a short discussion about if statements in  
[https://github.com/FluxML/Hydra.jl/issues/13](https://github.com/FluxML/Hydra.jl/issues/13)

---

<div class="post-metadata">

**Author:** ![Elrod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/elrod/32/22461_2.png) [@Elrod](https://discourse.julialang.org/u/Elrod)\
**Post date:** [January 6, 2021, 8:30am UTC](https://discourse.julialang.org/t/ann-bootlegcassette-jl/52909/10 "2021-01-06T08:30:28Z")

</div>

I knew of the library (and that it was abandoned), but hadn’t looked for a while or seen your issue, thanks.  
Out of curiosity, have you seen [this series](https://pharr.org/matt/blog/2018/04/18/ispc-origins.html) of blog posts?  
I can confirm that for a few benchmarks, ISPC was many times faster than a comparable Julia program. I’d like to fix that 😉 , but more generally, it’d be great to make packages like MonteCarloMeasurements faster, as well as use this to SIMD DIfferentialEquations ensemble solves (and obviously LoopVectorization wants this too).

Still, it seems like Hydra is probably advanced enough to at least look at for a starting point.

---

<div class="post-metadata">

**Author:** ![baggepinnen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/baggepinnen/32/693_2.png) [@baggepinnen](https://discourse.julialang.org/u/baggepinnen)\
**Post date:** [January 6, 2021, 5:12pm UTC](https://discourse.julialang.org/t/ann-bootlegcassette-jl/52909/11 "2021-01-06T17:12:21Z")

</div>

I had not seen it before, but having read the linked entry, I find it quite interesting.

> And God help you when they release a new version of the compiler with changes to the auto-vectorizer’s implementation.

I’ve felt this, and similar issues like `@inferred` etc., before, so julia is not free from it. I guess anyone relying on an auto vectorizer or type inference should really consider contributing meaningful tests and benchmark tests to julia.

I also enjoyed the story on the PR adding ARM support in the ispc repo 🙂 I can imagine the awkward feelings for everyone involved!

---

<div class="post-metadata">

**Author:** ![Zach\_Christensen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/zach_christensen/32/7220_2.png) [@Zach\_Christensen](https://discourse.julialang.org/u/Zach_Christensen)\
**Post date:** [January 6, 2021, 11:44pm UTC](https://discourse.julialang.org/t/ann-bootlegcassette-jl/52909/12 "2021-01-06T23:44:24Z")

</div>

> [@Elrod](#):
>
> Okay, I’ll probably wait for AbstractInterpreter, but replacing `if ... else ...` statements is definitely something I’d like to be able to do.

I was trying to figure out something for this exact thing today (among other stuff) and eventually was let to this post. If/when there is more info on using all the magic in “julia/base/compiler/” I’m going to probably disappear from life for a week while I digest it.

---

<div class="post-metadata">

**Author:** ![Mason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mason/32/2423_2.png) [@Mason](https://discourse.julialang.org/u/Mason)\
**Post date:** [January 7, 2021, 7:05pm UTC](https://discourse.julialang.org/t/ann-bootlegcassette-jl/52909/13 "2021-01-07T19:05:31Z")

</div>

Just an update, Cassette.jl has been updated to work on 1.6 and nightly! [https://github.com/JuliaLabs/Cassette.jl/releases/tag/v0.3.4](https://github.com/JuliaLabs/Cassette.jl/releases/tag/v0.3.4)

Bravo @simeonschaub!

---

<div class="post-metadata">

**Author:** ![oxinabox](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oxinabox/32/206603_2.png) [@oxinabox](https://discourse.julialang.org/u/oxinabox)\
**Post date:** [January 7, 2021, 7:29pm UTC](https://discourse.julialang.org/t/ann-bootlegcassette-jl/52909/14 "2021-01-07T19:29:58Z")

</div>

This is really cool, in and of its own right, because it is Cassette built on IRTools.

Have you done any benchmarking?

---

<div class="post-metadata">

**Author:** ![Mason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mason/32/2423_2.png) [@Mason](https://discourse.julialang.org/u/Mason)\
**Post date:** [January 7, 2021, 7:39pm UTC](https://discourse.julialang.org/t/ann-bootlegcassette-jl/52909/15 "2021-01-07T19:39:55Z")

</div>

> [@oxinabox](#):
>
> Have you done any benchmarking?

Yes, it’s pretty slow and I’m not exactly sure what’s going on. Here’s an interesting example of something that’s wrong and causing a performance problem:

```julia
using BootlegCassette
using BootlegCassette: @context, overdub

@context Ctx

let x = Ref(1), y = Ref(2)
    @btime overdub(Ctx(), *, $x[], $y[])
end;

#+RESULTS:
  16.292 ns (0 allocations: 0 bytes)

```

Investigating the typed code doesn’t reveal anything amis:

```julia
@code_typed overdub(Ctx(), *, 1, 2)

#+RESULTS:
CodeInfo(
1 ─ %1 = Base.getfield(args, 3)::Int64
│ %2 = Base.getfield(args, 4)::Int64
│ %3 = (Core.Intrinsics.mul_int)(%1, %2)::Int64
└── return %3
) => Int64

```

but the LLVM code tells a different story:

```julia
@code_llvm overdub(Ctx(), *, 1, 2)

#+RESULTS:
; @ /home/mason/.julia/packages/IRTools/aSVI5/src/reflection/dynamo.jl:114 within `overdub'
define nonnull {}* @japi3_overdub_1970({}* %0, {} **%1, i32 %2, {}** %3) #0 {
top:
  %4 = alloca {}**, align 8
  store volatile {} **%1, {}*** %4, align 8
  %5 = icmp ugt i32 %2, 2
  br i1 %5, label %pass, label %fail

fail: ; preds = %top
  %6 = sext i32 %2 to i64
  call void @jl_bounds_error_tuple_int({}** %1, i64 %6, i64 3)
  unreachable

pass: ; preds = %top
  %.not = icmp eq i32 %2, 3
  br i1 %.not, label %fail1, label %pass2

fail1: ; preds = %pass
  call void @jl_bounds_error_tuple_int({}** %1, i64 3, i64 4)
  unreachable

pass2: ; preds = %pass
  %7 = getelementptr inbounds {}*, {}** %1, i64 2
  %8 = bitcast {} **%7 to i64**
  %9 = load i64*, i64** %8, align 8
  %10 = getelementptr inbounds {}*, {}** %1, i64 3
  %11 = bitcast {} **%10 to i64**
  %12 = load i64*, i64** %11, align 8
; @ /home/mason/.julia/packages/IRTools/aSVI5/src/reflection/dynamo.jl within `overdub' @ int.jl:88
; ┌ @ /home/mason/Dropbox/Julia/BootlegCassette/src/BootlegCassette.jl:49 within `overdub_pass'
; │┌ @ /home/mason/Dropbox/Julia/BootlegCassette/src/BootlegCassette.jl:55 within `overdub'
    %13 = load i64, i64* %9, align 8
    %14 = load i64, i64* %12, align 8
    %15 = mul i64 %14, %13
    %16 = call nonnull {}* @jl_box_int64(i64 signext %15)
    ret {}* %16
; └└
}

```

Somehow, LLVM doesn’t seem to know how many arguments there are to the function or something.

---

<div class="post-metadata">

**Author:** ![Elrod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/elrod/32/22461_2.png) [@Elrod](https://discourse.julialang.org/u/Elrod)\
**Post date:** [January 8, 2021, 1:52am UTC](https://discourse.julialang.org/t/ann-bootlegcassette-jl/52909/16 "2021-01-08T01:52:25Z")

</div>

Did you try `args::Vararg{Any,K}` instead of `args...` to force specialization on the number of arguments (in the definition of `overdub`)?

---

<div class="post-metadata">

**Author:** ![Mason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mason/32/2423_2.png) [@Mason](https://discourse.julialang.org/u/Mason)\
**Post date:** [January 8, 2021, 2:04am UTC](https://discourse.julialang.org/t/ann-bootlegcassette-jl/52909/17 "2021-01-08T02:04:11Z")

</div>

Yes and it didn’t help, however that led me to discover this:

```julia
using IRTools
IRTools.@dynamo foo(x::Int, y::String, z::Symbol, w::Int...) = nothing

methods(foo)

#+RESULTS:
 # 1 method for generic function "foo":
 [1] foo(args...) in Main at /home/mason/.julia/packages/IRTools/aSVI5/src/reflection/dynamo.jl:114

```

So it may be an issue with how `@dynamo` is creating the function.

---

<div class="post-metadata">

**Author:** ![Elrod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/elrod/32/22461_2.png) [@Elrod](https://discourse.julialang.org/u/Elrod)\
**Post date:** [January 8, 2021, 2:10am UTC](https://discourse.julialang.org/t/ann-bootlegcassette-jl/52909/18 "2021-01-08T02:10:20Z")

</div>

Looks like it’s time to hack on IRTools then 😉  
Hopefully changing how it generates the function solves it.

---

<div class="post-metadata">

**Author:** ![Tomas\_Pevny](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tomas_pevny/32/25466_2.png) [@Tomas\_Pevny](https://discourse.julialang.org/u/Tomas_Pevny)\
**Post date:** [January 18, 2021, 10:51am UTC](https://discourse.julialang.org/t/ann-bootlegcassette-jl/52909/19 "2021-01-18T10:51:08Z")

</div>

Can I ask, where I can read about AbstractInterpretter?