[ANN] Traceur.jl: Make Your Julia Code Not Totally Slow

Community
1

If, like me, you’re too lazy to learn to read @code_typed, or all of the performance gotchas in the manual, you might find Traceur.jl interesting. It’s extremely simple to use: you throw your function at it, it yells at you, you fix the problems. When it’s done yelling at you your code is probably reasonably fast.

julia> y = 1

julia> f(x) = x+y

julia> @trace f(1)
f(::Int64) at none:1
    uses global variable Main.y
    dynamic dispatch to x + Main.y at line 1
    returns Any

Notice how type issues propagate here – the root issue is a global variable access, and this creates further problems down the line. Make y constant and everything else goes away!

Right now it has a basic set of obvious things, like unstable variable types and dynamic dispatch. Reports of funky behaviour or extra things it should catch are welcome.

92 Likes
2

is it possible to integrate into vscode?

4 Likes
3

It’s probably not that easy to do, as it would require VS code to hook into a running process, which it’s not designed for. (It’s basically very similar to profiling.)

We will definitely have some integration in Juno though.

11 Likes
4

@pfitzseb already managed to hack together a UI for this by abusing our error message display :slight_smile:

39
39960×720 60.5 KB

A bit raw right now, but you can imagine how nice it’ll be with a real linter-like interface.

12 Likes
5

This is great.

I’ve been working on something similar - https://github.com/RelationalAI-oss/Analyzer.jl. It traces the call-graph statically, mostly because I didn’t know about Vinyl :smiley:

I can see us having uses for both styles of tracing. Maybe I can get rid off the rest of my code and fold the static tracing into Traceur?

I also had plans for

  • highlighting potential heap allocations
  • running assertions over the resulting trace (eg this functions should have no dynamical calls, this function should not cause heap allocations)
5 Likes
6

This looks great indeed,

Unfortunately, I cannot install the package.
It appears to be officially registered but when I do:

julia> Pkg.add("Traceur")

I get:

ERROR: unknown package Traceur
macro expansion at ./pkg/entry.jl:53 [inlined]
(::Base.Pkg.Entry.##1#3{String,Base.Pkg.Types.VersionSet})() at ./task.jl:335
Stacktrace:
 [1] sync_end() at ./task.jl:287
 [2] macro expansion at ./task.jl:303 [inlined]
 [3] add(::String, ::Base.Pkg.Types.VersionSet) at ./pkg/entry.jl:51
 [4] (::Base.Pkg.Dir.##4#7{Array{Any,1},Base.Pkg.Entry.#add,Tuple{String}})() at ./pkg/dir.jl:36
 [5] cd(::Base.Pkg.Dir.##4#7{Array{Any,1},Base.Pkg.Entry.#add,Tuple{String}}, ::String) at ./file.jl:70
 [6] #cd#1(::Array{Any,1}, ::Function, ::Function, ::String, ::Vararg{String,N} where N) at ./pkg/dir.jl:36
 [7] add(::String) at ./pkg/pkg.jl:117

Should I install through Pkg.clone()?
Many thanks in advance,
Olivier

7

Have tried Pkg.update() first?

1 Like
8

Yep indeed,

Sorry for the noise. I ran Pkg.update() last week and thought it was ok.
Thanks again!

9

Yes, it’s a very new package :slight_smile:

Absolutely, you’d be very welcome to. Looks like you have some useful passes in Analyzer and I like the sound of the other ideas, so it’d definitely be a good idea to combine forces there. It may well make sense to have multiple interfaces (static/dynamic) to that analysis as well.

5 Likes
10

What kind of integration were you thinking about?

I don’t understand this comment. Why wouldn’t be able to provide the same kind of integration in VS Code that you think can be provided in Juno?

11

For the exact same reason you don’t have debugger integration.

12

@MikeInnes, always coming out with incredibly useful stuff! (which should at least be advertised advertised here so that people know about it and don’t wallow unnecessarily in macro purgatory for a year :wink:).

5 Likes
13

That makes no sense to me. What has the VS Code debug adapter architecture to do with this?

2 Likes
14

Hello, thanks a lot your package looks very great but I can’t make it work.
Is it an error on my side or should I open an issue, this is error with latest julia:

ERROR: LoadError: ArgumentError: invalid type for argument val in method definition for print_var at /home/gregory/.julia/v0.7/DebuggerFramework/src/DebuggerFramework.jl:14
Stacktrace:
 [1] top-level scope
 [2] include at ./boot.jl:306 [inlined]
 [3] include_relative(::Module, ::String) at ./loading.jl:1072
 [4] include(::Module, ::String) at ./sysimg.jl:29
 [5] top-level scope
 [6] eval at ./boot.jl:309 [inlined]
 [7] top-level scope at ./<missing>:3
in expression starting at /home/gregory/.julia/v0.7/DebuggerFramework/src/DebuggerFramework.jl:13
ERROR: LoadError: Failed to precompile DebuggerFramework to /home/gregory/.julia/compiled/v0.7/DebuggerFramework.ji.
Stacktrace:
 [1] error at ./error.jl:33 [inlined]
 [2] compilecache(::Base.PkgId) at ./loading.jl:1206
 [3] _require(::Base.PkgId) at ./loading.jl:979
 [4] require(::Base.PkgId) at ./loading.jl:879
 [5] require(::Module, ::Symbol) at ./loading.jl:874
 [6] include at ./boot.jl:306 [inlined]
 [7] include_relative(::Module, ::String) at ./loading.jl:1072
 [8] include(::Module, ::String) at ./sysimg.jl:29
 [9] top-level scope
 [10] eval at ./boot.jl:309 [inlined]
 [11] top-level scope at ./<missing>:3
in expression starting at /home/gregory/.julia/v0.7/ASTInterpreter2/src/ASTInterpreter2.jl:4
ERROR: LoadError: LoadError: LoadError: LoadError: Failed to precompile ASTInterpreter2 to /home/gregory/.julia/compiled/v0.7/ASTInterpreter2.ji.
Stacktrace:
 [1] error at ./error.jl:33 [inlined]
 [2] compilecache(::Base.PkgId) at ./loading.jl:1206
 [3] _require(::Base.PkgId) at ./loading.jl:1008
 [4] require(::Base.PkgId) at ./loading.jl:879
 [5] require(::Module, ::Symbol) at ./loading.jl:874
 [6] include at ./boot.jl:306 [inlined]
 [7] include_relative(::Module, ::String) at ./loading.jl:1072
 [8] include at ./sysimg.jl:29 [inlined]
 [9] include(::String) at /home/gregory/.julia/v0.7/Vinyl/src/Vinyl.jl:1
 [10] top-level scope
 [11] include at ./boot.jl:306 [inlined]
 [12] include_relative(::Module, ::String) at ./loading.jl:1072
 [13] _require(::Base.PkgId) at ./loading.jl:998
 [14] require(::Base.PkgId) at ./loading.jl:879
 [15] require(::Module, ::Symbol) at ./loading.jl:874
 [16] include at ./boot.jl:306 [inlined]
 [17] include_relative(::Module, ::String) at ./loading.jl:1072
 [18] _require(::Base.PkgId) at ./loading.jl:998
 [19] require(::Base.PkgId) at ./loading.jl:879
 [20] require(::Module, ::Symbol) at ./loading.jl:874
 [21] top-level scope at /home/gregory/workspace/StructDOM.jl/test/benchmark.jl:4
 [22] include at ./boot.jl:306 [inlined]
 [23] include_relative(::Module, ::String) at ./loading.jl:1072
 [24] include(::Module, ::String) at ./sysimg.jl:29
 [25] exec_options(::Base.JLOptions) at ./client.jl:327
 [26] _start() at ./client.jl:455
in expression starting at /home/gregory/.julia/v0.7/Vinyl/src/interpret.jl:1
in expression starting at /home/gregory/.julia/v0.7/Vinyl/src/Vinyl.jl:6
in expression starting at /home/gregory/.julia/v0.7/Traceur/src/Traceur.jl:4
in expression starting at /home/gregory/workspace/StructDOM.jl/test/benchmark.jl:4

I’ve recompiled julia with latest commit, and updated all my packages, it makes no differences.

with julia 0.6 (pkg updated) I also have an error:

(Core.kwfunc)(::#Pattern) at boot.jl:237
  returns Any
ERROR: LoadError: UndefRefError: access to undefined reference
Stacktrace:
 [1] lookup_var(::ASTInterpreter2.JuliaStackFrame, ::SSAValue) at /home/gregory/.julia/v0.6/ASTInterpreter2/src/interpret.jl:3
 [2] lookup_var_if_var(::ASTInterpreter2.JuliaStackFrame, ::SSAValue) at /home/gregory/.julia/v0.6/ASTInterpreter2/src/ASTInterpreter2.jl:147
 [3] lookup(::ASTInterpreter2.JuliaStackFrame, ::SSAValue) at /home/gregory/.julia/v0.6/Vinyl/src/interpret.jl:21
 [4] broadcast_t(::Function, ::Type{Any}, ::Tuple{Base.OneTo{Int64}}, ::CartesianRange{CartesianIndex{1}}, ::ASTInterpreter2.JuliaStackFrame, ::Array{Any,1}) at ./broadcast.jl:256
 [5] broadcast_c(::Function, ::Type{Array}, ::ASTInterpreter2.JuliaStackFrame, ::Array{Any,1}, ::Vararg{Array{Any,1},N} where N) at ./broadcast.jl:319
 [6] broadcast(::Function, ::ASTInterpreter2.JuliaStackFrame, ::Array{Any,1}) at ./broadcast.jl:434
 [7] callargs(::DebuggerFramework.DebuggerState) at /home/gregory/.julia/v0.6/Vinyl/src/interpret.jl:29
 [8] runall(::Traceur.Trace, ::DebuggerFramework.DebuggerState) at /home/gregory/.julia/v0.6/Vinyl/src/interpret.jl:34
 [9] overdub(::Traceur.Trace, ::Function, ::Array{Any,1}, ::Vararg{Any,N} where N) at /home/gregory/.julia/v0.6/Vinyl/src/interpret.jl:53
 [10] macro expansion at /home/gregory/.julia/v0.6/Traceur/src/trace.jl:18 [inlined]
 [11] primitive(::Traceur.Trace, ::Function, ::Array{Any,1}, ::Vararg{Any,N} where N) at /home/gregory/.julia/v0.6/Vinyl/src/hooks.jl:24
 [12] runall(::Traceur.Trace, ::DebuggerFramework.DebuggerState) at /home/gregory/.julia/v0.6/Vinyl/src/interpret.jl:37
 [13] overdub(::Traceur.Trace, ::Function, ::String, ::Vararg{String,N} where N) at /home/gregory/.julia/v0.6/Vinyl/src/interpret.jl:53
 [14] macro expansion at /home/gregory/.julia/v0.6/Traceur/src/trace.jl:18 [inlined]
 [15] primitive(::Traceur.Trace, ::Function, ::String, ::Vararg{String,N} where N) at /home/gregory/.julia/v0.6/Vinyl/src/hooks.jl:24
 [16] runall(::Traceur.Trace, ::DebuggerFramework.DebuggerState) at /home/gregory/.julia/v0.6/Vinyl/src/interpret.jl:37
 [17] overdub(::Traceur.Trace, ::Function) at /home/gregory/.julia/v0.6/Vinyl/src/interpret.jl:53
 [18] trace(::Function, ::Function) at /home/gregory/.julia/v0.6/Traceur/src/trace.jl:23
 [19] warntrace(::Function) at /home/gregory/.julia/v0.6/Traceur/src/trace.jl:36
while loading /home/gregory/workspace/StructDOM.jl/test/benchmark.jl, in expression starting on line 27
15

Does @trace not conflict with the macro of the same name in TraceCalls.jl?

1 Like
16

Hi

I also get an error during installation:

julia> using Pkg

julia> Pkg.update()
  Updating registry at `~/.julia/registries/General`
  Updating git-repo `https://github.com/JuliaRegistries/General.git`
 Resolving package versions...
  Updating `~/.julia/environments/v1.0/Project.toml`
 [no changes]
  Updating `~/.julia/environments/v1.0/Manifest.toml`
 [no changes]

julia> Pkg.add("Traceur")
 Resolving package versions...
 Installed Vinyl ─── v0.1.1
 Installed Traceur ─ v0.1.1
  Updating `~/.julia/environments/v1.0/Project.toml`
  [37b6cedf] + Traceur v0.1.1
  Updating `~/.julia/environments/v1.0/Manifest.toml`
  [37b6cedf] + Traceur v0.1.1
  [83e47e11] + Vinyl v0.1.1

julia> using Traceur
[ Info: Precompiling Traceur [37b6cedf-1f77-55f8-9503-c64b63398394]
ERROR: LoadError: UndefVarError: Nullable not defined
Stacktrace:
 [1] top-level scope at none:0
 [2] include at ./boot.jl:317 [inlined]
 [3] include_relative(::Module, ::String) at ./loading.jl:1038
 [4] include(::Module, ::String) at ./sysimg.jl:29
 [5] top-level scope at none:2
 [6] eval at ./boot.jl:319 [inlined]
 [7] eval(::Expr) at ./client.jl:389
 [8] top-level scope at ./none:3
in expression starting at /Users/ps/.julia/packages/DebuggerFramework/9yZZM/src/DebuggerFramework.jl:13
ERROR: LoadError: Failed to precompile DebuggerFramework [67417a49-6d77-5db2-98c7-c13144130cd2] to /Users/ps/.julia/compiled/v1.0/DebuggerFramework/0bKxs.ji.
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] macro expansion at ./logging.jl:313 [inlined]
 [3] compilecache(::Base.PkgId, ::String) at ./loading.jl:1184
 [4] _require(::Base.PkgId) at ./logging.jl:311
 [5] require(::Base.PkgId) at ./loading.jl:852
 [6] macro expansion at ./logging.jl:311 [inlined]
 [7] require(::Module, ::Symbol) at ./loading.jl:834
 [8] include at ./boot.jl:317 [inlined]
 [9] include_relative(::Module, ::String) at ./loading.jl:1038
 [10] include(::Module, ::String) at ./sysimg.jl:29
 [11] top-level scope at none:2
 [12] eval at ./boot.jl:319 [inlined]
 [13] eval(::Expr) at ./client.jl:389
 [14] top-level scope at ./none:3
in expression starting at /Users/ps/.julia/packages/ASTInterpreter2/b1fAq/src/ASTInterpreter2.jl:4
ERROR: LoadError: LoadError: Failed to precompile ASTInterpreter2 [e6d88f4b-b52a-544c-a8d3-7a4f12cb39c3] to /Users/ps/.julia/compiled/v1.0/ASTInterpreter2/XrlMa.ji.
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] macro expansion at ./logging.jl:313 [inlined]
 [3] compilecache(::Base.PkgId, ::String) at ./loading.jl:1184
 [4] _require(::Base.PkgId) at ./logging.jl:311
 [5] require(::Base.PkgId) at ./loading.jl:852
 [6] macro expansion at ./logging.jl:311 [inlined]
 [7] require(::Module, ::Symbol) at ./loading.jl:834
 [8] include at ./boot.jl:317 [inlined]
 [9] include_relative(::Module, ::String) at ./loading.jl:1038
 [10] include at ./sysimg.jl:29 [inlined]
 [11] include(::String) at /Users/ps/.julia/packages/Vinyl/OVtz4/src/Vinyl.jl:1
 [12] top-level scope at none:0
 [13] include at ./boot.jl:317 [inlined]
 [14] include_relative(::Module, ::String) at ./loading.jl:1038
 [15] include(::Module, ::String) at ./sysimg.jl:29
 [16] top-level scope at none:2
 [17] eval at ./boot.jl:319 [inlined]
 [18] eval(::Expr) at ./client.jl:389
 [19] top-level scope at ./none:3
in expression starting at /Users/ps/.julia/packages/Vinyl/OVtz4/src/interpret.jl:1
in expression starting at /Users/ps/.julia/packages/Vinyl/OVtz4/src/Vinyl.jl:6
ERROR: LoadError: Failed to precompile Vinyl [83e47e11-4a7a-5bf8-892c-11f81bf9b223] to /Users/ps/.julia/compiled/v1.0/Vinyl/5xrHi.ji.
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] macro expansion at ./logging.jl:313 [inlined]
 [3] compilecache(::Base.PkgId, ::String) at ./loading.jl:1184
 [4] _require(::Base.PkgId) at ./logging.jl:311
 [5] require(::Base.PkgId) at ./loading.jl:852
 [6] macro expansion at ./logging.jl:311 [inlined]
 [7] require(::Module, ::Symbol) at ./loading.jl:834
 [8] include at ./boot.jl:317 [inlined]
 [9] include_relative(::Module, ::String) at ./loading.jl:1038
 [10] include(::Module, ::String) at ./sysimg.jl:29
 [11] top-level scope at none:2
 [12] eval at ./boot.jl:319 [inlined]
 [13] eval(::Expr) at ./client.jl:389
 [14] top-level scope at ./none:3
in expression starting at /Users/ps/.julia/packages/Traceur/JHUAr/src/Traceur.jl:4
ERROR: Failed to precompile Traceur [37b6cedf-1f77-55f8-9503-c64b63398394] to /Users/ps/.julia/compiled/v1.0/Traceur/QT21m.ji.
Stacktrace:
 [1] macro expansion at ./logging.jl:313 [inlined]
 [2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1184
 [3] macro expansion at ./logging.jl:311 [inlined]
 [4] _require(::Base.PkgId) at ./loading.jl:941
 [5] require(::Base.PkgId) at ./loading.jl:852
 [6] macro expansion at ./logging.jl:311 [inlined]
 [7] require(::Module, ::Symbol) at ./loading.jl:834
 [8] #29 at ./boot.jl:319 [inlined]
 [9] with_logstate(::getfield(Main, Symbol("##29#31")), ::Base.CoreLogging.LogState) at ./logging.jl:397

With best regards
Plamen

17

Traceur.jl isn’t yet supported on Julia v1.0, but it looks like it will be soon: Julia 1.0 compatability · Issue #13 · JunoLab/Traceur.jl · GitHub

1 Like
18

Thanks to @pfitzseb this now works on 1.0 – Juno integration coming shortly!

14 Likes
19

Was the Juno integration finished? I can’t find anything in the Juno docs.

20

Yeah, mostly, but Traceur has bitrotted a bit:
grafik

3 Likes