# Using the Intel VTune Profiler with julia

**URL:** <https://discourse.julialang.org/t/using-the-intel-vtune-profiler-with-julia/34327>\
**Category:** Tooling\
**Tags:** profiling\
**Created:** [February 7, 2020, 9:55pm UTC](https://discourse.julialang.org/t/using-the-intel-vtune-profiler-with-julia/34327 "2020-02-07T21:55:14Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![carstenbauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carstenbauer/32/4981_2.png) [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)\
**Post date:** [February 7, 2020, 9:55pm UTC](https://discourse.julialang.org/t/using-the-intel-vtune-profiler-with-julia/34327/1 "2020-02-07T21:55:14Z")

</div>

Hi all,

I’m trying to profile the following piece of Julia code, taken from [ProfileView.jl](https://github.com/timholy/ProfileView.jl), using the external [Intel VTune Profiler](https://software.intel.com/en-us/vtune). Concretely, the code is

```julia
# vtune.jl
function profile_test(n)
    for i = 1:n
        A = randn(100,100,20)
        m = maximum(A)
        Am = mapslices(sum, A; dims=2)
        B = A[:,:,5]
        Bsort = mapslices(sort, B; dims=1)
        b = rand(100)
        C = B.*b
    end
end

profile_test(1)
profile_test(10)

```

As described in the relevant section of the Julia documentation, I compiled Julia (release-1.3) with `USE_INTEL_JITEVENTS=1` in the Make.user file and set the environment variable `ENABLE_JITPROFILING=1`. Specifically, since I’m on Windows I cross compiled Julia using the recommended Cygwin-to-MinGW path.

To profile the code I fired up VTune and created a simple analysis (inspired by the Python tutorial [here](https://software.intel.com/en-us/vtune-help-python-code-analysis)):

 ![image](https://global.discourse-cdn.com/julialang/original/3X/3/b/3bd9eb7e9929dc30780f068137d8782cdaaefc50.png)

Inspecting the “Bottom-up” analysis results I see all kinds of low level `jl_*` stuff but cannot find anything about my vtune.jl file or the included `profile_test` function:

 ![image](https://global.discourse-cdn.com/julialang/original/3X/5/1/5185791fb1423998cf8095e215751a972dc06bd9.png)

Is there something else I need to do? Is VTune expected to work with Julia? If yes, to what extend? In the python tutorial (linked above) both the script file `runtime.py` as well as the function `black_scholes` could be clearly identified as a bottleneck.

I’d appreciate any comments/hints on how to get better profiling results with VTune + Julia!

Best,  
Carsten

---

<div class="post-metadata">

**Author:** ![carstenbauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carstenbauer/32/4981_2.png) [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)\
**Post date:** [February 8, 2020, 12:22pm UTC](https://discourse.julialang.org/t/using-the-intel-vtune-profiler-with-julia/34327/2 "2020-02-08T12:22:09Z")

</div>

Short update, I did the same experiment on a linux machine with similar outcome: no mentioning of my file or function as far as I can see.

---

<div class="post-metadata">

**Author:** ![christianl](https://avatars.discourse-cdn.com/v4/letter/c/e274bd/32.png) [@christianl](https://discourse.julialang.org/u/christianl)\
**Post date:** [March 9, 2020, 2:51pm UTC](https://discourse.julialang.org/t/using-the-intel-vtune-profiler-with-julia/34327/3 "2020-03-09T14:51:47Z")

</div>

Any advances on this topic?

I am experiencing the same issue. Compiled v1.3.1. with `USE_INTEL_JITEVENTS:=1` in `Make.user` and running it in vtune profiler 2020 with

```julia
Application: /usr/bin/env
Application Parameters: ENABLE_JITPROFILING=1 /usr/bin/julia intelitttest.jl

```

I am using the “instrumentation and tracing technology (ITT) API” to profile just the `f01()` function in

```julia
include(joinpath(@ __DIR__ ,"IntelITT.jl"))
using Main.IntelITT

N = 256
A = rand(UInt32,1024*1024*N) .% UInt32(17)

function f01()
  B = cumsum(A)
end

precompile(f01,())

__itt_resume()

f01()

__itt_detach()

```

But vtune is not able to show the stack

 ![Auswahl_168](https://global.discourse-cdn.com/julialang/original/3X/4/2/4243694dcd554b7508660faff11635e310dceb19.png)

---

<div class="post-metadata">

**Author:** ![christianl](https://avatars.discourse-cdn.com/v4/letter/c/e274bd/32.png) [@christianl](https://discourse.julialang.org/u/christianl)\
**Post date:** [March 11, 2020, 11:20am UTC](https://discourse.julialang.org/t/using-the-intel-vtune-profiler-with-julia/34327/4 "2020-03-11T11:20:20Z")

</div>

got it working now

 ![Auswahl_169](https://global.discourse-cdn.com/julialang/original/3X/b/5/b5bf63d6244b2c5a4e43890e462aaafb0ab81c33.png)

the issue was, that the precompiled binary of LLVM is compiled without LLVM\_USE\_INTEL\_JITEVENTS

I found a log file `logs/LLVM.log` documenting this

```julia
CMake Warning:
  Manually-specified variables were not used by the project:

    USE_INTEL_JITEVENTS
    USE_OPROFILE
    USE_PERF

```

so maybe this is a bug in the provided binary.

Setting `USE_BINARYBUILDER=0` (and also `USE_SYSTEM_LLVM=0` which might be the default anyways) results in a recompilation of LLVM (takes some time).

---

<div class="post-metadata">

**Author:** ![carstenbauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carstenbauer/32/4981_2.png) [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)\
**Post date:** [March 11, 2020, 3:05pm UTC](https://discourse.julialang.org/t/using-the-intel-vtune-profiler-with-julia/34327/5 "2020-03-11T15:05:52Z")

</div>

Thanks for sharing this. Will try it on Windows later today.

> [@christianl](#):
>
> I am using the “instrumentation and tracing technology (ITT) API” to profile just the `f01()` function in

Would you mind explaining what the “IntelITT.jl” business is about?

---

<div class="post-metadata">

**Author:** ![carstenbauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carstenbauer/32/4981_2.png) [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)\
**Post date:** [March 11, 2020, 3:10pm UTC](https://discourse.julialang.org/t/using-the-intel-vtune-profiler-with-julia/34327/6 "2020-03-11T15:10:48Z")

</div>

> [@christianl](#):
>
> so maybe this is a bug in the provided binary.

If not than it should definitely be mentioned in the docs. Mind opening an issue over on github for discussing this?

---

<div class="post-metadata">

**Author:** ![christianl](https://avatars.discourse-cdn.com/v4/letter/c/e274bd/32.png) [@christianl](https://discourse.julialang.org/u/christianl)\
**Post date:** [March 11, 2020, 4:02pm UTC](https://discourse.julialang.org/t/using-the-intel-vtune-profiler-with-julia/34327/7 "2020-03-11T16:02:33Z")

</div>

There seems to be a library for something called [Instrumentation and Tracing Technology APIs](https://software.intel.com/en-us/vtune-help-instrumentation-and-tracing-technology-apis) (ITT) which gives the ability to pass debug information from an application to the Vtune profiler.

- you can start an application with vtune in “pause” mode
- and let the application call `__itt_resume()` which makes the profiler start to collect data
- you might pass string-annotations for what-is-going-on to the API which can be used in the profiler to group events in an analysis later-on
- The application can call `__itt_detach()` which tells vtune to finish the data collection

I just found out about this myself, tried to hack a Julia binding for the `libittnotify` library and [uploaded it for you on github](https://github.com/mchristianl/IntelITT.jl). Haven’t tested anything more specific, but it seems to work at least for this simple case.

---

<div class="post-metadata">

**Author:** ![christianl](https://avatars.discourse-cdn.com/v4/letter/c/e274bd/32.png) [@christianl](https://discourse.julialang.org/u/christianl)\
**Post date:** [March 11, 2020, 4:07pm UTC](https://discourse.julialang.org/t/using-the-intel-vtune-profiler-with-julia/34327/8 "2020-03-11T16:07:40Z")

</div>

Of course 🙂

… Although I am not quite sure where this bug report would belong to. Is it the [LLVMBuilder](https://github.com/staticfloat/LLVMBuilder) repository or the [BinaryBuilder.jl](https://github.com/JuliaPackaging/BinaryBuilder.jl) repository or the [julia](https://github.com/JuliaLang/julia) repository?

Do you have any advice which one to choose?

EDIT: Further, as far as I understood, this feature of LLVM is based on the [JIT Profiling API](https://software.intel.com/en-us/vtune-help-jit-profiling-api) and unfortunately the disassembly seems broken. I can open the disassembly section in vtune but it displays the same single assembly instruction in each line. This might be a bug in [LLVM](https://github.com/llvm/llvm-project) not reporting the assembly correctly; or it is not even intended to work; or it is a misconfiguration on my side. So this needs some further observation.

---

<div class="post-metadata">

**Author:** ![carstenbauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carstenbauer/32/4981_2.png) [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)\
**Post date:** [March 11, 2020, 5:57pm UTC](https://discourse.julialang.org/t/using-the-intel-vtune-profiler-with-julia/34327/9 "2020-03-11T17:57:02Z")

</div>

> [@christianl](#):
>
> Although I am not quite sure where this bug report would belong to. Is it the [LLVMBuilder](https://github.com/staticfloat/LLVMBuilder) repository or the [BinaryBuilder.jl](https://github.com/JuliaPackaging/BinaryBuilder.jl) repository or the [julia](https://github.com/JuliaLang/julia) repository?

I’m not sure. I’d go with JuliaLang/julia for now.

---

<div class="post-metadata">

**Author:** ![carstenbauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carstenbauer/32/4981_2.png) [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)\
**Post date:** [March 11, 2020, 10:00pm UTC](https://discourse.julialang.org/t/using-the-intel-vtune-profiler-with-julia/34327/10 "2020-03-11T22:00:34Z")

</div>

@giordano Maybe you can point us to the right repo here?

---

<div class="post-metadata">

**Author:** ![giordano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giordano/32/2166_2.png) [@giordano](https://discourse.julialang.org/u/giordano)\
**Post date:** [March 11, 2020, 10:29pm UTC](https://discourse.julialang.org/t/using-the-intel-vtune-profiler-with-julia/34327/11 "2020-03-11T22:29:36Z")

</div>

LLVM nowadays is built in [Yggdrasil](https://github.com/JuliaPackaging/Yggdrasil/)

---

<div class="post-metadata">

**Author:** ![carstenbauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carstenbauer/32/4981_2.png) [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)\
**Post date:** [March 11, 2020, 10:53pm UTC](https://discourse.julialang.org/t/using-the-intel-vtune-profiler-with-julia/34327/12 "2020-03-11T22:53:42Z")

</div>

Unfortunately building with `USE_BINARYBUILDER=0` in cygwin on Windows fails for me, so I can’t get it to work. I’m wondering whether compiling LLVM with `LLVM_USE_INTEL_JITEVENTS` would be safe as a default (does it come with a performance penalty or something similar?)

I [opened an issue](https://github.com/JuliaPackaging/Yggdrasil/issues/616) to discuss whether it makes sense to set the flag for the prebuilt binaries.

---

<div class="post-metadata">

**Author:** ![christianl](https://avatars.discourse-cdn.com/v4/letter/c/e274bd/32.png) [@christianl](https://discourse.julialang.org/u/christianl)\
**Post date:** [March 12, 2020, 9:01am UTC](https://discourse.julialang.org/t/using-the-intel-vtune-profiler-with-julia/34327/13 "2020-03-12T09:01:26Z")

</div>

I think this already was the intention, but fails due to a typo of using `USE_INTEL_JITEVENTS` instead of `LLVM_USE_INTEL_JITEVENTS` (which is why the pre-build-cmake log contains this warning)

---

<div class="post-metadata">

**Author:** ![johnh](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/johnh/32/3615_2.png) [@johnh](https://discourse.julialang.org/u/johnh)\
**Post date:** [March 12, 2020, 11:08am UTC](https://discourse.julialang.org/t/using-the-intel-vtune-profiler-with-julia/34327/14 "2020-03-12T11:08:32Z")

</div>

Not adding anything to the discussion. Using Intel VTune with Julia is impressive! Its something I though about recently. I hope to get hands on with some bigger systems soon so could help with this.

Also Yggdrasil was the original Linux distribution on floppy disks. I thought it was long dead. Is the Julia builder related in some way?

---

<div class="post-metadata">

**Author:** ![giordano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giordano/32/2166_2.png) [@giordano](https://discourse.julialang.org/u/giordano)\
**Post date:** [March 12, 2020, 11:16am UTC](https://discourse.julialang.org/t/using-the-intel-vtune-profiler-with-julia/34327/15 "2020-03-12T11:16:11Z")

</div>

> [@johnh](#):
>
> I thought it was long dead. Is the Julia builder related in some way?

No

---

<div class="post-metadata">

**Author:** ![pengtu](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pengtu/32/19411_2.png) [@pengtu](https://discourse.julialang.org/u/pengtu)\
**Post date:** [August 16, 2022, 4:48pm UTC](https://discourse.julialang.org/t/using-the-intel-vtune-profiler-with-julia/34327/16 "2022-08-16T16:48:43Z")

</div>

With the latest VTune, I also need to set the following environment variable to see the JITed codes showing up in the profile: INTEL\_JIT\_BACKWARD\_COMPATIBILITY=1

---

<div class="post-metadata">

**Author:** ![Amro](https://avatars.discourse-cdn.com/v4/letter/a/4491bb/32.png) [@Amro](https://discourse.julialang.org/u/Amro)\
**Post date:** [October 3, 2022, 7:13pm UTC](https://discourse.julialang.org/t/using-the-intel-vtune-profiler-with-julia/34327/17 "2022-10-03T19:13:06Z")

</div>

I tried to work on Intel VTune profiler with, but I did not know how to connect it to Julia (run via VS code).  
Any step-by-step documentation for it? or could anyone help me in this please ?  
Thank you very much!

---

<div class="post-metadata">

**Author:** ![vchuravy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/vchuravy/32/8_2.png) [@vchuravy](https://discourse.julialang.org/u/vchuravy)\
**Post date:** [November 16, 2022, 6:27pm UTC](https://discourse.julialang.org/t/using-the-intel-vtune-profiler-with-julia/34327/18 "2022-11-16T18:27:47Z")

</div>

Played around this morning with it.

```julia
ENABLE_JITPROFILING=1
INTEL_JIT_BACKWARD_COMPATIBILITY=0

```

I see:

 ![image](https://global.discourse-cdn.com/julialang/original/3X/3/5/35b9ef1f883256f46b122a17a56b0e75d17edeb3.jpeg)

With

```julia
ENABLE_JITPROFILING=1
INTEL_JIT_BACKWARD_COMPATIBILITY=1

```

I see:

 ![image](https://global.discourse-cdn.com/julialang/original/3X/8/8/8844ce2b1062b008583f16f4a63d79041d0d7af7.jpeg)

This is on Intel VTune 2022 and [Julia 1.9](https://github.com/JuliaLang/julia/pull/47586)

So the biggest annoyance is that it doesn’t automatically find the source file.

---

<div class="post-metadata">

**Author:** ![vchuravy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/vchuravy/32/8_2.png) [@vchuravy](https://discourse.julialang.org/u/vchuravy)\
**Post date:** [November 16, 2022, 6:32pm UTC](https://discourse.julialang.org/t/using-the-intel-vtune-profiler-with-julia/34327/19 "2022-11-16T18:32:02Z")

</div>

Would be interesting to see if [GitHub - intel/ittapi: Intel® Instrumentation and Tracing Technology (ITT) and Just-In-Time (JIT) API](https://github.com/intel/ittapi) is enough and we could use Yggdrasil to provide the libs instead of having to pick them up from VTune in the environment.

---

<div class="post-metadata">

**Author:** ![gbaraldi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gbaraldi/32/22101_2.png) [@gbaraldi](https://discourse.julialang.org/u/gbaraldi)\
**Post date:** [November 16, 2022, 6:51pm UTC](https://discourse.julialang.org/t/using-the-intel-vtune-profiler-with-julia/34327/20 "2022-11-16T18:51:14Z")

</div>

Did you run a julia script from the GUI or the CLI?

[Next page](https://discourse.julialang.org/t/using-the-intel-vtune-profiler-with-julia/34327.md?page=2)