# Has PGO for (re)compiling methods been considered?

**URL:** <https://discourse.julialang.org/t/has-pgo-for-re-compiling-methods-been-considered/97426>\
**Category:** Internals & Design\
**Created:** [April 13, 2023, 11:36am UTC](https://discourse.julialang.org/t/has-pgo-for-re-compiling-methods-been-considered/97426 "2023-04-13T11:36:09Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![nsajko](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nsajko/32/221187_2.png) [@nsajko](https://discourse.julialang.org/u/nsajko)\
**Post date:** [April 13, 2023, 11:36am UTC](https://discourse.julialang.org/t/has-pgo-for-re-compiling-methods-been-considered/97426/1 "2023-04-13T11:36:09Z")

</div>

It just crossed my mind that it would be very nice if Julia provided some kind of profile-guided optimization (PGO) API.

Something like:

1. The user gathers a performance profile for some of their Julia code

2. The user is now able to request a recompilation of their Julia code, feeding the profile to the Julia compiler to enable better optimizations, like, for example, [control height reduction](https://reviews.llvm.org/D50591) or [hot-cold splitting](https://groups.google.com/g/llvm-dev/c/RUegaMg-iqc/m/wFAVxa6fCgAJ).

Step 2 could look like this:

```julia
pgo_anonymous_function = recompile_with_pgo(my_function, (ArgType1, ArgType2, ...), my_profile)

# Now `pgo_anonymous_function` may be used as a more performant version of `my_function`!

```

The nice thing (I think) is that LLVM already supports PGO, so I guess this would mostly be a matter of expanding the profiling functionality and plugging into LLVM, and then just letting LLVM take care of the optimization?

Has this been considered already? Is it really such a low-hanging fruit like it seems? Perhaps this could even be implemented as a package outside the standard library?

---

<div class="post-metadata">

**Author:** ![ffevotte](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ffevotte/32/6587_2.png) [@ffevotte](https://discourse.julialang.org/u/ffevotte)\
**Post date:** [April 13, 2023, 12:11pm UTC](https://discourse.julialang.org/t/has-pgo-for-re-compiling-methods-been-considered/97426/2 "2023-04-13T12:11:48Z")

</div>

> [@nsajko](#):
>
> Perhaps this could even be implemented as a package outside the standard library?

Do you know about [`Catwalk.jl`](https://github.com/tisztamo/Catwalk.jl)? It targets another type of optimizations than what you mention (namely, it optimizes dynamic dispatch) and is currently designed more as a JIT kind of thing, but I think the approach it takes could very well be reused in a PGO context.

---

<div class="post-metadata">

**Author:** ![giordano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giordano/32/2166_2.png) [@giordano](https://discourse.julialang.org/u/giordano)\
**Post date:** [April 13, 2023, 12:13pm UTC](https://discourse.julialang.org/t/has-pgo-for-re-compiling-methods-been-considered/97426/3 "2023-04-13T12:13:37Z")

</div>

> <https://github.com/JuliaLang/julia/pull/45641>
>
> Adds a convenient way to enable PGO+LTO on Julia and LLVM together:
> 
> 1. \`cd co…ntrib/pgo-lto\`
> 2. \`make -j$(nproc) stage1\`
> 3. \`make clean-profiles\`
> 4. \`./stage1.build/julia -O3 -e 'using Pkg; Pkg.add("LoopVectorization"); Pkg.test("LoopVectorization")'\`
> 5. \`make -j$(nproc) stage2\`
> 
> \<details\>
> \<summary\>\* Output looks roughly like as follows\</summary\>
> 
> \`\`\`c++
> $ make -C contrib/pgo-lto top 
> make: Entering directory '/dev/shm/julia/contrib/pgo-lto'
> llvm-profdata show --topn=50 /dev/shm/julia/contrib/pgo-lto/profiles/merged.prof | c++filt
> Instrumentation level: IR entry\_first = 0
> Total functions: 85943
> Maximum function count: 7867557260
> Maximum internal block count: 3468437590
> Top 50 functions with the largest internal block counts: 
> llvm::BitVector::operator|=(llvm::BitVector const&), max count = 7867557260
> LateLowerGCFrame::ComputeLiveness(State&), max count = 3468437590
> llvm::hashing::detail::hash\_combine\_recursive\_helper::hash\_combine\_recursive\_helper(), max count = 1742259834
> llvm::SUnit::addPred(llvm::SDep const&, bool), max count = 511396575
> llvm::LiveRange::overlaps(llvm::LiveRange const&, llvm::CoalescerPair const&, llvm::SlotIndexes const&) const, max count = 508061762
> llvm::StringMapImpl::LookupBucketFor(llvm::StringRef), max count = 505682177
> std::map\<llvm::BasicBlock\*, BBState, std::less\<llvm::BasicBlock\*\>, std::allocator\<std::pair\<llvm::BasicBlock\* const, BBState\> \> \>::operator\[\](llvm::BasicBlock\* const&), max count = 395628888
> llvm::LiveRange::advanceTo(llvm::LiveRange::Segment const\*, llvm::SlotIndex) const, max count = 384642728
> llvm::LiveRange::isLiveAtIndexes(llvm::ArrayRef\<llvm::SlotIndex\>) const, max count = 380291040
> llvm::PassRegistry::enumerateWith(llvm::PassRegistrationListener\*), max count = 352313953
> ijl\_method\_instance\_add\_backedge, max count = 349608221
> llvm::SUnit::ComputeHeight(), max count = 336604330
> llvm::LiveRange::advanceTo(llvm::LiveRange::Segment\*, llvm::SlotIndex), max count = 331030109
> llvm::SmallPtrSetImplBase::insert\_imp(void const\*), max count = 272966545
> llvm::LiveIntervals::checkRegMaskInterference(llvm::LiveInterval&, llvm::BitVector&), max count = 257449540
> LateLowerGCFrame::ComputeLiveSets(State&), max count = 252096274
> /dev/shm/julia/src/jltypes.c:has\_free\_typevars, max count = 230879464
> ijl\_get\_pgcstack, max count = 216953592
> LateLowerGCFrame::RefineLiveSet(llvm::BitVector&, State&, std::vector\<int, std::allocator\<int\> \> const&), max count = 188013152
> /dev/shm/julia/src/flisp/flisp.c:apply\_cl, max count = 174863813
> /dev/shm/julia/src/flisp/builtins.c:fl\_memq, max count = 168621603
> \`\`\`
> \</details\>
> 
> 
> This results quite often in spectacular speedups for time to first X as
> it reduces the time spent in LLVM optimization passes by 25 or even 30%.
> 
> Example 1:
> 
> \`\`\`julia
> using LoopVectorization
> function f!(a, b)
> @turbo for i in eachindex(a)
> a\[i\] \*= b\[i\]
> end
> return a
> end
> f!(rand(1), rand(1))
> \`\`\`
> 
> \`\`\`console
> $ time ./julia -O3 lv.jl
> \`\`\`
> 
> Without PGO+LTO: 14.801s
> With PGO+LTO: 11.978s (-19%)
> 
> Example 2:
> 
> \`\`\`console
> $ time ./julia -e 'using Pkg; Pkg.test("Unitful");'
> \`\`\`
> 
> Without PGO+LTO: 1m47.688s
> With PGO+LTO: 1m35.704s (-11%)
> 
> Example 3 (taken from issue #45395, which is almost only LLVM):
> 
> \`\`\`console
> $ JULIA\_LLVM\_ARGS=-time-passes ./julia script-45395.jl
> \`\`\`
> 
> Without PGO+LTO:
> 
> \`\`\`
> ===-------------------------------------------------------------------------===
> ... Pass execution timing report ...
> ===-------------------------------------------------------------------------===
> Total Execution Time: 101.0130 seconds (98.6253 wall clock)
> 
> ---User Time--- --System Time-- --User+System-- ---Wall Time--- --- Name ---
> 53.6961 ( 54.7%) 0.1050 ( 3.8%) 53.8012 ( 53.3%) 53.8045 ( 54.6%) Unroll loops
> 25.5423 ( 26.0%) 0.0072 ( 0.3%) 25.5495 ( 25.3%) 25.5444 ( 25.9%) Global Value Numbering
> 7.1995 ( 7.3%) 0.0526 ( 1.9%) 7.2521 ( 7.2%) 7.2517 ( 7.4%) Induction Variable Simplification
> 6.0541 ( 5.1%) 0.0098 ( 0.3%) 5.0639 ( 5.0%) 5.0561 ( 5.1%) Combine redundant instructions #2
> \`\`\`
> 
> With PGO+LTO:
> 
> \`\`\`
> ===-------------------------------------------------------------------------===
> ... Pass execution timing report ...
> ===-------------------------------------------------------------------------===
> Total Execution Time: 72.6507 seconds (70.1337 wall clock)
> 
> ---User Time--- --System Time-- --User+System-- ---Wall Time--- --- Name ---
> 36.0894 ( 51.7%) 0.0825 ( 2.9%) 36.1719 ( 49.8%) 36.1738 ( 51.6%) Unroll loops
> 16.5713 ( 23.7%) 0.0129 ( 0.5%) 16.5843 ( 22.8%) 16.5794 ( 23.6%) Global Value Numbering
> 5.9047 ( 8.5%) 0.0395 ( 1.4%) 5.9442 ( 8.2%) 5.9438 ( 8.5%) Induction Variable Simplification
> 4.7566 ( 6.8%) 0.0078 ( 0.3%) 4.7645 ( 6.6%) 4.7575 ( 6.8%) Combine redundant instructions #2
> \`\`\`
> 
> Or -28% time spent in LLVM.
> 
> \`perf\` reports show this is mostly fewer instructions and reduction in icache misses.
> 
> \---
> 
> Finally there's a significant reduction in binary sizes. For libLLVM.so:
> 
> \`\`\`
> 79M	usr/lib/libLLVM-13jl.so (before)
> 67M	usr/lib/libLLVM-13jl.so (after)
> \`\`\`
> 
> And it can be reduced by another 2MB with \`--icf=safe\` when using LLD as
> a linker anyways.
> 
> \- \[x\] Two out-of-source builds would be better than a single in-source build, so that it's easier to find good profile data

---

<div class="post-metadata">

**Author:** ![nsajko](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nsajko/32/221187_2.png) [@nsajko](https://discourse.julialang.org/u/nsajko)\
**Post date:** [April 13, 2023, 12:28pm UTC](https://discourse.julialang.org/t/has-pgo-for-re-compiling-methods-been-considered/97426/4 "2023-04-13T12:28:20Z")

</div>

The linked PR is a different topic, it’s about compiling Julia itself (the implementation), while I’m talking about compiling user Julia code.

---

<div class="post-metadata">

**Author:** ![tim.holy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tim.holy/32/52_2.png) [@tim.holy](https://discourse.julialang.org/u/tim.holy)\
**Post date:** [April 13, 2023, 12:42pm UTC](https://discourse.julialang.org/t/has-pgo-for-re-compiling-methods-been-considered/97426/5 "2023-04-13T12:42:40Z")

</div>

There are some tools for a related analysis: [Profile-guided despecialization · SnoopCompile](https://timholy.github.io/SnoopCompile.jl/stable/pgdsgui/)

The aim there, though, is to figure out what _isn’t_ worth optimizing, given that Julia optimizes everything by default.

---

<div class="post-metadata">

**Author:** ![giordano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giordano/32/2166_2.png) [@giordano](https://discourse.julialang.org/u/giordano)\
**Post date:** [April 13, 2023, 12:50pm UTC](https://discourse.julialang.org/t/has-pgo-for-re-compiling-methods-been-considered/97426/6 "2023-04-13T12:50:34Z")

</div>

I mean, that affects the Julia _ **compiler** _…