# Track progress of multiple threads
**URL:** https://discourse.julialang.org/t/track-progress-of-multiple-threads/94921
**Category:** General Usage
**Tags:** question
**Created:** [February 20, 2023, 4:39pm UTC](https://discourse.julialang.org/t/track-progress-of-multiple-threads/94921 "2023-02-20T16:39:31Z")
**Posts on this page:** 18
**Page:** 1
### Author:  [@gitboy16](https://discourse.julialang.org/u/gitboy16)
#### Post date: [February 20, 2023, 4:39pm UTC](https://discourse.julialang.org/t/track-progress-of-multiple-threads/94921/1 "2023-02-20T16:39:31Z")
Hi,
I have the following code
```julia
function pb(i, k, lpad, LEN, PBSTR, PBWID,nb)
perc = i/(LEN);
lpad = floor(Int,perc * LEN);
print("\r Processing data chunk ", k," of $nb : [",first(PBSTR,lpad),first(PBWID,LEN-lpad),"] ",trunc(Int,perc*100),"%");
nothing
end
function progr(;nb::Int=3)
LEN = 40
PBSTR = '━'^LEN
PBWID = ' '^LEN
print("\e[?25l")
for k in 1:nb
for i in 1:LEN
sleep(0.01*nb)
pb(i, k, lpad, LEN, PBSTR, PBWID, nb)
end
print("\n")
end
print("\e[?25h")
end
progr()
```
I would like to get the 3 progress bars to print progress at the time.
Each task would be running in its own thread.
I have tried to look at packages like ProgressMeter but was not able to figure out how I can do it. Please know that I am not a computer scientist and I am relatively new to Julia. I would like to understand how to do it and understand what is happening.
Thank you for your time.
---
### Author:  [@ffevotte](https://discourse.julialang.org/u/ffevotte)
#### Post date: [February 20, 2023, 5:09pm UTC](https://discourse.julialang.org/t/track-progress-of-multiple-threads/94921/2 "2023-02-20T17:09:28Z")
I don’t have much first-hand experience with it myself, but I believe [`Term.jl`](https://fedeclaudi.github.io/Term.jl/) would be a good fit for you. Especially look at its documentation regarding progress bars:
[https://fedeclaudi.github.io/Term.jl/stable/adv/progressbars/](https://fedeclaudi.github.io/Term.jl/stable/adv/progressbars/)
Adapting your example, it could look like this:
```julia
using Term.Progress
function progr(;nb::Int=3)
LEN = 40
pbar = ProgressBar()
jobs = [addjob!(pbar; N=LEN) for _ in 1:nb]
with(pbar) do
for k in 1:nb
for i in 1:LEN
sleep(0.01*k)
update!(jobs[k])
end
end
end
end
progr()
```
* * *
Maybe @FedeClaudi will have more insight about this.
---
### Author:  [@gitboy16](https://discourse.julialang.org/u/gitboy16)
#### Post date: [February 20, 2023, 5:15pm UTC](https://discourse.julialang.org/t/track-progress-of-multiple-threads/94921/3 "2023-02-20T17:15:35Z")
Thanks. Yes probably both Term.jl and ProgressMeter.jl do it. I wanted to understand how they achieve it because when I read the source code, I am just completely lost and don’t understand what is happening.
---
### Author:  [@FedeClaudi](https://discourse.julialang.org/u/FedeClaudi)
#### Post date: [February 20, 2023, 5:43pm UTC](https://discourse.julialang.org/t/track-progress-of-multiple-threads/94921/4 "2023-02-20T17:43:11Z")
@gitboy16 we’ve been discussing this on [GitHub](https://github.com/FedeClaudi/Term.jl/issues/202). Is there in particular you’d like for me to explain about how Term achieves it?
---
### Author:  [@gitboy16](https://discourse.julialang.org/u/gitboy16)
#### Post date: [February 20, 2023, 5:47pm UTC](https://discourse.julialang.org/t/track-progress-of-multiple-threads/94921/5 "2023-02-20T17:47:22Z")
I woukd like to k ow how you manage to print all 3 progress bars as the same time while also tracking the progress of each. In my code above, I only manage to print 1 progress bar at a time.
Thank you.
---
### Author:  [@FedeClaudi](https://discourse.julialang.org/u/FedeClaudi)
#### Post date: [February 20, 2023, 7:52pm UTC](https://discourse.julialang.org/t/track-progress-of-multiple-threads/94921/6 "2023-02-20T19:52:42Z")
The `ProgressBar` type has a filed [`jobs`](https://github.com/FedeClaudi/Term.jl/blob/c0ed8c06ccc1db05d779d40fd79acd2e6dd8a8c7/src/progress.jl#L255) which stores all currently active `ProgressJob`s assigned to the bar. These can be added/removed at any time with `addjob!, removejob!`.
Every `n` milliseconds `render!(::ProgressBar)` gets (see [here](https://github.com/FedeClaudi/Term.jl/blob/c0ed8c06ccc1db05d779d40fd79acd2e6dd8a8c7/src/progress.jl#L476)) called when a progress bar is active and this, in turn, calls `render(::ProgressJob)` for each `job` in `progress.jobs`. Each `job` is then printed to a separate line.
Each `ProgressJob` track’s it’s own progress which gets updated with `update!(::ProgressJob)` and when `render!` is called produces the visualization to display the current progress.
Note that generally `with` spawns a separate thread to handle the progress bar updates (see [here](https://github.com/FedeClaudi/Term.jl/blob/c0ed8c06ccc1db05d779d40fd79acd2e6dd8a8c7/src/progress.jl#L563)) separate from the rest of the code.
So I guess the general idea is to use `struct`s to store the information relative to each task and an overarching `struct` that coordinates the whole thing. Using functions only I don’t think you can achieve what you’re after, or at least not as easily.
Hope this helps, let me know if you have questions!
---
### Author:  [@gitboy16](https://discourse.julialang.org/u/gitboy16)
#### Post date: [February 20, 2023, 10:07pm UTC](https://discourse.julialang.org/t/track-progress-of-multiple-threads/94921/7 "2023-02-20T22:07:31Z")
Thank you @FedeClaudi , I guess I am going to have to dive into the code and try to understand what each line of code does. I feel it is going to be a long and painful process… not knowing whether it is going to be succesful are not …
---
### Author:  [@FedeClaudi](https://discourse.julialang.org/u/FedeClaudi)
#### Post date: [February 20, 2023, 10:12pm UTC](https://discourse.julialang.org/t/track-progress-of-multiple-threads/94921/8 "2023-02-20T22:12:42Z")
I’m sure you can do it! Happy to provide a more in depth explanation.
Are you doing this out of curiousity or with the aim to develop something that Term doesn’t yet offer? Term’s progress bar can do a lot already!
---
### Author:  [@gdalle](https://discourse.julialang.org/u/gdalle)
#### Post date: [February 20, 2023, 10:25pm UTC](https://discourse.julialang.org/t/track-progress-of-multiple-threads/94921/9 "2023-02-20T22:25:49Z")
As for ProgressMeter.jl, the key is to use the `update!` function, which is thread-safe as specified in the readme ([GitHub - timholy/ProgressMeter.jl: Progress meter for long-running computations](https://github.com/timholy/ProgressMeter.jl))
---
### Author:  [@ffevotte](https://discourse.julialang.org/u/ffevotte)
#### Post date: [February 20, 2023, 11:39pm UTC](https://discourse.julialang.org/t/track-progress-of-multiple-threads/94921/10 "2023-02-20T23:39:25Z")
> [@gdalle](#):
>
> As for ProgressMeter.jl, the key is to use the `update!` function
Do you know whether ProgressMeter.jl can display several progress bars at once?
---
### Author:  [@gdalle](https://discourse.julialang.org/u/gdalle)
#### Post date: [February 21, 2023, 7:08am UTC](https://discourse.julialang.org/t/track-progress-of-multiple-threads/94921/11 "2023-02-21T07:08:35Z")
I don’t think it can, what I usually do is update the main progress bar with all threads
---
### Author:  [@gitboy16](https://discourse.julialang.org/u/gitboy16)
#### Post date: [February 21, 2023, 9:27am UTC](https://discourse.julialang.org/t/track-progress-of-multiple-threads/94921/12 "2023-02-21T09:27:04Z")
Thank you for your help.
It is more than curiosity. For me it is about learning how to solve a specific problem which I can maybe later use to solve other problems.
I am not planning to develop a package and if a functionnality is missing then (if I have the knowledge) I would try to add it to an existing package.
---
### Author:  [@MarcMush](https://discourse.julialang.org/u/MarcMush)
#### Post date: [May 31, 2023, 2:43pm UTC](https://discourse.julialang.org/t/track-progress-of-multiple-threads/94921/13 "2023-05-31T14:43:59Z")
I’ve been working on adding this functionnality to [ProgressMeter.jl](https://github.com/timholy/ProgressMeter.jl) but it has been paused for some time
You can try this PR with `]add ProgressMeter#a5c5b2c`, and I would appreciate feedback
>
>
> following my answer in #156 , this adds two types of progressbars that mimic \`Pr…ogress\`:
>
> \`ParallelProgress\` is pretty much the example in the readme and works like that :
> \`\`\`julia
> using Distributed
> addprocs()
> @everywhere using ProgressMeter
> prog = ParallelProgress(10; desc="test ")
> pmap(1:10) do i
> sleep(rand())
> next!(prog)
> return myid()
> end
> \`\`\`
>
> \`MultipleProgress\` spawns multiple progress bars using \`offset\`
> \`\`\`julia
> using Distributed
> addprocs(2)
> @everywhere using ProgressMeter
> p = MultipleProgress(\[Progress(10; desc="task $i ") for i in 1:5\], Progress(50; desc="global "))
> pmap(1:5) do i
> for \_ in 1:10
> sleep(rand())
> next!(p\[i\])
> end
> sleep(0.01)
> myid()
> end
> \`\`\`
> \`\`\`plain
> global 100%|██████████████████████████████████████████████████████████| Time: 0:00:24
> task 4 100%|██████████████████████████████████████████████████████████| Time: 0:00:05
> task 5 100%|██████████████████████████████████████████████████████████| Time: 0:00:05
> \`\`\`
>
> It is also possible to add progressbars from another worker with \`addprogress!\`
>
> \`\`\`julia
> p = MultipleProgress(Progress(10, "tasks done "); count\_finishes=true)
> sleep(0.1)
> pmap(1:10) do i
> L = rand(20:50)
> addprogress!(p\[i\], Progress, L, desc=" task $i ")
> for \_ in 1:L
> sleep(0.05)
> next!(p\[i\])
> end
> end
> \`\`\`
>
> the main progress can be a \`Progress\`, either counting all individual steps, or counting finished tasks. It can also be a \`ProgressUnknown\`
>
> the other progresses can be \`Progress\`, \`UnknownProgress\` or \`ProgressTresh\`
>
> \*\*feel free to try this PR with \`\]add ProgressMeter#a5c5b2c\` and give feedback and suggestions\*\*
> Everything \_should\_ be explained in \`?MultipleProgress\`
>
> closes #9, closes #96, closes #97, closes #125, closes #156, closes #253
---
### Author:  [@gitboy16](https://discourse.julialang.org/u/gitboy16)
#### Post date: [June 1, 2023, 10:26am UTC](https://discourse.julialang.org/t/track-progress-of-multiple-threads/94921/14 "2023-06-01T10:26:57Z")
Thank you @MarcMush, this is great!
Just a few comments.
1/ in your PR, the very first example does not work for me. I am running Julia 1.9. The rest runs fine.
2/ would you have an example using Threads?
Thank you again.
---
### Author:  [@MarcMush](https://discourse.julialang.org/u/MarcMush)
#### Post date: [June 3, 2023, 12:36pm UTC](https://discourse.julialang.org/t/track-progress-of-multiple-threads/94921/15 "2023-06-03T12:36:24Z")
I fixed it, I forgot an `i` in the example
I don’t know much about threads but it should be working fine
Similarly, use `ParallelProgress` if you want only one global progressbar or `MultipleProgress` if each thread should have its own progressbar
---
### Author:  [@MarcMush](https://discourse.julialang.org/u/MarcMush)
#### Post date: [June 3, 2023, 12:54pm UTC](https://discourse.julialang.org/t/track-progress-of-multiple-threads/94921/16 "2023-06-03T12:54:46Z")
Here are 2 examples with `Threads.@threads`:
one global progressmeter:
```julia
@show Threads.nthreads() # 4
using ProgressMeter
p = ParallelProgress(10)
Threads.@threads for i in 1:10
sleep(rand())
next!(p)
end
sleep(0.1)
```
10 tasks of unknown lengths, spawned with `addprogress!`:
```julia
p = MultipleProgress(Progress(10); count_finishes=true)
Threads.@threads for i in 1:10
r = rand(10:100)
addprogress!(p[i], Progress, r; desc="Prog $i ")
for _ in 1:r
sleep(0.02)
next!(p[i])
end
end
sleep(0.1)
```
---
### Author:  [@liuyxpp](https://discourse.julialang.org/u/liuyxpp)
#### Post date: [June 4, 2023, 1:48am UTC](https://discourse.julialang.org/t/track-progress-of-multiple-threads/94921/17 "2023-06-04T01:48:19Z")
Is this PR merged?
---
### Author:  [@MarcMush](https://discourse.julialang.org/u/MarcMush)
#### Post date: [June 4, 2023, 9:49am UTC](https://discourse.julialang.org/t/track-progress-of-multiple-threads/94921/18 "2023-06-04T09:49:22Z")
No, but you can use it with `]add ProgressMeter#a5c5b2c`