# Continuous provision of current logs

**URL:** <https://discourse.julialang.org/t/continuous-provision-of-current-logs/93143>\
**Category:** General Usage\
**Created:** [January 18, 2023, 9:35am UTC](https://discourse.julialang.org/t/continuous-provision-of-current-logs/93143 "2023-01-18T09:35:02Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![max](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/max/32/31135_2.png) [@max](https://discourse.julialang.org/u/max)\
**Post date:** [January 18, 2023, 9:35am UTC](https://discourse.julialang.org/t/continuous-provision-of-current-logs/93143/1 "2023-01-18T09:35:02Z")

</div>

Hello all,

I want to continuously provide the logs in a cloud instance that are the output in a long running processes (optimization problems). For this I tried to run asynchronously and also on a separate thread the function that drives these continuous updates. During this update I upload the file to a GCP bucket to access the log from another resource.

Here is a code example:

```julia
using Logging

io = open("/home/logs.txt", "w+")
logger = SimpleLogger(io)
global_logger(logger)

function run_logger!(io::IOStream)
    counter = 0
    while isopen(io)
        # flush the io stream to the file
        @info("Updated $counter times on thread $(Threads.threadid())")
        flush(io)
        ###
        # uploading to gcp
        ###
        # only updates every 5 second
        sleep(5)
        counter += 1
    end
    @info("Parallel Logger closed.")
    nothing
end

# async test
@async run_logger!(io)

# multithreaded test
Threads.@spawn run_logger!(io)

# async and multithreaded test
Threads.@spawn @async run_logger!(io)

```

If I now run the instance, the log is updated 2 times in parallel at the beginning. After that it will be updated again after all processes are finished via another function. Also the while loop is not closed during this time. Can anyone tell me what the problem is or how I manage to provide the logs continuously?

---

<div class="post-metadata">

**Author:** ![fredrikekre](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/fredrikekre/32/1688_2.png) [@fredrikekre](https://discourse.julialang.org/u/fredrikekre)\
**Post date:** [January 18, 2023, 10:32am UTC](https://discourse.julialang.org/t/continuous-provision-of-current-logs/93143/2 "2023-01-18T10:32:00Z")

</div>

I don’t really understand what you are trying to examplify with the `@spawn` and `@async`, but perhaps all of that is moot if you simply use a [`FileLogger`](https://julialogging.github.io/reference/loggingextras/#LoggingExtras.FileLogger-Tuple%7BAny%7D) that automatically flushes every message.

```julia
julia> Threads.nthreads()
4

julia> using Logging, LoggingExtras

julia> logger = FileLogger("logs.txt");

julia> global_logger(logger);

julia> @sync for i in 1:4
           Threads.@spawn @info "Hello from thread $(Threads.threadid())"
       end

shell> cat logs.txt
┌ Info: Hello from thread 1
└ @ Main REPL[5]:2
┌ Info: Hello from thread 3
└ @ Main REPL[5]:2
┌ Info: Hello from thread 4
└ @ Main REPL[5]:2
┌ Info: Hello from thread 2
└ @ Main REPL[5]:2

```

---

<div class="post-metadata">

**Author:** ![max](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/max/32/31135_2.png) [@max](https://discourse.julialang.org/u/max)\
**Post date:** [January 18, 2023, 11:09am UTC](https://discourse.julialang.org/t/continuous-provision-of-current-logs/93143/3 "2023-01-18T11:09:38Z")

</div>

Thanks for the reply @fredrikekre .  
Sadly this does not solve my issue. What I didn’t mention in my request is that I’m also uploading this file to access it from another resource. This currently happens right after the ‘flush(io)’ statement.

I update my initial post, so this will become more clear.

---

<div class="post-metadata">

**Author:** ![fredrikekre](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/fredrikekre/32/1688_2.png) [@fredrikekre](https://discourse.julialang.org/u/fredrikekre)\
**Post date:** [January 18, 2023, 11:37am UTC](https://discourse.julialang.org/t/continuous-provision-of-current-logs/93143/4 "2023-01-18T11:37:28Z")

</div>

Okay, I don’t really understand what you are trying to do then. Can you come up with a self-contained example which demonstrates the issue? E.g. instead of uploading the log file you copy it to disk or something like that.

---

<div class="post-metadata">

**Author:** ![max](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/max/32/31135_2.png) [@max](https://discourse.julialang.org/u/max)\
**Post date:** [January 19, 2023, 6:21pm UTC](https://discourse.julialang.org/t/continuous-provision-of-current-logs/93143/5 "2023-01-19T18:21:38Z")

</div>

Here is an example with copying the file.

```julia
using Logging

io = open("/home/logs.txt", "w+")
logger = SimpleLogger(io)
global_logger(logger)

function copy_log()
    f = open("/home/online_log.txt", "w")
    write(f, read("/home/logs.txt", String))
    close(f)
end

function run_logger!(io::IOStream)
    counter = 0
    while isopen(io)
        # flush the io stream to the file
        @info("Updated $counter times on thread $(Threads.threadid())")
        flush(io)
        ###
        copy_log()
        ###
        # only updates every 5 second
        sleep(5)
        counter += 1
    end
    @info("Parallel Logger closed.")
    nothing
end

# async test
@async run_logger!(io)

# multithreaded test
Threads.@spawn run_logger!(io)

# async and multithreaded test
Threads.@spawn @async run_logger!(io)

```

---

<div class="post-metadata">

**Author:** ![max](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/max/32/31135_2.png) [@max](https://discourse.julialang.org/u/max)\
**Post date:** [January 19, 2023, 6:38pm UTC](https://discourse.julialang.org/t/continuous-provision-of-current-logs/93143/6 "2023-01-19T18:38:43Z")

</div>

Since the while loop is not closed in the instance I assume it might be an similar issue to this one:

> [@Work on main thread delays spawned task](https://discourse.julialang.org/t/work-on-main-thread-delays-spawned-task/75297):
>
> Consider the following example: function ex1() task = @Threads.spawn begin st = time() sleep(1.0) return time() - st end st2 = time() while time() - st2 \< 5 end println("task slept for $(fetch(task)) seconds") end When I start Julia 1.7.1 with julia -t2 and run ex1(), the result is “task slept for 5.0 seconds”. With the following code, the result is “task slept for 1 seconds”: function ex2() task = @Threads.spawn begin st = time() …