Mutating function works in REPL but has no effect in tests

New to Julia
1

Hi folks,

I thought I’d see how far I could get at creating a Prometheus client for Julia, to cut my teeth and learn about package development. This involves juggling stateful objects (metrics) which are mutated in place. I’ve got the very beginnings of a repo here but the stripped-down core idea is below:

mutable struct MetricValue
    value::Float64
    lock::ReentrantLock
end

abstract type Metric end

mutable struct Gauge <: Metric
    name::String
    value::MetricValue
end

function inc(metric::Metric, amount::Real = 1.0)
    @async begin
        lock(metric.value.lock)
        metric.value.value = metric.value.value + amount
        unlock(metric.value.lock)
    end
    return nothing
end

function get(metric::Metric)
    lock(metric.value.lock)
    current_value = metric.value.value
    unlock(metric.value.lock)
    current_value
end

function Gauge(name::String; value::Real = 0)
    Gauge(name, MetricValue(value))
end

Now, the following code snippet works just fine in a repl:

using Test
g = Gauge("test") # initialised with a value of 0
inc(g)
@test get(g) == 1.0

But when I try to place this snippet in a test (as commented out in my repository) I get entirely different behaviour: the inc(g) has no effect at all, and the value of g stays at its initial value of 0.0.

Normally I’d attribute this to my lack of understanding of how scoping works in Julia, but the fact that everything is okay in a REPL really throws me off here. Do you know what I might be doing wrong?

Additionally, have I handled the @async and lock stuff as I should have? The idea I was going for is to do all metric updating asynchronously, but if two attempts to manipulate a single metric clash then the second one has to wait its turn.

Julia Version 1.5.0
Commit 96786e22cc (2020-08-01 23:44 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Core(TM) i7-7500U CPU @ 2.70GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-9.0.1 (ORCJIT, skylake)
2

Arrays and References are pointers to mutable data, so even when the pointer is stored in a struct, the stored values can be modified.

When you want to control the creation of composite types to maintain invariants or provide defaults, Inner Constructors are handy.

Julia has atomic intrinsics that enforce thread-safe access.

I think what went wrong with the test is that @async starts a new task to go increment the value, but you never wait for that task to complete. So, in the REPL, enough time passed between lines for the task to finish, but during test was scheduled after the comparison.

Try this

using Test

abstract type Metric end

struct Gauge <: Metric
    name::String
    # A thread-safe reference to a Float64
    value::Threads.Atomic{Float64}
    # An inner constructor with a default value. 
    Gauge(name, initial=0.0) = new(name, Threads.Atomic{Float64}(initial))
end

function inc(metric::Metric, amount::Float64=1.0)
    # safe modification of the atomic reference
    Threads.atomic_add!(metric.value, amount)
    return nothing
end

function get(metric::Metric)
    metric.value[]
end

@testset "Gauge inc" begin
    g = Gauge("test")
    inc(g)
    @test get(g) == 1.0
end
1 Like
3

Of course it’s the async! Concurrent programming is a big weakness of mine. Thank you so much!

The inbuilt atomics look to be a lot simpler, although I was very proud of what I thought were my functioning locks. It seems I have a lot to learn about this stuff.