MQTT message topic parsing optimization

Hello,

The problem: I am designing an MQTT client using GitHub - rweilbacher/MQTT.jl: An asynchronous MQTT client library for julia and am trying to determine the best way to handle incoming messages. messages are handled by a callback function with a topic and data both strings. my data is protobuf encoded, and each topic has a specific message encoding, so once I know what type of message I have I can parse the data into a protobuf data type.

The question: Is there a better way than a if/else chain for handling each message topic type? it seems like there should be a more “julia” way to do this (with multiple dispatch/meta programming) but I don’t know if anything is more efficient than just an if statement because the topic is a string.

example:

my_client = Client((topic, package) -> begin
    if topic == "topic1"
        parsed_package = proto_one_format(package)
        handle_new_message(parsed_package)
   elseif topic == "topic2"
        parsed_package = proto_two_format(package)
        handle_new_message(parsed_package)
   elseif topic == "topicN"
        parsed_package = proto_N_format(package)
        handle_new_message(parsed_package)
   end
end)

function proto_N_format(data)
     # parse raw data to protobuf data
end

function handle_new_message(proto_data::ProtoType1)
    # do something with the data
end 
function handle_new_message(proto_data::ProtoType2)
    # do something else with the data
end 

This is more or less how I do it now. would love to hear if there is a better way to be doing this.

Background: while there are multiple topics that will be handled, this is for doing processing on data streams, so the vast majority of all data comes in on a small subset of the potential topics, likely only one or two. The actual data packets are relatively small (sensor readings) but can be at a relatively high frequency (50Hz).

Thanks

You could use more dispatch and do something like this:

abstract type Topic end
struct TopicA <: Topic end
struct TopicB <: Topic end

abstract type Package end
struct FormatAPackage <: Package 
  x::Int64
end
struct FormatBPackage <: Package 
  y::Float64
end

parse_package(::TopicA, package) = FormatAPackage(1)
parse_package(::TopicB, package) = FormatBPackage(2.0)

handle_new_message(p::FormatAPackage) = println("format A package ", p.x)
handle_new_message(p::FormatBPackage) = println("format B package ", p.y)

const topic_map = Dict("topic1" => TopicA(), "topic2" => TopicB())

function handler(topic, package) 
  t = topic_map[topic]
  parsed_package = parse_package(t, package)
  handle_new_message(parsed_package)
end

Then calling it:

julia> handler("topic1", "abcd")
format A package 1

julia> handler("topic2", "abcd")
format B package 2.0

I replaced the sequence of if conditions with a Dictionary and used dispatching to select the correct method for parsing the package. However, it feels like over-engineering, and I am not even sure about the performance implications. You would have to benchmark it.

Thanks for the response!
I took the code (expanded for 10 topics) and ran a benchmark comparison and unfortunately it ends up being a bit slower. It does seem like it should be a better solution though.

# @benchmark str = handle_new_message_dict("topic$(rand(1:10))", randstring(16))
"""
julia> @benchmark str = handle_new_message_dict("topic$(rand(1:10))", randstring(16))
BenchmarkTools.Trial: 10000 samples with 195 evaluations.
 Range (min … max):  481.836 ns …  10.591 μs  ┊ GC (min … max): 0.00% … 94.89%
 Time  (median):     511.318 ns               ┊ GC (median):    0.00%
 Time  (mean ± σ):   562.296 ns ± 451.526 ns  ┊ GC (mean ± σ):  3.52% ±  4.31%

  ▂▆█▆▃▁▁▁▂▂▂▂▁▁                                                ▂
  ██████████████▇▇███▇▆▆▇███▇▇▆▆▇█▆▇▇▅▆▆▅▅▆▅▄▄▃▄▄▄▁▄▅▁▄▃▄▃▅▁▃▄▅ █
  482 ns        Histogram: log(frequency) by time       1.18 μs <

 Memory estimate: 512 bytes, allocs estimate: 9.
 """
# @benchmark str = handle_new_message_ifelse("topic$(rand(1:10))", randstring(16))
"""
julia> @benchmark str = handle_new_message_ifelse("topic$(rand(1:10))", randstring(16))
BenchmarkTools.Trial: 10000 samples with 259 evaluations.
 Range (min … max):  305.332 ns …   7.591 μs  ┊ GC (min … max): 0.00% … 95.00%
 Time  (median):     329.878 ns               ┊ GC (median):    0.00%
 Time  (mean ± σ):   359.609 ns ± 358.605 ns  ┊ GC (mean ± σ):  5.23% ±  5.02%

  ▁▃▅▆███▆▄▃▂▂▁   ▁▂▂   ▁▁  ▁                                   ▂
  ██████████████▇█████▇▇██████▇▆▇█▇▆▆▆▆▆▆▆▅▅▆▆▅▆▅▅▅▆▅▆▆▅▂▂▅▅▃▅▅ █
  305 ns        Histogram: log(frequency) by time        566 ns <

 Memory estimate: 496 bytes, allocs estimate: 8.
"""
function handle_new_message_dict(topic, package)
    t = topic_map[topic]
    parsed_package = parse_package(t, package)
    do_something(parsed_package)
end

function handle_new_message_ifelse(topic, package)
    if topic == "topic1"
        do_something(FormatAPackage(package))
    elseif topic == "topic2"
        do_something(FormatBPackage(package))
    elseif topic == "topic3"
        do_something(FormatCPackage(package))
    elseif topic == "topic4"
        do_something(FormatDPackage(package))
    elseif topic == "topic5"
        do_something(FormatEPackage(package))
    elseif topic == "topic6"
        do_something(FormatFPackage(package))
    elseif topic == "topic7"
        do_something(FormatGPackage(package))
    elseif topic == "topic8"
        do_something(FormatHPackage(package))
    elseif topic == "topic9"
        do_something(FormatIPackage(package))
    elseif topic == "topic10"
        do_something(FormatJPackage(package))
    else
        @error "Failed"
    end
en

my guess is that I will just have to accept that it’s an if/else block and move on.