Hello,
The problem: I am designing an MQTT client using GitHub - rweilbacher/MQTT.jl: An asynchronous MQTT client library for julia and am trying to determine the best way to handle incoming messages. messages are handled by a callback function with a topic and data both strings. my data is protobuf encoded, and each topic has a specific message encoding, so once I know what type of message I have I can parse the data into a protobuf data type.
The question: Is there a better way than a if/else chain for handling each message topic type? it seems like there should be a more “julia” way to do this (with multiple dispatch/meta programming) but I don’t know if anything is more efficient than just an if statement because the topic is a string.
example:
my_client = Client((topic, package) -> begin
if topic == "topic1"
parsed_package = proto_one_format(package)
handle_new_message(parsed_package)
elseif topic == "topic2"
parsed_package = proto_two_format(package)
handle_new_message(parsed_package)
elseif topic == "topicN"
parsed_package = proto_N_format(package)
handle_new_message(parsed_package)
end
end)
function proto_N_format(data)
# parse raw data to protobuf data
end
function handle_new_message(proto_data::ProtoType1)
# do something with the data
end
function handle_new_message(proto_data::ProtoType2)
# do something else with the data
end
This is more or less how I do it now. would love to hear if there is a better way to be doing this.
Background: while there are multiple topics that will be handled, this is for doing processing on data streams, so the vast majority of all data comes in on a small subset of the potential topics, likely only one or two. The actual data packets are relatively small (sensor readings) but can be at a relatively high frequency (50Hz).
Thanks
You could use more dispatch and do something like this:
abstract type Topic end
struct TopicA <: Topic end
struct TopicB <: Topic end
abstract type Package end
struct FormatAPackage <: Package
x::Int64
end
struct FormatBPackage <: Package
y::Float64
end
parse_package(::TopicA, package) = FormatAPackage(1)
parse_package(::TopicB, package) = FormatBPackage(2.0)
handle_new_message(p::FormatAPackage) = println("format A package ", p.x)
handle_new_message(p::FormatBPackage) = println("format B package ", p.y)
const topic_map = Dict("topic1" => TopicA(), "topic2" => TopicB())
function handler(topic, package)
t = topic_map[topic]
parsed_package = parse_package(t, package)
handle_new_message(parsed_package)
end
Then calling it:
julia> handler("topic1", "abcd")
format A package 1
julia> handler("topic2", "abcd")
format B package 2.0
I replaced the sequence of if conditions with a Dictionary and used dispatching to select the correct method for parsing the package. However, it feels like over-engineering, and I am not even sure about the performance implications. You would have to benchmark it.
Thanks for the response!
I took the code (expanded for 10 topics) and ran a benchmark comparison and unfortunately it ends up being a bit slower. It does seem like it should be a better solution though.
# @benchmark str = handle_new_message_dict("topic$(rand(1:10))", randstring(16))
"""
julia> @benchmark str = handle_new_message_dict("topic$(rand(1:10))", randstring(16))
BenchmarkTools.Trial: 10000 samples with 195 evaluations.
Range (min … max): 481.836 ns … 10.591 μs ┊ GC (min … max): 0.00% … 94.89%
Time (median): 511.318 ns ┊ GC (median): 0.00%
Time (mean ± σ): 562.296 ns ± 451.526 ns ┊ GC (mean ± σ): 3.52% ± 4.31%
▂▆█▆▃▁▁▁▂▂▂▂▁▁ ▂
██████████████▇▇███▇▆▆▇███▇▇▆▆▇█▆▇▇▅▆▆▅▅▆▅▄▄▃▄▄▄▁▄▅▁▄▃▄▃▅▁▃▄▅ █
482 ns Histogram: log(frequency) by time 1.18 μs <
Memory estimate: 512 bytes, allocs estimate: 9.
"""
# @benchmark str = handle_new_message_ifelse("topic$(rand(1:10))", randstring(16))
"""
julia> @benchmark str = handle_new_message_ifelse("topic$(rand(1:10))", randstring(16))
BenchmarkTools.Trial: 10000 samples with 259 evaluations.
Range (min … max): 305.332 ns … 7.591 μs ┊ GC (min … max): 0.00% … 95.00%
Time (median): 329.878 ns ┊ GC (median): 0.00%
Time (mean ± σ): 359.609 ns ± 358.605 ns ┊ GC (mean ± σ): 5.23% ± 5.02%
▁▃▅▆███▆▄▃▂▂▁ ▁▂▂ ▁▁ ▁ ▂
██████████████▇█████▇▇██████▇▆▇█▇▆▆▆▆▆▆▆▅▅▆▆▅▆▅▅▅▆▅▆▆▅▂▂▅▅▃▅▅ █
305 ns Histogram: log(frequency) by time 566 ns <
Memory estimate: 496 bytes, allocs estimate: 8.
"""
function handle_new_message_dict(topic, package)
t = topic_map[topic]
parsed_package = parse_package(t, package)
do_something(parsed_package)
end
function handle_new_message_ifelse(topic, package)
if topic == "topic1"
do_something(FormatAPackage(package))
elseif topic == "topic2"
do_something(FormatBPackage(package))
elseif topic == "topic3"
do_something(FormatCPackage(package))
elseif topic == "topic4"
do_something(FormatDPackage(package))
elseif topic == "topic5"
do_something(FormatEPackage(package))
elseif topic == "topic6"
do_something(FormatFPackage(package))
elseif topic == "topic7"
do_something(FormatGPackage(package))
elseif topic == "topic8"
do_something(FormatHPackage(package))
elseif topic == "topic9"
do_something(FormatIPackage(package))
elseif topic == "topic10"
do_something(FormatJPackage(package))
else
@error "Failed"
end
en
my guess is that I will just have to accept that it’s an if/else block and move on.