You were right. Just encoding the raw content of the file is all that it needed:
using Base64
using Downloads
using FileIO
path = Downloads.download("https://github.com/MakieOrg/Makie.jl/raw/master/assets/doge.png")
img = read(path)
encoded_img = base64encode(img)
w, h = 100, 100
svg = """
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
<image width="$w" height="$h" xlink:href="data:image/png;base64,$encoded_img"/>
</svg>
"""
write("/tmp/mwe.svg", svg)
Thank you!