If you just want to work on bytes you should use codeunits instead of collect:
str = "aa <12> bb <c> ąą <123> dd"
r = r"<.+?>"
mask = BitArray((0, 1, 1))
matches = collect(eachmatch(r, str))[mask]
ranges_to_remove = [m.offset:(m.offset + sizeof(m.match)) for m in matches]
ranges_to_keep = setdiff(1:sizeof(str), reduce(vcat, ranges_to_remove))
String(codeunits(str)[ranges_to_keep])
# "aa <12> bb ąą dd"