Ok well I should have thought for two minutes more, using the SubstitutionString constructor does what I want. I’ll leave this here lest it should be useful for someone else:
foo(rs::String, t::String) = replace(rs, r"(aa|bb)" => SubstitutionString("\\1$(t)"))