Using VSCode, Debugger.jl is reasonably fast in my experience. The trick is to make sure that packages you don’t plan to need to step into are running in compiled mode. Often this just means that Base should be run in compiled mode (which is true by default). In practice just having base in compiled mode seems to be sufficiently fast. (In the past Base was not run in compiled mode by default, and if you use Debugger.jl outside of VScode I think this is still the case, but haven’t checked recently). But it can also be good to mark large well used and supported packed as being in compiled mode.
In spite of that, I often find myself just using Infiltrator since it is easy to use and suffices for many purposes, especially with @exfiltrate. This is a better interface I’d you want to mess around a lot with things in the REPL.
Many times I will often just use @debug or @show, that can work better for debugging, e.g. when evaluating an algorithms data flow.