How to use Flux.stop()

I think Flux.stop() is not meant to stop any outer loop around Flux.train!().

But for your example you do not actually need a loop around Flux.train!(). Instead what you can do is to directly provide your dataloader to Flux.train!() (the loop over minibatches happens inside Flux.train!()).
See the minimal example here: Training · Flux

Btw., Flux.train!() and Flux.stop() are no magic. The latter just throws a custom error, that the former catches within the minibatch loop: https://github.com/FluxML/Flux.jl/blob/ef04fda844ea05c45d8cc53448e3b25513a77617/src/optimise/train.jl#L63-L124

So if you replace your training loop (your second code block) by:

Flux.train!(loss, parameters, loader_train, opt, cb = function cb()
     global times[:elapsed] = time() - times[:initial]
     "Time elapsed = $(round(times[:elapsed], digits=2))" |> println
     if times[:elapsed] > time_limit
         println("Trained for $(round(times[:elapsed], digits=2)), stopping")
         Flux.stop()
     end
 end
 )

this produces what you want:

...
Time elapsed = 0.94
Time elapsed = 0.95
Time elapsed = 0.97
Time elapsed = 1.01
Trained for 1.01, stopping
1 Like