Yes, I agree, that makes total sense, and explicitly calling GC.gc() (what BenchmarkTools presumably does, before benchmarking any foreign code) is likely a very sensible thing to do, in that context. That also doesn’t say anything about how often the gc actually gets called in “regular” code, when we don’t benchmark it. However my example does highlight, just how much time is being spent, in the gc-logic, even when there is very little to be collected (in this example just some meager ~170 Bytes, after the first 2 calls, each). And the gc-call does cost more than 1/4 sec. for collecting, in these cases, which, to me, is an indication, that there is considerable logic, it is going trough, independently of what it actually ends up collecting. Granted my laptop is not the fastest, but it is much more, than what I would expect, if I was to take my general impression of voices about the gc at face value. 
This is getting past my understanding, but I think the time it takes to do a full GC run depends mostly on how many objects there are in the session, not how many can it successfully gc. (While a not-“full” run only looks at “young” objects and is much quicker). Which is why there’s a bunch of heuristics to govern when GC should run, and when to do a “full” run (much more rarely) and in particular the scenario of running 4 full runs in a row without collecting much is kinda pathological because the runtime would probably never decide to actually do that (it’s only due to the manual calls in BenchmarkTools).
So I think it totally makes sense to track/benchmark GC time, but BenchmarkTools isn’t really the tool for that job. BenchmarkTools is mostly for microbenchmarks, but I think GC time makes more sense over longer function calls.
But FYI there are benchmarks specifically to test Julia’s garbage collector, GitHub - JuliaCI/GCBenchmarks. They use their own macro, which looks similar to @time and co. in that it doesn’t re-run the code many times like @btime, and also they run each benchmark in a separate Julia session.
Note:
help?> GC.gc
GC.gc([full=true])
Perform garbage collection. The argument full determines the kind of collection: A full collection (default) sweeps all objects, which makes the next GC scan much slower, while an incremental collection may only sweep so-called young
objects.
│ Warning
│
│ Excessive use will likely lead to poor performance.
julia> @time GC.gc()
0.377440 seconds (99.79% gc time)
julia> @time GC.gc()
0.138331 seconds (99.97% gc time)
julia> @time GC.gc()
0.135032 seconds (99.97% gc time)
julia> @time GC.gc(false)
0.001329 seconds (95.83% gc time)
julia> @time GC.gc(false)
0.001351 seconds (95.88% gc time)
Full is the default when you invoke it manually, but not when the GC runs implicitly. Then it always runs the the “much” faster non-full version i.e. only sweeps “so-called young objects”. I’m not sure it ever does ever does the full version. Or possibly only, as a fallback, if it realizes about to run out of memory?
Full is the default when you invoke it manually, but not when the GC runs implicitly. Then it always runs the the “much” faster non-full version i.e. only sweeps “so-called young objects”. I’m not sure it ever does ever does the full version. Or possibly only, as a fallback, if it realizes about to run out of memory?
Thx, for this info, that makes total sense. Also a great reminder, for me, to just use the repl-help, more often, as that sometimes seems to be the best (+fastest !) way, to query some basic info. I’m forgetting this, all to often, reading multiple discourse-threads, and then ending with the open question: Is this still up-to-date, or maybe not? REPL-help seems to be the solution for all of these questions, really. 
This thread landed to very interesting insights, but my point as a Julia user was much more simple: understand which kind of code patterns lead to expensive allocations and which do not.
As there seems not to be a general rule for it (using mutable vs immutable data structures seems although a good proxy) perhaps it would be nice to have some examples of common code patterns with explanations on where allocation arises or doesn’t…
I once started a common allocation mistakes thread here: Common allocation mistakes