If TTFX is very inconvenient to you, while developing, you can try it:
But, IMHO, it is more convenient to deal with that using a Revise-based worflow. I would not recommend start building system images to everyone.
At the end you could deliver a binary for users of your package, without the TTFX at all (and that could be used for benchmarking in some contexts). But I personally think that is a necessity that first needs to be demonstrated in the specific domain/application in question.
Concerning the original question of the post, my opinion is that “performance” and " development responsiveness" are two different things. Of course there is a TTFX in Julia that you may or may not include in the benchmark. Yet, I would not call that a measure of “performance”, at least in my field, things that take a few seconds to run are not important, and we care about performance for things that take hours and days to run, for which TTFX and compilation are irrelevant.