Function specialization in Printf

My code, after compiled with PackageCompiler, show some memory allocation in logging functions. With the help of --trace-compile, I’ve found that when the compiled binary is running, the following functions are precompiled:

  1. Printf.fmt
  2. Printf.plength
  3. Printf.format
  4. Printf.computelen
  5. Dates.format
  6. Dates.DateFormat
  7. one custom function, which is just
function my_printf!(io, format::Printf.Format, args...)
    pos = Printf.format(LOG_DATA_BUFFER, 1, format, args...)
    GC.@preserve LOG_DATA_BUFFER unsafe_write(io, pointer(LOG_DATA_BUFFER), pos - 1)
    return
end

where

const LOG_DATA_BUFFER = Base.StringVector(4096)

However, when compiling the binary, I used the precompile_execution_file argument, and the exactly same function with exactly same input is called in the file as the compiled binary.
If all functions are specialized, I think they should all be compiled into the binary, and there should be no precompiling when the binary is executed? So, the problem should be, that the aformentioned functions are not specialized?
However, after checking the source code, I find that the @nospecialize macro is not used in Prinf, so why will some functions be not specialized?
I’ve found some discussion, like this and this, on this topic. But I have to confess this is too hard for me to understand.

Let’s put the question clear: what should I do to make the compiled binary not precompiling these functions when executing? Is the precompiling really the reason for the allocation?
I’ve already made a custom MyLogger <: AbstractLogger, and implemented the Logging.handle_message method for it, in which I basicly copied the implementation of ConsoleLogger, but removed the @nospecialize macro.

Because of this:

As a heuristic, Julia avoids automatically specializing on argument type parameters in three specific cases: Type , Function , and Vararg .

Your args... is an untyped Vararg, so Julia doesn’t specialize.

I modified the code according to the referencence like the following:

function my_printf!(io, format::Printf.Format, args::Vararg{Any, N}) where {N} = tuple(args...)
    pos = Printf.format(LOG_DATA_BUFFER, 1, format, args...)
    GC.@preserve LOG_DATA_BUFFER unsafe_write(io, pointer(LOG_DATA_BUFFER), pos - 1)
    return
end

But this does not compile:

ERROR: LoadError: syntax: unexpected "="

where did I do wrong?

ps: I’ve somehow succeed by another way: declaring the my_printf! with @inline. Most of the allocations have been removed, I’ll dig further for the remainings.


I’ve found two things.
First, @inline is a hint to compiler, but it will not force the compiler to do so. I didn’t find a way to do so, and for an example complicated as this one, it will be hard to follow the ideas in this post to make it concrete. My temporary solution is to split the long printf into several short ones, so that they will be inlined.

Second, I’ve used this way in my code:

const FORMAT_STH = Printf.Format(format_string)
function do_sth()
    my_printf!(io, FORMAT_STH, someargs)
end

I suppose in this way, it will all be compiled. However, when there are both %s and %d in the format_string, it will not be specialized. The compiled binary will need to compile the whole my_printf! for this case. Again, my temporary solution is to split the printf into several shorter ones, each containing %s or %d only.