In DataFrames.jl we see a strong trend that people want more and more powerful functions allowing for transformation of data frames (to match dplyr, data.table, pandas, …).
This leads to a situation when code base becomes really complex - which means long compilation. Additionally transformation functions most often take anonymous functions as arguments (so this is a “run once” case very often - most of the time the user will not call the transformation twice with the same argument types as every anonymous function has its own new type).
We experiment with various options how to use e.g.
@nospecialize to reduce compilation latency, while ensuring that the code runs fast for large data.
However the thing I struggle with is a convenient way to check what and how gets compiled when a call to a function with a new signature happens. It would be great to hear back from the experts in the field what are the recommended approaches to it (currently I do it mostly based on my general understanding how things work and trial-and-error).