First of all, wow @Albert_Zevelev, it’s really cool that you’ve already engaged with this package so much - thank you! I’m sorry for the delayed response, was away for the past couple of days and have come down w/ Covid.
Just some high-level thoughts on this form my side:
I’m finding it difficult to think about conformal predictive distributions in the same way that I’m used to thinking about posterior predictive distributions (in the Bayesian setting as outlined by @Sourish). In particular, since CP is inherently free of prior distributional assumptions, it does not follow the standard Bayesian paradigm where we update prior beliefs about parameter distributions using data/evidence to then form posterior beliefs. If your goal is to derive a functional form for the predictive distribution, I think the Bayesian context is perhaps more useful.
As for theoretical properties of the resulting predictive distribution in CP, the literature seems to talk about validity guarantees, but not so much about the functional properties of the resulting distribution. The paper I linked here and related papers (see here and here) appear to derive empirical predictive distributions along the lines of what you’ve done here. This is useful because:
Their advantage over the usual conformal prediction intervals is that a conformal predictive distribution Q_n contains more information and can produce a plethora of prediction intervals.
But again, not sure how much (if anything) can really be said about the functional form of Q_n.
At this point, for my own research I am primarily interested in instance-based post-hoc uncertainty estimates (for which conformal predictors are sufficient it seems). Nonetheless, recovering predictive distributions is of course very interesting, so I’ll definitely keep the issue open.
Sorry that I can’t be of more help and thanks again for this!