Well, median is more robust in the presence of outliers. The standard error will still be usually higher though. The true error of the estimator is always unknown in any case.
I can’t comment on the optimal estimator for software benchmarking though.