Quality measure for predictive Highest Density Regions

Question

An alternative to point, interval and density forecasts/predictions would be "predictive highest density regions (pHDRs)", i.e., HDRs for the conditional density of a yet-unknown future observable.

A natural question would be that of evaluating a pHDR once we have observed the corresponding observable. This is analogous to point forecast error measures or prediction interval scores. (Note that the interval score cannot be applied to one-dimensional pHDR, which may be the union of multiple intervals.)

Is there such a quality measure for pHDRs? The best I could think of is to test the coverage achieved against the nominal value, but this disregards the volume of the pHDR, which we want to be as small as possible.

Answer 1

Maybe a variation of the Winkler score would work. Let the $100(1-\alpha)$% HDR be given by $R_\alpha$. Then the score could be $$s_\alpha + \frac{2}{\alpha}1(y\not\in R_\alpha)$$ where $s_\alpha$ is the total size of the region (i.e., the sum of the lengths of the sub-intervals).

However, note that the HDR is the smallest region with specified coverage by definition. So just checking the coverage is probably ok.