Does a density forecast add value beyond a point forecast when the loss function is given?

Question

Density forecasts are more universal than point forecasts; they provide information on the whole predicted distribution of a random variable rather than on a concrete function thereof (such as predicted mean, median, quantile, etc.). Availability of a density forecast allows different users pick out relevant elements -- point forecasts -- that are of their interest. Some users will focus on the predicted mean, others on the predicted median, etc., depending on the loss function by which the forecast is evaluated (and which may differ from user to user). Given a density forecast, every user's needs will be satisfied regardless of the loss function, because the density forecast contains all probabilistic information about the random variable.

However, if we have a concrete user in mind and know his/her loss function, then

• Does the density forecast provide any added value over a point forecast tailored to the loss function?
• If the answer is No in general, what are the conditions to make it a Yes?

---

P.S. @hejseb draws an interesting parallel between a point forecast tailored to the loss function and a sufficient statistic; perhaps this can inspire an answer.

Answer 1

I can think of one-and-a-half more or less realistic situations where a full density is better than a point forecast, even if the loss function is known.

• The nitpicky situation is the one where the user's loss function depends not only on the point forecast, but on a two-sided prediction-interval, or even the entire density, i.e., the loss function is a scoring-rules.

  Yes, a loss function is typically defined to depend on a single point forecast, so I'm loose with nomenclature here. Nevertheless situations like these do occur, e.g., in financial volatility forecasting. Or where I work, in retail replenishment forecasting: we may want to achieve a 95% service level, so on the face of it, we may only be interested in that (point) quantile forecast. However, a 95% quantile forecast may be 4, while we may be constrained to replenish in pack sizes of 8. In such a situation, it can be valuable to know what percentage 8 units correspond to.

• The more relevant situation is one where we are interested in functions of predictive densities. Again, consider retail forecasting: because of the delivery schedule, our replenishment order may need to cover three days, Tuesday to Thursday. However, we forecast on daily granularity. So we may be interested in the 95% quantile forecast of the sum of the demands, and for the convolution, we need the full densities. (We could also try to forecast on three-day bucket granularity, but that becomes problematic if, say, a promotion starts in the middle of the bucket.)

Answer 2

Background (may be skipped)

I will be thinking in decision-theoretic terms as follows. A user must choose an action $a$ among a set of possibilities $A$. The action will bring him/her some "utility" (a notion commonly used in economics) $u(a;s)$ depending on the state of nature $s$ that will be realized in the future, where $s \in S$, a set of all possible states. (Utility is basically the negative of loss, and what follows could be reformulated equivalently either in terms of utility or loss.) The user aims at maximizing the expected utility (or equivalently, minimizing the expected loss) w.r.t. the action, $$ \max_{a \in A} \mathbb{E}_{S} u(a;s). $$

The choice of action is based on the forecast of the state of nature to be realized. Given a density forecast $\hat f_S(\cdot)$, a user can calculate the expected utility of a particular action by integrating the utility of that action over the predicted distribution of the states of nature, $$ \mathbb{E}_{\hat S} u(a;s) = \int u(a;s) \hat f_S(s) ds. $$ Then he/she chooses the action (among all possible ones) that maximizes this expected utility, $\hat a^* := \arg\max_{a \in A} \mathbb{E}_{\hat S} u(a;s)$. The expected value of utility at this action, for this density forecast is $\hat u^*:=u(\hat a^*)$.

If the utility function has a unique maximum (loss function has a unique minimum), the optimal action is unique. If the state of nature is a continuous random variable, there exists a point in the distribution (a state of nature) that yields exactly $\hat u^*$. That point defines the target of the "relevant" point forecast. Hence, the user will get exactly the same maximized (over all possible actions) expected utility regardless of whether the forecast he gets is a density forecast or the "relevant" point forecast (a unit probability mass on a certain state of nature), provided the quality of the two forecasts is "equally good" (the easiest to intuitively understand the latter is to consider the case where both the point and the density forecast are perfect).

Main part (see background for more details)

I think it is reasonable to assume that the usefulness of a forecast is fully reflected by the loss it incurs to a given user. Then the aim of a user is to choose a forecast that minimizes the expected loss. Hence, given a predicted distribution, the user will take a concrete function thereof (e.g. predicted mean) that minimizes the expected loss. The rest of the predicted density will not have any added value to the user.

If the loss function has a unique minimum, the function will be single-valued, and that value will be the point forecast relevant for the user. For example, if the user's loss function is quadratic (which has a unique minimum at the mean of the true distribution), he/she will only care about the forecast of the mean. If another user is facing absolute loss (which has a unique minimum at the median of the true distribution), he/she will only care about the forecast of the median. Providing a density forecast for either of these users in addition to forecasts of mean and median, respectively, will be of zero added value to them.

Elliott and Timmermann (2016a) write on p. 423-424 (regarding evaluation of density forecasts):

One way to [evalute a density forecast] would be to convert the density forecast into a point forecast and use the methods for point forecast evaluation. This simple approach to evaluating density forecasts might be appropriate for a number of reasons. <...> [D]ensity forecasts can be justified on the grounds that there are multiple users with different loss functions. Any one of these users might examine the performance of a density forecast with reference to the specific loss function deemed appropriate for their problem. The relevant measure of forecast performance is the average loss calculated from each user’s specific loss function.

Moreover, given a known loss function, a density forecast may even be inferior to a relevant point forecast, for the following two reasons. First, density forecasts are typically more difficult to produce than point forecasts. Second, they might trade off precision/accuracy at a particular point (say, mean or median) for precision/accuracy across the whole distribution that is being predicted. That is, if one is predicting the whole density, one might have to sacrifice some precision/accuracy for the forecast of the mean so as to get greater precision/accuracy elsewhere. As Elliott and Timmermann (2016b) write,

[T]he relationships between the scoring rules popular in the literature and the underlying loss functions for individual users is not clear. Thus, it could well be that the scoring rule used provides a poor estimate of the feature of the conditional distribution that some users wish to construct.

A similar quote can be found in Elliott and Timmermann (2016a), p. 277-278:

It would seem that provision of a predictive density is superior to reporting a point forecast since it both (a) can be combined with a loss function to produce any point forecast; and (b) is independent of the loss function. In classical estimation of the predictive density, neither of these points really holds up in practice. <...> [I]n the classical setting the estimated predictive distributions depend on the loss function. All parameters of the predictive density need to be estimated and these estimates require some loss function, so loss functions are thrown back into the mix. The catch here is that the loss functions that are often employed in density estimation do not line up with those employed for point forecasting which can lead to inferior point forecasts. <...> Moreover, conditional distributions are difficult to estimate well, and so point forecasts based on estimates of the conditional density may be highly suboptimal from an estimation perspective.

Hence, when a loss function is given, it might make sense to focus on forecasting the particular point tailored to the loss function rather than attempt to forecast the whole distribution. This might be easier to do and/or more accurate.

A critical question to myself: may it be that the "relevant" point forecast cannot be expressed as a function of the unknown density but rather be different (as a function, not just its value) for different densities? Then a density forecast would be needed to find out which point forecast one is interested in, making a density forecast an inevitable step in the point forecasting process.

References:

• Elliott, G., & Timmermann, A. (2016a). Economic forecasting. Princeton: Princeton University Press.
• Elliott, G., & Timmermann, A. (2016b). Forecasting in economics and finance. Annual Review of Economics, 8, 81-110.