Cross validating quantile regression

Question

I applied quantile regression on some data and did it for tau = 0.25, 0.5, 0.75. After i got the estimates of each model, i did some cross validation on my hold out data. When i used the estimates for my hold out data i calculated in what quantile each of my independent variables lie, 25%, 50% or 75%. For instance if my x-values (independent variable) looks as follows,

x -> 1 2 3 4 5 6 7 8 9 11 10 12 13 14 15

I calculated my quantiles using the R function cut as cut(x,4) and got the following quantile ranges (0.986; 4.49], (4.49; 8], (8; 11.15], (11.15; 15]. I then divide the x vector into the correct quantile range and depending on what range it falls determine what quantile regression estimates i use, either 0.25, 0.5 or 0.75. So for the x-values 1 to 4 i will use the estimates of the 0.25, 5 to 8 i use the estimates of 0.5 and the rest i used 0.75.

I do get very accurate results, i.t.o. forecast accuracy, but can someone please tell if this is the correct way to do my quantile cross validation.

I am a little unclear about a couple things: 1) Usually people use Y for the dependent variable. It is the dependent variable that should be cut. 2) If I understand your question, it is whether to use the quantiles of the main data set or the hold out data set when testing the quantile regression, right? That's a good question, but I just wanted to be sure it was your question. — Peter Flom, Jun 22 '14 at 11:27
As long as you bin the response variable without considering the values of the covariables, I doubt that this validation is useful. — Michael M, Jun 22 '14 at 12:36

score 2 · Answer 1 · edited Aug 15 '17 at 06:06

You have to understand that in quantile regression, the quantile of interests is not for the predictor variables but the response variable so it is pointless to do a cross-validation using the quantile of your predictors. Here is your model for $\tau^{th}$ quantile: $$Q_\tau (Y|X=x) = X\beta(\tau)$$ In classical regression, you evaluate the model using a loss function. It's the same for the quantile regression, you need a loss function to evaluate how good your model is. In the case of quantile regression, your loss function is: $$\rho_\tau(u)=u(\tau - I(u<0)).$$

score 1 · Answer 2 · answered Aug 15 '17 at 07:50

I like the previous answer. However, let me outline another perspective.

It is well known that Bayesian inference provides an alternative to cross validation: Best approach for model selection Bayesian or cross-validation?
There is a Bayesian version of quantile regression: http://www.sciencedirect.com/science/article/pii/S0167715201001249

Without having elaborated on details, I assume that the you will get the results by means of MCMC, e.g. using jags.

Cross validating quantile regression

2 Answers2