13

I have a study where many outcomes are represented like percentages and I'm using multiple linear regressions to asses the effect of some categorical variables on these outcomes.

I was wondering, since a linear regression assume that the outcome is a continuous distribution, are there methodological problems in applying such model to percentages, which are limited between 0 and 100?

amoeba
  • 93,463
  • 28
  • 275
  • 317
Bakaburg
  • 2,293
  • 3
  • 21
  • 30
  • 1
    Are these percentages continuous (like the percentage of cream in milk, for example), or discrete (like binomial proportions, a count in some category out of a total count)? – Glen_b Jun 17 '14 at 23:24
  • 1
    Uhm... i don't get the difference. Aren't they both continuous? Anyway I think the second describe better my data, since we are speaking about people out of at total. – Bakaburg Jun 18 '14 at 14:50
  • The distribution of counts divided by counts is definitely discrete. Indeed, the numerator is usually modelled as a binomial, the denominator is conditioned on (treated as constant), so the ratio is usually treated as a scaled binomial. However, even if the denominator was alse a random variable, the ratio would still be discrete since its sample space is countable – Glen_b Jun 18 '14 at 21:28

3 Answers3

19

I'll address the issues relevant to either discrete or continuous possibility:

  1. A problem with the description of the mean

    You have a bounded response. But the model you're fitting isn't bounded, and so can blast right through the bound; some of your fitted values may be impossible, and predicted values eventually must be.

    The true relationship must eventually become flatter than it is at the middle as it approaches the bounds, so it would be expected to bend in some fashion.

  2. A problem with the description of the variance

    As the mean approaches the bound, the variance will tend to decrease as well, other things being equal. There's less room between the mean and the bound, so the overall variability tends to reduce (otherwise the mean would tend to be pulled away from the bound by points being on average further away on the side not close to the bound.

(Indeed, if all the population values in some neighborhood were exactly at the bound, the variance there would be zero.)

A model that deals with such a bound should take such effects into consideration.

If the proportion is for a count variable, a common model for the distribution of the proportion is a binomial GLM. There are several options for the form of the relationship of the mean proportion and the predictors, but the most common one would be a logistic GLM (several other choices are in common use).

If the proportion is a continuous one (like the percentage of cream in milk), there are a number of options. Beta regression seems to be one fairly common choice. Again, it might use a logistic relationship between the mean and the predictors, or it might use some other functional form.

See also Regression for an outcome (ratio or fraction) between 0 and 1.

Glen_b
  • 257,508
  • 32
  • 553
  • 939
  • 1
    +1 and I took the liberty to add a link to what could perhaps be seen as our "master" thread on this topic (gung's answer there also covers beta and logistic options). – amoeba Feb 15 '17 at 09:26
  • 2
    An easy general argument is if that the mean is 0 that is only possible if all values are 0, and similarly with 1 = 100% and all values being 1. So the variance must be 0 at the extremes regardless of whether proportions are based on counting or measuring. Although it is possible that all other values are some constant, in practice that is very rare. Hence variance will be highest for some value between 0 and 1. – Nick Cox Feb 15 '17 at 16:25
  • would you be able to provide some references for the 2 issues described? – user1607 Jun 08 '19 at 09:33
4

This is exactly the same thing as the case when the outcome is between 0 and 1, and that case is typically handled with a generalized linear model (GLM) like logistic regression. There are lots of excellent primers for logistic regression (and other GLMs) on the internet, and there is also a well-known book by Agresti on the topic.

Beta regression is a viable but more complicated alternative. Chances are logistic regression would work fine for your application, and would typically be easier to implement with most statistical software.

Why not use ordinary least squares regression? Actually people do, sometimes under the name "linear probability model" (LPM). The most obvious reason why LPMs are "bad" is that there's no easy way to constrain the outcome to lie within a certain range, and you can get predictions above 1 (or 100% or any other finite upper bound) and below 0 (or some other lower bound). For the same reason, predictions near the upper bound tend to be systematically too high, and predictions near the lower bound tend to be too low. The math underlying linear regression explicitly assumes that tendencies like this don't exist. There typically isn't a great reason to fit an LPM over logistic regression.

As an aside, it turns out that all OLS regression models, including LPMs, can be defined as a special kind of GLM, and in this context LPMs are related to logistic regression.

shadowtalker
  • 11,395
  • 3
  • 49
  • 109
  • 5
    Although overall much of this answer looks worthwhile, it contains some misinformation that could confuse readers. The account of logistic regression in the first paragraph sounds like a description of a log-like transformation of the dependent variable followed by linear regression: it is not logistic regression. The interpretation of the coefficients is not quite right, either. A more important problem with "LPMs" is that when data are near the extremes, they likely exhibit asymmetric distributions of the residuals, which is an important violation of the iid assumption of regression. – whuber Jun 17 '14 at 21:50
  • I didn't think it was worth getting into odds ratios and such. I'll just strip that stuff out and let the OP read up on it then. Also good point about the residuals. – shadowtalker Jun 17 '14 at 22:59
  • (+1) Thank you for your constructive responses! – whuber Jun 18 '14 at 00:28
2

It might be worth investigating beta regression (for which I understand there is an R package), which seems well suited to such problems.

http://www.jstatsoft.org/v34/i02/paper

Dikran Marsupial
  • 46,962
  • 5
  • 121
  • 178
  • 7
    You answer would be even better if you hit on some of the major reasons why linear regression suffers when the outcome is a percentage. – Alexis Jun 17 '14 at 18:36