What is the reason the log transformation is used with right-skewed distributions?

Question

I once heard that

log transformation is the most popular one for right-skewed distributions in linear regression or quantile regression

I would like to know is there any reason underlying this statement? Why is the log transformation suitable for a right-skewed distribution?

How about a left-skewed distribution?

Answer 1

Economists (like me) love the log transformation. We especially love it in regression models, like this: \begin{align} \ln{Y_i} &= \beta_1 + \beta_2 \ln{X_i} + \epsilon_i \end{align}

Why do we love it so much? Here is the list of reasons I give students when I lecture on it:

1. It respects the positivity of $Y$. Many times in real-world applications in economics and elsewhere, $Y$ is, by nature, a positive number. It might be a price, a tax rate, a quantity produced, a cost of production, spending on some category of goods, etc. The predicted values from an untransformed linear regression may be negative. The predicted values from a log-transformed regression can never be negative. They are $\widehat{Y}_j=\exp{\left(\beta_1 + \beta_2 \ln{X_j}\right)} \cdot \frac{1}{N} \sum \exp{\left(e_i\right)}$ (See an earlier answer of mine for derivation).
2. The log-log functional form is surprisingly flexible. Notice: \begin{align} \ln{Y_i} &= \beta_1 + \beta_2 \ln{X_i} + \epsilon_i \\ Y_i &= \exp{\left(\beta_1 + \beta_2 \ln{X_i}\right)}\cdot\exp{\left(\epsilon_i\right)}\\ Y_i &= \left(X_i\right)^{\beta_2}\exp{\left(\beta_1\right)}\cdot\exp{\left(\epsilon_i\right)}\\ \end{align} Which gives us: That's a lot of different shapes. A line (whose slope would be determined by $\exp{\left(\beta_1\right)}$, so which can have any positive slope), a hyperbola, a parabola, and a "square-root-like" shape. I've drawn it with $\beta_1=0$ and $\epsilon=0$, but in a real application neither of these would be true, so that the slope and the height of the curves at $X=1$ would be controlled by those rather than set at 1.
3. As TrynnaDoStat mentions, the log-log form "draws in" big values which often makes the data easier to look at and sometimes normalizes the variance across observations.
4. The coefficient $\beta_2$ is interpreted as an elasticity. It is the percentage increase in $Y$ from a one percent increase in $X$.
5. If $X$ is a dummy variable, you include it without logging it. In this case, $\beta_2$ is the percent difference in $Y$ between the $X=1$ category and the $X=0$ category.
6. If $X$ is time, again you include it without logging it, typically. In this case, $\beta_2$ is the growth rate in $Y$---measured in whatever time units $X$ is measured in. If $X$ is years, then the coefficient is annual growth rate in $Y$, for example.
7. The slope coefficient, $\beta_2$, becomes scale-invariant. This means, on the one hand, that it has no units, and, on the other hand, that if you re-scale (i.e. change the units of) $X$ or $Y$, it will have absolutely no effect on the estimated value of $\beta_2$. Well, at least with OLS and other related estimators.
8. If your data are log-normally distributed, then the log transformation makes them normally distributed. Normally distributed data have lots going for them.

Statisticians generally find economists over-enthusiastic about this particular transformation of the data. This, I think, is because they judge my point 8 and the second half of my point 3 to be very important. Thus, in cases where the data are not log-normally distributed or where logging the data does not result in the transformed data having equal variance across observations, a statistician will tend not to like the transformation very much. The economist is likely to plunge ahead anyway since what we really like about the transformation are points 1,2,and 4-7.

Answer 2

First let's see what typically happens when we take logs of something that's right skew.

The top row contains histograms for samples from three different, increasingly skewed distributions.

The bottom row contains histograms for their logs.

You can see that the center case ($y$) has been transformed to symmetry, while the more mildly right skew case ($x$) is now somewhat left skew. One the other hand, the most skew variable ($z$) is still (slightly) right skew, even after taking logs.

If we wanted our distributions to look more normal, the transformation definitely improved the second and third case. We can see that this might help.

---

So why does it work?

Note that when we're looking at a picture of the distributional shape, we're not considering the mean or the standard deviation - that just affects the labels on the axis.

So we can imagine looking at some kind of "standardized" variables (while remaining positive, all have similar location and spread, say)

Taking logs "pulls in" more extreme values on the right (high values) relative to the median, while values at the far left (low values) tend to get stretched back, further away from the median.

In the first diagram, $x$, $y$ and $z$ all have means near 178, all have medians close to 150, and their logs all have medians near 5.

When we looks at the original data, a value at the far right - say around 750 - is sitting far above the median. In the case of $y$, it's 5 interquartile ranges above the median.

But when we take logs, it gets pulled back toward the median; after taking logs it's only about 2 interquartile ranges above the median.

Meanwhile a low value like 30 (only 4 values in the sample of size 1000 are below it) is a bit less than one interquartile range below the median of $y$. When we take logs, it's again about two interquartile ranges below the new median.

It's no accident that the ratio of 750/150 and 150/30 are both 5 when both log(750) and log(30) ended up about the same distance away from the median of log(y). That's how logs work - converting constant ratios to constant differences.

It's not always the case that the log will help noticeably. For example if you take say a lognormal random variable and shift it substantially to the right (i.e. add a large constant to it) so that the mean became large relative to the standard deviation, then taking the log of that would make very little difference to the shape. It would be less skew - but barely.

---

But other transformations - the square root, say - will also pull large values in like that. Why are logs in particular, more popular?

I touched on one reason just at the end of the previous part - constant ratios tend to constant differences. This makes logs relatively easy to interpret, since constant percentage changes (like a 20% increase to every one of a set of numbers) become a constant shift. So a decrease of $-0.162$ in the natural log is a 15% decrease in the original numbers, no matter how big the original number is.

A lot of economic and financial data behaves like this, for example (constant or near-constant effects on the percentage scale). The log scale makes a lot of sense in that case. Moreover, as a result of that percentage-scale effect. the spread of values tends to be larger as the mean increases - and taking logs also tends to stabilize the spread. That's usually more important than normality. Indeed, all three distributions in the original diagram come from families where the standard deviation will increase with the mean, and in each case taking logs stabilizes variance. [This doesn't happen with all right skewed data, though. It's just very common in the sort of data that crops up in particular application areas.]

There are also times when the square root will make things more symmetric, but it tends to happen with less skewed distributions than I use in my examples here.

We could (fairly easily) construct another set of three more mildly right-skew examples, where the square root made one left skew, one symmetric and the third was still right-skew (but a bit less skew than before).

---

What about left-skewed distributions?

If you applied the log transformation to a symmetric distribution, it will tend to make it left-skew for the same reason it often makes a right skew one more symmetric - see the related discussion here.

Correspondingly, if you apply the log-transformation to something that's already left skew, it will tend to make it even more left skew, pulling the things above the median in even more tightly, and stretching things below the median down even harder.

So the log transformation wouldn't be helpful then.

See also power transformations/Tukey's ladder. Distributions that are left skew may be made more symmetric by taking a power (greater than 1 -- squaring say), or by exponentiating. If it has an obvious upper bound, one might subtract observations from the upper bound (giving a right skewed result) and then attempt to transform that.

Answer 3

The log function essentially de-emphasizes very large values. Look at the image below which shows $y = ln(x)$. Notice how large values on the $x$-axis are relatively smaller on the y-axis.

Now, in a right-skewed distribution you have a few very large values. The log transformation essentially reels these values into the center of the distribution making it look more like a Normal distribution.

Answer 4

All of these answers are sales pitches for the natural log transformation. There are caveats to its use, caveats that are generalizable to any and all transformations. As a general rule, all mathematical transformations reshape the PDF of the underlying raw variables whether acting to compress, expand, invert, rescale, whatever. The biggest challenge this presents from a purely practical point of view is that, when used in regression models where predictions are a key model output, transformations of the dependent variable, Y-hat, are subject to potentially significant retransformation bias. Note that natural log transformations are not immune to this bias, they're just not as impacted by it as some other, similar acting transformations. There are papers offering solutions for this bias but they really don't work very well. In my opinion, you're on much safer ground not messing with trying to transform Y at all and finding robust functional forms that allow you to retain the original metric. For instance, besides the natural log, there are other transformations that compress the tail of skewed and kurtotic variables such as the inverse hyperbolic sine or Lambert's W. Both of these transformations work very well in generating symmetric PDFs and, therefore, Gaussian-like errors, from heavy-tailed information, but watch out for the bias when you try to bring the predictions back into the original scale for the DV, Y. It can be ugly.

Answer 5

Many interesting points have been made. A few more?

1) I would suggest that another issue with linear regression is that the 'left hand side' of the regression equation is E(y) : the expected value. If the error distribution is not symmetrical, then merits for the study of the expected value are weak. The expected value is not of central interest when the errors are asymmetrical. One could explore quantile regression instead. Then the study of, say, the median, or other percentage points might be worthy even if the errors are asymmetrical.

2) If one elects to transform the response variable, then one may wish to transform one of more of the explanatory variables with the same function. For example, if one has a 'final' outcome as response, then one might have a 'baseline' outcome as an explanatory variable. For interpretation, it makes sense the transform 'final' and 'baseline' with the same function.

3) The main argument for transforming an explanatory variable is often around the linearity of the response - explanatory relationship. These days, one can consider other options like restricted cubic splines or fractional polynomials for the explanatory variable. There is certainly often a certain clarity if linearity can be found though.