7

I am interested in interpreting (back transforming) the effect of a one standard deviation (sd) increase in a log transformed on the non-transformed variable.

So let's say I have a variable Y:

Y= # of likes

ln(Y)= log transformed # of likes

mean(ln(Y))=7.7

sd(ln(Y))=0.8

Now I want to relate the sd(ln(Y)) back to non-transformed units Y. In other words, how many (#) likes is a one standard deviation increase of the log transformed units( sd(ln(Y))=0.8) equal to?

I thought maybe I could simply compare the change in the transformed data mean with the standard units) and back transform using the exponential function

e^(7.7+0.8)- e^(7.7)≈2706

So a one standard deviation increase of the log-transformed variable translates to 2,706 likes. Is this ok? Or should I be using another formula to calculate this?

Thomas
  • 71
  • 1
  • 1
  • 2

3 Answers3

8

The proposed interpretation in your last paragraph is incorrect -- that increase only applies at the mean. If you started lower, it would be a smaller increase and if you started higher it would be a larger increase.

$e^{a+0.8}- e^a=e^{a}(e^{0.8}- 1)\approx 1.2255 e^a$

It's better to think in terms of percentage increase.

$\frac{e^{a+0.8}- e^a}{e^a}\approx 1.2255$, or about 122.5% increase.

However, I am concerned about your use of logs on a count that could be zero (count of "likes").

Nick Cox
  • 48,377
  • 8
  • 110
  • 156
Glen_b
  • 257,508
  • 32
  • 553
  • 939
  • Thanks, this is very helpful. Yeah I avoid the zero boundary issue with logs by adding a very small amount. I see, so I can use that general function calculate a percentage increase. This is really helpful. Thanks! – Thomas Feb 15 '16 at 23:44
  • Also isn't it a 122.5% increase not 22.5% increase?? – Thomas Feb 15 '16 at 23:51
  • 1
    @Thomas Yes, you're right; in effect I subtracted the 1 *twice*; that's what I get for writing a post on two hours sleep. Sorry about that. – Glen_b Feb 16 '16 at 15:31
5

So a one standard deviation increase of the log-transformed variable translates to 2,706 likes. Is this ok?

You were careful to formulate your statement with 'increase of of log-transformed variable' qualifier. I think this eliminates misunderstanding that could have occurred to a reader who may assume that you're trying to calculate the standard deviation of $Y$. You're clearly not trying to do that. You use a word 'translates', which is not a standard term thus indicating that you're not transforming variables and converting the statistics between these variables by 'standard' means.

Compare your procedure to what's described in "SAS/ETS 12.1 Users Guide", p.252

The log transformation is often used to convert time series that are nonstationary with respect to the innovation variance into stationary time series. The usual approach is to take the log of the series in a DATA step and then apply PROC ARIMA to the transformed data. A DATA step is then used to transform the forecasts of the logs back to the original units of measurement. The confidence limits are also transformed by using the exponential function.

The highlighted [by me] sentence essentially describes what you're doing.

Hence, what you are doing is not wrong, whether it's right is an interesting question. It depends on the interpretations and the intended use.

One more thing (c) The estimator of the mean of the original variable $Y$ is not necessarily $e^{\overline{\ln Y}}$. I'm using a soft language here, because there's this seemingly obvious estimator $$\hat\mu_Y=\exp\left(\hat\mu_{\ln Y}+\hat\sigma^2_{\ln Y}/2\right)$$ It is based on the exact relationship for log-normal distribution: $$E[Y]=\exp\left(E[\ln Y]+\sigma^2_{\ln Y}/2\right)$$

However, this estimator is not always the best one in practice for the variance $\sigma^2_{\ln Y}$ is unknown, and has to be estimated. Once you start using the estimator of the variance, things get complicated, as shown in the empirical paper by Helmut Lutkepohl and Fang Xu. "The role of the log transformation in forecasting economic variables." Empirical Economics, 42(3):619{638, 2012.

The following, naive, estimator of the mean may end up being the best in such cases: $$\hat\mu_Y'=\exp\left(\hat\mu_{\ln Y}\right)$$

I went to write about the means because when you talk about the 'translation' of the standard deviation increase, you need to mention what is the base. You assumed rather implicitly that the increase is from the point of the naive estimator above. As I wrote it is not wrong, but you have to clearly state that it's what you used, otherwise your reader may assume that you're correcting for the variance or that the 2,706 likes increase is from any point (which is not true). For instance, if you apply your equation to the base of 0, you get $$e^{0+0.8}-e^0=2.2$$

Aksakal
  • 55,939
  • 5
  • 90
  • 176
0

If I understand, you want the standard deviation of Y. The standard deviation of Y is NOT easily calculated from mean(ln(Y)) and sd(ln(Y)), so your formula is not okay. The easy solution is to ignore the log-transform when calculating the standard deviation of Y: i.e. sd(Y) or sd(e^ln(Y)).

R Greg Stacey
  • 2,202
  • 2
  • 15
  • 30
  • Sorry for the confusion. I am not interesting in recovering the standard deviation of Y. I simply want to convert the sd(ln(Y)) back to the units of # of likes (non-transformed). – Thomas Feb 15 '16 at 23:39
  • You can use e^sd(ln(Y)). This will be the number of likes that's equal to "a one standard deviation increase of the log transformed units". But note that this number is NOT the standard deviation of Y. – R Greg Stacey Feb 15 '16 at 23:48
  • @Qroid, OP's formula's better reflection of the likes jump than than $e^{sd[\ln Y]}$. Consider this: $e^{0.8}=2.2$. Just barely 2 likes? The reason is that you're starting off $e^0=1$, while OP starts from $e^\mu=2208$, which is probably more representative – Aksakal Feb 16 '16 at 20:03