21

I have two variables which don't show much correlation when plotted against each other as is, but a very clear linear relationship when I plot the logs of each variable agains the other.

So I would end up with a model of the type:

$$\log(Y) = a \log(X) + b$$ , which is great mathematically but doesn't seem to have the explanatory value of a regular linear model.

How can I interpret such a model?

StubbornAtom
  • 8,662
  • 1
  • 21
  • 67
Akaike's Children
  • 1,251
  • 7
  • 15
  • 5
    I've nothing substantial to add to the existing answers, but a logarithm in the outcome and the predictor is an elasticity. Searches for that term should find some good resources for interpreting that relationship, which is not very intuitive. – Upper_Case Mar 26 '19 at 18:30
  • 3
    The complementary log-log link is an ideal GLM specification when the outcome is binary (risk model) and the exposure is cumulative, such as number of sexual partners vs. HIV infection. https://www.jstor.org/stable/2532454 – AdamO Mar 27 '19 at 15:58
  • The interpretation of a log-log model, where the dependent variable is log(y) and the independent variable is log(x), is: $\%Δ=β_1\%Δx$. – Bob Mar 27 '19 at 15:52
  • Thank you @AdamO ! Do you know if there are any circumstances where logit, probit, or complementary log-log links give substantively different results, either in point estimates, or in standard errors? As the authors you cited wrote "Since the logit of *P* closely approximates the complementary log- log of *P* over a wide range, such an analysis is likely to lead to qualitatively similar results to the methods described here," I wonder if there's a *strong* reason to prefer any in given circumstances? I know that the econometricians like probit links as natural for things like elasticities. – Alexis Mar 27 '19 at 20:47
  • 2
    @Alexis you can see the sticky points if you overlay the curves. Try `curve(exp(-exp(x)), from=-5, to=5)` vs `curve(plogis(x), from=-5, to=5)`. The concavity accelerates. If the risk of event from a single encounter was $p$, then the risk after the second event should be $1-(1-p)^2$ and so on, that's a probabilistic shape logit won't capture. High high exposures would skew logistic regression results more dramatically (falsely according to the prior probability rule). Some simulation would show you this. – AdamO Mar 27 '19 at 20:52
  • 1
    @AdamO There's probably a pedagogical paper to be written incorporating such a simulation which motivates how to chose a particular dichotomous outcome link out of the three, including situations where it does and does not make a difference. – Alexis Mar 27 '19 at 21:05

5 Answers5

27

You just need to take exponential of both sides of the equation and you will get a potential relation, that may make sense for some data.

$$\log(Y) = a\log(X) + b$$

$$\exp(\log(Y)) = \exp(a \log(X) + b)$$

$$Y = e^b\cdot X^a$$

And since $e^b$ is just a parameter that can take any positive value, this model is equivalent to:

$$Y=c \cdot X^a$$

It should be noted that model expression should include the error term, and these change of variables has interesting effects on it:

$$\log(Y) = a \log(X) + b + \epsilon$$

$$Y = e^b\cdot X^a\cdot \exp(\epsilon)$$

That is, your model with a additive errors abiding to the conditions for OLS (normally distributed errors with constant variance) is equivalent to a potential model with multiplicative errors whose logaritm follows a normal distribution with constant variance.

Pere
  • 5,875
  • 1
  • 13
  • 29
  • 3
    OP may be interested to know that this distribution has a name, the log-normal: https://en.wikipedia.org/wiki/Log-normal_distribution – gardenhead Mar 26 '19 at 19:40
  • 2
    What about the effect of Jensen's inequality? Generally for convex g, $E[g(X)]≥g(E[X])$ – Stats Mar 26 '19 at 21:17
14

You can take your model $\log(Y)=a\log(X)+b$ and calculate the total differential, you will end up with something like : $$\frac{1}YdY=a\frac{1}XdX$$ which yields to $$\frac{dY}{dX}\frac{X}{Y}=a$$

Hence one simple interpretation of the coefficient $a$ will be the percent change in $Y$ for a percent change in $X$. This implies furthermore that the variable $Y$ growths at a constant fraction ($a$) of the growth rate of $X$.

Chaos
  • 421
  • 3
  • 12
  • So if the log-log plot is linear, that would imply a constant growth rate? – dimitriy Mar 26 '19 at 17:22
  • Not actually, the growth rate of $Y$ will be constant if and only if $a=0$. – Chaos Mar 26 '19 at 18:18
  • Not over time, the growth rate with respect to the growth in x. – dimitriy Mar 26 '19 at 18:21
  • reordering doesn't help, i'd remove it – Aksakal Mar 26 '19 at 18:21
  • 1
    @DimitriyV.Masterov Ok, then since the $\log(Y)$ is linear in $\log(X)$ it means that the variable $Y$ grows at a constant fraction of the growth rate of $X$. Is there something wrong with my answer according to you? – Chaos Mar 26 '19 at 18:36
  • @RScrlli I like your answer very much, and +1ed already. I just wanted to emphasize what linearity means in the log-log setting. – dimitriy Mar 26 '19 at 18:46
  • I see,@DimitriyV.Masterov I was struggling to find out what was wrong, since it is obvious you already knew the answer :-) – Chaos Mar 26 '19 at 18:47
7

Intuitively $\log$ gives us the order of magnitude of a variable, so we can view the relationship as the orders of magnitudes of the two variables are linearly related. For example, increasing the predictor by one order of magnitude may be associated with an increase of three orders of magnitude of the response.

When plotting using a log-log plot we hope to see a linear relationship. Using an example from this question, we can check the linear model assumptions:

log-log

Anon1759
  • 3
  • 2
qwr
  • 352
  • 1
  • 16
  • 3
    +1 for an intuitive answer to an unintuitive concept. However, the image you have included clearly violates constant error variance across the predictor. – Frans Rodenburg Mar 27 '19 at 07:24
  • 1
    The answer is right, but authorship attribution is wrong. The image shouldn't be attributed to Google Images but, at least, to the web page where it is to be found, that can be find out just by clicking in Google images. – Pere Mar 27 '19 at 08:42
  • @Pere I cannot find the original source of the image unfortunately (at least using reverse image search) – qwr Mar 27 '19 at 09:33
  • It seems to come originally from http://diagramss.us/ though that site is down and most of its pages are not in the Web Archive apart from [its homepage](http://web.archive.org/web/20171118204828/http://diagramss.us/) – Henry Mar 28 '19 at 09:03
4

Reconciling the answer by @Rscrill with actual discrete data, consider

$$\log(Y_t) = a\log(X_t) + b,\;\;\; \log(Y_{t-1}) = a\log(X_{t-1}) + b$$

$$\implies \log(Y_t) - \log(Y_{t-1}) = a\left[\log(X_t)-\log(X_{t-1})\right]$$

But

$$\log(Y_t) - \log(Y_{t-1}) = \log\left(\frac{Y_t}{Y_{t-1}}\right) \equiv \log\left(\frac{Y_{t-1}+\Delta Y_t}{Y_{t-1}}\right) = \log\left(1+\frac{\Delta Y_t}{Y_{t-1}}\right)$$

$\frac{\Delta Y_t}{Y_{t-1}}$ is the percentage change of $Y$ between periods $t-1$ and $t$, or the growth rate of $Y_t$, say $g_{Y_{t}}$. When it is smaller than $0.1$, we have that an acceptable approximation is

$$\log\left(1+\frac{\Delta Y_t}{Y_{t-1}}\right) \approx \frac{\Delta Y_t}{Y_{t-1}}=g_{Y_{t}}$$

Therefore we get

$$g_{Y_{t}}\approx ag_{X_{t}}$$

which validates in empirical studies the theoretical treatment of @Rscrill.

Alecos Papadopoulos
  • 52,923
  • 5
  • 131
  • 241
2

A linear relationship between the logs is equivalent to a power law dependence: $$Y \sim X^\alpha$$ In physics such behavior means that the system is scale free or scale invariant. As an example, if $X$ is distance or time this means that the dependence on $X$ cannot be characterized by a characteristic length or time scale (as opposed to exponential decays). As a result, such a system exhibits a long-range dependence of the $Y$ on $X$.

Itamar
  • 789
  • 4
  • 10