0

I have difficulty of understanding p-value from
An Introduction to Statistical Learning by Gareth James • Daniela Witten • Trevor Hastie Robert Tibshirani (2015) : 67

Consider simple linear regression equation $$Y= β_0+ β_1 X$$

Null hypothesis states Y doesn't relate to X, Mathematically $β_1\approx0$
To reject Null hypothesis, we should prove $β_1 >> 0$
After performing linear regression I calculated coefficient to be $\hat{β_{1}}$ .But since number of samples are limited, There is no confidence to reject null hypothesis.

So I calculate t-statistic as $$t_{statistic}=\frac{\hat{β_{1}} - 0}{SE(\hat{β_{1}})}$$
It is trivial to understand that higher t-statistic implies more confidence in $β_{1}$ since $β_{1}$ is larger than error in calculation of itself, hence we can reject Null hypothesis for high t-statistic values

From: https://en.wikipedia.org/wiki/Student%27s_t-distribution $$t=\frac{\hat{β_{1}} - {β_1}}{SE(\hat{β_{1}})}=\frac{Error\,in\,calculated\,β_{1}}{(Estimated\,Variance)/(\sqrt{n})}$$

Note here that ${β_1}$ & ${SE(\hat{β_{1}})}$ are random variables and vary with batch of samples observed.
Let's denote t-distribution as $T(t,n)$ and ${t\,statistic}$ as $T_{st}$

Probability of observing ${t_{obs}}\ge{t\,statistic}$ is p-value $${p}{-}{value} = \int_{|t|>|T_{st}|}T(t,n)$$
What is mathematical derivation for ${p}{-}{value}\approx 0 \implies β_1 >> 0$ is large ?

I have a quantitative explanation that t-statistic follows t-distribution, if we assume $H_{0}$ is true, and ${p}{-}{value}$ i.e. $\int_{|t|>|T_{st}|}T(t,n)$ denotes probability of getting a value of "t" $\ge T_{st}$ which being less implies hypothesis is false

  • 1
    Please tell us what you mean by "$P(\beta_1).$" It *looks* like you intend this to be some kind of probability. If that's the case, then it's based on a misconception. You can clear that up by consulting our threads on p-values, such as https://stats.stackexchange.com/questions/31. – whuber Aug 22 '18 at 12:52
  • I have edited it. I have consulted the threads prior posting question –  Aug 22 '18 at 12:56
  • 2
    It's still not correct. P(β1>>0) does not make sense because β1 is not a random variable, it's a parameter. And that's not the correct definition of a p-value anyway, the p-value is concerned with the probability of the data under the null hypothesis, when β1 = 0. I think you need to read up on hypothesis testing first. – Denziloe Aug 22 '18 at 13:12
  • Thanks . But still is there any mathematical derivation proving smaller p-value implies higher β1 ?. I'll edit question to that effect. Please provide me relevant material for reading. ISLR is providing more quantitative explanations –  Aug 22 '18 at 13:15
  • The link I gave in my first comment is highly relevant material. You may supplement that by [searching our site](https://stats.stackexchange.com/questions/tagged/p-value?sort=votes). – whuber Aug 22 '18 at 13:21
  • Can I define p-value as probability of observing a certain t-statistic assuming Null hypothesis and if such a probability is lesser implies hypothesis is false? –  Aug 22 '18 at 13:33
  • No, you can't, sorry, that's not what a p-value is ... you can define a *likelihood* (or more technically a likelihood density), but you will need several more steps in order to do inference in terms of likelihood. – Ben Bolker Aug 22 '18 at 13:34
  • Even if β1 is parameter, why P(β1>>0) does not make sense?. Since all we know about β1 is from a smaller sample set we observed. I can state it as likelihood distribution P(β1>>0 | observed samples set) >> 0 . –  Aug 22 '18 at 13:51

0 Answers0