0

So suppose after a simpple OLS regression we want to know what the chance(P-value) is that the Beta coefficient is 0 . First we assume that many random processes caused the errors ($\epsilon\!_i$), meaning that the residual is normally distributed white noise with mean of μ and standard deviation of σ.

  1. How can we deduct the shape and standard deviation of the beta coefficient's probability distribution using this information. IE. How does the second formula presented below (also equaling the standard deviation of the errors) arise from this fact?

  2. How does increasing the sample size affect the confidence that the beta coefficient is 0, when the population mean and variance are known (that is Z distribution is used)?

Here are the relevant mathematical formulas:

Simple regression:

$X=Y+\epsilon\!_i$

B's standard error(=standard deviation of B): $\sqrt[2]{\dfrac{\sum_{1}^{n}(\epsilon\!_i)^2}{N}}$ = $\sqrt[2]{\dfrac{\sum_{1}^{n}(Y_i-Ý_i)^2}{N}}$

Z-value =$B/S(B)$ = $B$$\sqrt[2]{\dfrac{N}{\sum_{1}^{n}(Y_i-Ý_i)^2}}$

If $(Y-Ý)$ is constant the standard error won't change as a result of increasing N and thus the p-value won't change either. Is this reasoning correct?

Tony
  • 303
  • 2
  • 11
  • This post, as currently written, is likely to engender debate and confusion. You need to distinguish more clearly between a coefficient and its estimate, as well as indicate whether you are conceiving of OLS in a Bayesian sense (with a prior probability distribution for the coefficient) or whether you really mean to be asking about the *p-values* of tests of the coefficients instead of *probabilities*. Have you searched our site for related posts? This material is *very extensively* discussed here. – whuber May 28 '15 at 21:45
  • @whuber I tried searching for an answer and found nothing. If you can find an answer to the question I will gladly delete this post. Yes I am talking about p-values... So I am indeed talking about confidence rather than probabilities. – Tony May 29 '15 at 00:34
  • 2
    Are you aware of the result that a linear combination of multivarate normal random variables (which includes independent normals) will itself be normally distributed? – Glen_b May 29 '15 at 01:03
  • The result is discussed on wikipedia here [*Multivariate normal distribution - Affine transformation*](http://en.wikipedia.org/wiki/Multivariate_normal_distribution#Affine_transformation). Outlines of proofs can be found on site, for example [here](http://stats.stackexchange.com/questions/137518/proof-for-linear-combination-of-multivariate-normal-x). That you need joint normality is discussed in many places on site, e.g. [here](http://stats.stackexchange.com/questions/22879/linear-combination-of-two-dependent-multivariate-normal-random-variables). ... (ctd) – Glen_b May 29 '15 at 01:54
  • 1
    (ctd) ... A google search using relevant keywords turns up several sets of notes relating to this result. If you don't know the result, there's no reason to suspect that parameter estimates should be jointly normal (but if you do, it's easy). I think your question needs further clarification, however. Are you only talking about simple (one-x) regression? Where do your formulas come from? – Glen_b May 29 '15 at 01:54
  • @Glen_b My bad I typed the formula without squaring it. The formulas can be had for example here: http://stattrek.com/regression/slope-test.aspx. I checked the links and either they go over my head, or I can't relate them to my question. – Tony May 29 '15 at 11:11

0 Answers0