4

I am trying to understand the intricacies of Linear Regression (1 predictor, 1 response). I am using Istanbul Stock Exchange dataset (https://www.kaggle.com/uciml/istanbul-stock-exchange) to do the same. I am using the column FTSE as my predictor variable and USD BASED ISE as my response variable.
I am writing my own Python (v 3.5.6) methods for all mathematical computation tasks viz. calculating $\beta$s ($\beta_{0}$ and $\beta_{1}$ in my case), calculating MSE, calculating $R^{2}$ etc. I have successfully calculated $\beta$s and could correctly estimate MSE, $R^{2}$ values.
I am trying to understand Boostrap and use the same to estimate

  1. Confidence intervals for coefficients $\beta_{0}$ and $\beta_{1}$
  2. Calculate t-statistic for $\beta_{0}$ and $\beta_{1}$
  3. Calculate p-value for $\beta_{0}$ and $\beta_{1}$

I am trying to calculate the above mentioned statistics without using any statistical package. I have written a function that takes my input dataset and generates 1,00,000 bootstrap samples out of it. For each of these samples I have obtained $\beta_{0}$ and $\beta_{1}$ values. In a nutshell, I have 1,00,000 $\beta_{0}$ and $\beta_{1}$ values. Using these list of values, how do I calculate statistics mentioned in (1), (2) and (3)

1 Answers1

0

There are in general multiple ways to calculate bootstrap intervals, see Wikipedia for an overview. The case of bootstrap-based confidence intervals for a linear regression is described in detail here.

For instance, let $\hat{\beta}_{1(1)}, \ldots, \hat{\beta}_{1(b)}$ be your $b=1,000,000$ bootstrap estimates of $\beta_1$. Then, a $(1-\alpha)$-confidence interval for $\beta_1$ can be defined by the $\alpha/2$ and the $(1-\alpha/2)$ quantiles of your bootstrap estimates $\hat{\beta}_{1(1)}, \ldots, \hat{\beta}_{1(b)}$.

Edit: Provided more details.

Nussig
  • 470
  • 1
  • 4
  • 11
  • Regarding the p-values, I've seen it described like that before, but other answers tend to point out that quantile-based bootstrapping tends to be supoptimal (see https://stats.stackexchange.com/a/277391/60613) – Firebug Oct 22 '20 at 14:16