Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [weighted-regression]

Weighted least squares regression is a generalization of OLS regression used when different data points have different importance, or "weights". See also [weighted-data].

Weighted least squares regression is a generalization of ordinary least squares regression. It can be used when it is believed that the data are not all equally informative about the parameters, e.g., when there is heteroscedasticity.

323 questions
57
votes
9 answers

Reference book for linear algebra applied to statistics?

I have been working in R for a bit and have been faced with things like PCA, SVD, QR decompositions and many such linear algebra results (when inspecting estimating weighted regressions and such) so I wanted to know if anyone has a recommendation on…
Palace Chan
  • 759
  • 3
  • 9
  • 17
29
votes
2 answers

How do you find weights for weighted least squares regression?

I am a bit lost in the process of WLS regression. I have been given dataset and my task is to test whether there is heteroscedascity, and if so I should run WLS regression. I have carried out the test and found evidence for heteroscedascity, so I…
m3div0
  • 569
  • 1
  • 7
  • 11
17
votes
2 answers

R: glm function with family = "binomial" and "weight" specification

I am very confused with how weight works in glm with family="binomial". In my understanding, the likelihood of the glm with family = "binomial" is specified as follows: $$ f(y) = {n\choose{ny}} p^{ny} (1-p)^{n(1-y)} = \exp \left(n \left[ y \log…
FairyOnIce
  • 755
  • 1
  • 7
  • 15
16
votes
1 answer

Ratios in Regression, aka Questions on Kronmal

Recently, randomly browsing questions triggered a memory of on off-hand comment from one of my professors a few years back warning about the usage of ratios in regression models. So I started reading up on this, leading eventually to Kronmal 1993. I…
Affine
  • 2,307
  • 19
  • 25
14
votes
0 answers

How can I measure model performance with weighted logistic regression?

I am working with some survey data that uses probability weights. A number of sources explain that likelihood-based tests and fit statistics like likelihood-ratio, AIC, and BIC are not valid in the context of the weighted MLE. Are there other tests,…
biased_estimator
  • 311
  • 2
  • 7
13
votes
2 answers

Harmonic mean minimizes sum of squared relative errors

I am looking for a reference where it is proven that the harmonic mean $$\bar{x}^h = \frac{n}{\sum_{i=1}^n \frac{1}{x_i}}$$ minimizes ( in $z$) the sum of squared relative errors $$\sum_{i=1}^n \left( \frac{(x_i - z)^2}{x_i}\right).$$
Martin Van der Linden
  • 739
  • 1
  • 6
  • 13
13
votes
2 answers

The theory behind the weights argument in R when using lm()

After a year in grad school, my understanding of "weighted least squares" is the following: let $\mathbf{y} \in \mathbb{R}^n$, $\mathbf{X}$ be some $n \times p$ design matrix, $\boldsymbol\beta \in \mathbb{R}^p$ be a parameter vector,…
Clarinetist
  • 3,761
  • 3
  • 25
  • 70
12
votes
3 answers

Weighted generalized regression in BUGS, JAGS

In R we can "prior weight" a glm regression via the weights parameter. For example: glm.D93 <- glm(counts ~ outcome + treatment, family = poisson(), weights=w) How can this be accomplished in a JAGS or BUGS model? I found some paper discussing…
user28937
  • 141
  • 1
  • 5
12
votes
1 answer

Intuitive explanation for inverse probability of treatment weights (IPTWs) in propensity score weighting?

I understand the mechanics of calculating the weights using the propensity scores $p(x_i)$: \begin{align} w_{i, j={\rm treat}} &= \frac{1}{p(x_i)} \\[5pt] w_{i, j={\rm control}} &= \frac{1}{1-p(x_i)} \end{align} and then applying the weights in…
RobertF
  • 4,380
  • 6
  • 29
  • 46
9
votes
2 answers

Adding weights for highly skewed data sets in logistic regression

I am using a standard version of logistic regression to fit my input variables to binary output variables. However in my problem, the negative outputs (0s) far outnumber the positive outputs (1s). The ratio is 20:1. So when I train a classifier, it…
arahant
  • 141
  • 1
  • 5
9
votes
2 answers

How to determine weights for WLS regression in R?

I am trying to predict age as a function of a set of DNA methylation markers. These predictors are continuous between 0 and 100. When performing OLS regression, I can see that variance increases with age. Thus, I decided to fit a weighted regression…
I. Smeers
  • 111
  • 1
  • 2
  • 9
9
votes
1 answer

Weighted least square weights definition: R lm function vs. $\mathbf W \mathbf A\mathbf x=\mathbf W \mathbf b$

Could anyone tell me why I am getting different results from R weighted least squares and manual solution by matrix operation? Specifically, I am trying to manually solve $\mathbf W \mathbf A\mathbf x=\mathbf W \mathbf b$, where $\mathbf W$ is the…
Haitao Du
  • 32,885
  • 17
  • 118
  • 213
8
votes
2 answers

How to make the rare events corrections described in King and Zeng (2001)?

I have a dataset with a binary (survival) response variable and 3 explanatory variables (A = 3 levels, B = 3 levels, C = 6 levels). In this dataset, the data is well balanced, with 100 individuals per ABC category. I already studied the effect of…
Aurelie
  • 141
  • 7
8
votes
1 answer

Weights argument in lm and lme very different in R- am I using them correctly?

So, it seems to me that the weights function in lm gives observations more weight the larger the associated observation's 'weight' value, while the lme function in lme does precisely the opposite. This can be verified with a simple simulation.…
colin
  • 862
  • 1
  • 11
  • 27
8
votes
3 answers

Using regression weights when $Y$ might be measured with nonzero-mean measurement error

Suppose we observe data $Y, X$ and would like to fit a regression model for $\mathbf{E}[Y \,|\, X]$. Unfortunately, $Y$ is sometimes measured with errors whose mean is nonzero. Let $Z \in \left\{\text{unbiased}, \text{biased}\right\}$ indicate…
Adrian
  • 3,754
  • 1
  • 18
  • 31
1
2 3
…
21 22