3

When using IPTW, one can easily get weights 10 or even 20 for the observations.

For instance, in logistic regression, weight 10 for an observation means that we have not one, but 10 observations identical to this one. Thus, if we are allowed to assign weights > 1 to observations, then we can take, e.g., a sample of 100 observations, assign weight 10 to each one of them, and thus obtain a sample of 1000 observations, with totally lesser p-values etc.. But that will be evidently illegal.

Why assigning weights > 1 is legal in IPTW? Is IPTW with weights >1 legal?

Pavel Ruzankin
  • 121
  • 5
  • 1
    Don't confuse frequency weights with probability weights. – AdamO Aug 04 '21 at 16:52
  • @AdamO You are right, of course. But frequency weights and probability weights have very similar meanings. They both denote that an observation is "n times more significant" than another one. – Pavel Ruzankin Aug 04 '21 at 16:57
  • 1
    And they also mean that one observation becomes "n times less significant" than another one. Weights simply say that one observation should be made more important and others less important in the estimation procedure (e.g. individuals who are unlikely to receive treatment should be given the same weight as individuals who are very likely to receive treatment to remove confounding bias and mimic a randomized trial (where everyone has equal weight to receive the treatment)). However, weights do not mean your inference should act like you have a much larger sample size, opposite is usually true. –  Aug 04 '21 at 17:16
  • Also, inverse probability weights are always larger than one, so it is only the normalized weights that have any meaning. You might very well see that for the normalized weights, you are actually down-weighting most observations. –  Aug 04 '21 at 17:22
  • @LarsvanderLaan There is subtle moment. Even if we "exaggerate the importance" of a very small portion of data "deminishing the importance" of all the remaining observations, then it will still be incorrect. Even a small portion of data cannot mean more than it is. Otherwise we have to prove the correctness explicitly. – Pavel Ruzankin Aug 04 '21 at 17:30
  • 2
    What you say isn't true/does not make sense. To be blunt, I do not think you understand what IPW estimators are/do/etc. There is plenty of literature online that explains what IPW estimators are and why they are used, which I recommend searching for. IPW estimators are not some philosophical object, they are simply a way to estimate certain target parameters. For some target parameters, we are interested in parts of the data that are not well-supported. So in fact, some observations can most definitely be more important than others. –  Aug 04 '21 at 17:42
  • For example, suppose I want to know how well a treatment will perform for my population (e.g. the USA). Suppose there is an observational trial that is 90% males which has recorded the treatment assignment and outcome for each subject. If I wish to understand how this treatment will do on my population (which is roughly 50% male and 50% female), I better give more weight to females in my analysis. Otherwise, I will have extremely biased treatment effect estimates that capture how well the treatment does for males, and does not capture how well it does for females, and thus isn't representative –  Aug 04 '21 at 17:46
  • Consider that IPW estimators are invariant to a constant multiplicative factor on the weights. Multiply all your weights by .001, and estimate the potential outcome means using the Hajek (WLS) estimator. Use a sandwich standard error. You will see that the answer doesn't change. – Noah Aug 05 '21 at 04:44
  • @Noah It is one of the points of my question. If an estimator is invariant to scaling the weights, then there is still an assumed total sample size somehow participating in the estimator. If the sample size is assumed to be equal to the number of observations, then some "frequency" weights are assumed to be >1, since the weights vary. – Pavel Ruzankin Aug 09 '21 at 06:46

1 Answers1

9

That is not how it works. The inference based on logistic regression is not correct when you incorporate weighting. You need to estimate the variance of the IPTW estimator, which happens to be inversely related to the propensity score. So large weights also lead to large variance estimates and thus larger p-values.

(Also, with IPTW, all weights are larger than one since it is the inverse of a probability).

Here is a ultra mini-lesson on IPW estimators. Suppose you observe the data structure $(X,A,Y)$ where $X$ is a vector of covariates, $A$ is a binary treatment, and $Y$ is some outcome. Let $\pi_0(x):= P(A=1|X=x)$ be the propensity score. Suppose we are interested in estimating the treatment-specific mean parameter $Psi := E_XE[Y|A=1,X]$. Consider the identity $$E_XE[Y|A=1,X] = E_X \left[\frac{E[Y|A,X]1(A=1)}{\pi_0(X)}\right] = E_X \left[\frac{Y1(A=1)}{\pi_0(X)}\right],$$ which follows from a conditioning argument. This suggests the following IPW estimator of $\Psi$: $$\hat \Psi_n := \frac{1}{n} \sum_{i=1}^n \frac{Y_i1(A_i=1)}{\pi_0(X_i)}$$ where we unrealistically assume that $\pi_0$ is known. Since $\hat \Psi_n$ is just an average of a random variable, inference is easy. We have $$\sqrt{n}(\hat \Psi_n - \Psi) \rightarrow_d N\left(0, \text{Var}_0\{Y1(A=1)/\pi_0(X)\} \right).$$ Now, note that the variance depends on the weight $w_0=\frac{1}{\pi_0(X)}$, so a larger weight gives a larger variance.

You are free to replace $\pi_0$ with a logistic regression estimate of $\pi_n$, but this is usually not a $\sqrt{n}$-consistent or asymptotically normal estimator without very strong conditions.

Note, $E[Y|A=1]$ and $E_XE[Y|A=1,X]$ are two very different parameters.

  • So, the results of IPTW can be used in IPTW estimator only? – Pavel Ruzankin Aug 04 '21 at 16:23
  • IPTW is a sort-of estimate adjustment method, but it also changes the method of inference. I recommend reading into some introductory causal inference on IPW estimators. –  Aug 04 '21 at 16:35
  • I am not aware of introductory courses that state which analyzes can and which cannot be used with IPTW. Are there such courses? Is there a course where the impossibility of using logistic regression is explained? – Pavel Ruzankin Aug 04 '21 at 16:40
  • I added a short discussion about IPW estimators to my answer, but I am afraid that there is a bit too much to this topic to answer your questions. –  Aug 04 '21 at 17:12