Why do we use a fixed design analysis of regression coefficients, even for observational data, where the design is not fixed?
For instance: $Var[\hat \beta]=(X'X)^{-1}\sigma^2$ is conditional on $X$. Since $X$ is random in observational studies, this is an under estimate of the true $Var[\hat \beta]$.
Edit: As pointed out by @christoph-hanck, $(X'X)^{-1}$ cannot be, by definition, systematically smaller than $\mathbb{E}(X'X)^{-1}$. Question remains: why do we use fixed design standard errors, instead of random design standard errors?