The meaning of two sided (double tailed) p-values

Question

A p-value is "the probability for a given statistical model that, when the null hypothesis is true, the statistical summary would be the same as or of greater magnitude than the actual observed results." Source

Thus, the p-value is defined as $\mathbb P_r(X ≥ x|H )$ for a one sided (right tailed) test. For a two sided (double tailed) test, this probability is doubled, meaning $ p = 2 \mathbb P_r(X ≥ x|H ) $ and then compared to $\alpha$ (e.g .05). However, how can this two sided p-value be interpreted as the probability as defined above? The real probability of getting a stat. summary of greater magnitude than the observed result (e.g. +2) is $ \mathbb P_r(X≥2|H)$ and not $2 \mathbb Pr(X≥2|H)$. Yet, statisticians double the right-hand p-value. There is, of course, an error probability on the left side too ($\alpha/2$), but this should be of no concern for us, as our observed value is positive and thus creates a probability only on its right side to $ +\infty $ below the curve.

The question, therefore, is: Would it not be more mathematically accurate to compare $ \mathbb P_r(X≥2|H)$ to $\alpha/2$ $(=.025)$ instead of $2 \mathbb P_r(X≥2|H)$ to $\alpha$ $(=.05)$ in a two sided test? The results would of course be the same in symmetric distributions, but not in asymmetric ones!

Furthermore, if we say that p for double tailed tests is indeed $2 \mathbb P_r(X≥x|H)$, then this $p$ is not the probability of $X$ being of greater magnitude than $x$, but rather this probability times $2$.

Answer 1

I would prefer to consider extreme rather than magnitude

If you are expecting the value $0$ for something thought to have a symmetric distribution but in fact see the value $2$

then other values that you might have seen which would be as extreme as $2$ or more extreme than $2$ would include all values greater than or equal to $2$ and all values less than or equal to $-2$

so the probability of seeing $2$ or a more extreme value can be expressed as $\mathbb P(|X| \ge 2) = \mathbb P(X \ge 2) + \mathbb P(X \le -2) = 2\mathbb P(X \ge 2)$ by symmetry

In asymmetric distributions, you need to decide what extreme means in the context of your experiment

If you had a lightbulb which had an exponentially distributed lifetime L expected to be $1000$ hours, you might decide that extreme means that seeing a lifetime of $4000$ hours or more would be as extreme as seeing a lifetime of about $18.5$ hours or less, or you might take a different view. You could still say $\mathbb P(L \ge 4000) + \mathbb P(L \le 18.5) \approx 2\mathbb P(L \ge 4000) \approx 0.0366$

You should decide what you mean by as extreme or more extreme before you see the actual observation, and this is especially important with a distribution believed to be asymmetric