In Example 5 here, the authors discuss polling. Voting for candidate 'A' is given the numerical value of 1, and voting for 'B' has the value of 0. The CLT says that the average of a sample of $n$ people has the approximate distribution $$\overline{X} \approx N(p_0,\sigma/\sqrt{n}) $$ where $p_0$ is the true proportion of people who would vote 'A'. From this, and some general knowledge of the normal distribution they write:
This means that we can conservatively say that in 95% of polls of $n$ people the sample mean $\bar{X}$ is within $1/\sqrt n$ of the true mean. The frequentist statistician then takes the interval $\bar{X} ± 1/\sqrt n$ and calls it the 95% confidence interval for $p_0$.
I'm fine with this. However they write further
A word of caution: it is tempting and common, but wrong, to think that there is a 95% probability the true fraction $p_0$ is in the confidence interval. This is subtle, but the error is the same one as thinking you have a disease if a 95% accurate test comes back positive. It’s true that 95% of people taking the test get the correct result. It’s not necessarily true that 95% of positive tests are correct.
This remark has me puzzled. Earlier they said that in 95% of polls (of $n$ people) the confidence interval would contain $p_0$. Why doesn't this mean that $p_0$ would be in the interval with a probability of 95%?
Also, I can't see the analogy with the disease/test scenario (which sounds like the base rate fallacy). I'd appreciate some elaboration on that too.
Thank you!