A seeming paradox with rational agents not coming to the same conclusion given the same data

Question

So one day after a tasty dinner full of bananas, an idea comes to your mind (you are person A) - "What if eating bananas can cure cancer?". Being a scientist at heart, you conduct a double blind study and your data shows you that bananas cure cancer with probability at least 99.99%. "WOW!" - you think, - "I am onto something important here!". You pack your stuff and hastily head towards the central organization for cancer research in your country. "People!", you exclaim entering the building, "I found the cure for cancer!".

They direct you to the head of the organization (person B). You show him your data, and he agrees that your conclusion based on your data is correct. "But, you see," - says the guy, - "this year we got at least 100,000 people like you coming to us and reporting their findings of curing cancer with various kinds of fruits. And we expect at least some of them to have data showing that they cure cancer with probability of 99.99% purely by chance. Therefore I am unconvinced, and you should conduct further research.". "So what?" - you say, - "I didn't conduct all those experiments, I only conducted one experiment, and it showed me that bananas cure cancer with probability 99.99%. Therefore I am bound to believe that bananas cure cancer with probability 99.99%. All the other studies used different fruits, and none of them used bananas, so they are irrelevant for this matter".

Assume that both person A and person B are absolutely rational and they completely trust each other. It is generally assumed that two rational agents given the same knowledge must come to the same conclusions. In this case though it appears to me that there is nothing that person B can tell person A that would lead person A to accept the person B's position. What will happen I think, is that person A will accept that person B should indeed hold the position that it is not clear whether bananas cure cancer or not. And person B will accept that person A is justified in believing that bananas cure cancer with probability 99.99%.

So this appears to be a paradox to me: from one side rational agents given same information must come to the same conclusion, and from the other hand we have here that rational agents have the same information but come to different conclusions. Do you see any resolution to this?

Answer 1

I am afraid that you fell victim to the usual misinterpretation of the (essentially vacuous) dictum "rational agents given the same information must come to the same conclusion". "Information" in this context is not just "data". It includes also the information-processing procedures a rational agent will use. Also, it includes the structure of preferences or attitudes towards uncertainty.

Let's say person A and person B have the same data. Do they process the data in the same way? If not, they do not have "same information" (and there are many different ways to process the data, all compatible with rationality). If yes, do they quantify the costs of making wrong inferences in the same way? If not, (and there are many "cost-quantification" approaches compatible with rationality -say, a risk-averse person is no less rational than a risk-neutral one), they do not have "same information". If yes, they are a priori identical (for all aspects relevant to the issue at hand), and that's why the dictum becomes vacuous, and so no paradox emerges: persons A and B in your specific example, obviously have not the same information, in this more general and complete sense.

"Rationality" is a very minimum set of requirements, having mainly to do with internal consistency. There are a lot of different inferential setups that are internally consistent, and so they all are "rational" -and they lead to different conclusions.

Answer 2

Let me first point out that you appear to have a common misunderstanding about the meaning of p-values. In conventional (frequentist) statistical analysis, the p-value is the probability of getting a sample statistic (say a sample mean) as far or further from the proposed null value as yours, if the null value is the true value. Importantly, there is no such thing as (e.g.) "bananas cure cancer with probability at least 99.99%". The fact that a p-value might be $< 0.0001$ very much does not imply that there is a 99.99% probability the alternative hypothesis is true (or a 0.01% probability the null hypothesis is true). For more on this topic, it may help you to read this CV thread: What is the meaning of p values and t values in statistical tests?

That having been said, it is possible to assert a (subjective) probability associated with the null hypothesis within the Bayesian framework. Bayes' rule is:
$$ Pr(H_0|D) = \frac{Pr(D|H_0)}{Pr(D)}Pr(H_0) $$ In words, the probability that the null hypothesis is true that you should believe after having seen some data is equal to the distinctiveness of the data with respect to the null hypothesis (indexed by the quotient on the RHS) multiplied by the probability that the null hypothesis is true that you believed before having seen the data in question. To make this easier, consider the following example¹:

MAMMOGRAPHY
A reporter for a women's monthly magazine would like to write an article about breast cancer. As a part of her research, she focuses on mammography as an indicator of breast cancer. She wonders what it really means if a woman tests positive for breast cancer during her routine mammography examination. She has the following data:
The probability that a woman who undergoes a mammography will have breast cancer is 1%.
If a woman undergoing a mammography has breast cancer, the probability that she will test positive is 80%.
If a woman undergoing a mammography does not have breast cancer, the probability that she will test positive is 10%.
What is the probability that a woman who has undergone a mammography actually has breast cancer, if she tests positive?

How can we figure out that probability? We must revise the a priori probability that a woman who undergoes a mammography has breast cancer, p(cancer) which according to the text is 1% or p=.01, in light of the new information that the test was positive. That is, we are looking for the conditional probability of p(cancer|positive). The probability of a positive result given breast cancer, p(positive|cancer), is 80% or p=.8, and the probability of a positive result given no breast cancer, p(positive|no cancer), is 10% or p=.1.

Thus, we have:
$$ Pr({\rm cancer|positive}) = \frac{0.80}{\underbrace{0.80\!\times\! 0.01}_{Pr(D)\text{ w/ cancer}}\;+\;\underbrace{0.10\!\times\! 0.99}_{\Pr(D)\text{ w/o cancer}}} 0.01 = 0.075 $$ (The denominator of the fraction in Bayes' rule is often hard for people to understand. In this case, it is possible to enumerate the possible probabilities of the data, and $Pr(D)$ is simply the sum of all the individual enumerated probabilities. For greater clarity, I annotated them here. Often, the set of possible probabilities is much harder to determine. In practice, people often ignore the denominator and replace the equals sign with $\propto$, 'proportional to'.)

Now in this example, the cancer rate is known beforehand. To make this example more like your new research finding example, let's imagine that no one knows exactly what the cancer rate is, but two different doctors believe the cancer rate is 1%, and 5% respectively. If we use the latter value in the equation above, we get:
$$ Pr({\rm cancer|positive}) = \frac{0.80}{0.80\!\times\! 0.05\;+\;0.10\!\times\! 0.95} 0.05 = 0.296 $$ The probability is now 29.6%, which is very different from the 7.5% above. So who is right? We don't really know, but the important part is that both doctors are rational in believing their (very different) probabilities that their patient has breast cancer. To put this a different way, what is rational isn't the probability that each believes, but rather the manner in which they change their belief in light of new evidence. Since both doctors changed their belief using a correct application of Bayes' rule, both are rational, even though they came to different conclusions. The reason they didn't end up with the same probability is because they didn't believe in the same probability beforehand; this is what @AlecosPapadopoulos meant by 'they do not have "same information"'.

_{1. This example is copied from: Sedlmeier, Improving Statistical Reasoning, pp. 8-9.}

Answer 3

The reason that your question appears to be both difficult and strange is that conventional accounts of statistics do not include the philosophy of statistics. In particular for your question we need to consider two normative principles. It is the likelihood principle that suggests that identical evidence should lead to equivalent inferences and it is the repeated sampling principle that suggests that we should assess statistical methods on the basis of how frequently they lead us astray. Both principles are reasonable, but they conflict. Thus we end up in a position where your question can exist.

Inferential methods can be fully compliant with the likelihood principle or with the repeated sampling principle but, often, not with both. Rational minds can disagree about which principle is more important.

Inferences are made by people, not by statistical algorithms, and rational minds can disagree about the acceptable degree of subjectivism and the mechanism for its incorporation into inferential considerations.