# Extraordinary claims require extraordinary evidence
source: https://ift.tt/w0Yyi8F tags: #literature uid: 202212101125 — In the context of the bayes theorem 202212092211 readings.
What makes evidence extraordinary?
The likelihood ratio is defined as:
Likelihood ratio = Probability of seeing evidence, assuming claim is true
----------------------------------------------------
Probability of seeing evidence, assuming claim is false
To obtain an extreme likelihood ratio, the bottom of the fraction has to be very low. The top of the fraction being very high doesn’t help much. If the top of the fraction is 99% and the bottom is 70%, that’s still not a very extreme ratio, and it doesn’t help much if the top is 99.9999% instead.
So to get extremely strong evidence, we need to see an observation which is very improbable, given “business as usual,” but fairly likely according to the extraordinary claim. This observation would be deserving of the title, “extraordinary evidence”.
What makes claims extraordinary?
An obvious next question is what makes a claim ‘extraordinary’ or ‘ordinary’. This is a deep separate topic, but as an example, consider the claim that the Earth is becoming warmer due to carbon dioxide being added to its atmosphere.
To evaluate the ordinariness or extraordinariness of this claim:
- We don’t ask whether the future consequences of this claim seem extreme or important.
- We don’t ask whether the policies that would be required to address the claim are very costly.
- We ask whether “carbon dioxide warms the atmosphere” or “carbon dioxide fails to warm the atmosphere” seems to conform better to the deep, causal generalizations we already have about carbon dioxide and heat.
- If we’ve already considered the deep causal generalizations like those, we don’t ask about generalizations causally downstream of the deep causal ones we’ve already considered. (E.g., we don’t say, “But on every observed day for the last 200 years, the global temperature has stayed inside the following range; it would be ‘extraordinary’ to leave that range.”)
- These tests suggest that “Large amounts of added carbon dioxide will incrementally warm Earth’s atmosphere” would have been an ‘ordinary’ claim in advance of trying to find any evidence for or against it - it’s just how you would expect a greenhouse gas to work, more or less. Thus, one is not entitled to demand a prediction made by this hypothesis that is wildly unlikely under any other hypothesis before believing it.