Extraordinary claims and extraordinary evidence

[mcc1789]

Any person who knows of scientific skepticism will likely have heard or read the phrase "extraordinary claims require extraordinary evidence", usually attributed to Carl Sagan. The problem naturally arises in deciding what counts as extraordinary claims or extraordinary evidence. Writing of a conversation with Richard Dawkins, Rupert Sheldrake says he retorted to this with by saying that because (according to him) most people in the UK believe they have experienced telepathy, it is not an extraordinary claim.

So what defines either of these things? If a majority believe something is true (such as telepathy) does that make it an "ordinary" claim which "ordinary" evidence will suffice for? Is the entire concept unworkable because it is overly subjective, or sets up a double standard, as some have suggested? I submit Dr. Sheldrake's contention is in danger of becoming a bandwagon appeal-a majority believes telepathy is real, therefore it is.

Telepathy is a difficult example, however. So perhaps we should examine a more profound example to examine whether this concept holds up. Take resurrection of the dead, for instance, a quite verifiable phenomenon, assuming it is true. For instance, few people would not take someone at their word in saying that they had seen a car accident while driving to the store. If a person claimed to have witnessed a dead person rise, though, they would likely be far more skeptical.

I submit that this is quite reasonable, for in our experience car accidents happen frequently while resurrections do not, if at all. I do not mean to claim that, again, because most people have no experience of something that makes it untrue. However, in terms of judging extraordinariness versus ordinariness it does seem that resurrections qualify as extraordinary, and car accidents ordinary. 

Now, as to the extraordinary evidence part, I am reminded of David Hume's view, that testimony cannot establish miracles like resurrection, unless it would be more miraculous for those giving testimony to be mistaken. Unfortunately, it is not clear what he meant by this exactly, but perhaps one example would be with so many independent witnesses relating the same exact facts that it would be more extraordinary for them all to be lying or in some way mistaken. 

So what do you think? Is this concept valid, or not?

[unkleE]

I think I agree with your opening - who gets to define extraordinary?

If it's what the majority believe, then a lot of beliefs (in God, in ESP, in angels, in good luck, etc) would not be extraordinary. If it's based on the frequency of occurrence of such events, then we'd all disagree on that because it more or less assumes the answer. So in the end it usually comes down to what we feel is extraordinary, which is highly variable. I think it turns out that the saying is smart, catchy, but glib and in the long run a little useless.

The car vs resurrection example is fair in establishing relativities, but it doesn't establish absolutes as required by the saying. As a christian, I believe in the resurrection of Jesus, but I don't think resurrections are a common event, so I would need reasonable evidence to believe in a modern day resurrection. But I wouldn't require "extraordinary" evidence, just reasonable evidence, because I believe God can do such things, even if he doesn't do it often.

In the end, the only reasonable way, I think, is the obvious way we all use - which of the possible explanations of an event is the most plausible? Whether the evidence is extraordinary or not, if it is the most plausible, that's the one we should accept, at least provisionally. The probabilities of all options have to add to 1.0, but often we are sceptical of all explanations, which is unreasonable.

[mcc1789]

Well as I discussed, it seems like going by our experiences we can find a guideline. Even if the majority believes something, it can still be extraordinary if that doesn't just happen on a day-to-day basis in most cases. If it did, then the claim would not be extraordinary at all and probably already established.

I'm not sure whether the saying is intended to be an absolute anyway, but I don't think it has to be. Would you need any different evidence for belief in a modern day resurrection than you need to believe in Jesus' resurrection? I guess it depends on why you believe the resurrection happened. Perhaps a better saying would be simply "reasonable claims require reasonable evidence", though of course what's "reasonable" would then become an issue.

Plausibility seems to be what the saying is aiming at, but it's problematic for the reasons we both lay out, as people will have different views of what's plausible. So thus personally I don't find resurrections or telepathy very plausible (though not impossible) while others do. I also think "I don't know" can be a perfectly acceptable answer (although obviously not very satisfying, so the temptation of coming up with one anyway arises).

[unkleE]

Dec 1, 2015 14:17:45 GMT mcc1789 said:

Well as I discussed, it seems like going by our experiences we can find a guideline. Even if the majority believes something, it can still be extraordinary if that doesn't just happen on a day-to-day basis in most cases.

But take divine healing as an example. Naturalists say it can't happen, some Pentecostals say it happens all the time, and I say it happens occasionally in an obvious way and likely more often in less obvious ways. So we disagree at the start whether it is "extraordinary" or not. (This also shows there are different meanings to the word extraordinary, from "beyond the ordinary" to "amazingly difficult to believe".)

I think, further, that these differences are based on differences in assumptions and assessment of evidence. To generalise rather drastically, naturalists tend to dismiss all miracle claims without investigation (of course some do hunt out obviously false claims, but still tend to assume miracles can't happen when they dismiss some of the better justified claims), Pentecostals and others for whom healing is an integral part of their faith, tend to believe stories without any more than hearsay evidence. I think both those approaches are not fair to the evidence, and I have tried to "collect" miracle stories that are well-evidenced and believable.

So I think each viewpoint concludes according to its assumptions, and until we can agree on some basics, the division will continue.

Would you need any different evidence for belief in a modern day resurrection than you need to believe in Jesus' resurrection? I guess it depends on why you believe the resurrection happened.

I think it depends on all aspects of each case. Modern claimed resurrections may have medical evidence which we don't have for Jesus, but modern day resurrections don't involve a person who was believed to do miracles, speak on God's behalf and implicitly claimed to be "son of God", nor are they instrumental in starting a movement that changed people's lives and changed the world dramatically, and to boast 2 billion plus followers. It is easy to find reasons to believe either way, and we each make our choice.

I also think "I don't know" can be a perfectly acceptable answer (although obviously not very satisfying, so the temptation of coming up with one anyway arises).

(1) There are some questions where "I don't know" is an acceptable answer (e.g. Who do you think will win the football? or Will the universe collapse in on itself one day?) but (2) there are others where the urgency or risk make answering more imperative (e.g. What is the best way out of this burning building? or Will we abort this baby?), or (3) where "I don't know" isn't an answer at all (e.g. Will you marry me?). The question then is, in which category are questions like Does God exist? and Was Jesus resurrected?

I think those questions are closer to #2 & #3 than to #1. If Jesus spoke the truth, God wants people who believe in him and act in the world in a way consistent with his teaching - he is looking for a response. "I don't know" is an acceptable response for a while, but like GK Chesterton said "The object of an open mind is the same as an open mouth - to close it on something solid." So I think we have to accept uncertainty and go with the option we think is most plausible. If I think (as I do) that the evidence points to God existing and Jesus being resurrected as being more likely than not, it is rational and sensible of me to act on that conclusion even if the evidence is merely "most likely" and not "extraordinary".

I think this is a useful discussion, and I appreciate the opportunity to interact with you. Thanks.

[mcc1789]

I think divine healing can be divided on the same lines. Growing back limbs, for instance, seems to happen rarely if ever (I know of one claim, though it was from centuries ago). Even among miracle claims, therefore, it seems extraordinary. If someone says extraordinary means impossible to believe or near enough, that seems to be the wrong definition. For me, it just means "out of the ordinary." Not as likely, but hardly impossible. Claiming something is impossible seems like a bad idea in general.

Why do those things make claims for a resurrection more credible? Respectfully, I think this may also be dangerously close to the bandwagon appeal.

In some cases of course, failing to decide will cause the decision to be made through inaction (i.e. in your abortion example, failing to decide will result in the pregnancy continuing to birth, assuming miscarriage does not happen). I didn't mean to say that one can't make a plausible conclusion, even if it's very possibly wrong. To me it's more of a counsel against being overly certain or leaping to conclusions.

You're welcome, I've searching for places like this with fruitful discussion for some time. It's good finding this.

[mcc1789]

Dec 1, 2015 21:31:16 GMT mcc1789 said:

I think divine healing can be divided on the same lines. Growing back limbs, for instance, seems to happen rarely if ever (I know of one claim, though it was from the 1600s. Even among miracle claims, therefore, it seems extraordinary. If someone says extraordinary means impossible to believe or near enough, that would seem to be the wrong definition. For me, it just means "out of the ordinary." Not as likely, but hardly impossible. Claiming something is impossible seems like a bad idea.

Why do those things make claims for a resurrection more credible? Respectfully, I think this may also be dangerously close to the bandwagon appeal.

In some cases of course, failing to decide will cause the decision to be made through inaction (i.e. in your abortion example, failing to decide will just result in the pregnancy continuing to birth, assuming miscarriage does not happen). I didn't mean to say that one can't make a plausible conclusion, even if it's very possibly wrong. To me it's more of a counsel against being overly certain or leaping to conclusions.

You're welcome, I've searching for places like this with fruitful discussion for some time. It's good finding this.

[unkleE]

Dec 1, 2015 21:31:16 GMT mcc1789 said:

I think divine healing can be divided on the same lines. Growing back limbs, for instance, seems to happen rarely if ever (I know of one claim, though it was from centuries ago). Even among miracle claims, therefore, it seems extraordinary. If someone says extraordinary means impossible to believe or near enough, that seems to be the wrong definition. For me, it just means "out of the ordinary." Not as likely, but hardly impossible. Claiming something is impossible seems like a bad idea in general.

So where does this leave us? We seem to agree that extraordinary doesn't mean impossible or near enough, but more like low probability (I presume that's what "out of the ordinary" means). But we're still left with deciding how we assess that. If we investigate 1000 cases of claimed divine healing, and find that only one involved resurrection, while 500 involved restoration of hearing, we might reasonably say that resurrection claims are infrequent and hearing claims are relatively frequent. Or if we investigate 10,000 claims of healing and find 20% of healing of deafness claims cannot be easily explained medically, while none of the resurrection claims can be supported medically, we might again draw a conclusion on probability. But failing that, we are left with personal experience, and if a christian attends lots of healing meetings while an atheist attends lots of debunking meetings, they will easily come to very different assessments.

I feel we can agree that claims that are broadly less probable require more evidence before we believe them, but I can't see practically how this can be applied until someone does an enormous amount of research, and both sides accept the methodology and results. In other words, I think it will remain a standoff that unfortunately ends up too often in name-calling.

Why do those things make claims for a resurrection more credible? Respectfully, I think this may also be dangerously close to the bandwagon appeal.

It's all a matter of context. If I told you I went out in the street and did a sample of people's heights, and in Sydney the average height was 6 ft 6 ins, you would doubt my result. But if you found out that I had sampled outside a basketball stadium as the players were leaving, you would accept the result though doubt my common sense. Context changes assessments.

So if someone says Benny Hinn or James Randi were resurrected, I'd have my doubts. I'd think it more likely that there was fraud or misrepresentation or a trick. But if someone says the son of God was resurrected, I'd want to ask why they think that person was the son of God, and if I thought their reasons were logical, I'd think it less likely there would be fraud, misrepresentation or a trick, and if there was good evidence, then I'd be willing to believe it.

In some cases of course, failing to decide will cause the decision to be made through inaction (i.e. in your abortion example, failing to decide will result in the pregnancy continuing to birth, assuming miscarriage does not happen). I didn't mean to say that one can't make a plausible conclusion, even if it's very possibly wrong. To me it's more of a counsel against being overly certain or leaping to conclusions.

Yes, I understand and agree. But I still think it is the case that christian claims are too monumental to justify saying "there's nothing wrong with saying 'I don't know'". It's sort of like a different form of Pascal's Wager and Sagan's statement that we're discussing - "High risk claims require a decision even if it can't be certain."

[mcc1789]

Yes, of course a lot of study would have to be done. This concept is a guideline at best.



It would depend on the evidence, yes. My reply was specifically about your reference to the billions who believe Christianity. I don't think numbers of believers will show whether it's true. Islam has almost the same numbers, for instance. Assuming those ever surpass Christianity, that will say nothing by itself about if Islam is true (or false).



I wasn't referring to Christianity specifically when saying that. For myself, I don't think Christianity is true on the balance of probabilities, but it would be more correct to say "I don't believe, but I'm not certain." I'm not sure this requires a decision, however. The evidence for this and other claims can be very difficult to assess, so it may be best to delay making a decision until you have more facts either way before doing so. Regarding Pascal's Wager, for instance, there seem to be too many excluded possibilities that would need weighing. 

[unkleE]

Dec 2, 2015 13:53:06 GMT mcc1789 said:

It would depend on the evidence, yes. My reply was specifically about your reference to the billions who believe Christianity. I don't think numbers of believers will show whether it's true. Islam has almost the same numbers, for instance. Assuming those ever surpass Christianity, that will say nothing by itself about if Islam is true (or false).

I think the "success" of christianity is part of the evidence. If I had to set up criteria to guide how I would judge whether a religion is true, historical evidence would be an important part, but I'd also expect it to have some success, otherwise it's hard to see that God is behind it. Of course this is very subjective, and one can reasonably ask what level of success is my measure? And I don't know. But I can say that I would find it hard to believe that Jesus was a special messenger from God if his teachings totally died out in a generation!

I'm not sure this requires a decision, however. The evidence for this and other claims can be very difficult to assess, so it may be best to delay making a decision until you have more facts either way before doing so.

Obviously taking time to decide is wise, but of course none of us knows how much time we have. If we think a belief is the most likely but still very uncertain, we may be able take it up as the most likely option until and if new evidence comes along. If it's true, we should be confirmed in our choice, and vice versa. That is how science works, and I think belief can work that way too. Of course we wouldn't want to keep changing all the time, so we'd want to give every step good consideration, but I think waiting too long ends up nowhere.

I'm not sure how practical those thoughts are as we're all human beings, not decision-making robots. But Jesus seemed OK with a guy who prayed "I believe, help my lack of belief."

Regarding Pascal's Wager, for instance, there seem to be too many excluded possibilities that would need weighing. 

I was only referring to my suggestion that a better aphorism is: "High risk claims require a decision even if it can't be certain." - which is sort of analagous to Pascal's Wager.

[mcc1789]

Dec 3, 2015 7:45:03 GMT unkleE said:

Dec 2, 2015 13:53:06 GMT mcc1789 said:

I think the "success" of christianity is part of the evidence. If I had to set up criteria to guide how I would judge whether a religion is true, historical evidence would be an important part, but I'd also expect it to have some success, otherwise it's hard to see that God is behind it. Of course this is very subjective, and one can reasonably ask what level of success is my measure? And I don't know. But I can say that I would find it hard to believe that Jesus was a special messenger from God if his teachings totally died out in a generation!

    
Well yes, if it had died out entirely, there would be no claim of course. I'm just noting that something can be both successful and factually untrue, as I feel sure you'd agree. 

Obviously taking time to decide is wise, but of course none of us knows how much time we have. If we think a belief is the most likely but still very uncertain, we may be able take it up as the most likely option until and if new evidence comes along. If it's true, we should be confirmed in our choice, and vice versa. That is how science works, and I think belief can work that way too. Of course we wouldn't want to keep changing all the time, so we'd want to give every step good consideration, but I think waiting too long ends up nowhere. I'm not sure how practical those thoughts are as we're all human beings, not decision-making robots. But Jesus seemed OK with a guy who prayed "I believe, help my lack of belief."

I guess I'm not used to thinking that belief has an urgency about it due to future consequences, but from a Christian perspective it's sensible.
Was that Augustine of Hippo that you're referencing?

I was only referring to my suggestion that a better aphorism is: "High risk claims require a decision even if it can't be certain." - which is sort of analagous to Pascal's Wager.

All right then, your aphorism makes sense.

[ignorantianescia]

I agree that defining "extraordinary" in a satisfying way is a significant snag for the motto. I also think that "extraordinary evidence" is more fatally ambiguous than "extraordinary claims". We can use science as a fairly objective frame of reference for the latter and identify "extraordinary claims" as those claims that run strongly counter to scientific knowledge. Sure, it's imprecise, but I think we have a working sense of that part. However, it implicitly defines methodological naturalism as its scope and the question about what to do when a claim moves beyond that scope is a valid one.

But what is "extraordinary evidence"? It seems to be varyingly used as any point on a scale running from "plausible" to "sufficient" to "cogent" to "supernatural". I think a more modest definition is preferable. For instance, if somebody claims that America was inhabited 5 million years ago by humans who had mastered fire and tool working, that's certainly a extraordinary claim, tremendously so even. But if remains of hominids of that age are found alongside fire-hardened tools or remains of cremations, that is excellent evidence for the claim. But is it extraordinary? In a sense, but on the same token it's also very mundane. I think a phrase like "extraordinary evidence requires sufficient evidence" would be fairer, but it 1) shifts the semantic argument to "sufficient", 2) would be regarded as a dumb truism by many and 3) clearly lacks the rhetorical fireworks that made the original almost a cliché.

Dec 3, 2015 13:49:48 GMT mcc1789 said:

Was that Augustine of Hippo that you're referencing?

It's a reference to Mark 9: 24.

[mcc1789]

Dec 3, 2015 17:19:05 GMT ignorantianescia said:

I agree that defining "extraordinary" in a satisfying way is a significant snag for the motto. I also think that "extraordinary evidence" is more fatally ambiguous than "extraordinary claims". We can use science as a fairly objective frame of reference for the latter and identify "extraordinary claims" as those claims that run strongly counter to scientific knowledge. Sure, it's imprecise, but I think we have a working sense of that part. However, it implicitly defines methodological naturalism as its scope and the question about what to do when a claim moves beyond that scope is a valid one.

But what is "extraordinary evidence"? It seems to be varyingly used as any point on a scale running from "plausible" to "sufficient" to "cogent" to "supernatural". I think a more modest definition is preferable. For instance, if somebody claims that America was inhabited 5 million years ago by humans who had mastered fire and tool working, that's certainly a extraordinary claim, tremendously so even. But if remains of hominids of that age are found alongside fire-hardened tools or remains of cremations, that is excellent evidence for the claim. But is it extraordinary? In a sense, but on the same token it's also very mundane. I think a phrase like "extraordinary evidence requires sufficient evidence" would be fairer, but it 1) shifts the semantic argument to "sufficient", 2) would be regarded as a dumb truism by many and 3) clearly lacks the rhetorical fireworks that made the original almost a cliché. 

I'm beginning to see that aphorisms like this don't tell you enough. There's another one that I like by Philip K. Dick: "Reality is that which, when you stop believing in it, doesn't go away." That has the same problems when I start to evaluate it however. While your example is extraordinary to a degree, it seems to be aimed at the more paranormal or supernatural claims (though defining those also has problems). For most, there is usually only anecdotes, often second or third hand. I don't find that sufficient myself. Of course, it's hard to turn "most such claims, in my opinion, lack sufficient evidence to believe them" into a catchy saying. However, I do believe that such claims, in theory at least, can be proven. I don't agree with the concept of methodological naturalism-it seems like a copout. Assuming supernatural claims have empirical evidence for them which can be evaluated, they should be examined like the rest (again, without attempting to define supernatural at this point). 

It's a reference to Mark 9: 24.

Ah, I see. It reminded me of Augustine's prayer "Lord, give me chastity and continence, but not yet."

[ignorantianescia]

This aphorism is by the way a pet peeve of William Lane Craig, who has responded to it several times (usually in relation to the resurrection).



He also addressed it in writing (with Craig's emphases):

www.reasonablefaith.org/stephen-law-on-the-non-existence-of-jesus-of-nazareth

1. Extraordinary claims require extraordinary evidence.

This sounds so commonsensical, doesn’t it? But in fact it is demonstrably false. Probability theorists studying what sort of evidence it would take to establish a highly improbable event came to realize that if you just weigh the improbability of the event against the reliability of the testimony, we’d have to be sceptical of many commonly accepted claims. Rather what’s crucial is the probability that we should have the evidence we do if the extraordinary event had not occurred.3 This can easily offset any improbability of the event itself. In the case of the resurrection of Jesus, for example, this means that we must also ask, “What is the probability of the facts of the empty tomb, the post-mortem appearances, and the origin of the disciples’ belief in Jesus’ resurrection, if the resurrection had not occurred?” It is highly, highly, highly, improbable that we should have that evidence if the resurrection had not occurred.

In other words, the aphorism fails to follow Bayes' Theorem according to Craig.

[mcc1789]

I'd be more surprised if he liked the aphorism. However, I'm not familiar with Bayes' Theorem or probability in general. So how does one calculate that? A resurrection seems pretty improbable from any common-sense standard, more improbable than having a number drawn from the lottery. We know random numbers are selected from lotteries very commonly. Resurrections, however, appear to be rare claims even according to believers. I'm not sure what amount of evidence would make that more or less improbable.

[unkleE]

Dec 3, 2015 13:49:48 GMT mcc1789 said:

Well yes, if it had died out entirely, there would be no claim of course. I'm just noting that something can be both successful and factually untrue, as I feel sure you'd agree.

Yes I do, of course. But I find it difficult to believe that a religion God started (i.e. factually true) would be totally unsuccessful.

I guess I'm not used to thinking that belief has an urgency about it due to future consequences, but from a Christian perspective it's sensible.
Was that Augustine of Hippo that you're referencing?

If you're referring to "I believe, help my unbelief", then its Mark9:24 as already said.

I'm beginning to see that aphorisms like this don't tell you enough.

If the matter being referenced is complex, the aphorism can hardly fail to be simplistic. But I guess some aphorisms will be both snappy (as this one surely is) and actually true.

However, I do believe that such claims, in theory at least, can be proven. I don't agree with the concept of methodological naturalism-it seems like a copout. Assuming supernatural claims have empirical evidence for them which can be evaluated, they should be examined like the rest (again, without attempting to define supernatural at this point).

The difficulty, I think, is that the naturalist will always believe that a natural explanation is to be preferred, even if none is currently available. Take Intelligent Design. Suppose it is true that God didn't just start the universe off, but that he interferes in it from time to time to help evolution along (e.g. abiogenesis, the eye, the disappearance of dinosaurs and the beginning of primates able to develop language). I don't believe this myself, but let's say it was true. The ID-ers could never really prove that God did any of these because his actions would not be observable or measurable, so the effects of his actions cannot be linked to him by observing cause and effect. ID could say that there is no other explanation, but I doubt they can prove there cannot possibly be any natural explanation. And the naturalists will keep saying there is an explanation there, even though we haven't yet found it (abiogenesis is an example of them trying many explanations but none of them have been proven yet, as far as I know.) So the situation remains a stalemate.

If we could run the experiment over and over again (which we can't do with evolution, but we can do with, say divine healings) then we may resolve the question by testing how often prayer led to healing and didn't, and how often no prayer led to healing (i.e. spontaneous remission) or didn't. It would take a lot of data, and would be muddied by questions like How do we know no-one outside the experiment is praying? and Which God are they praying to? and Maybe God doesn't answer prayers from that person? and Are all healings equally probable?, but in the end we would have enough data to use Bayes Theorem. But short of that, it is pretty difficult.

I'm not familiar with Bayes' Theorem or probability in general. So how does one calculate that?

Bayes Theorem takes a hypothesis with an assigned prior probability and modifies that probability in the light of new information. The new information is evaluated by asking two questions: 1. How likely is this information to have occurred if the hypothesis is true? and 2. How likely is this information to have occurred if the hypothesis is false? If the information is more likely if the hypothesis is true, the hypothesis is thereby made more likely and a new probability can be calculated.

In the case of divine healing, we might say the prior probability (before evidence) of the hypothesis that God exists and heals is 1 in a hundred (just to pick a number). Suppose we found that of a thousand people prayed for, 50 were healed and 950 were not, whereas of the 1000 not prayed for, 1 fully recovered and 999 did not (that is the new information). Then, if we assume that the second figure (0.001) is the probability of a person being healed if God doesn't exist (i.e. by natural means) and the first figure (0.050) is the probability of a person being healed if God does exist and answers prayers, we can calculate a new probability for the hypothesis that God heals. In this case, if my maths is correct, the probability that God exists has moved from 1 in 100 to 1 in 3 (0.336). Obviously a different prior probability and different experimental results would give different answers.

The matter is further complicated by the fact that the same data could be used to test the hypothesis that prayer has a natural effect, like a placebo or some other psychological effect. The prior probability might be different, but a new probability could be calculated in the same way, but it would be assigned to a different hypothesis. This shows that the supernatural is hard to test. A reasonable person is left to decide what they think without the greater "certainty" of "proof".

Craig hasn't (I presume) tried to put numbers to his resurrection argument, but the logic is the same. If the evidence he refers to, which is accepted by the majority of NT scholars, christian or not, is more likely to have occurred if Jesus was resurrected than if he wasn't, then that evidence increases the probability that Jesus really was resurrected. How much it increases it depends on the assumptions we make about probabilities.