Richard Carrier

The issue of Craig's use of the McCullagh criteria in historical argument was raised in my Historical Method thread, which derived from my announcement that I will be giving a talk on historical method in Sacramento this coming weekend for Atheists and Other Freethinkers. I have long noted in correspondence the faults in Craig's abuse of these criteria, but I think it would be useful to air my views here--these issues can be further explored in person this weekend. The following is by no means a thorough or deep analysis, but just a survey of problems I believe lie on the surface.

For an online example, see Visions of Jesus: A Critical Assessment of Gerd Lüdemann's Hallucination Hypothesis. This is not a very good example, since in reality it only argues against one particular version of one particular theory regarding one particular fact related to the resurrection, and not to the whole Resurrection as such. Of course, Craig inappropriately turns his argument into the latter after only accomplishing the former--still a logical fallacy, no matter how much Craig openly admits this problem, as in fact he does here, as apologetically as ever.

But never mind that serious methodological flaw. Much could be said about his argument besides, but I will confine my remarks here to his misapplication of the McCullagh criteria in this one essay, as a mere example of what I mean when I say that he doesn't know what he is doing when he deploys them--rather like an amateur scientist who goofs up scientific method. His errors do not count against the validity of the criteria--when used correctly, they are sound enough, and very useful. It is just that Craig does not use them correctly.

The criteria in question come from C. Behan McCullagh, an expert in the philosophy of history, who draws his paradigms from numerous actual uses by real historians. I will be following McCullagh's own explication and exemplification of the criteria, as well as my own professional experience with history and historians.

The criteria are six in all. I am leaving aside a seventh, which is not a criterion of degree, but merely of qualification: that a hypothesis must entail other observation statements not shared by a competing hypothesis in order for the two hypotheses to be compared. This is little more than an issue of definition, rather like the sometimes-dubbed "Fourth Law" of Thermodynamics. It doesn't really measure any difference between hypotheses, apart from the bare fact of their being different.

But the other six criteria are what one would call "predictors": the more a theory meets each criterion, the more strongly that criterion predicts the theory's truth. Theoretically, one could test the historical analysis on each criterion against quasi-historical claims that can be confirmed or refuted by direct observation (after the fact, hence conceivably producing a double-blind experiment), and see how strongly (i.e. how frequently) each criterion predicted a correct theory, and how this predictive success was affected by multiple criteria working together, and so on. In actual practice, this kind of test has only been done in the arena craftsman's experience: historians have after centuries centered on these criteria as the most commonly-found predictors of the truth.

But it must be remembered that as mere predictors they never entail a theory's truth: success on many predictors can at best produce a sufficient probability to believe one theory over another, but at worst will often produce no definite conclusion at all. But what value they do have derives solely from their being applied correctly. Like any tool, inept use produces only inept results.

So here is how Craig misuses them:

(1) Explanatory Scope (ES): Resurrection (R), he says, "has great explanatory scope" relative to the Luedemann hypothesis (L), because R "explains why the tomb was found empty, why the disciples saw post–mortem appearances of Jesus, and why the Christian faith came into being" whereas L does not explain all these things. Now, Craig almost has the right idea of what ES measures, but he isn't counting all the apples, and isn't always counting the right ones.

First, L does explain the empty tomb story, the existence of which is the only actual fact in evidence. That the story was caused by a tomb actually being empty has to be a historical inference. It therefore cannot be treated as a fact requiring explanation itself. To the contrary, that the tomb was really empty is a part of Craig's hypothesis R, it is not a part of the evidence-set that R must explain. This is also true of the appearances, even though different issues apply to that question.

Second, there are things L explains but R does not, such as why Jesus didn't appear to everyone on earth (a far more rational way to save mankind) but only to a very limited set of Jews, why the testimony Craig relies on comes so late, from such uncertain authors, with so much discrepancy (whereas on a genuine resurrection we should expect things like Peter's memoires--or at the very least Paul's--as well as inscriptions, etc., with lots of remarkable agreement), why the appearance narratives have certain peculiar features (like the unrecognizability of Jesus), why the details of those narratives never get any mention in the Pauline epistles, and so on. This does not mean Craig cannot explain these things--but he has to admit they need explaining, and his theory (R) must actually explain them or else he must concede L wins on ES. But to beat L on ES by providing such explanations, R must then face the consequences of every modification necessary to accomplish that explanation. It is a Catch-22 that Craig must grit his teeth and bear, as every historian must.

Therefore:

Third, to apply this or any criterion correctly, the entire theory and all its implications must be considered, and facts (pieces of evidence) must be correctly distinguished from causal hypotheses. Craig is unwilling to concede anything to L, which is dogmatically unrealistic, and he doesn't seem to grasp the difference between evidence and theory. It is as if he denies there are any peculiar problems with his position at all, including the problem (not a great problem, but an important one all the same) that most of his "facts" are actually hypothetical inferences. It may still be, and I suspect it is so, that R can beat L on this criterion (ES) even after a fair reckoning, given a correct construction of R and the set of all actual facts E, but Craig doesn't even give us these things. It doesn't even seem that he knows how to.

Fourth, it must be noted that ES is the weakest of all six criteria in the McCullagh ABE. It is true that, all else being equal, any (H)ypothesis that explains a larger set of (E)vidence than its competitors (assuming substantial overlap in explananda) is more probably true. But this is, in my experience, only slightly so (it is a weak predictor), and so this advantage is easily overcome by contrary indicators (i.e. the other criteria), as well as by complexification of a competing hypothesis. For instance, just as Craig must greatly complexify R in order to account for all the peculiar facts (such as those I point out above that would otherwise favor L), so can an advocate of L complexify L so as to include explanatory features that account for, e.g., an actual empty tomb. In other words, multiple hypotheses can be merged into a single hypothesis, and in fact Craig himself is required to do this when defining R appropriately, so he should not assume his opponents can't do the same. History isn't easy.

(2) Explanatory Power (EP): Craig says R "has great explanatory power" because "it explains why the body of Jesus was gone, why people repeatedly saw Jesus alive despite his earlier public execution, and so forth," whereas L has insufficient "power to account for the post–mortem appearances and the origin of the disciples' belief in Jesus's resurrection," in particular it weakly explains the "the diversity of the appearances" and "the origin of the disciples’ belief in Jesus's resurrection." Again, Craig almost has the right idea of what EP measures, but he isn't addressing the full implications of both theories (L and R).

First, EP is a measure of what is identified as P(e|h&k) in Bayes Theorem, or the (P)robability that we would have a particular set of (E)vidence if our stated (H)ypothesis is true, given all our relevant Bac(K)ground knowledge. It is thus unlike ES, in that ES measures the number of specific and indivisible facts in E, whereas EP measures the probability that we would have that set E given H and K. This is what Craig has right when he describes L as failing on EP because, though it "explains" certain things like the (D)iversity of appearances and the (O)rigination of resurrection belief, it does so only weakly. That is, though Craig implicitly admits that L could cause D and O, Craig is implying (correctly, in my opinion) that L does not make D and O as probable as R would if R were true.

Second, however, and again, the entire theory and all its implications must be considered. I agree that E is not very probable on L (or in fact on any theory). That is commonly the case in historical analysis--indeed, I would say almost routinely the case. Bayes Theorem can, and often does, start with a very low EP, even something like 0.1 (= 1 in 10), yet still produce a result with extraordinarily high final epistemic certainty (like 0.99 = 99%). This is because, when all competing hypotheses are considered, and the math is worked out correctly, one hypothesis can still be so much superior as an explanation than the others. But this requires (a) considering all competing hypotheses, and (b) correctly assessing each theory's EP. The former, (a), is a mistake we already observed Craig makes throughout. The latter, (b), is a new problem: for it requires addressing everything in E that relates to your favored H. In other words, you can't leave crucial evidence out of account.

Hence:

Third, by ignoring the necessary implications of his own theory, Craig ignores the basic rules of inductive logic. Every proposition entailed by a hypothesis must be as credible as the hypothesis itself (a chain is only as strong as its weakest link). But the Resurrection hypothesis entails at the very least the existence of an (A)ctivist miracle-working God. Craig cannot ignore this implication of the theory in assessing that theory on the McCullagh criteria. Yet he does ignore it, thus destroying the validity of his application of those criteria.

For example, since R requires A, then E must contain all facts pertinent to A. And there's the rub. What is the probability, for example, that (W) the world would be enmeshed in widespread natural evil without any clear act of God to prevent or correct or alleviate or compensate for it, and where, in addition, that God is basically as silent and vague about what he wants as a coy school girl. This does not fit an activist and miracle-working God at all. The question here, then, is not even whether there is a God (though that does introduce still other problems Craig cannot ignore), but whether his hypothesis R, which entails A, actually fits E.
It is clear that it does not. Though A can cause W (just as L can cause D and O), it is certainly not very probable that A would cause W, at least given all that we really know. We cannot normally argue that this probability is higher because there are things we don't know. For that would only establish the possibility, which has already been conceded. It would not affect what we can assess to be the epistemic probability that W on A, which can only be based on what we know, not what we don't (however, see below).

Fourth, that is only one example of a mountain of facts that seem improbable on A, and therefore on R. So it does not seem likely that R exceeds L on EP. The central mistake here bears repeating: Craig ignores the entirety of this problem--because he cheats (consciously or not). By ignoring the implications of his own theory, but harping on the implications of his opponent's, he generates the false air of theoretical superiority. But pull up the rug, and expose all the problems he has slid under there, and suddenly things don't look so clear cut.

Fifth, it should now be clear how peculiar R is as a historical theory. R comes with tons of baggage most historical theories (like L) avoid. L does not entail anything as astonishing as A. For example, L entails (P) that ordinary people hallucinate, which seems unlikely at first glance--but unlike A, P is a scientifically demonstrated fact (especially in cultures more closely resembling that of the ancient disciples). If Craig had that kind of assurance for A, then A would not present a problem for R. But Craig does not have this.
Hence, for all its faults (and there are many, some mentioned by Craig and others besides), L is still not as challenged as R when it comes to fitting the entire set E of all relevant evidence. Craig obscures this fact by pointing out that L does not make E very probable, and then ignoring the even greater problem faced by R. Craig also does not seem aware of the fact that a low EP is not fatal to a theory--a very low EP can still produce a very high final probability that a theory is true. For EP, like all six criteria, is in the final analysis compared relatively. For example, a low EP for any given H can still be ten times higher than all competing hypotheses.

Sixth, and last, it is possible to "jockey" a theory around these difficulties. For example, Craig can posit as a theoretical element some specific reason that A would cause W, and thereby avoid the problem that W is improbable on R (he would thus make W probable by, again, complexifying the theoretical structure of A and hence R). But L can do this, too: make up some "additional" assumption whereby D and O are more probable than Craig argues. In the end, both R and L must face the consequences of this enormous complexification. Already, to beat L, Craig must make R so incredibly complex that it explains a large array of peculiar facts, and also makes those facts more probable than present human knowledge would suggest. That's a hell of a lot of tinkering. And the consequences aren't pretty. Nevertheless they must be honestly faced.

(3) Plausibility (PP): Craig says R "is plausible" because "given the historical context of Jesus's own unparalleled life and claims, the resurrection serves as divine confirmation of those radical claims," whereas L is implausible because "there are insufficient data to do" the "psycho–analysis of Peter and Paul" that L requires, and the evidence suggests that such analysis is mistaken anyway (he adds similar arguments regarding the assessment of the appearances, but the same issues apply so I will only address the psychoanalytic point by way of example). Here Craig doesn't even grasp what PP measures, much less measure it correctly.

First, PP corresponds in Bayes Theorem to P(h|k), the (P)robability that our chosen (H)ypothesis could be true given all our relevant Bac(K)ground knowledge. In short, it measures the prior probability of H, regardless of (most of) what we know about E. Craig's analysis shows no grasp of this at all. Instead, he treats PP as if it were just a repeat of EP. But they are almost exactly the opposite. EP measures the probability of E (the specific evidence) assuming H is true, whereas PP measures the probability of H without any reference to E (the specific evidence), hence not assuming H is true (or even that E is for that matter).

More specifically:

Craig says that (J) "Jesus's own unparalleled life and claims" increases the PP of R. But J is a part of E, not K, and PP = P(h|k). At best a theoretical element of R that explains J can improve EP for R, or if one can establish certain facts pertaining to J this might perhaps decrease EP for L, but neither has any effect on PP.

Craig also says that because L lacks the data it needs to prove its psy(C)hoanalytic claims, that therefore L has a lower PP. But C has nothing to do with PP. Just as Craig must necessarily import dozens of theoretical elements that have no evidential support (like a solution to the A-->W problem, among many other things already noted above, and many others besides), it would be a major foot-in-mouth for him to criticize L for doing exactly the same. And, moreover, the effect this has on PP is different than what Craig seems to understand. But the point for now is that it is perfectly valid to posit as a hypothesis a certain psychoanalytic state as a cause of an observed event, just as it is valid to posit as a hypothesis an activist miracle-working God as a cause of that same event.

Craig says that L has a lower PP because it contradicts certain evidence related to C. Though that is precisely the point of contention between him and Luedemann--both are interpreting the evidence tendentiously, so the issue is not clear-cut for either of them--I will assume it is true. But even if it is true, it would not lower PP, but EP. For PP must be calculated regardless of the specific evidence, and C is specific evidence.

So:

Third, okay, so Craig has totally screwed the pooch on this one--but what really is the relative PP of R and L? A theory is "plausible" to the precise degree that it makes sense in the socio-historical context and specific circumstances containing or leading to E, excluding reference to E itself (beyond what E entails regarding the circumstances of H). Thus, to assess the PP of L, we must assess the prior probability that all its elements could (a) have occurred in the time and place in question (1st century Judaea, etc.) and (b) occur together in the same time and place. And this must be assessed relative to all other possible explanations of the same E (so that all PP's sum to 100% and no more or less--another reason why Craig cannot just address L and arrive at R).

For example, how often did people have religiously-inspiring hallucinations around then? And so on. Taking assessments like that into account, how likely is it that E would be produced by our hypothesis as opposed to the sum of all other possibilities? This is no easy thing to assess, and guestimation is often the order of the day (hence historical conclusions can rarely carry the conviction many like Craig want them to). I think L is certainly plausible--though I think it needs a lot of work to make it respectably so, and even on its best possible formulation, I doubt PP for L would be very high. It would be lower than 50% I am certain, and possibly as low as 10% or even 1%. That is, given all our background knowledge, and ignoring all our specific evidence (except that pertaining to establishing the circumstances of E and hence of H), what is the likelihood of something like L happening in Judaea in the 1st century? That is the question Craig should be asking and attempting to answer, but he isn't even in the ballpark--he isn't even playing the same sport! I think the answer is that this probability is low for any theory, even Craig's. That is why we need the relative probability. But again, Craig is oblivious here.

Fourth, given all our background knowledge, and ignoring our specific evidence (except that pertaining to establishing the circumstances of E and hence of H), what is the likelihood of something like R happening in Judaea in the 1st century? Now we're on very troubled water. What is the prior probability of a resurrection? Can Craig even present any relevant background evidence that would help determine such a thing? Even if that were possible (and I doubt it), it is certainly not anything he actually has done here, and thus he is not applying PP correctly. It seems the only thing Craig could do (if he were to assess PP correctly, but he doesn't) is offer some pure speculation as to the PP of an R, not any actual evidence that R's happen, much less how frequently. But an argument relying on a premise of pure speculation will only produce a conclusion of pure speculation. That has no place in historical analysis.

Fifth, PP for R is based on all the required elements of R, not just R itself. And here is where that complexification referred to earlier starts to hurt real bad. For example, what is the prior probability of A? Whatever it is, the prior probability of R given A must be multiplied by the prior probability of A, thus reducing PP further. And so on for every element of R that Craig has to adopt in order to beat L on ES and EP: every one has its own prior probability, which affects the total prior probability of the entire theory R, and the more elements that had to be added to R, the less likely all those elements would have been conjoined, and consequently the lower PP must be for R. Thus, all the baggage attached to R starts to sink it quite soundly on the PP criterion, whereas the baggage attached to L, though not negligible, is much more manageable. But Craig sidesteps all these problems, by not even assessing any actual PP, but just coming up with another EP (and a specious one at that).

(4) Ad Hocness (AH): Craig says R "is not excessively ad hoc or contrived" because "it requires only one additional hypothesis: that God exists" whereas L already concedes this, and adds at least four ad hoc suppositions of its own. Here Craig totally fails to count any of the apples in his own cart: he doesn't apply the criterion to R at all. For as we have seen, with our discussion of ES and EP above, R is chock full of ad hoc hypotheses, none of which Craig acknowledges here. But, once again, all the consequences of a theory must be taken into account. And on any proper analysis, R clearly entails far more unproven assumptions than L, no matter how many unproven assumptions L does entail (and it does entail quite a few). Just to get R to fit W Craig has to bloat R into a wildly complicated theory, never mind all the other peculiar facts I noted that L explained but R did not--unless, again, R is bloated with many more ad hoc theses. There does not seem any way R can win on AH, unless Craig can present direct and convincing evidence for each assumption that he must adopt in order to explain away all these otherwise embarrassing facts.

(5) Accord with Accepted Beliefs (AA): Craig says R "is in accord with accepted beliefs" because "the hypothesis: 'God raised Jesus from the dead' does not in any way conflict with the accepted belief that people don’t rise naturally from the dead," while L, in contrast, goes against at least six generally accepted beliefs. Now, Craig is fibbing a bit when he claims all six beliefs he lists are "generally accepted," much less that they are all generally accepted with any appreciable certainty. Likewise, that he points to flaws unique to L, which don't exist for other variants of L's theory, only reminds us of the fundamental flaw we noted at the outset, which undermines Craig's entire case (the failure to compare his hypotheses against all viable contenders). But we will brush these aside and assume Craig is right about L and that this is just as true of all other viable variants of ~R.

First, L does not go against the facts, namely that stories arose of a certain nature--it can explain why those stories exist as they do. So even if L went against what was commonly inferred from those facts (and strongly inferred--since weakly inferred beliefs hardly rate on this criterion), that would indeed count against it on AA, but it would not count against it as strongly as Craig seems to think. Moreover, to the same extent that L suffers on AA on such an account, R suffers equally as much. For example, most people do not accept the belief that a God exists who would raise Jesus from the dead (the world is far larger than many Christians want us to think), and, as we've already noted, R entails A which implies ~W, but we observe W, therefore R forces us to reject the natural inference from a universal observation.

In the final analysis, I think R and L are well-matched on AA. Though a lot of expert opinion does go against L, it is not very strongly against it, nor universally against it, and much of that opposition is circular (i.e. based on prior belief in R), while there is plenty of expert opinion that goes against R, and elements of R (such as A, among many other things). So a stalemate is the best one can get here. But Craig does not mention any of these issues. He glosses the problem of circularity, of lack of consensus, of expert opposition to R (and necessary elements of R) comparable to that against L. Thus, he misapplies AA.

(6) Extent of Superiority (S): Finally, Craig says that R "far outstrips any of its rival theories in meeting" the other criteria. Well, this just caps all the same errors: (a) Craig examines only one theory (L), and then concludes he has defeated all of them; and (b) though he has misapplied all the criteria to one extent or another, and on a more proper analysis none of the criteria provide a very large victory, if any, for R, whereas many produce very evident failures, even against the relatively weak competitor L, Craig nevertheless claims R "far" outstrips L on every criterion--a conclusion that can only be produced by abusing, not using the McCullagh criteria.

Such is my observation at any rate.

gregor

Tremendous

Any chance of anyone writing a book using these criteria to compare the, say, four leading contenders for the explanation of the belief in a risen Jesus. R v. L v. Spiritual v. other?

Basically a book concluding the relative likelihood of accuracy of the four best hypotheses?

Richard Carrier

Cavin and Colombetti have written a book entitled (I believe) Double Cross that does this using Bayes Theorem, but not the McCullagh criteria (Cavin despises the McCullagh methodology). They are seeking a publisher now. I'm pretty sure it will end up advertised in the SW bookstore when it does come out, if it isn't also made Book of the Month and advertised on the SW home page, or in banner ads. But I have not seen their manuscript, so I can't comment beyond that. I do know that they argue pretty strongly for a theft-and-conspiracy conclusion, even against competing naturalistic explanations, and that they are already writing another book specifically directed in more detail against the supernatural explanations. I am skeptical. But I am awaiting its publication to see what they do with the evidence.

Vorkosigan

You can't be serious. How can any scholar do that in this day and age?

IMHO the best explanation is that the Jerusalem crowd manufactured the visions of Jesus as ammunition for in-house political struggles among nascent cult factions and members, just like many other cults in history.

Vorkosigan

Richard Carrier

I am strongly inclined to agree with you, especially in your skepticism of the theft-conspiracy "proof." I am very skeptical indeed, but they are adamant. So I will just have to wait for the book to come out and see why they are so adamant.

As to the most probable theory: there is an open question regarding whether visions (in all the cult histories you intend as examples) are actually fabrications or hallucinations, or some combination thereof. This is an issue Evan Fales and I have been tackling, and he has come out in a position of agnosticism (he is, unlike me, an expert on the comparative anthropology of religious experience, so he knows a lot about this question). Any one of the three scenarios is possible in just about every case on record--since the evidence is consistent with any one of them (and that includes the scientific evidence as to what humans are capable of and tend toward, and the evidence for historical generalizations as to the probability of fabrication vs. hallucination vs. mixing the two).

Fales and I will argue for our conclusions on this issue in separate chapters in the forthcoming Prometheus book Jesus Is Dead. I am relatively charitable and present the case for sincere hallucination (if a bit "dressed up" and refined in the telling) followed by honest symbolic exegesis (i.e. embellishments that are not even meant to be taken as historical facts). Fales takes a middle ground, supporting either that or your scenario equally well. In his other published articles on the subject of religious experience (already available in print) he defends a slightly more cynical position, though always reserves room for the sincerity scenario. All his work on this subject is excellent.

But either way, I am fairly certain he is right about one thing: it all boils down to authority, and the means by which it is demonstrated and secured among one's peers. Of course, there has to be something you believe in that you need that authority to promote. It does happen that people seek authority simply to enjoy power, but without a genuine or sincere vision of what they want to do with that power, they are unlikely to do anything with it that will ever be spiritually remarkable. In some cases, political structure can direct from above the energies of a power monger toward a coherent agenda, but this cannot explain the behavior of the alpha males in such a structure (such as Paul and Peter). They must have had some internal motive that must have aligned in some significant way with what they actually did and preached.

In other words, it is probable Peter and Paul believed in something, something that Jesus represented to them, something the movement meant to them vis-a-vis its goals or projected effect on society. In Peter's case we may never know, since we have no trustworthy sources on his agenda, except from Paul, who gives us scarce data--beyond the fact that Peter did not like the idea of a mission to the Gentiles, which does tell us a lot about what Jesus originally taught, or was originally held to have taught (and it isn't encouraging to modern Christians). In Paul's case, though, we have a lot of data--not always the best (since there has been tampering with the source material), but enough to go on. And an analysis of what Paul thought he was doing, what he wanted to accomplish with the authority of the Christian movement, is something I provide in Jesus Is Dead.

To sum up: in my opinion, the theft scenario is at least plausible, as is a misplaced-body scenario (I also argue for those two theories in different chapters in Jesus Is Dead), but a visionary-authority movement with an ulterior moral-ideological agenda is the most probable explanation of the origins of the religion.

Vorkosigan

I shall certainly look up Fales, and can't wait to read your book.

That is a good point and one I try and constantly remind myself to remember. I am often too cynical about the motives of the early Christians.

I can't wait.

I do not believe the theft scenario is plausible at all, because I believe those stories are complete fictions. I suppose if you must have a naturalistic explanation, and take the story at more or less face value, in some basic form (some group of disciples claim to have buried Jesus and then later found the Tomb empty), then it could be plausible. I don't think what the early believers knew about Jesus' life mattered much -- you only have to look at the modern Lubavitchers to see that.

Richard Carrier

This is a good point to bring up. I agree.

The theft/relocation scenario is a kind of "save the phenomena" sort of explanation, which follows the believer's own methods. Ironically, that means it is more plausible if one accepts the principle held by most believers that the text is largely historical and interpretable historically, and yet theft/relocation is one of the explanations many believers find the most "repulsive" (which Craig translates into "implausible" by some sort of neo-Freudian transference). In contrast, the "authority movement" scenario requires treating the biblical documents very differently than most believers want to (namely, more ahistorically and symbolically than is palatable to many of them), but is actually the more historically and culturally probable account of those documents and the movement itself.

In the one case, they accept our method but can't abide the conclusion, whereas in the other they are almost willing to accept our conclusion but can't abide the method. Go figure.