Edit: Some people have complained that I haven't given enough credit to Den Beste- that his description of deduction was technically accurate. That definition, as quoted below, is that "It requires sufficient information of high reliability, and when that is available it yields an answer which is nearly certain". The former qualification is indeed true- it does require "sufficient information of high reliability" to yield an answer. The problem, however, is that it can yield multiple valid answers. This doesn't leave deductive reasoning unusable, even in the social sciences, but it does mean that one must watch out for these multiplicities of valid answers that are proven by the evidence at hand. This is what satisficing is: satisficing is when somebody "proves" one hypothesis (even when using deduction), but it's actually only one of many valid ones consistent with the evidence at hand. It also doesn't necessarily require mathematics, although mathematics certainly make the process easier. After all, the deductive chain "All men die/Socrates is a man/therefore Socrates will die" is valid whether it's described that way or as "A=D,B=A, therefore B=D (Well, actually, I guess a fully accurate mathematical description of that chain would involve sets, but you get the point.)

So: Den Beste's qualifications are valid, but don't go far enough. Satisfied?

Steven Den Beste seems to somehow believe that deductive reasoning can never come up with the wrong answer. Don't believe me? Check it out:

Deduction is prissy; it refuses to play unless it knows it can win. It requires sufficient information of high reliability, and when that is available it yields an answer which is nearly certain.

This would be true... if it weren't for the simple fact that it's extremely difficult to tell whether a) you have all the possible hypotheses, b) that your methodology for eliminating options is unimpeachably sound, c) the evidence is both accurate and means what you think it means and d) whether your evidence only supports one of them. One of the reasons I've made a point of dealing with the concept of satisficing (which is picking an answer not because it is *the* right answer, but that it's one of a series of "right" answers that you pick because it's the preferred answer) is to deal with this notion of unquestionable rightness in deduction, but how on earth could an engineer make this sort of elementary mistake? Well, perhaps because deductive reasoning's errors lie in mistakes made by the deducer, not in random chance... a lot of non-scientists who admire the scientific process get it into their heads that the human component is negligable, especially if that admiration is the basis of some sort of moral or ethical system. That's hardly true, of course, especially in the social sciences- but people still make that mistake.

To be fair, he does later acknowledge such a thing: "The real world doesn't cooperate with deduction. It's necessary to take chances, to make guesses, to work with inadequate information and information of doubtful validity." The problem is that people will deduce anyway and, in fact, they need to, in order to apply theoretical knowledge to the real world.

See, induction is normally used in the social sciences to generate theory- you observe phenomena and create a theory based on those observations, usually situating it within one of the broad explanatory paradigms in the field. If you manage to successfully demonstrate the validity of that theory, however, the whole reason it's useful is so that one can deduce with it- to take the general (the theory) and with it understand the specific (the situation at hand). It's not perfect, and definitely bears some methodological analysis before you use it, but the application of theory is a valid tool for understanding events... both past events, and future ones.

"Future events", you say? How is that possible? Well, that's the job of the analyst- to figure out what's going to happen, instead of explaining what's already happened (that's the job of the historian). Steven goes into some depth as to the nature of analysis, saying:

And if it is not possible to always be right even about normal events, where the only limitations are inadequate access to information, then how much more difficult must it be in military intelligence, where you're trying to determine the intentions and capabilities and plans of a determined enemy who is doing his best to fool you?

Actually, Stephen, I wouldn't say it's difficult per se, so much as precise. Well, my analysis knowledge comes from the recommended methods for CIA analysts, not military analysts, but the basic concept is the same: you develop as many possible hypotheses as you can (usually using the resources of as many people as you can nail down) and then find evidence that eliminates these hypotheses until you get down to the one hypothesis that is accurate. You aren't proving hypotheses, you're trying to disprove them- that way, you don't have to worry about evidence that supports several conclusions (which leads to satisficing) because that isn't what you're using it for, and you can't satisfice elimination. Since satisficing is incredibly dangerous in real-world analysis, this is really the only proper way to do it (and the fact that people don't is one of your bigger analysis problems right now.)

You use a mixture of three different techniques to do so:

1) Situational analysis. Basically you look at the situation as it stands, all the evidence that you have, and use that to look ahead and see what is likely to happen. The advantage is that there's no way you can mix up what you think is going on and what is actually going on, because they're one and the same (assuming your intel is accurate); the disadvantage is that it's incredibly time consuming and doesn't allow you to use any sort of theoretical or comparative tool to understand what's going on or what to do.

2) Historical comparisons. You compare the situation going on right now with a well-understood and similar situation that has happened in the past- what you know of what happened is used to explain what's going on. It's really useful for finding parallels and trends, but there's no theory working here and there's a huge danger in creating equivalencies that aren't there. Steven does this a lot in relation to WWII- since he knows WWII so well and thinks of diplomacy and statecraft in fairly strict military terms, it's an ideal comparison for him... although for the rest of us, the problem is that what's going on right now isn't precisely equivalent, and WWII is a dubious comparison at best.

3) Lastly, you have the application of theory... this is where you take some sort of theoretical model and apply it to the situation at hand so as to recognize what's going on. Popular with academics (whose lives revolve around the creation and application of theory), and provided that the theory works and can be applied to the specific situation, it's pretty damned useful. The problem, of course, is that any specific situation will only fit theory to a certain degree, and recognition of that degree is absolutely vital to successfully applying theory- you have to fit the theory to the situation, not the other way around. Steven tends to use Clausewicz for this- I don't see much application of political theory aside from this, and while Steven applies his own theories, this isn't a very valid way of doing analysis- other people spend their lives making theories and others reconfirming them. There's little reason to simply apply your own (poorly proven) theories when there are much better, more comprehensive and better proven theories ones out there. (Hence the reason Paul Krugman is always railing about pseudo-economists who have their own half-baked economic theories that demonstrate only ignorance of the theoretical field that really exists.)

All three are useful, but all three have their faults.

So why am I going on about this? Because Stephen has written an exceptionally long article that purports to take down the arguments of myself and Hesiod- the one that I linked to above. One of the arguments he made throughout is that he's working from "inductive reasoning" that "..part of induction is to decide not just what the chance is that a given conclusion is wrong, but also what the consequences are for a false positive or a false negative, and decide based on that which way to err." This is good as far as it goes, but shows that Stephen is going about analysis precisely the wrong way... he's attempting to prove a pre-existing conclusion, instead of taking all the possible hypotheses and using the evidence to figure out which one is valid.

Now, part of my problem with Steven's response is that for such a long response, it's actually pretty short... he only quotes me directly once, and only on the question of whether or not Saddam has WMDs. (He does use what I've said as an adjunct to one of Hesiod's arguments.) So already he has implied no contradiction with anything else I've written, and the critiques I've written of his reasoning extend far, far beyond whether Saddam has WMDs, through the question of International law, all the way down to the way that Steven engages in analysis himself, which is rife with satisficing and poor use of the three techniques I mentioned above. (He's obviously not aware of them- not surprising, as although he's someone interested in politics, statecraft, and warcraft, he's not exactly a professional and is probably unaware of just how dangerous his satisficing is for real analysis.) Throughout the entire argument, he defends his arguments based on the idea that they support his conclusions... but as I've shown above, that's meaningless, because they could support other conclusions as well, and as long as they do it's faulty analysis. By considering the question of satisficing and these three ideas of analysis, one can discover how each of his arguments in turn is fatally flawed.

-he says that his argument that Iraq has WMDs "is based on induction and might be wrong", but that it doesn't matter because he might be wrong and he might be right, and there are reports that hint that he's right. He believes that we might as well operate as if we know that WMDs are in operation there, and make analysis as if there are weapons there. This is an example of satisficing- he's already come to his conclusion, and is looking for things to prove it's true. He admits that all the evidence he cites might not be accurate but that it leans towards his analysis of the situation... but because they don't eliminate the hypothesis that "Iraq doesn't have weapons, but would really like to have them", To proceed from that reasoning is fatally flawed- you haven't really eliminated anything.

-He misreads the situation as it exists, partially by trying to set up neat little contradictions like this one:

There are really only two ways to acquire real proof: invade the place and look around without the Iraqi government impeding us in any way, or wait until a weapon goes off somewhere. Given that we're trying to decide whether to invade, and trying to prevent such a detonation, we must act before proof exists.

This is exceptionally faulty reasoning, because there are lots of questions begging. First, are there only two ways? Why? Why not three or four, and is it really a good idea to limit the hypotheses artificially right off the bat, like he's doing here? (Again, could lead to satisficing.) Is invasion of Iraq the only way of looking around without the Iraqis impeding? Considering my observation that Iraq is afraid of the U.S. using inspectors as plants, it's not a given conclusion that invasion is the only way to get unfettered access. It's not a given conclusion anyway... it's just an assumption, used to build up a chain of reasoning. You can't do that with assumptions... not in analysis, not without recognizing that it is an assumption. (Seems like Steven does this a lot.) The other question begging is whether or not that weapon will be detonated, which is surely not a proven fact in the slightest. He's engaging in faulty situational analysis here- he has to be, because there's no historical precedent that I can easily think of, and he's not applying any theory, that's for damned sure. That's fine so far as it goes, but he's not looking at the whole situation. Even if he weren't engaging in rampant satisficing that undermines his whole chain of reasoning, this sort of argument leaves any situational analysis fatally flawed.

-He misrepresents the costs of the invasion... which is surprising, because for someone who was planning on refuting my arguments he pretty much ignored a fundamental aspect of them. He said "if we're incorrectly pessimistic, it means we'll fight a war that probably wasn't necessary. That's certainly very bad for whatever nation we attack, but the cost to us of such an outcome is much lower than the effects of having one of our cities nuked." First, this isn't remotely proven, and is certainly questionable when one extends the definition of "us". "Us" as in "the United States" isn't proven, because we will have showed that the United States has no regard for international treaties and bodies, even fundamental and ancient ones like the Treaty of Westphalia. A nuked city is a very bad thing, but the breakdown of the international system may be far worse. (Or it may not, but Steven isn't even considering the point, which weakens his analysis.) If you extend the "us" to those outside the United States, then the problems of invasion become far greater, because Saddam may decide to curry the favor of a radical nation he plans to secretly escape to by using his chemical and biological weapons (assuming they exist, and Steven does) against Israel; the "us" that is Israel is therefore much worse off (nukes are bad, but there's a reason the others are called "WMDs" too). If you extend "us" to the planet, then the precedent of the United States ignoring the U.N. security council because it feels threatened by "immanent attack" means that there won't be a country on the planet that doesn't use the same excuse for preemptive attack on a weaker (but theoretically dangerous) neighbour, and assuming the U.S. doesn't care about the situation, that weaker nation is pretty much screwed. The United States would have set the stage for a true Hobbesian state of nature on the international scene, something that international law is supposed to mitigate. This might not be the case, but Steven didn't even consider it, making his analysis fatally flawed.

-his argument that Iraq kicked out the inspectors because "they were too close to finding the truth, irrespective of whether or not the Americans were using them to spy or not" is another faulty argument, preceding from an unproven assumption. First he asserts that the WMDs exist... although that's a fine one to make when making a point, when coming to a conclusion it's weak. He said that he'll proceed from that, and I don't accept it. A more important point that he doesn't address, though, is the order of importance that he places the two arguments in- saying that the first priority was to protect the WMDs, and the second to prevent the U.S. of spying. He deliberately portrays the second as an "excuse", not a "reason", but that's not an assumption he's entitled to make, as it's the very sort of argument that he uses to attack the International Criminal Court on behalf of the United States! He argues that the ICC would be "used by foreign officials to 'get' the US"; others argue that "the U.S. doesn't want to be prosecuted for all the ethically dubious things it's done in the past". Both may be right, and he acknowledges that, but in the latter situation, he reverses the "reason" and the "excuse" for no better reason than the identity of the country involved! Fine for him, an American... less so for a reader in the third world, where American realpolitik might have turned their country into a living hell for a decade or more and that has a definite interest in seeing, say, Henry Kissinger brought before the ICC and tried for war crimes. After all, if Iraq doesn't have WMDs, then the entire basis of his ordering becomes useless, and he has already acknowledged that that possibility exists. The evidence supports multiple conclusions (Iraq doesn't have WMD and is paranoid about spying, Iraq does have WMD and is paranoid about spying, Iraq does have WMD and isn't paranoid about spying but doesn't want them found), and Steven has arbitrarily chosen a conclusion based on evidence that only eliminates one possibility: that Iraq doesn't have WMDs, and doesn't care if its spied on. His analysis, then, is fatally flawed.

-he pulls a silly little side-issue into the fray, which is whether some think "the US is the problem, not Iraq". He attacks those who think that the U.S. is "too powerful, too large, too rich, too greedy... we do too much, own too many weapons, consume too much, and because of that we unbalance the world". First, he's using a simplistic cartoonish version of that argument to try to dismiss it, but that much is obvious- there's less strawmen in a Wizard of Oz audition line. The problem with this line of argument, though, is that even if they do, so what? The U.S. invading Iraq is a problem... the question is whether the U.S. has any legitimate basis for doing so. If they're criticizing the invasion based on those grounds, then deal with that, but I know I'm not, I know Hesiod isn't, and frankly the question of whether the U.S. is extraordinarily powerful is entirely beside the point except in that it's a valid data point to consider when one considers the ramifications of unilateral American action and what it would mean for collective security. It's not the only one to consider, but it's valid. Remove the "too"s inserted in there and there's nothing objectionable there... the U.S. is large, is rich, is powerful, and is greedy (how one reacts to that depends on whether or not you consider greed wrong or not). It does do a lot, it does have a lot of weapons, it does consume more than most, and there are probably ramifications to all of this. Unfortunately, it's besides the point, and any attempt to defend his arguments based on this sort of ad hominem nonsense shows more of their weakness than anything else. This isn't even an analysis error- it's just pointless red-baiting.

-Finally, he pulls out the old gambit: that "they don't think like we do", and therefore we should invade Iraq. He doesn't like that I and Hesiod use the word "insane" to describe how he thinks of Hussein, but that's because he's ascribing fundamentally insane beliefs and actions to Hussein, in that they're meaninglessly self-destructive. Hussein is not self-destructive- he's no fundamentalist, and probably doesn't even care about Islam that much except as a tool. In fact, he makes a huge, unbelievably huge error that would get most analysts fired: he takes unlikes and turns them into likes.

Which brings me, finally, to the article which inspired this entire ridiculously long post. Much of this analysis, by everyone involved, makes a fundamental assumption that Saddam, and other leaders of Muslim and Arab nations and groups, think more or less like we do -- or that they are insane.

Already, he has ruined his argument, before I could even touch it. Amazing trick, really, and I'm quite impressed. Mixing together "Muslim" and "Arab" is fantastically wrong for obvious reasons, but so is mixing together fundamentalist leaders and strongman dictators like Hussein. The entire reason Hussein has been in power as long as he has is because he isn't some kind of Islamic extremist- that's why he was a U.S. ally before the Gulf War and why he was backed by the U.S. during the Iran/Iraq war. Arguments that apply to Hussein don't apply to Al Qaeda and vice versa. At all.

(Heck, this sort of thing is usually an imperfect fit at best- Steven's trying to use theoretical analysis here, and doing a pretty poor job of it, making bloody elementary mistakes.)

Much of the rest of his article deals with Al Qaeda and their philosophy. Unfortunately, I have no problem with the war with Al Qaeda. So that's pretty much a wash. That's the real war that should be prosecuted, and which Iraq is a dangerous distraction from.

Later on he makes the distinction (which really doesn't help that much; he shouldn't have made the error in the first place) between Al Qaeda and Hussein, but once again is trying to pull together whatever he can to fit a pre-determined conclusion- that Hussein is indeed like Al Qaeda. Unfortunately, it could also mean "the evidence implies that they're similar", which does not eliminate the possibility that the evidence implies many different things, that other evidence implies that they're dissimilar that he isn't including, etc. etc. etc. Again, satisficing.

The argument he uses is a familiar one- Iraq has nukes, it might leak them to terrorists, and therefore they're dangerous. Hesiod annihilated this argument by simply asking "what if Saddam realizes that they might decide to use them on Saddam as well?" Steven didn't deal with this point except to say that Saddam would do it anyway as long as "there remains a higher chance of them getting us instead of him", which ignores the reality of "maximin"- they idea where people will reduce the penalty of a bad call instead of the reward for a good call, and which Saddam would definitely think about were his own ass and country to be on the line. Hesiod's point remains uncontradicted, and certainly valid. Steven gussies this basic theory up with a lot of utterly unsubstantiated assumptions and theories with little basis in reality, but it remains contradicted by Hesiod's basic objection, which Steven can't honestly answer, because he doesn't understand the varying interests here. (He seems to have a problem with that.) It's woefully inadequate analysis.

I mean, let's look at the risk/reward here for the action of "giving terrorists a Bomb".

Possible rewards:
-United States gets nuked. Somewhat likely, although not so much as you'd think.
-nuking of United States brings down U.S. Less likely- requires both that U.S. is successfully nuked and that it really hurts U.S.
-nuking of United States pulls U.S. out of region. Even less likely- requires that U.S. is successfully nuked, and that there's an effect, and that the U.S. doesn't go seeking revenge.
-U.S. pullout allows for pulling together of "Arab empire". Utterly unlikely.. requires all the previous data points, plus successful conquest of region, plus no other power arising to take place of U.S.

Possible risks:
-U.S. is not nuked (bomb is wasted, but possibility of getting caught still exists.)
-U.S. believes that Iraq is responsible, retaliates. (Very likely- in fact, difficult to deal with unless Iraq can somehow implicate someone else, which is unlikely)
-Terrorists attack Iraq instead (more likely- it's an easier target with little western support, and if Baghdad gets nuked Saddam is utterly screwed)
-U.S. is nuked, doesn't believe that Iraq is responsible, but becomes even more interested in region (as he's a U.S. opponent, that would be a risk, not a reward for him.)
-Israel is nuked instead (doesn't mean much to Saddam, and the U.S. will definitely become more interested in the region, which is a net minus for Saddam. He's also possibly implicated, which means that U.S. nukes him)

Sorry, Steven, but the top group doesn't justify the bottom group in the slightest. Even if Saddam had this dream, nobody can stay top dog for as long as he has without being able to unconsciously or consciously make this sort of analysis- if he does either, he'll realize that it's more trouble than it's worth. This also applies to conventional attacks- if he conventionally attacks any country in the region, he'll get thrust out- and if he uses nukes to defend his holdings, he'll get nuked in retaliation by the U.S. So that won't work either-it isn't even MAD, just AD- assured destruction. The only reason he could really have for having nukes is so that he could use them if he had nothing left to lose- to defend his current borders against invasion. And the only legitimate reason we have for invasion is, well, that he has WMDs. It's a circular argument, and very poor analysis. It's the application of theory, but application of theory that only he actually ascribes to instead of theory within the field itself. To apply untested and unproven theory is a fundamental error (even if mixing up likes and unlikes weren't), and leaves his arguments fatally flawed.

Ok, so what's the point of this long article? That Steven makes bad arguments? Well, kind of, but there's a specific reason- he talks about analysis without seeming to really understand it. He satisfices constantly, and doesn't seem to understand that other hypotheses both exist and are supported by the evidence he brings forth, or at the least aren't eliminated by the evidence he brings forth. Each of his arguments is fundamentally flawed in different ways, but all betray the dread form of satisficing... picking the most desirable among multiple possible conclusions. I'm not saying he's intellectually dishonest, but that his form of reasoning is (as I've repeated) fatally flawed. It's pretty common, actually- part of the reason the CIA recommends that its analysts adopt the forms of analysis I mentioned earlier is that they too have the same kinds of problems, and it really hurts their analysis. The fact that it is relatively common, however, doesn't change the fact that it's near-useless.

Honestly, I'm somewhat disappointed. I was expecting a grand debate on the concepts of international law and I.R. theory, on how that applies to Iraq, on the nature of the War on Terrorism, and everything else that I've critiqued Steven on for what must be going on a month now. I was actually a little nervous, because I had thought that he had this going for a while, and that I'd be crushed like a bug between the hammer of logic and the anvil of evidence. Instead, I got a weak chain of satisficing and faulty analysis, less notable for its persuasiveness than as an instructional tool for explaining the difference between good analysis and bad.

I've spent my morning rebutting this, and I have to say that I've seen better arguments in my own comments threads- I doubt Hesiod will even bother, considering how far superior his arguments are to anything I've read and responded to today. I also have no doubt that more people will read Steven's argument than my response, because he's more widely read than I am, is permalinked on more pages, and is reconfirming the beliefs of many E.C. bloggers out there, who will be far more likely to wade through a poorly-argued entry that supports their own prejudices than one that uses the flaws of said entry to make a point about analysis. Still, it needs to be said, and with any luck it will inspire some to more rigorous analysis in the future.

Lord knows, if this is the best that's out there, we need it.