We may be thinking about it all wrong

“What should we do about North Korea?” asked an article [1] in the September 8 edition of the Washington Post. What made me read the article, however, was not the title but the subtitle:  “We may be thinking about it all wrong.” Might the article be of relevance to the psychology of reasoning? The author, Bradley DeWees is both a captain in the US Air Force and a doctoral student in decision science and public policy at the Harvard Kennedy School. And, yes, his article is relevant to understanding the mutual relationships between psychology of reasoning and decisions on public affairs.

The United States could aim at containing a nuclear North Korea, or they could opt for military action against it. DeWees writes:

“Selecting from options such as containment or military action can take two forms — what decision scientists call “choice” or “matching.” [*] … Choice would lead a decision-maker to ask why containment is better than military action (or vice versa). This “why?” question triggers a search for good reasons to adopt or reject an option. Matching, by contrast, would lead a decision-maker to ask how much (or how little) military action would have to cost before it was as desirable as containment. This “how much?” question triggers a quantification of the dimensions common to each option. The two processes can lead to different outcomes because choice — and the concomitant search for “good reasons” — can bias a decision-maker in several ways.”

In our book The Enigma of Reason [2], Hugo Mercier and I have argued that, contrary to standard views, reasoning has primarily a social function. It consists either in producing reasons in order to convince others of a conclusion already reached or in evaluating reasons others produce to convince us. In the same vein, “choice,” as described by DeWees (who links to an earlier article of ours [3]), seems focused on finding a simple one-criterion reason to justify one’s decision.

“Choice pushes people toward the option scoring highest on the most important criterion, even if that option leads to a worse overall outcome. The criteria at stake in this decision include lives, money, living under the risk of a nuclear North Korea and maintaining the credibility of public threats. … If maintaining the credibility of threats … is the most important criterion to the administration, choice would favor a military option. Choice also would predispose a military option if preventing a nuclear North Korea is the most important criterion to the administration… In both cases, acting on one’s most important criterion constitutes a good reason.”

What makes one criterion among several “the most important one”? The ”choice” procedure doesn’t include a sub-procedure to determine what are the relevant criteria and which one is the most important (and it is not clear what this sub-procedure could be). In practice, this may be left to intuitive bias or emotional commitment. Since, as DeWees suggests, the decision to attack or to contain may intuitively follow from a preferred criterion, the main if not the unique role of reasoning in “choice” seems to be to justify a decision rather than to make it.

Actually, a decision to use diplomatic or military means and, in the latter case, which military means to use, is a matter not just of military strategy but also of politics: being able to justify one’s decisions in the eye of public opinion is a political gain; being unable to do so is a political loss, and these gain or losses have to be factored in in the overall decision.

What happens if the criterion that has in fact led to the decision fails to provide compelling arguments? Imagine for instance that Trump would decide to attack in order to maintain the credibility of his threat that North Korea “will be met with fire and fury like the world has never seen.” American public opinion might not see maintaining Trump’s credibility as a good enough reason to go to war. So what? Trump could with more success defend his decision by arguing that attacking North Korea now is the best way to prevent a nuclear North Korea attacking the US in the future, even if this criterion is not what actually swayed his decision. True, there are decisions so hard to justify by any criterion that they end up not being pursued, but where demagoguery works, reasons are cheap.

So “choice” is less a method for choosing than for justifying choices. What about “matching”?

“Matching,” DeWees writes, “could lead to a different decision even if the relative importance of each criterion remained unchanged. Using matching, the task would be to say how much, in lives and money, one values avoiding the risk of a nuclear North Korea or maintaining the credibility of public threats. What containment lacks on these criteria it could make up for in saving lives and money.”

Decisions based on matching seem more careful, better considered than decisions based on choice. Moreover, they seem to be more objective:

“Choice based on reasons factors a decision-maker’s audience into the decision. What counts as a good reason depends on what a decision-maker’s audience thinks is a good reason. … What counts as a good reason for one group may not necessarily be good for another. Matching, on the other hand, is more stable across audiences — the audience is unlikely to affect the numbers underlying the decision.”

DeWees makes matching sounds definitely better than choice, but is it really? Here are two problems with matching, one internal the other external. The first, internal problem is well-known: in order to match different options you need a common metric for all the criteria considered. In most real life case—what to do about North Korea for instance—relevant criteria may be quite heterogeneous: expected numbers of casualties, financial costs, loss of credibility and so on. The weighing of these factors, moreover, should take into account time discounting, which is different for different people and which may vary across criteria (the expected number of casualties from a future atomic war may be much greater than from an immediate conventional attack, but, because of time discounting its weight in decision may be lesser). It is very far from obvious (1) that there is an objective metric across heterogeneous criteria, (2) that even if such a metric exists, it can effectively be worked out, and (3) that even if such an objective metric can be worked out, it will be generally understood and accepted.

Point (3) leads to the second, external problem with matching. Whatever way you arrive at a decision on an issue such as the best strategy towards a hostile country developing atomic weapons, the reaction of political actors and public opinion should be and will be a factor in evaluating this decision. Except in a strict dictatorial system (North Korea maybe?), political decision-makers have to give reasons for their decisions. As we suggested, what DeWees calls “choice” is the normal way of producing reasons for decisions however they have been arrived at. Is there an alternative to this normal way? One can imagine two: one alternative way would be to justify one by one the steps in the matching decision. Note that a step-by-step justification would require more intellectual work and more communication than either decision-makers or ordinary citizens are generally able and willing to go for. Still, suppose justification is done in that way. Chances are that one would end up falling back on single-criterion, “choice”-type reasons for most if not for all of these steps taken individually. The other way to justify a matching-based decision is to appeal to the competence, the expertise of the decision-makers. This again is typically made by invoking a single-criterion argument of authority.

“Choice” and “matching” may be evaluated not just in terms of efficiency and objectivity as decision procedures—and, I have argued, such an evaluation is far from straightforward—but also in term of the role they might play in a democratic process.

Democracy is not well served if a choice procedure based on a criterion that the decision-maker considers paramount (say, protecting Trump’s credibility) is defended by invoking whatever criterion—the same or a different one—public opinion might accept as paramount (say, protecting American cities from the risk of nuclear attacks). Democracy is not well-served either if matching procedures are carried out by experts who defend their decisions by asking the people to trust their expertise (demonstrated by the very complexity of the procedure they use).

For more democratic decision making, the people should be truly informed of the major steps involved and have a voice in the process. Decisions informed by public deliberation are in themselves more democratic than decisions imposed from above and justified by arousing popular passions or by invoking the expertise of the decision-makers. (I am aware, of course, that they are issues, military ones for instance, where we cannot but rely on expert, but even if unavoidable, this is still a limit to democracy.)

In a public deliberation, most participants would probably be focused on producing reasons for some single-criterion preference. Still, they would jointly be considering a variety of criteria and evaluating a diversity of reasons. The interaction among them would lead to a collective and interactive version of a matching process. If (a very big “if,” I know) there would be a high enough level of public understanding of the issues and a variety of point of view combined with a sufficient degree of agreement about the common interest, this would tend to lead to better decisions (as Hugo and I have argued, with evidence).

Deliberative democracy is an idea worth exploring [4], but my point here is not to defend it. It is rather to suggest that, before examining decisions procedures from a normative point of view, it makes sense to look empirically at the social practices of decision making. This empirical examination, interesting in itself, may also usefully inform the normative assessment.

At one level, DeWees’ discussion of choice vs. matching reads like a detached, normative argument in favour of matching. At another level, this discussion can be viewed as a “choice”-type argument based on the single criterion of (apparent) objectivity for adopting the matching method in deciding what policy to adopt towards North Korea. The implicit consequence seems to be that this decision should be made by experts rather than by the President, resulting in a policy of containment rather than of preventive war. For many readers and maybe for the author himself, this implicit consequence may have been the main point of the article.

Or, as I have tried to do, we can think about the whole issue differently.

(*) The choice/matching distinction was outlined, as DeWees mentions, by Amos Tversky, Shmuel Sattath, and Paul Slovic in their 1988 article, “Contingent weighting in judgment and choice,” [5] where matching, (which roughly corresponds to “maximising expected utility” as prescribed by rational choice theory) comes out as the way to go. Eduard Brandstätter, Gerd Gigerenzer, and Ralph Hertwig in a 2006 article, “The Priority Heuristic: Making Choices Without Trade-Offs,” [6] (extending to decision-making Gigerenzer’s well-known “take-the best heuristic” in inference, see Gerd Gigerenzer and Daniel Goldstein 1996 article, “Reasoning the fast and frugal way: Models of bounded rationality” [7]) can be read as a fleshing out and a realistic defence of choice. These articles, however, are not particularly focused on the social and political dimension of decision making, which DeWees highlights and which I am discussing in this post.

[1] Perspective _ What should we do about North Korea_ We may be thinking about it all wrong

[2] Mercier, H., & Sperber, D. (2017). The enigma of reason. Harvard University Press.

[3] Why do humans reason? Arguments for an argumentative theory

[4] What Deliberative Democracy Means

[5] Tversky, A., Sattath, S., & Slovic, P. (1988). Contingent weighting in judgment and choice. Psychological Review, 95(3), 371–384.

[6] Brandstätter, E., Gigerenzer, G., & Hertwig, R. (2006). The priority heuristic: making choices without trade-offs. Psychological review113(2), 409.

[7] Gigerenzer, G., & Goldstein, D. G. (1996). Reasoning the fast and frugal way: models of bounded rationality. Psychological review103(4), 650.


  • comment-avatar
    Christophe Heintz 21 September 2017 (22:05)

    Against deliberative democracy
    Thanks Dan for providing us with this opportunity to talk about deliberative democracy. I naively take it to be the goal of involving citizens to deliberate for taking decisions that are generally taken by policy makers alone. I see two limits about deliberative democracy and was wondering whether Gutman and Thomson have answers to that.
    First limitation: democracy as a process for distributing cognitive tasks

    One key aspect of democracy that seems to be set aside by democratic deliberative processes is delegation. Democracy is not just the power of the people, it is also a process through which tasks, especially cognitive tasks, are distributed. They are plenty of reasons why one would to delegate cognitive tasks to others, including:
    – Coordinating activity: in such case, a decision has to be taken, but the content of the decision is somewhat less important than its coordinating effect. For instance, a leader decides that people should drive on the right, that school starts at 8:30 am, etc. Most political decisions have some coordinating function, even when there are other things at stake.
    – Expertise: it is ok to trust others, even for important matters. They are multiple institutional processes through which trust is allocated, including medical degrees for decisions related to health, and voting for political decisions.
    – Unwillingness to contribute with cognitive resources and be held responsible. This is, I believe, a strong argument against deliberative democracy. Citizens, even good citizens, are not willing to dedicate a lot of time and effort to studying a political problem. For instance, who wanted to read the Treaty establishing a constitution for Europe (TCE)? In many cases, people are better off not thinking about specific problems and not taking responsibility for the solution implemented. There is only so much that one can ask from citizens. After all, it’s the job of policymakers’ to take hard decisions and bear responsibility. Asking citizens can be counterproductive if they don’t dedicate sufficient cognitive resources to reasoning about the problem.

    Democracy is a type of process for distributing cognitive tasks. Democratic deliberative processes challenge this distribution of cognitive tasks. There is, therefore, an argument for saying that some deliberative processes detract rather than empower democracy.

    Second limitation: how manipulable is uninformed reasoning?

    Dan says:
    ‘’Democracy is not well served if a choice procedure based on a criterion that the decision-maker considers paramount is defended by invoking whatever criterion—the same or a different one—public opinion might accept as paramount.’’

    The problem Dan is raising, if I understand well, is that the criterion is chosen because of its effect on opinion rather than because of its objective importance. Dan presents this problem as a weakness of democratic processes, which could be fixed by deliberation. But I fear that the problem is instead pointing at a weakness of deliberative processes themselves. Indeed, deliberations happen within frames that specify the questions and the stakes. Who sets the frame? In Hungary at the moment, the government is proud to implement systematic public consultations. But the consultation inevitably makes use of frames. The main question asked to citizens is ‘would you like to be invaded by illegal immigrants and thus increase the risk of terrorist attacks?’
    Does the argumentative theory of reasoning predict that citizens would, during deliberation, think outside of the frames provided to them? Shouldn’t they have some preliminary knowledge about what options and what costs and benefits there are? In other words, shouldn’t experts be leading the deliberation?

  • comment-avatar
    Hugo Mercier 22 September 2017 (05:39)

    (Mild) defense of deliberative democracy
    Deliberative democracy was not exactly the main topic of the initial post, so I’ll only write a few words in response to Christophe. Deliberative democracy doesn’t necessarily mean letting citizens directly making policy decisions. In most cases, it is an exercise in helping them develop more enlightened opinions, which they can then use to guide their political choices (such as voting). So deliberative democracy isn’t incompatible at all with representative democracy, as you suggest.

    As for framing, there’s work suggesting that framing effects can disappear during deliberation, but when several framings were available from the start. I think it’s a reasonable assumption in most cases, unless you look at frames that are quasi-universal (the it’s wrong to kick puppies frame?). So as long as at least some of the people deliberating aren’t taken in by the government’s framing, then deliberation should help.

  • comment-avatar
    Brad DeWees 26 September 2017 (02:32)

    Response to Initial Post
    Dear Professor Sperber — thank you for taking the time to read the initial op-ed and write this piece. Your post touches on a wide range of topics, I’d like to keep my response here fairly narrow, however.

    Judging from the asterisk at the bottom of the post, I seem to have stepped into a debate that precedes my time and exceeded my goals in the op-ed. My goal was not, as you state above, to make “ matching sounds definitely better than choice” for all decisions. My approach was to ask whether, for this decision, in this political environment, and for this administration, choice would better serve than matching. Would the security of the United States — not to mention the lives of tens of millions of people in North and South Korea — really be worse off if the President employed matching? I could not in good conscience make that argument.

    As you deftly point out, even in an article extolling the virtues of matching, I am forced to set up a choice. That is a lesson in itself. The op-ed environment, with its length requirements and need to avoid complexity in carrying the reader to a clear lesson-learned, calls for choice. “Get to the point — which is better?” To risk understating the point, avoiding complexity would be inappropriate for decisions involving nuclear weapons. Matching may be ill-suited to much, if not most, of life, but the stakes in this decision are beyond anything the human mind can naturally conceive; it’s fitting to call for an unnatural decision process.