Conviction, persuasion and manipulation: the ethical dimension of epistemic vigilance
In today’s political climate moral outrage about (alleged) propaganda and manipulation of public opinion dominate our discourse. Charges of manipulative information provision have arguably become the most widely used tool to discredit one’s political opponent. Of course, one reason for why such charges have become so prominent is that the way we consume information through online media has made us more vulnerable than ever to such manipulation. Take a recent story published by The Guardian, which describes the strategy of information dissemination allegedly used by the British ‘Leave Campaign’:
"The strategy involved harvesting data from people’s Facebook and other social media profiles and then using machine learning to ‘spread’ through their networks. Wigmore admitted the technology and the level of information it gathered from people was ‘creepy’. He said the campaign used this information, combined with artificial intelligence, to decide who to target with highly individualised advertisements and had built a database of more than a million people."
This might not just strike you as “creepy” but as simply unethical just as it did one commentator cited in the article who called these tactics “extremely disturbing and quite sinister”. Here, I want to investigate where this intuition comes from.
We can distinguish different ways one can go about changing someone else’s mind:
(1) by providing arguments,
(2) by appealing to epistemic trust (i.e. claims to expertise and epistemic authority),
and
(3) by exploiting how our cognitive mechanisms have evolved/developed to process information in general. For example, by directing attention in a certain way or by creating/triggering semantic associations between different contents (e.g. pairing an image of a happy family with a specific brand of car or by using terms such as ‘flood’ to describe migratory movements of people).
While in (1) and (2) one interacts with one’s audience’s cognitive machinery charged with evaluating critically what information to endorse (so-called epistemic vigilance mechanisms) in (3) one bypasses this machinery by exploiting the way that our cognitive system is designed to process information in general. For the purposes of this entry, I want to call (1) conviction, (2) persuasion and (3) manipulation. Put in these terms, the question I want to address here is why manipulation in contrast to conviction and persuasion often strikes us as ethically wrong.
One thing that bears mentioning right away is that the distinction between (1), (2) and (3) is not simply one between verbally communicated information and non-verbally communicated information. A political speech, say, can be manipulative while still employing arguments and appeals to epistemic trust explicitly. One distinction that is of importance here, however, is that of ostensively vs. non-ostensively communicated information. Manipulation, by definition, can never be ostensive since this would imply that the speaker intends her audience to know about her intention to manipulate. Manipulation is thus by default non-ostensive.
Why and how is manipulation ethically problematic?
Nonetheless, when I ask the question why manipulation should be ethically problematic, I am talking of course about intentional manipulation. In interacting with others, we always rely on their general cognitive capacities and the tendencies through which these capacities have evolved to process information. If we took ‘manipulation’ in this sense to be unethical, much of our interactions with others would be, too.
So why do we intuitively take intentional manipulation (i.e. non-ostensive forms of ‘mind changing’) to be ethically problematic? I want to suggest that this has to do with the kind of commitments that underlie communicative interaction. When I make an ostensive (i.e. explicit) claim, I thereby not only convey information but also take responsibility for its truth: I make myself accountable.
This analysis applies to conviction and persuasion in different ways. In conviction/argumentation, one provides an argument, which is intended to trigger specific inferences in one’s audience based on what that audience already believes. This minimizes the extent to which the audience has to trust the speaker. Nonetheless argumentation relies on assertions to some extent, which always force the speaker to commit herself to the truth of whatever she is stating.
Persuasion, on the other hand, works only because of the commitments involved: whenever a speaker claims epistemic authority of some kind, this claim is only effective to the extent to which she makes herself accountable for the truth of her utterances. Crucially, it is rare that conviction and persuasion occur entirely apart from each other: we commonly rely on epistemic authority in our arguments.
Manipulation differs exactly in that there is neither an explicit claim to be evaluated nor does the manipulator incur any commitment towards her target. When I create an implicit association between immigration and terrorism, for example, by mentioning both words in the same sentence, I have thereby not committed to any such association indeed existing. In this sense, manipulation crucially differs from lying since a lie always consists in an explicit claim, which commits the speaker to its truth. Manipulation intentionally shortcuts those cognitive mechanisms designed to evaluate and keep track of such commitments. The reason we take manipulation to be unethical then is because it comes with some of the benefits of persuasion and conviction without their respective costs.
Dan Sperber 10 March 2017 (13:19)
Is Johannes manipulating us?
Ostensive communication involves a layer of communication about the communication itself. The ostensive communicator not only communicates some content, but also that she wants her audience to attend to her communication and to accept its content (to believe what she states or to do what she requests). Ostensive communication is typically and specifically human. When bees or baboons communicate, they do not do so ostensively. They may do what it takes to get the attention of their conspecifics and to have the content of their communication accepted, but, for this, they don’t communicate that they want attention, let alone that they want the content of their communication accepted. They communicate non-ostensively. As Johannes points out, ostension goes with responsibility for what you communicate, or accountability – typically human aspects of social interaction.
Still, much of human communication is non-ostensive. By the way you dress, for instance, you communicate a lot about yourself to the people you interact with. Aspect of the way you dress may be ostensive – say you wear green clothes on St Patrick’s day – but most of it is not. Even if not ostensive, you may use it intentionally and for instance dress so as to communicate that you are quite informal about the way you dress. Is all this – at least when it is intentional – manipulation? If so manipulation is unavoidable, in fact pretty useful, and not necessarily objectionable. Or else is this just non-ostensive communication that we tend to call “manipulation” when we object to it?
Even when you communicate ostensively, in the very act of doing so you also convey a lot of information non-ostensively by your tone of voice, your body posture, your movements, the social register of your speech. Of course much of this can be done ostensively, but it typically is not. This may well affect the way your audience interprets the ostensive part of your communication and again it looks like manipulation in Johannes’ sense, except that it is hardly objectionable.
Take, as an example, Johannes new blog: http://cognitionandculture.local/blog/johannes-mahrs-blog. It is illustrated with a picture of our new blogger with a t-shirt and, a beard, an a hair-style that communicate informality and spontaneity, and with a smile that communicates a friendly disposition. Couldn’t this picture affect our interpretation of his post as much as, to take his own example “pairing an image of a happy family with a specific brand of car” in an advert. Aren’t we subtly nudged to trust Johannes’ post because of this “manipulation”?
The display of this picture, to quote Johannes “differs exactly [from what he otherwise communicates ostensively] in that there is neither an explicit claim to be evaluated nor does [he] incur any commitment towards [his] target.” Still, I for one, don’t feel manipulated – or at least not in an objectionable sense. In fact I am grateful for Johannes’ nice picture, and more generally, for so much that people communicate non ostensively: it helps me better interpret what is, at the same time, ostensively communicated.
This comment by the way is not an objection to Johannes’ main point, which I find insightful and convincing; it is a suggestion on how to broaden the perspective.
Johannes Mahr 10 March 2017 (15:50)
Counterfactual agreeing
Thank you for your comment, Dan! You are right, of course, that my analysis is incomplete. Indeed, as I pointed out in the post, any interaction between humans carries with it aspects which can be described as manipulation. In order to say anything, you have to say it in a certain way carrying myriads of semantic associations. In order to wear anything you have to wear something specific which in itself also carries information for those who care to make the relevant inferences. So why not intentionally design these aspects of our interactions so as to do work for us in others' minds? In fact, a lot of the time people might be entirely aware that many aspects of their informational environment are intentionally designed in a way that makes certain inferences more likely than others and are completely fine with it.
My attempt to differentiate the ethically relevant forms of manipulation via their intentionality might therefore only be part of the story. One thing that is missing, I think, is the fact that manipulations which we find morally questionable carry information which we would not be inclined to endorse when expressed explicitly/ostensively. Thus, maybe you do not find it objectionable that my picture accompanies my blog post and that it carries a certain impression of me because you might have agreed with a similar description of me had it been given to you ostensively.
Paulo Sousa 10 March 2017 (19:48)
Manipulation, morality and perception of selfishness
Many thanks for the interesting post and discussion. As Dan argued (and Johannes acknowledged), manipulation as characterized in the post is not deemed unethical simply by the fact that it involves transmission of information that is intentional but not ostensive. I would like to add that I tend to think that the perception of wrongdoing is not simply related to “the fact that manipulations which we find morally questionable carry information which we would not be inclined to endorse when expressed explicitly/ostensively”, as Johannes suggests in his reply. It seems to me that this has more to with the perception that the manipulator is not taking into account the interests of the "audience" (i.e., with the perception that the manipulator is being selfish in a significant way). Perhaps what I’m saying here was already inplicit when Johannes said in the original post that the “The reason we take manipulation to be unethical then is because it comes with some of the benefits of persuasion and conviction without their respective costs.” One could make it more explicit by including here something about the unjustifiable costs to the audience.