On the Use of Natural Experiments in Anthropology

Controlled and replicated laboratory experiment, in which an experimenter directly manipulates variables, is often considered the hallmark of the scientific method. It is virtually the only method employed in laboratory physical sciences and in molecular biology. Without question, this approach is uniquely powerful in establishing chains of cause and effect. But the cruel reality is that manipulative experiments are impossible in many fields widely admitted to be sciences, such as evolutionary biology, paleontology, epidemiology, historical geology, and astronomy. When one is studying bird communities, dinosaurs, smallpox epidemics, glaciers, or other planets, manipulative experiments are not possible. One therefore has to devise other methods of "doing science". A technique that frequently proves fruitful in these disciplines is the so-called natural experiment. A natural experiment is a naturally occurring instance of an observable phenomenon which approximates or duplicates the properties of a controlled experiment. In contrast to laboratory experiments, these events are not created or directly controlled by scientists. However, they can yield data that can be used to make causal inferences.

Jared Diamond, of Guns, Germs and Steel fame, has just edited a book [1] along with Harvard Economist James Robinson on natural experiments in history. This book reviews eight comparative studies drawn from history, archaeology, economics, economic history, geography, and political science. The studies cover a spectrum of approaches, ranging from a non-quantitative narrative style in the early chapters to quantitative statistical analyses in the later chapters. The studies range from a simple two-way comparison of Haiti and the Dominican Republic, which share the island of Hispaniola, to comparisons of 81 Pacific islands and 233 areas of India. The societies discussed are contemporary ones, literate societies of recent centuries, and non-literate past societies. Geographically, they include the United States, Mexico, Brazil, Western Europe, tropical Africa, India, Siberia, Australia, New Zealand, and other Pacific islands.

At a time where anthropologists are doing more and more experiments, this books reminds us that laboratory experiments are not the only way to rigorously test a theory.

One might think this is just the old comparative method. That would be misleading. Natural experiments are not just about comparing two situations, they are about finding the good situation to test a specific theory and it usually involves operationalizing the hypothesis in a way that allow statistical analysis. Actually, the studies as well as the methodological discussion in the Afterword suggest that natural experiments not only allow disentangling several hypotheses. It also allows being much more aware of the limitations of one's data and theories.

The whole book is interesting but the last two chapters are, I think, the most remarkable (I keep chapter 1 on Polynesian cultural evolution for a subsequent post). They describe natural experiments in which the historical consequences of a massive perturbation (British colonial rule in India and institutional changes accompanying French Revolutionary conquests) can be examined because the perturbation operated in a geographically irregular patchwork over a large region. When one compares the perturbed patches with the unperturbed patches, it is thus a plausible hypothesis that average societal differences observed between the two types of patches arose from the operation or the nonoperation of the perturbing factor rather than from some other differences between the patches. If however, the patches with and without the factor had instead been distributed in some geographically regular way (e.g. all the patches with the factor being in the south or at high altitude), it would have been an equally plausible hypothesis that those geographic differences rather than the presence of absence of the factor caused the observed societal differences. Of course, the studies must also address the question of the direction of cause and effect: did the perturbation really cause the observed differences, or might the instigators of the perturbations (British administrators, French conquerors) have instead chosen particular patches in a geographically irregular patchwork because of pre-existing differences that should be considered the real causes of the modern differences?

Social scientists have long emphasized the importance of institutions in nurturing economic growth and development (see in particular the work of Elinor Ostrom, recent Nobel prize). Abhijit Banerjee and Lskshmi Iyer examine the importance of institutions on development by studying the impact of British colonial institutions in India (the original article is here). They show that the land revenue system imposed by Britsih administrators was based on a serie of historical accident and that these accidents give us a test case to study the effect of institutions.

They find that areas of India that were historically under the control of big landlords (and consequently had much higher levels of land inequality - doesn't it sound familiar?) tended to have fewer schools and paved roads, lower literacy, and less use of domestic electricity than areas indirectly administered in the past (note that the administrative difference disappeared 60 years ago in the aftermath of the independence).

Daron Acemoglu, Davide Cantoni, Simon Johnson and James Robinson explore the debate concerning the effects of the massive institutional changes introduced by the French revolutionary armies and Napoleon into conquered areas of Europe (the original paper is here).

The authors compare areas of Germany with and without the massive institutional changes, and they describe the historical accidents that caused the changes to be applied in a geographically irregular patchwork over Germany. Those institutional changes led to increase urbanization but after a lag of several decades, owing to a lag in the arrival of the Industrial Revolution.

The conclusion of the book struck me, as I had the feeling that it could have been written for students of culture.

"Every field of scholarship, not just human history, experiences tension between narrowly focused case studies and broader synthesis or generalization. Practitioners of the case study method tend to decry syntheses as superficial, coarse-grained, and absurdly oversimplified; practitioners of syntheses tend to decry the case studies as merely descriptive, devoid of explanatory power, and unable to illuminate anything except one particular case study. Eventually, scholars in mature fields come to realize that scholarly understanding required both approaches. Without reliable case studies, generalists have nothing to synthesize; without sound syntheses, specialist lack a framework within which to place their case studies. (...)

Some specialist historians would respond with an implicit objection (...). An example of this objection could be phrased as follows: "I have devoted forty years of my professional life to studying the American Civil War, and I still don't fully understand it. How could I dare to discuss civil wars in general, or even just compare the American Civil War with the Spanish Civil war, to which I have not devoted forty years of study? And worse yet, isn't it outrageous that some scholar of the Spanish Civil War dares to trespass on my turf and to say something about the American Civil War?" Yes, if you study an event for a long time, that does give you one type of advantage. But you gain a different type of advantage by taking a fresh look at an event, and by applying to it the experience and insights that you have gained by studying other events. We hope that this book will offer useful guidelines to historians and social scientists desiring to exploit that advantage."

To finish, natural experiments seem particularly interesting for our field. Think for instance of natural experiments of cultural transmission. That would surely help to solve theoretical debates. Though, I have difficulties finding studies in cognition and culture that make use of natural experiments (I think to Joe Henrich's study on the relationships between demographics and cultural transmission in Tasmania, and to Richard Sosis's study on the longevity of religious and non religious communities, although I am not sure that it really counts as a natural experiments since the two groups differ on many other aspects than religion).

What about you dear readers? Do you have natural experiments of cognition and culture in mind? Would it be possible edit a book similar to Diamond and Robinson's in ethnology?

[1] Diamond, J., & Robinson, J. A. (Eds.). (2010). Natural experiments of history. Harvard University Press.