Abductive policy making. Draft section of report from research fellowship

Image from Policy Lab/Strange Telemetry project with Government Office for Science

Between discovery and justification

The aim of this section is to help clarify what kinds of experiments are going on in the work of Policy Lab. This section draws on the work of philosopher Charles Sanders Peirce who developed the term abduction. First it will describe what abduction is, and how abduction relates to the other kinds of inference, deduction and induction, which are well-established in policy development. It will then discuss Policy Lab’s work through the lens of abduction and show how policy experimentation via abductive reasoning intersects with the other logics shaping policy making.

Deduction and induction

The term abduction is associated with the philosopher CS Peirce, who explored the term over some decades[1]. Like other Pragmatists such as Dewey, Peirce’s approach has an orientation toward our experiences of what happens in practice, rather than proposing an idealized analysis. To explain why Peirce’s work is a useful contribution to understanding Policy Lab’s approach requires a brief detour into the two other logics through which reasoning is usually understood to proceed.

Deduction is the process of taking a principle (a rule) and then inferring a result in a particular case. For example:

Rule: People living in the Midlands are friendly.

Case: These people are from the Midlands.

Result: These people are friendly.

Deductions offer reliability if the initial statements are true, but Peirce argued they do not generate anything new.

Induction starts with surveying data (the case) and generalising across many observations (the result) to identify a pattern (the rule). For example:

Case: These people are from the Midlands.

Result: These people are friendly.

Rule: People from the Midlands are friendly.

Inductions indicate probability about patterns in the data. They suggest that something is the case. Depending on the research methods used, they offer some kind of validity. As with deduction, Peirce argued inductive reasoning does not generate new concepts or knowledge.

In the context of experimental research, knowledge building typically proceeds by developing a hypothesis based on the stock of existing knowledge via deduction and then seeking confirmation by induction if it holds in a particular case.

The logics of policy making

Much of the evidence used to inform policy making uses mixed methods based on deductive or inductive reasoning in various combinations. Neither are right or wrong – they do different things and offer different kinds of validity to different audiences, to allow policy officials and ministers to reach decisions. But in the culture of policy making, the deductive logic offers the allure of offering definitive evidence. For one civil servant, “Trials are the gold standard for policy making”[2] because they are able to prove whether something is true or not, providing sound evidence that decision makers want about whether to go ahead with a policy.

Deductive reasoning underpins work in the natural and physical sciences and also shapes research in the social sciences. For example the methodological approach “Test, Learn Adapt” advocated by the Behavioural Insights Team is grounded in deductive logic [3]. BIT helps civil servants design and construct trials of policy by demonstrating whether an intervention will achieve intended outcomes, informed by existing knowledge about human behavior. Randomised control trials (RCTs) are one way to test systematically, in a particular case, whether a hypothesis underpinning an intervention is true or not. In policy terms, RCTs can prove whether a proposed intervention will lead to the desired change in a particular case. It generates statistically valid data about changes to variables associated with the outcome that result from the intervention.

Inductive reasoning is also very familiar within the policy environment. It underpins much of the research in the social sciences and the humanities. Inductive research does not have to use qualitative data but it is strongly associated with it. Researchers specialising in research methods working within this logic make efforts to show to what extent their findings have validity. They make careful claims about whether they can show links between cause and effect and discuss the extent to which their findings are generalisable to other contexts.

Abduction

But where do hypotheses and new ideas come from in the first place? What happens in the context of massive uncertainty, when there is very little data, or much of it is in disagreement? What if you have a desired outcome that you want to achieve but are not sure of the constituent elements that might help you achieve it or how they relate? How do researchers get to the point that they are able to isolate an outcome variable which could be tested through a trial?

Recognising a gap in the philosophy of science, Peirce developed the idea of abduction as a logic of discovery within scientific inquiry, in contrast to the logic of justification associated with deductive reasoning. He argued that philosophers of science had paid insufficient attention to where ideas come from. Informed by ancient Greek thought, he developed the concept of abduction to explain how new concepts and hypothesis are created.

Abduction takes a result and a rule, and then jumps to making an interference that links the two. For example:

Rule: People from the Midlands are friendly.

Result: These people are friendly.

Case: These people are from the Midlands.

In abduction, we link things together in new ways. We can’t say if the interference is true or not, as is the case with deduction. Nor can we say it has strong validity because of the observations we made, as with induction. But with abductive reasoning, what we do get is a new insight or concept that we can explore further with the other two logics.

Hypotheses are not out there waiting to be discovered. Instead, Peirce argued, they are the outputs of a process of sensemaking. As we make observations through our own experience of the world, we compare these to the existing stock of knowledge. We may find something surprising that we can’t account for, resulting in a tentative guess – an embryonic hypothesis. Social researcher Jo Reichertz explains, “Something unintelligible is discovered in the data and, on the basis of the mental design of a new rule, the rule is discovered or invented and, simultaneously, it becomes clear what the case is.”[4] In contrast to deduction, which offers reliability, abduction offers possibility by generating something new, which can then be explored further through induction and deduction.

With the concept of abduction, Peirce was able reconnect creativity in science with the well-established idea that scientific reasoning can prove things. Other researchers such as Karl Popper separated the logic of discovery from the logic of justification. They focused on how science can make more reliable truth claims, paying less attention to how novel concepts are generated [5]. In Peirce’s own words, “Abduction is the process of forming an explanatory hypothesis. It is the only logical operation that introduces new ideas, for induction does nothing but determine a value, and deduction merely evolves the necessary consequences of a pure hypothesis.”[6] Table 1 shows the differences between the three logics, based on Peirce’s work. In his view there is an order: abduction precedes deduction and induction [7].

Table 1: Peirce’s ordering of the logics of scientific inquiry, developed from Hansen 2008.

Producing plausible, provisional results

Abduction reasons from effects to causes with incomplete data. It constructs plausible guesses and insights, shaped by our existing stocks of knowledge and in responses to effects gathered through observations or experiences. For management researcher Hans Hansen, “We take disparate elements and place them into relationships that are meaningful for us. Abduction generates hypothesis (sic) in the absence of any existing construct to interpret observation.”[8] Abduction shows something may be, but does not prove it, whereas deduction shows something is true in a particular case. Abductive inferences are plausible but are not justified by the structure of the argument. But they are plausible enough to move a project forward[9].

Abduction results in a new order that takes surprising observations and offers a way to make sense of them – for now – which is still productive. However for social researcher Jo Reichertz,

“The search for order is never definitively complete and is always undertaken provisionally. So long as the new order is helpful in the completion of a task it is allowed to remain in force: if its value is limited, distinctions must be made; if it shows itself to be useless, it is abandoned. In this sense, abductively discovered orders are neither (preferred) constructions nor (valid) reconstructions, but usable (re-) constructions.”[10]

At first glance there is a relationship here with the work of Karl Popper. He argued that science proceeds by hypotheses being challenged or upheld by subsequent research, a process he called falsification. But abductive interferences are never as firm as hypotheses in the first place. They are provisional, plausible constructs that are usable – they move a process of inquiry along – but do not offer a truth claim.

Participants in Policy Lab workshop discussing their hopes and fears in relation to a policy area

Creating the conditions for abduction

When developing his theory of abduction, Peirce discussed the conditions that gave rise it to it, which can be seen as broader principles for enabling the generation of new ideas. Rather than arguing that coming up with new insights is simply the result of chance, he identified strategies or enabling conditions for making it more systematic[11].

Peirce developed his thoughts on abduction as a result of a personal experience[12]. This was when a Tiffany watch was stolen from him after he left it behind by accident on a ship to New York. Initially he had no idea who was responsible[13]. In his account of the stolen watch, Peirce asked the captain of the ship to line up all the crew for him to talk to. At first he found himself unable to work out who might have taken his watch. Reflecting on what happened, Peirce described how this experience prompted him to reconsider how knowledge is generated.

The first condition conducive to the presence of developing an abductive inference is genuine doubt, uncertainty or great pressure to act[14]. The Tiffany watch was a gift, and a valuable one. But the fear motivating Peirce was not fear of its loss, but of professional disgrace for not being able at first to work out who might be guilty.

The second condition conducive to abduction Peirce identified was to let his mind wander with no specific goal – what he called “musement”. Instead of trying to use deductive logic to work out who had stolen his watch, Peirce gave into a state that was not controlled by his conscious mind. As a result, he concluded that his consciously working mind, which usually relied on logical rules, was outmaneovoured[15].

The third condition to make abduction more likely was to decide to act, even if the direction seemed arbitrary. As he tried to work out who had stolen the watch by walking up and down the lined up crew of the ship, Peirce concluded he must fasten on someone even through it would be almost a random choice[16]. The guess that Peirce made turned out later to be true and eventually he got the watch back. Management researcher Hans Hansen summarises, “At the point of being surprised by a surprising fact, if we can make a guess, any guess, we can make progress.”[17]

Recent interest in abduction

Researchers and practitioners in several domains are using abduction to help them distinguish between different kinds of research activity and practical experimentation. In business, Roger Martin[18] argued that managers need to use abductive as well as deductive and inductive reasoning as tools to achieve competitive advantage. In design[19], researchers have used the theory of abduction to explain how designers come up with new concepts. In social research, especially in fields such as nursing, researchers have turned to abduction to better understand how themes, codes and categories emerge during research[20]. In artificial intelligence and data science, there is a longstanding interest in abduction and induction and how they relate to one other in algorithmic machine learning[21].

Participants in a Policy Lab prototyping workshop

Abduction in policy making

Drawing these argument together, the concept of abduction helps those involved in policy experimentation distinguish between the logic of discovery and the logic of justification. What Peirce’s ideas do is highlight the often invisible work that goes on during what we might call the “fuzzy front end” of policy making[22].

This discussion highlights the different kinds of reasoning produce different results at different phases of the policy making cycle. They are not directly comparable and further, if Peirce is right, then there are interdependencies between them. Peirce’s view is that there is a sequence which starts with abduction. The exploratory insights and guesses produced through abductive reasoning with limited data but with are nonetheless plausible can then be further developed through deductive and inductive reasoning. Abduction produces the provisional insights and guesses linking things together in new ways, that become hypotheses that be tested through experimentation and other research based in deductive and inductive logics. Deductive research can answer if a policy intervention works or not; inductive research helps explain why it does[23]; but abductive reasoning enables the discovery of insights and guesses when there is not yet theory or evidence but a desired policy outcome.

This highlights the mostly invisible work that policy makers do, where they are required to rapidly gather and assess evidence and come up with options for ministers. Looking at this through the lens of Peirce’s work, we might pay more attention to this early, hypothesis-free, exploratory phase. Trials test policy interventions in particular kinds of case when there is a hypothesis to test. In contrast design and prototyping support policy inventiveness by making plausible links between elements of an issue to achieve an intended outcome. If RCTs are a robust way establishing if a policy is working[24], the abductive cycle of generating and exploring insights and guesses is the best way of developing and iterating plausible early-stage policy ideas.

This is where Policy Lab’s approach comes in. It is rooted in the practical experimentation of design going through cycles of rapid insight generation, idea generation and exploration via prototyping. Working within this tradition, Policy Lab foregrounds the abductive early stage of policy making – where “early stage” includes revisiting persistent, complex policy issues. Alongside data science, and agile approaches also used in government, Policy Lab reveals and supports the often neglected abductive work that policy makers do, but opens it up to a wider group of participants and new sources of evidence and inspiration which the next section will illustrate.

References (not complete or checked)

[1] Scholars make a distinction between his earlier work using the term and his later work, which is what I draw on here. See Roozenburg, N. (1993). On the pattern of reasoning in innovative design. Design Issues 14(1): 4-18.

[2] The speaker was a participant in a workshop organised by the What Works network held at the Institute for Government in June 2015.

[3] Haynes, Laura, Service, Owain, Goldacre, Ben and Torgerson, David. (2012). Test, Learn, Adapt: Developing Public Policy with Randomised Controlled Trial. London: Cabinet Office.

[4] Reichertz, Jo. (2010). Abduction: The Logic of Discovery of Grounded Theory. Forum: Qualitative Social Research. 1(13). Italics in original.

[5] Reichertz, Jo. (2010). Abduction: The Logic of Discovery of Grounded Theory. Forum: Qualitative Social Research. 1(13).

[6] CS Peirce cited in Hansen, Hans. (2008). Abduction. In Barry, David and Hansen, Hans (eds). The Sage Handbook of New Approaches in Management and Organization. p.456

[7] Table developed from Table 3.5.1 in Hansen, Hans. (2008). Abduction. In Barry, David and Hansen, Hans (eds). The Sage Handbook of New Approaches in Management and Organization. p.457.

[8] Hansen, Hans. (2008). Abduction. In Barry, David and Hansen, Hans (eds). The Sage Handbook of New Approaches in Management and Organization. p.457

[9] This is the underpinning to lean start up and agile software development.

[10] Reichertz, Jo. (2010). Abduction: The Logic of Discovery of Grounded Theory. Forum: Qualitative Social Research. 11(1). Article 13. http://nbn-resolving.de/urn:nbn:de:0114-fqs1001135.

[11] Reichertz, Jo. (2010). Abduction: The Logic of Discovery of Grounded Theory. Forum: Qualitative Social Research. 11(1). Article 13. http://nbn-resolving.de/urn:nbn:de:0114-fqs1001135.

[12] Reichertz, Jo. (2010). Abduction: The Logic of Discovery of Grounded Theory. Forum: Qualitative Social Research. 11(1). Article 13. http://nbn-resolving.de/urn:nbn:de:0114-fqs1001135.

[13] Boje, D. (2014). Storytelling Organizational Practices: Managing in the Quantum Age. Abingdon: Routledge. p.39-40.

[14] Reichertz, Jo. (2010). Abduction: The Logic of Discovery of Grounded Theory. Forum: Qualitative Social Research. 11(1). Article 13. http://nbn-resolving.de/urn:nbn:de:0114-fqs1001135.

[15] Reichertz, Jo. (2010). Abduction: The Logic of Discovery of Grounded Theory. Forum: Qualitative Social Research. 11(1). Article 13. http://nbn-resolving.de/urn:nbn:de:0114-fqs1001135.

[16] Boje, D. (2014). Storytelling Organizational Practices: Managing in the Quantum Age. Abingdon: Routledge. p.39-40.

[17] Hansen ibid p.461.

[18] Martin, Roger. (2009). The Design of Business: Why Design Thinking is the Next Competitive Advantage. Harvard Business Press.

[19] See for example, Roozenburg op cit; Kolko, John. (2010). Abductive Thinking and Sensemaking: The Drivers of Design Synthesis. Design Issues, 26(1); and Dorst, K. (2015). Frame Innovation: Create New Thinking By Design. MIT Press.

[20] See for example Tavory, Iddo and Timmermans, Stefan (2014). Abductive Analysis: Theorising Qualitative Research. Chicago University Press.

[21] See for example Flach, Peter and Kakas, Antonis (eds). (2000). Abduction and Induction. Essays on Their Relation and Integration. Springer.

[22] Fuzzy front end is a term describing the early stage of product development, introduced in Khurana, Anil and Rosenthal, S. (1997). Integrating the Fuzzy Front End of New Product Develpoment. Sloan Management Review, Winter. pp. 103-120.

[23] https://www.ipsos-mori.com/Assets/Docs/Scotland/Approach%20Summer%202012/SRI_Scotland_Newsletter_Summer2011_Test_Learn_Adapt.pdf

[24] Haynes, Laura, Service, Owain, Goldacre, Ben and Torgerson, David. (2012). Test, Learn, Adapt: Developing Public Policy with Randomised Controlled Trial. London: Cabinet Office. p.4.