Policy making as a collective inquiry. Draft section of report from research fellowship

Policy Lab prototyping workshop

This is a draft section from the report I’m writing for Policy Lab in which I’ve been embedded for 10 months. The primary audience will be policy makers and others involved in public sector experimentation. The final report will be published in late August/early September.

The workshop: Experiencing policy making as a collective inquiry

In a windowless conference room in the basement of the Department for Business, Innovation and Skills, about 20 civil servants are sitting on chairs arranged in rows facing a raised stage. On tables to one side are piles of brightly coloured materials such as pens, Play-Doh, straws, cardboard boxes and pipe cleaners. Andrea Siodmok, head of Policy Lab, and I arrive slightly late from running a workshop elsewhere. She checks with our Policy Lab colleagues who are already present if the Powerpoint slides she planned to use are ready on the computer to be projected. They are. This is a two-hour event on prototyping in government organised by Policy Lab as part of Open Policy 2015, a week-long series of practical workshops and talks aimed at policy makers.

 

Andrea does not go on to the raised stage but instead stays on the same level as the seated participants. She moves in front of the people and starts talking. She apologises for being slightly late, says the workshop will be mostly practical and invites people to ask what they want to find out today about prototyping in government. As she talks and listens she hands out some of the coloured Play-Doh to participants and starts molding some in her own hands. Pretty soon all the people in the room are molding Play-Doh in their hands. The mood is open, relaxed and expectant.

 

She leads a discussion on prototyping to which people in the room are contributing unselfconsciously and in an open manner. She instructs people to make a particular shape with the Play-Doh – a duck – and people do and then are willing to hold up and share what they’ve done with others. She then says “One of you is going to shout out what you are going to make next” and we hear someone say “monkey”. Andrea says “Ok, 15 seconds” and everyone quickly makes a monkey shape and again shares them. Then someone calls out “Fox”. One man holds up his shape: “It’s part fox, part monkey and I’m very proud of him” and we laugh.

 

About ten minutes in, it feels as if the workshop has not yet started but participants in the room continue to be attentive. Any skepticism they might feel is not evident in their behaviour. Andrea shares observations on prototyping drawing on the work of UK manufacturer Dyson as well as her early career as a product designer. She says “The purpose of prototyping is to come up with something tangible we can test, share and improve” and talks about how Policy Lab is working with departments to explore how to use this approach at the early stages of policy making, exploring what policy ideas might looks like at the point of delivery and experience. She then instructs, “One last prototype – in 15 seconds. Make a prototype of how to integrate health care and social care.” People laugh at the shift in register from the childlike to the very serious policy challenge, but carry out the request. Andrea then asks people to share their prototypes. “Mine’s a car crash – the incentives are misaligned,” says one woman. Another says, “I’m really trying to join them up.”

 

Andrea invites people to share reflections on what they think prototyping is. “It’s quick and easy – you can see the result,” says one woman. Another says, “There are lots of levels of abstraction”. Someone else points out that participants’ experiences were impacted by their capabilities with using Play-Doh. Another person reflects, “What it did was force us to do something – often in policy making we sit around trying to come up with the perfect idea.”

 

Participants in the workshop continue to be very engaged. Andrea moves to the stage and starts showing the Powerpoint slides which are projected on to a large screen. One of the slides includes a screengrab of the text of a recent speech by the Home Secretary in which she refers to prototyping a new service for people reporting crime, one of Policy Lab’s projects[1], possibly the first time a minister has used the term prototyping in public. Andrea shows photographs, many from local government projects, of exploring new policy or public service concepts at a very early stage by mocking up what the experience would be like for end users or citizens.

 

About an hour in to the workshop, Andrea sets a challenge for participants. This is to work together create a prototype of a new kind of GP surgery based on the principle of “the patient will see you now” rather than the current model of “the doctor will see you now”. Over the next hour participants work together in small teams of about five, producing ideas they share in a two-minute pitch to the rest of the group which is also an opportunity for feedback. To facilitate this, some of us arrange chairs around some large tables so that people can work together around a flat surface.

 

Most of the participants, all civil servants, are strangers to one another. They come from a range of departments and policy areas but they succeed in quickly self-organising into teams and collaborating to generate and prioritise ideas. They help themselves to the materials and get on with making small models of alternative GP surgeries, responding to the items made available such as cardboard boxes, pens, feathers and pipecleaners. As I move from group to group, I hear them discuss things like patients’ needs, frustrations they themselves have with visiting GP surgeries and concepts from other service contexts.

 

Everyone seems engrossed in the flow of exploring what a new primary healthcare experience might be and what the infrastructure and resources are required to deliver it. When they come to share their models, some of them use role play to show the new patient experience but they also refer to some of the resources and infrastructure associated with delivering it. One group communicates their concept through a woman performing as if in a TV advert. After each team presents we all have an opportunity to give feedback on the ideas they have presented or query details.

 

The workshop ends with a brief discussion led by Andrea. She emphasizes that prototyping is an early stage, exploratory and collaborative activity that can be done very early on when concepts are very malleable, as well as later on when concepts are more defined. It’s striking how she continues not to offer a definitive set of proposals as to what prototyping might mean in the context of policy making. Instead, the workshop has involved a practical exploration of the question in relation to a policy challenge. To conclude, she asks, “How many of you are creative?” Nearly everyone puts a hand up. The workshop is over. The Policy Lab team gather up the materials, put the room back to its arrangement of tidy rows of chairs facing the front, and leave.

There are many ways of discussing what was going on in this workshop. In what follows I will draw on the work of Pragmatist philosopher John Dewey to illuminate what made this workshop hang together: how it was that playing around with Play-Doh and pipecleaners became an appropriate and productive way of exploring the issue of prototyping in government and what it offers to conventional ways of thinking about policy making.

Making sense of Policy Lab: Policy making as inquiring

John Dewey made an important contribution to philosophy by focusing on how knowledge is developed in practice, rather than formulating concepts or generating facts without connecting with what goes on in the world. Dewey was one of several Pragmatist philosophers working in the early 20th century who challenged the well-established mind-body dualism in which thinking was maintained as separate from the world. Dewey’s thought was shaped by the idea that humans exist through interacting with their surroundings.

Much of Dewey’s work was in the realm of the theory of knowledge, although he preferred to use the term inquiry. For Dewey and other Pragmatists, what mattered was knowledge being put into use in the world to achieve human ends, rather than more abstract discussions of logic and truth. The point of inquiry is to provide a basis for action. Our knowing is a result of our interacting with our surroundings. Dewey’s work has been extremely influential, including shaping student-centred learning in education and professional development as well as action research and participatory community development.

Dewey’s definition of inquiry can be summarized as the controlled or directed transformation of an indeterminate situation into one that is determinate enough to hang together[2]. Dewey says the process of inquiry involves these steps

  • recognition that there is a problem, which could have a solution
  • working out what the constituents of the problem are
  • making observations about the problem
  • allowing a possible relevant solution to present itself; through so doing, more aspects of the problem become clear
  • exploring the meanings of possible solution to see if they are relevant to the problem at hand
  • finding facts to see if they link up with other facts to produce a coherent whole of problem and solution
  • further developing a new ordering of the facts which suggests a modified idea (or hypothesis), which results in new observations, which results in a new ordering; and
  • continuing this cycle until a new order is judged to be complete. In the course of this “the ideas that represent possible solutions are tested or ‘proved’.”[3]

So for Dewey, in inquiry the problem and a solution emerge together through practical interventions into and observations of the world. Instead of this being a linear process in which first a problem is defined and then solutions are found, in inquiry, the problems and solutions co-evolve together. Dewey’s philosophical argument has since been demonstrated through academic research into how designers approach their work, sometimes called “design thinking”. Researchers have shown that during designing, the problem and the solution emerge together[4].

However Dewey was writing about how knowledge is created in science. To explore how this is relevant to policy making which, like design, is a practical endeavor that involves lots of stakeholders, it’s worth going into a little more detail about how ideas are generated and how they are used.

“Because inquiry is a progressive determination of a problem and its possible solution, ideas differ in grade according to the stage of inquiry reached. At first, save in highly familiar matters, they are vague. They occur at first simply as suggestions; suggestions just spring up, flash upon us, occur to us. They may then become stimuli to direct an overt activity but they have as yet no logical status. Every idea originates as a suggestion, but not every suggestion is an idea. The suggestion becomes an idea when it is examined with reference to its functional fitness; its capacity as a means of resolving the given situation.”[5]

So for Dewey, ideas are not just abstractions; they act in and on the world. And in acting in the world, ideas help re-organize the current understanding of a problem. Through generating ideas and exploring them in relation to the problem, a better understanding emerges of its nature as well as its possible solution.

Dewey’s argument highlights the provisionality of ideas. What matters is how ideas are put into operation to check their fitness.

“Ideas are operational in that they instigate and direct further operations of observation; they are proposals and plans for acting upon existing conditions to bring new facts to light and to organize all the selected facts into a coherent whole.”[6]

To summarise, the key ideas from Dewey’s work relevant to this discussion about Policy Lab and its way of working are as follows.

  • An inquiry is a process of creating knowledge which is always purposeful, rather than concerned with generating abstractions that are not connected to practical situations.
  • The process of inquiry does not start with a problem and then move to solutions. Rather, the problem and the solution co-evolve together.
  • The constituents of an issue are not known in advance; they are discovered through inquiry.
  • Inquiring into a problem by generating ideas helps clarify the nature of the problem.
  • Ideas are always provisional and can be put to work.
  • Exploring ideas helps organize the understanding of the problem and its solution.

Dewey’s work offers a way of thinking about what Andrea was doing in the workshop that helps explain (a) Policy Lab’s experiential approach and (b) what prototyping is.

Staging a collective inquiry. Rather than talking from a position in which she held the knowledge, Andrea constructed the workshop as a collective inquiry. While Andrea is a Senior Civil Servant and has extensive experience of leading strategic design projects, she did not invoke her position in the hierarchy or her expertise to make an argument from authority. There was just one occasion where Andrea did use an argument from authority, when she showed on screen the text of a recent speech by the Home Secretary using the word “prototyping”. Other than this, from the outset of the workshop, she involved participants in exploring together what prototyping in government might be. Instead of telling them what she knew – a kind of one-way knowledge transfer – she enabled people to explore through discussion and practical, embodied interaction with the question of what prototyping in government is or could be. Instead of problem-solving for or with participants, she involved them in inventive “problem making” in the sense of collectively working out what the constituents of the problem might be[7]. Through so doing, she created space for participants to share their lack of knowledge along with their knowledge and ideas. Through the practical activities accompanying the discussion, she invited them to progressively develop their own understanding of the question.

A grounding in practice. Andrea’s design and facilitation of the workshop oriented participants towards action, rather than abstract discussion. For example she set them a challenge in which they could practically explore what prototyping might mean in the context of policy making. By choosing the topic of healthcare, this enabled people to draw on their own lived experience of healthcare. Since most likely everyone in the room had direct relevant experience of implications of the policy question, this enabled the workshop’s inquiry to proceed quickly, as everyone was a ‘user’ of a primary health care service. Further, when setting the challenge, Andrea asked people to make a model of the new GP surgery service and try to communicate what the experience it would offer to a patient. Rather than designing an ideal form such as a healthcare system, positioning themselves as outside of it, participants were asked to describe someone’s future experience of primary healthcare, and to communicate what it would be like inside of it.

Exploring the problem by iteratively exploring ideas and making observations towards coherence. Participants were empowered and supported to explore the question of prototyping in policy making by trying out the techniques Andrea introduced. Unexpected in the context of government, the Play-Doh exercise was extremely simple and accessible to everyone present. Through several rounds of making animal shapes, followed by a policy challenge of health and social care, the activity normalized the idea that it was possible and even worthwhile to give tangible form to your ideas. Then, through collectively making and sharing simple physical models of new kinds of GP services, participants developed a deeper understanding of the nature of healthcare service delivery and experience. By spending an hour making the models, they went through cycles of understanding the problem, making observations (based in some cases on their own experiences), generating ideas, exploring the ideas in relation to the problem, further developing the ideas, making new observations and building towards a (temporary) coherence between the problem and the solution.

Participatory leadership. The way the workshop was delivered resulted in handing over much of the responsibility for its success to participants. Throughout the workshop, participants were constructed as having to find their own answers to the question at the heart of the workshop about prototyping in government, rather than being persuaded (or not) by Andrea’s expert position.

All the way through, participants were invited to share what they wanted to learn, what they believed and what they knew. Andrea offered many opportunities for them to shape the activities, for example, calling out what animal to make next with Play-Doh, deciding which challenge they should work on together, or inviting them to move to another group if they wanted to.

Andrea’s use of humour and her self-deprecating stance further emphasized her rejection of holding an expert position. This opened up the question for participants about their degree of participation in the workshop.

Experiencing policy making. The design and facilitation of the workshop invited people to manipulate and organize material things – the Play-Doh, the cardboard boxes, the feathers. Andrea and the rest of us in the Policy Lab team intervened to reorganize the room to enable people to work together at tables, rather than being bounded by the conventional format of the room as we found it with rows of chairs facing a stage. Rather than operating in the domain of manipulating symbols, with which civil servants are very comfortable, the workshop foregrounded the material, the spatial and experiential. This decentred the civil servants from their own expertise. It also invited people to reflect on their own experiences as civil servants – the things they take for granted in the material and spatial organisation of their work and how it enables or inhibits how they approach problem finding and exploration in their day to day routines.

The next section will go on to explore what this approach offers to policy makers.

Footnotes

Note – the references are not yet checked or complete

[1] https://openpolicy.blog.gov.uk/2015/02/03/prototyping-an-online-crime-reporting-service-a-policy-lab-success-story/

[2] Dewey, J. Logic: The Theory of Inquiry. pp 104-105

[3] Dewey p 113-114

[4] Dorst and Cross 2001

[5] Dewey p 110

[6] Dewey pp 112-113

[7] In sociological research exploring the public understanding of science, Mike Michael and others have developed the concept of “inventive problem making” to describe bringing together distinct perspectives on an issue in material and discursive form. See Michael, M. (2012). “What Are We Busy Doing?” Engaging the. Idiot.” Science, Technology & Human Values 37, no. 5 (2012): 528-54.

Phases of the research: Phase 0 – Discovery

This blog is a public-facing vehicle for findings from my one-year, three-day a week AHRC research fellowship based in Policy Lab, a small team based within the Cabinet Office working with government departments. What I’ve discovered is that it was incredibly hard for me to make sense of this research while I am in the middle of it – at least in ways I want to share openly via a blog.

Now, as I write up some of my findings in various formats for sharing inside Policy Lab and within the policy profession, as well as with others outside government with an interest in design and policy experimentation, I am ready to do some blogging. But it’s the retrospective sense-making kind – a sort of short-form research lite – rather than near real-time reflections from the field which I had wanted to do. This is the first in a series of posts to chart what I now see as the main phases of the research.

Phase 0 – Discovery – September to December 2014

At the point of starting the fellowship I very little knowledge of policy making or how government works. Whitehall and Westminster were parts of London I rarely set foot in. I was more likely to see them on the news that go there. I already knew some of the Policy Lab team – Andrea Siodmok, who leads it, for example, has been involved with strategic design and policy issues for over a decade through her work at the Design Council and the DOTT Cornwall project and our paths had crossed several times. I had briefly met Beatrice Andrews of Policy Lab and Maria Nyberg of the Open Policy Making team (OPM) through my research on Mapping Social Design for the AHRC.

When I started, Policy Lab had been set up in April 2014 and was now mid way through its first demonstrator project with the Home Office on crime reporting. It existed in the form of a very small resource – some headcount (less than 3 people full-time), some budget, some desks to sit at – and what seemed like a constant stream of activities – workshops, discussions, meetings, writing up workshops, planning and drawing. Much of the discussion was about what Policy Lab could or should be and activities to bring it into being by trying things out. With blog posts on the Open Policy blog and a twitter feed it was already visible to the world outside government, while inside it was just as much in formation in relation to Cabinet Office and departmental priorities. The election in May 2015 was already on people’s minds.

Much of my participant observation consisted of turning up in the morning, and following the team around to their various workshops and meetings. In particular I sat in on several “Lab Lights”, short taster sessions during which Policy Lab would work with policy makers from a government department enabling them to try out using design methods on their challenges. Mostly I listened and asked some questions as I tried to understand what was going on. I wrote lots of notes (by hand) and tentatively took some photos (see the note on ethics below). I did some interviews which I sometimes recorded by audio. I did a few literature searches and nosed around the web.

The practicalities of getting inside the building where Policy Lab is based hampered my research. Policy Lab is part of the Government Innovation Group (GIG) team in the Cabinet Office, based in the HM Treasury building at 1 Horseguards Road. As a visitor to the building your host has to come down and escort you through the barriers and stay with you throughout your visit, which does not suit day-long visits, several days a week, where you might want to make tea, go for lunch, pop out of the building, or go to the toilet.

Then in mid-December, my security clearance came through and I finally got given a security pass giving me access to many government buildings. Around the same time the team switched to using a calendar system that meant I could see their diaries (even as I write this I am wondering how open to be about locations and technology providers, and erring on the side of caution). This meant I could see what was planned each day – I didn’t have to ask when I arrived what was happening and where and if I could come along. I was also given access to the digital drive where Policy Lab’s documents were stored which I could now roam around, much as I could now roam around the building unescorted. We talked about whether I should get a cabinetoffice.gov.uk email address too to have access to the email discussions where much of their work happened. But to get this I’d have to get a secure laptop and use it and I was not keen.

Working out what research ethics meant in this context too was a challenge. The Policy Lab team and the slightly larger OPM team who we sit next to and work closely with knew who I was and what I was doing. At meetings with others I made sure I that I introduced myself as a researcher and explained that I was taking notes and might be using them. I always emphasised that I would not attribute anything to anyone named without asking them which seemed to satisfy them, although even if I was not naming them or making them recognisable I was still researching them. Sometimes I took photos of workshops as did other members of the Policy Lab team and we said we might use these images publicly for Policy Lab – but where did Policy Lab end and my research begin? For my few formal interviews I brought a consent form for the participant to sign. But informed consent was tricky to work out for a long-term engagement when I was working consistently with some people, meeting many others each week, and had access to privileged information.

The blurred lines between research and my own strategic design practice emerged early on too. For example I helped facilitate one of the eight concurrent ideas days organised by Policy Lab for Northern Futures. For this I was paid as a facilitator on the same basis as the other facilitators. But unlike most of them as a researcher I was involved in helping shape and make sense of the ideas days and what learning they offered to Policy Lab and the Cabinet Office. My growing access to this world was had limits, of course. One of the people we worked closely with did not give me access to the summary of that event which was a report that was going to be seem by a minister. A line emerged – things that were for ministers were not things I could see.

A brief trip to Washington DC for a data science and humanities workshop, to New York to visit GovLab, the Parsons DESIS Lab and Public Policy Lab, and another short trip with some of the Policy Lab and OPM team to MindLab in Denmark, gave me access to other practitioner and practice-academic hybrids also experimenting with design and policy. By the Christmas break I had a sense of what Policy Lab was trying to do, I knew something of the culture of the civil service and the emerging “policy profession” and the OPM agenda. I was enjoying being part of this all-women (at that point) team. And I was excited by the idea that academic research could help shape organisational practice in the field.

My guides at this point were Dan Neyland’s Organisational Ethnography, Bent Flyvberg’s Making Social Science Matter and Geertz’s The Interpretation of Cultures. Re-reading these while visiting a central government department on regular basis helped me acknowledge the essential ambiguity inherent in participant observation. Just as Policy Lab was performing itself into being as a resource for the policy profession, so I too was performing myself into being as a practice-oriented researcher.

Ethnography in policymaking: Barriers and opportunities

Originally published on the Cabinet Office Open Policy blog on 27 March, this is my write up of an event I chaired during Open Policy 2015.

Example of ethnographically-informed research output from a Policy Lab/Department of Health/Department of Work and Pensions project about people in employment with health conditions: “Due to [the employer] not helping (they didn’t provide sick pay), I got into extreme financial difficulty. Benefits weren’t enough to cover everything and everything went wrong. The empty fridge represents no money as I couldn’t feed myself sometimes.” Picture and story from Katrina, circling in and out of work with a physical health condition, Sussex (name and details anonymised)
Example of ethnographically-informed research output from a Policy Lab/Department of Health/Department of Work and Pensions project about people in employment with health conditions: “Due to [the employer] not helping (they didn’t provide sick pay), I got into extreme financial difficulty. Benefits weren’t enough to cover everything and everything went wrong. The empty fridge represents no money as I couldn’t feed myself sometimes.” Picture and story from Katrina, circling in and out of work with a physical health condition, Sussex (name and details anonymised)
Increasingly on the agenda of policymakers is a need to understand the needs, capacities and perspectives of citizens, service users, beneficiaries and front line staff so that policies are fit for purpose and deliverable and public services are better designed. Ethnography is seen as one way to achieve this. Based on a methodology originally associated with anthropology, ethnographic approaches are now found within product development, innovation, strategy, marketing, and research and development in a wide range of organisations from Intel and Amazon to start-ups to central and local government. In the UK, the Government Digital Service (GDS) and government departments, as well as others such as Nesta and the King’s Fund have promoted ethnographically-informed approaches to doing user research.

But the value of ethnography is not simply that it’s a method for understanding people in the context of their own lives, although it does offer that. The real potential for ethnography in policymaking is to help reframe government’s understanding of its purposes and how the world in which it exists and which it shapes is changing. This insight emerged from a panel discussion organised by Policy Lab in the grand surroundings of the Churchill Room during Open Policy 2015, during which three people with different perspectives reflected on the opportunities and barriers for ethnography in government. An audience of over 50 people, the majority of whom were civil servants, gained a valuable overview from leading practitioners applying ethnographic approaches to contemporary organisational and social issues.

The first speaker, Dr Simon Roberts of Stripe Partners, set the scene, informed by his extensive knowledge of applied ethnography from consulting work and gained by twice co-chairing the international Ethnographic Praxis in Industry conference. He summarised what makes ethnographic research distinctive and illuminated the ongoing challenge of understanding its value and impact.

Next up was Lisa Rudnick from Interpeace, previously at the UN Institute for Disarmament Research, where she co-led the development of the Evidence-based Program Design tool and the Security Needs Assessment Protocol with Derek Miller, now of the Boston-based consultancy, The Policy Lab. Sharing perspectives gained from conducting ethnographic research in post-conflict contexts such as Nepal conducted with Ruth Edmonds,  she highlighted the importance of not just using the parts of ethnography that generate local descriptions, but of engaging ethnographic analysis of those descriptions as well as the basis from which to design, or implement, policy for shaping local action on the ground.

Example of ethnographically-informed research output from a Policy Lab/Department of Health/Department of Work and Pensions project about people in employment with health conditions: Photo taken by interviewee Kelly to highlight the amount of work required to manage health conditions at her place of work. (Details changed)
Example of ethnographically-informed research output from a Policy Lab/Department of Health/Department of Work and Pensions project about people in employment with health conditions: Photo taken by interviewee Kelly to highlight the amount of work required to manage health conditions at her place of work. (Details changed)


The third speaker, Rupert Gill, is a policymaker within the Department of Work and Pensions, currently using ethnography as well as data science approaches on a joint project with the Department of Health and Policy Lab. He shared some of the challenges civil servants face when trying to make use of ethnography in a policymaking culture which values particular kinds of argument and evidence.

What ethnographic research is

  • Today’s ethnographic research in organisations exists on a spectrum from hypothesis-free, exploratory research over several months into topics such as “ageing and mobility”, to targeted requirements gathering over a few days to inform the design of a service.
  • As a kind of qualitative research, ethnography investigates worldviews, socio-cultural structures and the practices that shape behaviours. It’s not just finding out what people think, listening to what they say or watching what they do.
  • Ethnographic research makes a commitment to being there with people in their worlds – which these days includes people’s digital lives. Its emblematic method since anthropologist Malinowski went to the Trobriand Islands a century ago is participant observation.

What ethnographic research produces

  • Ethnography is not just descriptive fieldwork. It’s a theory-building endeavour that makes use of research from across the social sciences. While the data might include stories about people’s lives, in their own language and categories, or observations about what people do, what is just as important is the interpretive analysis of that data. Or as Lisa Rudnick put it, “It’s not the story that matters for policymaking. The value is in what makes the story make sense.”
  • As a result, the output of ethnography is informed by people’s stories, and generates insights derived from people’s day to day experiences, but is better understood as an analysis of a social world within which people exist within and have relationships with others including organisations, governments and places.

The opportunity for using ethnography for government

  • The value of ethnographic research is how it creates (re)framings of a social world and helps an organisation understand what it exists for. Reflecting on the impact on technology firms such as Intel, which have made extensive use of ethnography over nearly two decades, Simon Roberts argued, “Ethnography has created a space and a possibility for organisations to reshape their understandings of the world and their understandings of how they have those understandings.”
  • The opportunity is for government to address the complexity of society by understanding people better in the context of their lives, and then changing the focus of policy responses, especially when things are changing. Rupert Gill said there was an appetite for this within the civil service. “We hope to get insights we wouldn’t get elsewhere and use them to create interventions we wouldn’t otherwise have thought of.”

Barriers and challenges

  • The culture within which policy making takes place is dominated by the need to produce evidence that is statistically valid, and not “policy by anecdote”. The small sample sizes associated with ethnographic research may not be seen as valid in this context. Minsters who have to give an account to parliament about their policies feel more confident about analysis from large data sets. But there is a contradiction here, in that ministers also get first hand access to, and are influenced by, stories from their constituents – a kind of field data with very small samples sizes.
  • What’s needed is to combine quantitative data with other approaches, recognising what each brings. “It’s real depth that we need and we can’t get this from numbers,” argued Rupert Gill.
  • Within the UN and peacebuilding contexts in which she works, Lisa Rudnick shared how the approach she co-developed (with Derek Miller) makes managers accountable for the data they use (or don’t) to shape their decisions. Like the policy tests being used in the civil service, this involves asking managers considering a proposal if they have the right kind information for the question at hand, enough information and whether it’s reliable. Posing such questions makes any commitment to action rest on research findings, not on data points or methods.


Policy Lab, GDS and government departments continue to explore ethnographic approaches in practical projects in policy making. If you are a civil servant, look out for guidance on the Open Policy Making Toolkit, Civil Service Learning short courses and Policy Lab workshops to try using the approach yourself.

AHRC-ESRC Design Research Fellowship at the UK Cabinet Office PolicyLab

Policy Lab Innovation Unit MOJ workshop

At the beginning of September I began a one-year research fellowship in the UK Cabinet Office’s PolicyLab, created by an open call by the AHRC and ESRC.

The Policy Lab was set up in April 2014 by the Open Policy Making team of the Cabinet Office, part of the UK Civil Service. It aims to experiment with bringing creative problem-solving, design thinking and user centred design approaches to policy-making in central government by undertaking projects with the 17 government departments funding the lab. Led by experienced designer Dr Andrea Siodmok, this is a one-year experimental pilot that aims to bring creative approaches to the early stages of generating policy, by focussing on people’s experiences, using data analytics, digital tools and prototyping to engage a more diverse group of people in policy-making.

The Policy Lab builds on similar activities elsewhere, notably in Denmark, which set up a cross-ministerial innovation unit MindLab over a decade ago, as well as Nesta’s Public Innovation Lab, France’s 27e Region, and The Australian Centre for Social Innovation. It sits alongside other related Cabinet Office initiatives such as the Behavioural Insights Team and the Government Digital Service.

Most of the existing literature on these initiatives currently comes from reports, conferences and blog posts. See for example the recent report on i-teams (innovation teams) by Nesta. There is comparatively little academic writing about using “design” approaches in policy making in central government. This research fellowship will draw on academic research in design studies, participatory design, participatory governance, participatory action research and management research to provide a thorough review of this emerging initiative. It will present an account of what such approaches do within central government policy-making, aimed in the first instance at the PolicyLab’s key stakeholders.

My research will anchor the Policy Lab’s activities within a strong evaluative framework supported by academic research and co-designed with its major stakeholders through a collaborative process. This framework will enable the Lab to evaluate and analyse the impact its approach and methods have on developing and implementing policy.

My two main outputs will be a report summarising the evaluation framework and a literature review that combines different fields to support further academic research into this emerging area.

Links
@PolicyLabUK