Note: This is an uncompleted draft chapter from a book in progress. This text is for review only, not to be quoted or referenced.
I argue that human practice and understanding in everyday life should be taken as the ontological and epistemological point of departure in inquiries into design and use of computer artifacts. Design becomes a concerted creative activity founded in our traditions, but aiming at transcendending them by anticipation and construction of alternative futures. (Ehn 1989, p.28, emphasis his)
We have discussed various methods and methodologies throughout the book. Participatory design, however, is a special case. As the name implies, the approach is just as much about design – producing artifacts, systems, work organizations, and practical or tacit knowledge – as it is about research. In this method, design is research. That is, although participatory design draws on various research methods discussed in earlier chapters – such as ethnographic observations, interviews, analysis of artifacts, and sometimes protocol analysis – these methods are always used to iteratively construct the emerging design, which itself simultaneously constitutes and elicits the research results as co-interpreted by the designer-researchers and the participants who will use the design. Like member checks in ethnographic research, participatory design’s many methods ensure that participants’ interpretations are taken into account in the research. Unlike member checks, however, these methods are shot through the entire research project; the goal is not just to empirically understand the activity, but also to simultaneously envision, shape, and transcend it in ways the workers find to be positive. In participatory design, participants’ cointerpretation of the research is not just confirmatory but an essential part of the process.
As we discussed in the Introduction, research methodologies, designs, and methods can be relinked or renetworked: brought into new relationships and transformed by them. That point is especially well illustrated in participatory design, which borrows methods from more established methodologies but, because of its very different orientation, deploys and employs them in very different ways. Participatory design is very much under development, making this reworking perhaps more evident than in more well established research approaches. And, as we discuss in the Conclusion, we expect it to continue to develop rapidly as it is taken up in writing studies.
Participatory design started in Scandinavia through a partnership between academics and trade unions. Since that time it has worked its way across the Atlantic, becoming an important approach for researchers interested in human-computer interaction, computer-supported cooperative work, and related fields. From there, it has begun to influence writing studies, particularly through technical communication as well as computers and composition. (See Spinuzzi 2002a for an overview.) As with the other methodologies we have discussed in this book, participatory design has undergone many changes – for instance, later variations have moved away from the Marxist underpinnings of the earlier work – but its core has remained more or less constant: it attempts to examine the tacit, invisible aspects of human activity; assumes that these aspects can be productively and ethically examined through design partnerships with participants, partnerships in which researcher-designers and participants cooperatively design artifacts, workflow, and work environments; and argues that this partnership must be conducted iteratively so that researcher-designers and participants can develop and refine their understanding of the activity. The result of the research typically consists of designed artifacts, work arrangements, or work environments.
As the quote at the beginning of this chapter suggests, participatory design attempts to steer a course “between tradition and transcendence,” that is, between participants’ tacit knowledge and researchers’ more abstract, analytical knowledge. The developers of participatory design believed that politically and ethically, the two types of knowledge must be bridged, with each being valued by all involved in the research. That’s especially true in studies of workers, for which participatory design was initially designed, but also in studies of end users and students.
In this chapter, we characterize participatory design as a way to understand knowledge by doing, that is, the traditional, tacit, and often invisible ways that people perform their everyday activities and how those activities might be shaped productively. (Other research approaches have been developed to meet these goals, such as usability testing; we discuss some of these briefly in the Conclusion.) To explore knowledge by doing, we address three questions: what, why, and how. We begin by defining and describing what participatory design research is. We describe participatory design research in terms of its paradigm, methodology, research design, and methods. With a definition and description as a framework, we next discuss why we should pursue participatory design studies. In this section, we discuss the benefits of knowledge by doing and provide evaluative criteria to use as guidelines for creating and assessing participatory design research. In the last section, we describe how we can construct, carry out, and critique knowledge by doing. To learn from the successful participatory design studies of the past, we analyze models of exemplary studies according to research design and methods, rhetoricality, and historical shifts. Then, focusing on a specific dissertation proposal that resulted in a strong dissertation, we set forth the procedures, the pitfalls, and the rewards of creating and reading knowledge by doing.
The participatory design approach was developed in the Scandinavian countries in the 1970s and 1980s. Scandinavian researchers such as Pelle Ehn and Morten Kyng based the approach on elements of action research and a Marxist commitment to democratically empowering workers and fostering democracy in the workplace; their avowedly political research aimed to form partnerships with labor unions that would allow workers to determine the shape and scope of new technologies introduced into the workplace. Up to that point, labor unions had little experience with computer technologies and had been forced to accept systems developed by management, systems that represented a sharp break from workers’ traditional ways of working; exerted a greater and greater control over increasingly fine details of their work; and automated large swathes of the workflow, putting people out of work (see Ehn, 1990; Zuboff, 1989). Since they did not know how to design computer technologies themselves, workers were put into the position of accepting these disempowering technologies or simply rejecting them. Ehn, Kyng, and others set out to develop a third way, an approach that provided a set of “language games” (Ehn & Kyng 1991, pp.176-177) that would allow software developers and workers to collaboratively develop and refine new technologies – thus allowing workers to retain control over their work.
In the 1970s, the Scandinavian countries (Denmark, Finland, Iceland, Norway, and Sweden) were, like the rest of the developed world, concerned about automating their industries. On the one hand, automation had the potential of making work easier, increasing productivity, and making Scandinavians wealthier. On the other hand, automation threatened the careful balance between capital and labor – employers and workers – because automation tended to radically remake jobs, severing the connection with workers’ craft traditions. Workers sometimes found to their dismay that their hard-earned experience with their craft was useless because the work itself was radically remade by the automated machinery being purchased by management. The case was not simply of pruning the labor force by reducing jobs, it was that of completely remaking the work around the new machines – and not always to the benefit of management, either, since the disruption of craft skills often meant that productivity declined while friction with unions increased. The Scandianvian countries have a strong tradition of workplace democracy – that is, the principle that workers should have a say in how their work is done – and automation threatened workplace democracy. At the same time, workers did not have experience in systems design, so they could not propose alternative systems that would be more appropriate for their work. Their only alternatives appeared to be either accepting management’s systems or simply resisting automation altogether.
Some Scandinavian academics became concerned and began to explore links with labor unions in the 1970s. Because of “two of the main barriers to effective participation in design – poor access to relevant information and [employers’] lack of appreciation for the knowledge employees had about their own work” (Clement & van den Besselaar 1993, p.33), these academics turned to participatory action research (PAR). PAR takes up traditional ethnographic methods such as observations and interviews, but enacts them differently, in ways that are more openly political and oriented to particular agendas, and links them to other methods that invite people to participate in studies of their own lives. Clement & van den Besselaar explain that “Unlike conventional research, which is directed primarily at producing results of interest to those beyond the immediate research site, an essential goal of action research is to achieve practical or political improvements in the participants’ lives (e.g., less routine work, greater autonomy, more effective tools). The researcher becomes directly involved in the ongoing work and feeds results back to the participants” (p.33). Citing participatory design researcher Kari Thoresen, they explain that action research involves the difficult work of alternating between practical work to support changes (such as design activities) on one hand, and systematic data collection and analysis on the other hand (p.33; see also Bertelsen 2000). Traditional ethnographic methods had focused on data collection and analysis – description – but the PAR orientation demanded movement toward design, and that orientation changed how the methods were enacted.
Taking participatory action research as their starting place, the Scandinavian academics developed the Collective Resources Approach (CRA), sometimes known as the “critical tradition” of Scandinavian information technology (Bansler 1989; Bjerknes 1992) or simply the “Scandinavian approach.” In 1971, Scandinavian academics and unions, led by Kristen Nygaard, began a series of projects aimed at empowering labor in its struggle with management, particularly in terms of the introduction of new technologies (Nygaard 1992; Bansler 1989). This struggle was framed in terms of democracy, based on Marx’s observation that in a capitalist society, democracy stops at the factory gates -- democracy in the workplace was counter to management’s goals of higher profits for less pay and its tactics of work rationalization and work intensification. Nygaard and his collaborators chose to work with unions in the manufacturing industry. At the time, Scandinavia was about 90% unionized, so unions had considerable power, yet they were not well equipped to proactively guide the introduction of technologies.
Nygaard’s work was overtly political; he argued that research is always political, but his work was overt because it advocated the case of labor rather than management (Nygaard 1992; Bansler 1989). Others built on this work, and eventually the Collective Resources Approach (CRA) took shape.
CRA projects dealt with how new technologies were introduced into workplaces and how they devalued workers’ traditional craft skills, deskilling jobs, removing workers’ ability to make decisions, and intensifying work. At first CRA projects involved educating workers about technology and organizing resistance to technologies that reduced industrial democracy. But in the UTOPIA project, the academics began to realize that workers had to be empowered to provide practical alternatives to management-based initiatives. That is, workers had to be able to describe a computer system that could automate work while still valuing their craft skills and upholding their autonomy. The problem was that the workers had no experience in systems design. Thus they could not begin to speculate on how to build such a system. So the UTOPIA team began experimenting with a range of techniques for discussing and exploring the possibilities, including mockups and other low-fidelity prototypes, future workshops, and organizational toolkits (Bødker et al. 1987).
The UTOPIA project was a critical success, if not a commercial one. Although the project failed to produce a working system, it did produce a design approach and a range of techniques for participatory design work. Based on UTOPIA and other projects that came after it, the Scandinavians issued the “Scandinavian challenge": to develop and use design approaches that encourage industrial democracy (Bjerknes, Ehn, & Kyng 1987). This call resulted in many approaches and techniques under the umbrella of participatory design, such as CARD, PICTIVE, cooperative interactive storyboard prototyping, and contextual design (see Muller, Wildman, & White 1993 for an exhaustive taxonomy). Some of these, such as contextual design, have become complex enough to be categorized on their own and, arguably, differentiated enough that they are no longer participatory design per se (see Spinuzzi 2002b). In the United States, because of relatively weak labor unions and a focus on functionality rather than workplace democracy, participatory design has tended to be implemented through nonintrusive methods: workplace microethnographies rather than walkthroughs and workshops, small-scale card-matching exercises rather than large-scale organizational games, and one-on-one prototyping sessions that focus on confirming developed ideas rather than group prototyping sessions that emphasize exploration.
In this section, we’ll discuss how participatory design defines and understands the object of its study: the tacit knowledge developed and used by those who work with technologies. We’ll define tacit knowledge and discuss how participatory design has made tacit knowledge its object of study.
Rationalist approaches to design, such as Taylorism, tend to assume that there is one best way to perform any activity. Thus rationalist design research therefore attempts to categorize and formalize activity at a fine grain, with the intention of finding the most optimal set of movements and forcing workers to perform those movements without deviation. According to Harry Braverman (a Marxist whose work profoundly influenced the academics who developed participatory design), Taylorism seeks to effect managerial control through “the dictation to the worker of the precise manner in which work is to be performed” (1974, p.90, emphasis his). That is, rather than allowing workers to determine how to accomplish their tasks -- and develop their own tacit craft skills and knowledge not possessed by management -- the Taylorist manager examines the work, then breaks it into discrete, formal tasks that can be optimized, regulated, and taught to new workers. All discretion and all decisions are taken away from the workers. Knowledge is made explicit, formalized, and regulated; workers’ craft traditions are judged inferior.
Participatory design opposes this notion of knowledge on both political and theoretical grounds. Politically, this notion of knowledge as wholly consisting of optimized tasks spells the death of workplace democracy: if it is accepted, workers cannot have a say in their own work because only trained researchers can determine the best way of performing that work. Theoretically, participatory design is founded on constructivism, which explicitly resists the notion that knowledge can be completely formalized and classified. (For overviews of the constructivist argument in writing studies, see Mirel 1998; Spinuzzi 2003). Knowledge is situated in a complex of artifacts, practices, and interactions; it is essentially interpretive, and therefore it cannot be decontextualized and broken into discrete tasks, nor totally described and optimized. In the constructivist view, participants’ knowledge is valorized rather than deprecated and their perspectives therefore become invaluable when researching their activity and designing new ways to enact that activity. “Knowing and learning,” as Barbara Mirel says, “take place in a dynamic system of people, practices, artifacts, communities, and institutional practices” (1998, p.13).
When we think of knowledge, we often think of explicit forms of knowledge: things that are formalized, written down, defined, categorized, systematized, or quantified. But to understand knowledge-making in participatory design, we have to understand that much knowledge tends to be tacit. Tacit knowledge is implicit rather than explicit, holistic rather than bounded, ad hoc rather than systematized; it is what people know without being able to articulate. As Ehn argues, participatory design takes a Heideggerian approach to knowledge in which “the fundamental difference between involved, practical understanding and detached theoretical reflection is stressed” (1989, p.28). This pragmatic approach involves alternating between the two by discovering tacit knowledge, then critically reflecting on it.
Since practical tacit knowledge was a main goal of early participatory design research, researchers adopted the tool perspective, the idea that “computer support is designed as a collection of tools for the skilled worker to use. The tool perspective takes the work process as its origin rather than data or information flow. This means: not detailed analysis, description, and formalization of qualifications, but the development of professional education based on the skills of professionals; not information flow analysis and systems description but specification of tools” (Bødker et al., 1987, p.261). The tool perspective involved recognizing the skilled craft that workers had developed over the years so that new tools would fit into that craft, rather than disrupting it: “a tool is developed as an extension of the accumulated knowledge of tools and materials within a domain” (p.261; see also Ehn 1989, pp.339-40). In contrast to design approaches favored by management that served Tayloristic goals (deskilling, work intensification), the tool perspective involved building “computer-based tools by which the craftsman [sic] can still apply and develop original skills” (p.261; see Ehn 1989, p.34; Ehn & Kyng 1987, p.34-38).
In his book on user-centered technology, Robert R. Johnson links tacit knowledge to metis. “Metis, or what is also called cunning intelligence, is the ability to act quickly, effectively, and prudently within ever-changing contexts” (1998, p.53). These ever-changing contexts are what Mirel points to when she talks about complex tasks (1998, 2004). In participatory design, tacit knowledge is not only explored, it is in many cases made material, as we saw with the tool perspective that participatory designers adopted. Workers find unconventional ways to use the tools that have been supplied to them, learn how to construct their own ad hoc tools (Spinuzzi 2003), and – if they are allowed the time and freedom to do so – eventually stabilize new tools and the ways they interact with them. One goal of participatory design is to preserve and institutionalize tacit knowledge, that is, to ensure that workers have a say in the design of new technologies so that those technologies fit into the existing web of tacit knowledge, work flow, and work tools – to the advantage of the workers. In contrast to rationalist studies that assume workers’ tasks can be broken down into their components, formalized, and made more efficient, participatory design assumes that tacit knowledge cannot be completely formalized; the task-and-efficiency orientation typical in many user-centered design methods (such as GOMS [Card, Moran & Newell 1983] and usability testing (Barnum 2002) can actually get in the way of the holistic activity.
Certainly some tacit knowledge can be made explicit and formalized, but “attempts at explication of such tacit knowledge must always be incomplete. The knowledge is too layered and subtle to be fully articulated. That is why action-centered skill has always been learned through experience (on-the-job training, apprenticeships, sports practice, and so forth). Actions work better than words when it comes to learning and communicating these skills” (Zuboff 1988, p.188). So tacit knowledge often remains invisible: since it is not made systematic or quantifiable, it passes unnoticed and often undervalued. (See Nardi & Engestrom 1999 for a collection of essays on this theme.) In particular, low-level workers are often not valued by management because their skills are invisible: the complexity, difficulty, and interconnectedness of their work are not recognized. One example is in Blomberg, Suchman, and Trigg’s (1994) study described below, in which document analysts (temporary workers who coded legal documents) were found to perform complex interpretive work. The attorneys who employed these workers did not recognize the work as being complex or interpretive, and consequently planned to outsource the work to lower paid workers in another country. Like others working in the participatory design tradition, Blomberg, Suchman, and Trigg attempted to demonstrate to management the tacit knowledge that workers brought to the activity, knowledge that had remained invisible up to that point, yet was vital to the continued success of the activity.
Since users’ tacit knowledge is valorized, participatory design focuses on exploring that tacit knowledge and taking it into account when building new systems. This task is accomplished with a strong political or ethical orientation: users’ knowledge is described so that it can be used to design new tools and workflows that empower the users. (What is meant by empowerment is sometimes different in the different participatory design approaches.) In this section, we’ll describe the paradigm that underpins participatory design, its methodology, research design, and methods.
Paradigm. Participatory design’s paradigm is constructivist. That is, it sees knowledge-making as occurring through the interaction among people, practices, and artifacts. Participatory design researchers tend to draw on Marxism and on monist, materialist theoretical approaches such as activity theory, actor-network theory, situated and distributed cognition, and symbolic interactionism. One of the most distinct and influential notions is that of the language game (Ehn 1989, p.17): bridging the worlds of researcher-designers and users by finding a common "language" or mode of interaction with which both parties feel comfortable.
Methodology. Participatory design’s methodology is derived from participatory action research or, as Ehn calls it, “practice research”: “Practical interventionistic investigations (as opposed to gathering of data) and parallel theoretical reflection (as opposed to detached theoretical reflections a posteriori)” (Ehn 1989, p.13). As discussed above, this activist brand of research has an explicit political-ethical orientation: to empower workers to take control over their work. To achieve that goal, participatory design emphasizes co-research and co-design: researcher-designers must come to conclusions in conjunction with users. So participatory design involves redesigning workplaces and work organization as well as work tools. And it is iterative, allowing workers and researchers to critically examine the impacts of these incremental redesigns in progress. This orientation reanimates or relinks the research design and methods.
Research Design. Participatory design is still developing and consequently its research design tends to be quite flexible. For instance, the early Scandinavian work tended to rely on union-sponsored workshops and games involving heavy direct interaction between designers and users, while later work in the U.S. has tended to rely on less intrusive methods such as observation and artifact analysis. But three basic stages are present in almost all participatory design research:
Stage 1: Initial exploration of work. In this stage, designers meet the users and familiarize themselves with the ways in which the users work together. This exploration includes the technologies used, but also includes workflow and work procedures, routines, teamwork, and other aspects of the work.
Stage 2: Discovery processes. In this stage, designers and users employ various techniques to understand and prioritize work organization and envision the future workplace. This stage allows designers and users to clarify the users’ goals and values and to agree on the desired outcome of the project. This stage is often conducted on site or in a conference room and usually involves several users.
Stage 3: Prototyping. In this stage, designers and users iteratively shape technological artifacts to fit into the workplace envisioned in Stage 2. Prototyping can be conducted on site or in a lab; can involve one or more users; and can be conducted on-the-job if the prototype is a working prototype.
The stages can be (and usually should be) iterated several times. Together, they provide an iterative co-exploration by designers and users.
Methods. Methods are grouped by stage.
Stage 1: Initial exploration of work. Since initial exploration tends to involve examining technology use on site, Stage 1 draws from ethnographic methods such as observations, interviews, walkthroughs and organizational visits, and examinations of artifacts. This stage is typically conducted on site, during the normal work day. In the earlier Scandinavian iterations, this initial exploration tended to be highly interactive and intrusive: the researchers generally aligned themselves with relatively powerful workers’ unions that believed in the projects and could insist on the sorts of disruptions caused by walkthroughs and organizational visits (see Bødker, Grønbaek, and Kyng 1993 for an overview). In North America, unions were much weaker and workers were not in a position to force participation, nor were they terribly interested in such projects. So researchers – particularly researchers at Xerox PARC, who pioneered much of the participatory design work in the United States – turned to less intrusive ethnographic and ethnomethodological techniques such as observations and interviews (see Wall & Mosher 1994 for an overview). Although the methods draw from ethnography, they are oriented toward design as well as description, so they tend to be focused and enacted differently, with more interaction in mind. Much of that interaction takes place during the second stage, in discovery processes.
Stage 2: Discovery processes. Stage 2 is where researchers and users interact most heavily, and it also typically involves group interactions. Again, discovery processes tended to be more interactive and intrusive in the earlier Scandinavian iterations than in the later North American iterations, but in all implementations they are more interactive than traditional ethnographies. Because of participatory design's orientation toward design, the goal is to cooperatively make meaning out of the work rather than to simply describe it. Methods used during this stage include organizational games (Bødker, Grønbaek, and Kyng 1993 , pp. 166-167), role-playing games (Iacucci, Kuutti & Ranta 2000), organizational toolkits (Tudor, Muller, & Dayton 1993; Ehn & Sjögren 1991; Bødker et al. 1987), future workshops (Bødker, Grønbaek, and Kyng 1993, p.164; Bertelsen 1996), storyboarding (Madsen & Aiken 1993), and workflow models and interpretation sessions (Beyer & Holtzblatt 1998).
Stage 3: Prototyping. Finally, this stage involves a variety of techniques for iteratively shaping artifacts. These techniques include mockups (Ehn 1989; Ehn & Kyng 1991; Bødker et al 1987), paper prototyping (Novick 2000), cooperative prototyping (Bødker & Grønbaek 1991; Grønbaek & Mogensen 1994); and PICTIVE (Muller 1991, 1993).
Finally, and just as importantly, results are disseminated in forms that users can understand and share – a continuation of the “language games” that allow researchers and users to collaborate, and a way to continue to support the empowerment and participation of users. The tone for this dissemination was set early on, in the UTOPIA project: results were discussed in everyday language in a union publication called Graffitti (Ehn 1989, pp.350-352). Another example is contextual design’s practice of “walking” through affinity diagrams and consolidated models with participants and of providing a room with diagrams and prototypes posted on the walls so that workers, managers, engineers, marketing people and customers can see the state of the project in progress (Beyer & Holtzblatt 1998, Ch.10).
Here, we’ll review some of the limitations of participatory design; discuss criteria for evaluating participatory design studies; and provide an overview of some exemplary participatory design studies.
Participatory design has strengths, but as with other research approaches, those strengths come with tradeoffs.
Methodological. Since participatory design aims to ground changes in traditional craft skills as a way of empowering workers, some argue that participatory design does not lend itself to radical change of the sort that sometimes must characterize new systems (Beyer & Holtzblatt 1998). Furthermore, some strains of participatory design – particularly later work that emphasizes functional empowerment over democratic empowerment – have a tendency to focus too narrowly on artifacts rather than overall workflow (Spinuzzi 2002b). And as participatory design has migrated across socioeconomic borders, from Scandinavia to North America, researchers had difficulty maintaining its methodological tenets, particularly its focus on democratic empowerment (Spinuzzi, under review; Spinuzzi 2002a).
Methodical. If more rigorous methods can be described as “measure twice, cut once,” participatory design methods can be described as “explore, get in the ballpark, then refine.” This essentially dissimilar methodological orientation – related to action research’s juggling act between the traditional researcher’s role of collecting and analyzing data vs. the activist’s role of initiating and sustaining significant change at the research site – tends to alter how researcher-designers apply established methods. For instance, participatory design researchers often draw on ethnographic methods to develop knowledge about the participants’ work, tools, and craft traditions. But these researchers, who often come from backgrounds in systems design, human-computer interaction, or technical communication, tend to apply these methods quite loosely in the eyes of trained ethnographers. Diana Forsythe, for instance, scathingly critiques these applications as “do-it-yourself ethnography” and complains that “superficial social research may confer the illusion of increased understanding when in fact no such understanding has been achieved.” She specifically takes to task a contextual design project “in which brief exercises in shadowing, observation, and interviewing have been undertaken from a common sense stance without engaging the questions that define ethnography as anthropologists understand it,” and warns that “such an exercise can result in a cognitive hall of mirrors. Without addressing basic issues such as the problem of perspective, researchers have no way of knowing whether they have really understood anything of their informants’ world view or have simply projected and then ’discovered’ their own assumptions in the data” (1999, p.136; see also Cooper et al., 1995; Nyce & Lowgren, 1995). Forsythe’s critique is valid if the aim of research is to extract knowledge in the mode of traditional research, pulling the data into another domain where it can be abstracted, analyzed, and used apart from the site. But participatory design research, properly done, continually brings the analysis back to the domain and shares it with the participants, who cointerpret it, co-analyze it, and co-design responses to it. That is, the traditional methods are – at least in the best examples – re-networked or reconfigured to meet the design orientation. As we say in the Introduction, the "same" methods can be enacted differently and take rather different shapes as they are attached to different methodologies and paradigms. In this case, the resulting research and designs do give up traditional research rigor, but they do so in order to gain reflexivity and agreement. (In the earlier, highly politicized Scandinavian work, that agreement took the shape of political representation; in later work, the focus shifted to ethical concerns in giving workers the tools needed to do their jobs, and agreement took the shape of consensus among representative users.) This tradeoff resembles “rolling” member checks.
For example, Muller (1999) describes using the participatory design technique CARD to study the work of telephone operators. CARD, he says, has less rigor and predictive power than more narrowly defined analytical techniques such as GOMS, but on the other hand it brings in benefits that are more important to participatory designers: “Its strengths lie in its ability to capture diverse information ..., its openness to the disconfirmation of assumptions ..., and its extensibility in the face of new information. Underlying all of these attributes is CARD’s enfranchisement of multiple stakeholders with differing disciplines, perspectives, and positions” (p.54; see also Bertelsen 2000). As with many user-centered design methods, rigor is difficult to achieve because researchers cede considerable control to their participants and share a “design language” with those participants which must by its nature be imprecise. On the other hand, the proof is in the pudding, so to speak – the design artifact both encapsulates the research results (as the material trace left by the design efforts) and elicits them (both during design sessions and afterwards, as it is introduced into the environment to be used as a stable work artifact). Wall and Mosher demonstrate that the same design artifacts can be used as records of a field study; tools for analysis; communication tools for a language game in which researcher-designers and users participate; and focal artifacts for co-design and co-development (1994). Rigor becomes something different in participatory design research: a desirable goal, but subordinated to users' control and aims.
Practical. In addition to the methodological and methodical critiques is the practical one: participatory design research takes an enormous amount of time, resources, and institutional commitment to pull off. That institutional commitment in particular can be hard to come by. From the standpoint of a profit-oriented business, participatory design seems to provide little structure and no deadlines (Wood & Silver 1995, pp.322-323). Researchers find that they have to cede considerable control to workers, who must be committed to the process and cannot be coerced. (Bertelsen [1996] ruefully recounts how some of his participants simply failed to show up to a future workshop, compromising the design developed in the workshop.) Finally, unlike ethnographic studies, participatory design studies typically require continuous critical participation by workers. Later participatory design variants such as contextual design [Beyer & Holtzblatt 1998] and customer partnering [Hackos, Hammar & Elser 1997] have compromised by sharply limiting users’ participation.
Participatory design is usually brought in at major turning points when work is to be automated and tools and workflows are to be changed. Since participatory design projects by definition involve design as well as research, the object of the research tends to be expressed in a purpose statement rather than a research question:
The purpose of this project ... is to design a number of computer applications for [an organization] and to develop a long-term strategy for decentralizing development and maintenance. (Bodker, Gronbaek & Kyng 1993, p.161)
The overall object of the project has been to contribute to the development of skill-enhancing tools for graphic workers. (Bodker et al. 1987, p.254)
The work-oriented design project was originally conceived to explore bringing together the worlds of corporate research, product development, and specific worksites ... in an effort to design more useful new technologies. (Blomberg, Suchman & Trigg 1997, p.269)
In concert with these types of research statements, participatory design has developed criteria that are also oriented toward development. Participatory design is still a relatively young approach, and at present it is more of a movement or research orientation than a coherent methodology, so it hasn’t developed evaluative criteria to the same level that, say, quantitative studies have. But nascent criteria do exist which we can draw from the methodological principles discussed earlier. Even though these criteria are not nearly as fully developed as, say, those of statistical studies, they are often difficult to meet. As Blomberg and Henderson illustrate, it’s easy to produce a study that looks like participatory design but that fails at all three of the criteria listed here. Participatory design projects, despite their ceding of power and analysis to users, still must rigorously apply these criteria to have internal integrity.
Criterion #1: Improvement of quality of life for workers
Most participatory designers would point to this criterion as the most important one. Participatory design is meant to improve workers’ quality of life both in terms of democratic empowerment (that is, workers’ control over their own work organization, tools, and processes) and functional empowerment (that is, workers’ ability to perform their given tasks with ease) (see Blomberg, Suchman, and Trigg 1997; Spinuzzi, Bowie, Rodgers, & Li 2003). In a participatory design study, workers critically reflect on their own practices, work organization, and tools. In the earlier Scandinavian iterations, this critical reflection usually involved examining ways that workers could better control the terms of their work; in later US-based iterations, critical reflection turned to an examination of tacit knowledge in order to more effectively meet the goals of the work. Either way, this methodological principle translates into an exploration of tacit knowledge, invisible work, and unstated individual and organizational goals.
To meet this criterion, participatory design studies strive for
Reflexivity and agreement between researchers and users. The two groups interact closely through interviews, focus groups, workshops, organizational games, prototyping sessions and other techniques to continually reassess the activity under investigation and to synchronize their interpretations.
Codetermination of the project by researchers and users. Specific project criteria are codetermined by researchers and users during the project. This way, researchers do not take total ownership of the project; users are also able to shape the project to reflect their values, goals, and ends.
Criterion #2: Collaborative development
Collaborative development is a key part of the effort to improve workers’ quality of life. As noted earlier, users’ work is often invisible and their knowledge is often tacit. Thus designers of information systems, educational websites, and documentation often assume that the work is simple, easily formalized, and (sometimes) easily automated. Collaborative development allows researcher-designers to avoid that trap by inviting participants to be co-researchers and co-developers. Doing so allows researcher-designers to elicit and explore the tacit knowledge and invisible practices that might otherwise have been lost, and simultaneously encourages workers to participate in their own empowerment.
In terms of a criterion for a study, this methodological principle translates into a requirement for mechanisms to ensure that data collection and analysis to be done in conjunction with participants. In the terms used by ethnography, participatory design uses member checks – but in participatory design, the member checks are continuous since the project is co-owned and co-enacted by the participants. To meet this criterion, participatory design studies strive for
Involvement. The successful study will provide mechanisms for participation and produce verifiable changes based on them. Participatory design studies are not a “listening tour” in which researchers hear the concerns of users, then go away and design a solution; they are participatory top to bottom and must include verifiable, regular avenues for group interaction and definite routines for ensuring that users’ concerns are methodically addressed in the resulting design.
Mechanisms for consensus/agreement and representation. In most cases, not every user can be involved in a participatory design study. For instance, if a participatory design study involved redesigning an interface used by 2000 workers, it’s simply not practical or manageable to involve every worker in workshops and prototyping sessions. Instead, workers must be represented in the same way that politicians are elected to represent the interests and views of their constituencies. In the earlier Scandinavian iterations of participatory design, representatives were assigned to projects by their unions, making them explicitly political representatives. In North America, however, unions hold considerably less power and no other ready-made mechanisms for political representation of workers exist; rather, workers are typically selected by management and are seen as functionally representative (i.e., “average users”). In any case, users must be given the opportunity to be broadly represented in the study, and the representatives should have a way to settle disagreements or come to consensus.
Common language games, such as contextual design’s work diagrams and PICTIVE’s pictures. To collaboratively develop solutions, users should be able to interact with researchers in a neutral “language” understood by both sides. It’s not enough to offer such a language game; researchers must also confirm that users are comfortable with the language game, able to understand it, and able to use it both to critique solutions and to express their own solutions.
Common aims codetermined by researchers and users in advance. Near the beginning of the project, researchers and users should be able to settle on a list of common aims that represent the users’ interests. That list must be flexible, as users will continue to critically evaluate their own aims.
Criterion #3. Iterative process
But to enact collaborative development, researcher-designers and participants must follow an iterative process. Tacit knowledge and invisible practices are by their nature difficult to tease out. A crude caricature of participatory design might involve gathering workers’ comments on current practice and their responses to a prototype, but without sustained, iterative reflection on and use of a designed artifact, workers may not be able to comment critically or respond effectively (see Hackos, Hammar, & Elser 1997). Each change in a prototype tends to unearth other invisible work practices and other tacit knowledge.
In terms of a criterion for a study, this methodological principle translates into a requirement for a series of opportunities to sustain the continuous member check. To meet this criterion, participatory design studies strive for
Continual participation. Users should be involved repeatedly or continually and offered mechanisms for co-design at multiple stages.
Revisiting stages. Rarely is one sweep through the stages enough because the stages are designed to inspire critical reflection on the work and turn up tacit knowledge. So a successful participatory design project should be flexible enough to revisit stages repeatedly and cyclically.
Sustained reflection. Finally, the continuous member check must go beyond simply reacting to the functionality of designs – a danger especially in the later stages of a project, when functioning prototypes take on the appearance of completeness and participants’ attention often turns to minor details. At all points, participants should be encouraged and given avenues to critically reflect on the implication of the research results for their own work.
Bansler, J. (1989). Systems development research in Scandinavia: Three theoretical schools. Scandinavian Journal of Information Systems, 1:3-20.
Bertelsen, O. W. (1996). The festival checklist: Design as the transformation of artefacts. In PDC '96 Proceedings of the Participatory Design Conference, pages 93-101. Computer Professionals for Social Responsibility, Palo Alto, CA.
Bertelsen, O. W. (2000). Design artefacts: Toward a design-oriented epistemology. Scandinavian Journal of Information Systems, 12:15-27.
Beyer, H. and Holtzblatt, K. (1998). Contextual design: Defining customer-centered systems. Morgan Kaufmann Publishers, Inc., San Francisco.
Bjerknes, G. (1992). Shared responsibility: A field of tension. In Floyd, C., Zullighoven, H., Budde, R., and Keil-Slawik, R., editors, Software development and reality construction, pages 295-301. Springer-Verlag, New York.
Bjerknes, G., Ehn, P., and Kyng, M., editors (1987). Computers and democracy - A Scandinavian challenge. Avebury, Aldershot, England.
Blomberg, J. and Henderson, A. (1990). Reflections on participatory design: Lessons from the trillium experience. In CHI '90 Human Factors in Computing Systems, pages 353-359. ACM, Seattle, Washington.
Blomberg, J., Suchman, L., and Trigg, R. (1994). Reflections on a work-oriented design project. In Trigg, R., Anderson, S. I., and Dykstra-Erickson, E., editors, Participatory Design Conference (PDC'94), pages 99-109, Palo Alto, CA. Computer Professionals for Social Responsibility, P.O. Box 717, Palo Alto, CA 94302-0717.
Blomberg, J., Suchman, L., and Trigg, R. (1997). Back to work: Renewing old agendas for cooperative design. In Kyng, M. and Mathiassen, L., editors, Computers and Design in Context, pages 267-288. MIT Press, Cambridge, MA.
Bødker, S., Ehn, P., Kammersgaard, J., Kyng, M., and Sundblad, Y. (1987). A Utopian experience: On design of powerful computer-based tools for skilled graphical workers. In Bjerknes, G., Ehn, P., and Kyng, M., editors, Computers and democracy - A Scandinavian challenge, pages 251-278. Avebury, Aldershot, England.
Bødker, S. and Grønbaek, K. (1991). Cooperative prototyping: Users and designers in mutual activity. International Journal of Man-Machine Studies, 34:453-478.
Bødker, S., Grønbaek, K., and Kyng, M. (1993). Cooperative design: Techniques and experiences from the scandinavian scene. In Schuler, D. and Namioka, A., editors, Participatory design: Principles and practices, pages 157-176. Lawrence Erlbaum Associates, Hillsdale, NJ.
Braverman, H. (1974). Labor and monopoly capital. Monthly Review Press, New York.
Card, S. K., Moran, T. P., and Newell, A. (1983). The psychology of human-computer interaction. Lawrence Erlbaum Associates, Hillsdale, NJ.
Clement, A. and Van den Besselaar, P. (1993). A retrospective look at PD projects. Communications of the ACM, 36(4):29-37.
Cooper, G., Hine, C., Rachel, J., and Woolgar, S. (1995). Ethnography and human-computer interaction. In Thomas, P. J., editor, The social and interactional dimensions of human-computer interfaces, pages 11-36. Cambridge University Press, New York.
Ehn, P. (1989). Work-oriented design of computer artifacts. Lawrence Erlbaum Associates, Hillsdale, NJ.
Ehn, P. and Kyng, M. (1987). The collective resource approach to systems design. In Bjerknes, G., Ehn, P., and Kyng, M., editors, Computers and democracy - A Scandinavian challenge, pages 17-58. Gower, Brookfield, VT.
Ehn, P. and Kyng, M. (1991). Cardboard computers: Mocking-it-up or hands-on the future. In Greenbaum, J. and Kyng, M., editors, Design at work: Cooperative design of computer systems, pages 169-196. Lawrence Erlbaum Associates, Hillsdale, NJ.
Ehn, P. and Sjögren, D. (1991). From system descriptions to scripts for action. In Greenbaum, J. and Kyng, M., editors, Design at work: Cooperative design of computer systems, pages 241-268. Lawrence Erlbaum Associates, Hillsdale, NJ.
Forsythe, D. E. (1999). "It's just a matter of common sense": Ethnography as invisible work. Computer Supported Cooperative Work, 8:127-145.
Grønbaek, K. and Mogensen, P. (1994). Specific cooperative analysis and design of general hypermedia development. In Trigg, R., Anderson, S. I., and Dykstra-Erickson, E., editors, Participatory Design Conference (PDC'94), pages 159-171, Palo Alto, CA. Computer Professionals for Social Responsibility, P.O. Box 717, Palo Alto, CA 94302-0717.
Hackos, J. T., Hammar, M., and Elser, A. (1997). Customer partnering: Data gathering for complex on-line documentation. IEEE Transactions on Professional Communication, 40(2):102-110.
Iacucci, G., Kuutti, K., and Ranta, M. (2000). On the move with a magic thing: Role playing in concept design of mobile services and devices. In DIS '00, pages 193-202, Brooklyn, NY. ACM, Inc.
Johnson, R. R. (1998). User-centered technology: A rhetorical theory for computers and other mundane artifacts. SUNY Press, New York.
Madsen, K. H. and Aiken, P. H. (1993). Experiences using cooperative interactive storyboard prototyping. Communications of the ACM, 36(4):57-66.
Mirel, B. (1998). "Applied constructivism" for user documentation. Journal of Business and Technical Communication, 12(1):7-49.
Mirel, B. (2004). Interaction design for complex problem solving: Developing usable and useful software. Morgan Kauffman, San Francisco.
Muller, M. J. (1991). PICTIVE - an exploration in participatory design. In CHI'91, pages 225-231. ACM, New Orleans, LA.
Muller, M. J. (1993). PICTIVE: Democratizing the dynamics of the design session. In Namioka, A. and Schuler, D., editors, Participatory design: Principles and practices. Lawrence Erlbaum Associates, Hillsdale, NJ.
Muller, M. J. (1999). Invisible work of telephone operators: An ethnocritical analysis. Computer Supported Cooperative Work, 8:31-61.
Muller, M. J., Wildman, D. M., and White, E. A. (1993). Taxonomy of PD practices: A brief practicioner's guide. Communications of the ACM, 36(4):26-27.
Nardi, B. A. and Engeström, Y. (1999). A web on the wind: The structure of invisible work. Computer Supported Cooperative Work, 8:1-8.
Novick, D. G. (2000). Testing documentation with ``low-tech'' simulation. In SIGDOC 2000, pages 55-68. ACM, Cambridge, MA.
Nyce, J. M. and Lowgren, J. (1995). Toward foundational analysis in human-computer interaction. In Thomas, P. J., editor, The social and interactional dimensions of human-computer interfaces, pages 37-47. Cambridge University Press, New York.
Nygaard, K. (1992). How many choices do we make? how many are difficult? In Floyd, C., Zullighoven, H., Budde, R., and Keil-Slawik, R., editors, Software development and reality construction, pages 52-59. Springer-Verlag, New York.
Smart, K. L. and Whiting, M. E. (2002). Using customer data to drive documentation design decisions. Journal of Business and Technical Communication, 16(2):115-169.
Spinuzzi, C. (2002a). A Scandinavian challenge, a US response: Methodological assumptions in scandinavian and us prototyping approaches. In ACM SIGDOC 2002 Conference Proceedings, pages 208-215. ACM, New York.
Spinuzzi, C. (2002b). Toward integrating our research scope: A sociocultural field methodology. Journal of Business and Technical Communication, 16(1):3-32.
Spinuzzi, C. (2003). Tracing genres through organizations: A sociocultural approach to information design. MIT Press, Cambridge, MA.
Spinuzzi, C., Bowie, J., Rogers, I., and Li, X. (2003). Open systems and citizenship: Developing a departmental website as a civic forum. Computers and Composition, 20(2):168-193.
Spinuzzi, C. (Under review). "Lost in the translation: Shifting claims in the migration of a research technique."
Tudor, L., Muller, M., and Dayton, J. (1993). A C.A.R.D. game for participatory task analysis and redesign: Macroscopic complement to PICTIVE. In INTERCHI'93. ACM, Inc, Amsterdam.
Wall, P. and Mosher, A. (1994). Representations of work: Bringing designers and users together. In Trigg, R., Anderson, S. I., and Dykstra-Erickson, E., editors, PDC'94: Proceedings of the Participatory Design Conference, pages 87-98, Palo Alto, CA. Computer Professionals for Social Responsibility, P.O. Box 717, Palo Alto, CA 94302-0717.
Wood, J. and Silver, D. (1995). Joint Application Development. John Wiley & Sons, New York, second edition.
Zuboff, S. (1988). In the age of the smart machine: The future of work and power. Basic Books, New York.