Zimmerman, J., J. Forlizzi, & S. Evenson. 2007. “Research Through Design as a Method for Interaction Design Research in HCI”, Carnegie Mellon University “Research Showcase @ CMU”
Zimmerman, Forlizzi, and Evenson’s article “Research Through Design as a Method for Interaction Design Research in HCI” seeks to make two contributions to the current way that human-computer interaction scientists can benefit from collaboration with design researchers. Firstly, they present a model of “interaction design research designed to benefit the HCI research and practice communities”, and; secondly, they present a “set of criteria for evaluating the quality of an interaction design research contribution” (p. 1). Their point of departure, which serves as the paradigmatic frame for understanding interaction design research, is in the principle of “research through design”, which is discussed in the article by Racine and Frankel, as a way to address “wicked problems” (p. 5). The paradigms proposed are based on a brief literature review and a sample of interviews with interaction designers and HCI researchers.
In that brief literature review of design research, the authors differentiate design and design research, situating the latter as something informed by reductionist scientific thinking, and heuristic problem-solving. In this context, the authors argue that HCI is primarily a research-oriented design practice which benefits from design perspectives and processes, such as design thinking, but which is nonetheless a scientific and research-based discipline (p. 4-5).
Interviews with interaction designers revealed a several findings which inform the model they present. Primarily, three significant patterns of thought emerged in the interviews on the subject of the contribution of design to HCI. Firstly, that interaction designers “brought a process for engaging in massively under-constrained problems” (p. 5); secondly, that designers brought a process of integrating ideas from “art, design, science, and engineering” (p. 5); thirdly, that designers brought “empathy for users” (p. 5). Secondarily, the interviews revealed a lack of clarity on what design research is or should be in the context of HCI. In probing this perceived deficit, however, a number of roles for design researchers to play emerged:
“(i) design researcher in service of a research community—working to help researchers ground and frame problems and communicate the impact; (ii) design researcher as critic of the HCI community—making artifacts that stimulate discussion of critical issues; and (iii) design researcher as pattern finder, finding patterns that lead to pattern languages” (p. 6).
Interviews with HCI researchers also revealed additional findings. While the general consensus of designers by HCI researchers was that design is focused solely on aesthetics of software and hardware (p. 6), HCI researchers often had no concrete ideas about what design researchers do (p. 6). Generally, they reported that design research was a product of interaction designers “working in research to invent what design research should be in the context of HCI” (p. 6).
In the model presented, interaction design researchers integrate theoretical knowledge as well as models about behaviour with the technical opportunities created by engineers. As Zimmerman et al write:
“Design researchers ground their explorations in real knowledge produced by anthropologists and by design researchers performing the upfront research for a design project. Through an active process of ideating, iterating, and critiquing potential solutions, design researchers continually reframe the problem as they attempt to make the right thing. The final output of this activity is a concrete problem framing and articulation of the preferred state, and a series of artifacts—models, prototypes, products, and documentation of the design process” (p. 6).
The process of framing and reframing the intersubjectivity of problems and producing valuable artefacts of knowledge in the “research through design” approach contributes to the HCI community in a number of ways, as the authors model. The interaction designers: 1) work with engineers to incorporate new technologies, and explore technical opportunities; 2) work with anthropologists to incorporate field data about context and meaning, but also contribute to that data; 3) work with behavioural scientists to apply models and theories about behaviour, but contribute to their development by exploring gaps in theory and unanticipated events/effects; 4) contribute to producing research artefacts which HCI practitioners use to fuel research and development (p. 7).
The authors’ second contribution, the criteria for evaluating interaction design contributions, is a four-part rubric:
Firstly, the contribution should be evaluated based on the process of the research. Examine the rigor of the methods applied and the rationale for their selection of specific methods. They must provide enough detail that the process they employed can be reproduced (though there is little hope that reproducing the process will yield identical results as in a scientific experiment).
Secondly, the research should constitute a significant intervention. The contribution must be a novel integration of subject matters to address a given situation. Extensive literature reviews must situate the work to be done and demonstrate how their contribution advances the current state of research.
Thirdly, the research must be relevant as opposed to valid. While it is likely impossible that two designers if given the same problem or problem framing would develop identical artifacts, interaction design research should frame the work within the real world in the vein of ethnography and must articulate the “preferred state” their design attempts to achieve in this context. They should provide support as to why this state should be considered preferred by the community (i.e. motivation, current situation, and preferred state).
Finally, the last criteria is extensibility, or the ability to build on the resulting outcomes of the research. Essentially, if the research can be leveraged in a future design problem and to what degree.
The authors’ model situates interaction designers as something different to what was revealed in interviews. Instead, the design research is akin to a design practitioner (p. 8). According to them, design researchers work to make the right things (contextually relevant), while design practitioners work to create commercially successful things; design researchers produce knowledge artefacts which are novel integrations of theory, technology, user need, and context, not simply refinements on products which exist already. The authors present four key criteria for evaluating the contribution of design researchers in this context (as addressed above, this is the second contribution to the field these authors hoped to make with this article). Given that the authors do not indicate the frequency of the reported roles of design researchers/interaction designers, we cannot assess the merit of their choices to incorporate certain activities into the model. Nor do they fully explain the basis for their selection of literature in the review portion of the article. Granted, we give them the benefit of the doubt that their model is based on an honest assessment of available data, and that the choices made to remark upon some comments were methodologically sound ways to explore both the frequency and the qualitative similarity of the interview comments.
Another shortfall of the article is the lack of rigor in the author’s own criteria for assessing the contribution of design research. For example, though they ask that interaction designers’ contributions be evaluated on the basis of a novel integration of technology and theory, they do not present a consistent framework for such an evaluation – novelty is certainly a relative assessment influence by a number of factors (available information, target audience or consumer market). Neither is novelty a marker of long-term success: for example, we could argue that Google’s Waymo car is more novel an integration of today’s technology and research, but in the autonomous vehicle market, there’s no indication that something more novel and by extension less familiar, will outperform (both functionally and commercially) something that is less novel and more familiar. Similarly, the criteria of relevance makes no distinction with respect to different community stakeholders, and how to evaluate an intervention which pleases some and displeases others (which is often the case within wicked problems). These are considerations worth discussing.
Zimmerman, Forlizzi, and Evenson’s article is an interesting exploration of how HCI can integrate the research through design paradigm into research and development applications. Additionally it makes the case for the lack of understanding about roles and responsibilities between HCI practitioners, interaction designers, and design researchers, and makes the case for what those roles can be.
 HCI is Human Computer Interaction