Design Research in Information Systems

Design Science Research in Information Systems

Section Editors: Vijay Vaishnavi and Bill Kuechler


[Overview of Design Science Research]  [Can Design be Research]  [Outputs of Design Science Research]
[ An Example of Community Determined Outputs] [The Philosophical Grounding of Design Science Research]
[ Design Science Research Methodology]  [A Design Science Research Example ]
[Design Science Research vs. Design] [Citations on Design Science Research]  [Resources For Design Science Research]


  Welcome

Welcome to the page on Design Science Research in Information Systems (IS). The intent of the page is to provide design science researchers in IS as well as others interested in design science research with useful information on understanding, conducting, evaluating, and publishing design science research.

  Introduction

This page is dedicated to design science research in Information Systems (IS). Design science research is yet another "lens" or set of analytical techniques and perspectives (complementing the Positivist and Interpretive perspectives) for performing research in IS. Design science research involves the design of novel or innovative artifacts and the analysis of the use and/or performance of such artifacts to improve and understand the behavior of aspects of Information Systems. Such artifacts include - but certainly are not limited to - algorithms (e.g. for information retrieval), human/computer interfaces and system design methodologies or languages. design science researchers can be found in many disciplines and fields, notably Engineering and Computer Science, using a variety of approaches, methods and techniques. In Information Systems, following a number of years of a general shift in IS research away from technological to managerial and organizational issues, an increasing number of observers are calling for a return to an exploration of the "IT" that underlies all IS research (Orlikowski and Iacono, 2001).

The page is organized as follows. We begin with a general overview of design science research, provide its philosophical and epistemological underpinnings, and contrast design science research in IS with traditional positivist and qualitative research in IS. This is followed by a section on design science research methods that includes an extended discussion of a published example of design science research in IS. Through the example we hope to make concrete all phases of the design science research method: artifact design, construction, analysis and evaluation. This is then followed by a number of resource sub-sections that relate to design science research in general as well as to design science research in IS: citation lists, links to resources on the Internet for design science researchers, calls for papers and links to conferences, workshops, journals and communities of practice for IS design science research.

The goal is to provide the IS community with useful information on design science research both in and outside of IS. The page contains numerous citations permitting the interested reader to easily access original material on and examples of this unique and dynamic IS research paradigm.

If you wish to cite this work, the complete citation information is included below. Please send suggestions for improvement to the Section Editors at: vvaishna@gsu.edu or kuechler@unr.edu

  Overview of Design Science Research

         Research

Drawing heavily from Kuhn (1996; first published in 1962) and Lakatos (1978), research can be very generally defined as an activity that contributes to the understanding of a phenomenon. In the case of design science research, all or part of the phenomenon may be created as opposed to naturally occurring. The phenomenon is typically a set of behaviors of some entity(ies) that is found interesting by the researcher or by a group - a research community. Understanding in most western research communities is knowledge that allows prediction of the behavior of some aspect of the phenomenon. The set of activities a research community considers appropriate to the production of understanding (knowledge) are its research methods or techniques. Historically, some research communities have been observed to have nearly universal agreement on the phenomenon of interest and the research methods for investigating it; in this page we term these paradigmatic communities. Other research communities are bound into a nominal community by overlap in sets of phenomena of interest and/or overlap in methods of investigation. We term these pre-paradigmatic or multi-paradigmatic research communities. As of the writing of this page Information Systems is an excellent example of a multi-paradigmatic community.

          Design

Design means "to invent and bring into being" [Webster's Dictionary and Thesaurus, 1992]. Thus, design deals with creating something new that does not exist in nature. The design of artifacts is an activity that has been carried out for centuries. This activity is also what distinguishes the professions from the sciences.  "Schools of architecture, business, education, law, and medicine, are all centrally concerned with the process of design" [Simon, 1996; first published in 1969]. However, in this century natural sciences almost drove out the design from professional school curricula in all professions, including business, with exceptions for management science, computer science, and chemical engineering -- an activity that peaked two or three decades after the Second World War [Simon, 1996].

Simon sets out a prescription for schools of business (in which most IS departments are housed) that has motivated this page to a considerable degree: ". . . The professional schools will reassume their . . . responsibilities just to the degree that they can discover a science of design, a body of intellectually tough, analytic, partly formalizable, partly empirical teachable doctrine about the design process . . . "

To bring the design activity into focus at an intellectual level, Simon [1996] makes a clear distinction between "natural science" and "science of the artificial" (also known as design science):

A natural science is a body of knowledge about some class of things -- objects or phenomenon -- in the world (nature or society) that describes and explains how they behave and interact with each other. A science of the artificial, on the other hand, is a body of knowledge about artificial (man made) objects and phenomena designed to meet certain desired goals.

Simon further frames sciences of the artificial in terms of an inner environment, an outer environment, and the interface between the two that meets certain desired goals. The outer environment is the total set of external forces and effects that act on the artifact. The inner environment is the set of components that make up the artifact and their relationships – the organization – of the artifact. The behavior of the artifact is constrained by both its organization and its outer environment. The bringing-to-be of an artifact, components and their organization, which interfaces in a desired manner with its outer environment, is the design activity. The artifact is “structurally coupled” to its environment and many of the concepts of structural coupling that Varela (1988) and Maturana (1987) have developed for biological entities are applicable to designed artifacts.

In a perspective analogous to considering design as the crafting of an interface between inner and outer environment, design can be thought of as a mapping from function space - a functional requirement constituting a point in this multidimensional space - to attribute space, where an artifact satisfying the mapping constitutes a point in that space (Takeda, et al, 1990). Design then, is knowledge in the form of techniques and methods for performing this mapping – the know-how for implementing an artifact that satisfies a set of functional requirements.

          Design Research vs Design Science Research

Design science research in IS is a rapidly evolving field. In the relatively short period from the time this web page was first initiated (late 2004) till now (9/2011) even the most commonly accepted name for the field has changed - from 'design research' (DR) to 'design science research' (DSR). As the DSR literature gained breadth and depth researchers came to understand the term 'design research' had a long prior history as the study of design itself and designers - their methods, cognition and education. DR is a much broader area than DSR spanning all design fields, and more importantly, does not have the defining feature of DSR: learning through building - artifact construction. IS Design Science researchers thus (in about 2005/6 as a scan of the literature will show) widely began to add the distinguishing word 'science' to the field designation. The distinction frequently expressed is that DR is research into or about design whereas DSR is research using design as a research method or technique.

DSR when defined as learning through building is not unique to IS. The fields of education, health care, computer science and engineering also make extensive use of DSR. DSR in education - where curriculula and learning programs are designed and empirically evaluated and in health care - where programs of treatment are designed and empirically evaluated - share the DSRIS concern with rigorous evaluation and especially the codification of design knowledge in design theories to a greater degree than do the technical disciplines of computer science and engineering (Kuechler and Vaishnavi, 2011). More information on the history of DSR, especially in North America is available at the link immediately below.

        A Short History of Design Science Research in Information Systems

�   Can Design Be Research

The question this page intends to answer in the affirmative is: can design (i.e. artifact construction) ever be considered an appropriate technique for conducting research into Information Systems? We will pursue the specific question in the next section. For the remainder of this section we discuss the question in the abstract – can design be research? – using as exemplars communities other than IS where the question of whether or not design is a valid research technique has for many years been a resounding Yes!



Owen (1997) discusses the relation of design to research with reference to a conceptual map of disciplines (Figure 1) with two axes: Symbolic/Real and Analytic/Synthetic. The horizontal axis of the map position disciplines according to their defining activities: disciplines on the left side of the map are more concerned with exploration and discovery. Disciplines on the right side of the map are characterized more by invention and making. The map’s vertical division, the symbolic/real axis, characterizes the nature of the subjects of interest to the disciplines – the nature of the phenomena that concerns the research community. Both axes are continua and no discipline is exclusively concerned with synthesis to the exclusion of analytic activities. Likewise, no activity is exclusively concerned with the real to the exclusion of the symbolic although the strong contrast along this axis between the physical science of chemistry (real) and the abstract discipline of mathematics (symbolic) is strongly and accurately indicated in the diagram.


The disciplines that lie predominantly on the synthetic side of the map are either design disciplines or the design components of multi-paradigmatic disciplines. Design disciplines have a long history of building their knowledge base through making – the construction of artifacts and the evaluation of artifact performance following construction. Architecture is a strongly construction-oriented discipline with a history extending over thousands of years. The architectural knowledge base consists of a pool of structural designs that effectively encourage the wide variety of human activities and has been accumulated largely through the post-hoc observation of successful constructions (Alexander, 1964). Aeronautical engineering provides a more recent example. From the Montigolfer balloon through WWI, the aeronautical engineering knowledge base was built almost exclusively by analyzing the results of intuitively guided designs – experimentation at essentially full scale.��



Owen (1997) further presents a general model for generating and accumulating knowledge (above) that is helpful in understanding design disciplines and the design science research process: “Knowledge is generated and accumulated through action. Doing something and judging the results is the general model . . . the process is shown as a cycle in which knowledge is used to create works, and works are evaluated to build knowledge." While knowledge building through construction is sometimes considered to lack rigor, the process is not unstructured. The channels in the diagram of the general model are the “systems of conventions and rules under which the discipline operates." They embody the measures and values that have been empirically developed as “ways of knowing” as the discipline has matured. They may borrow from or emulate aspects of other disciplines channels, but, in the end, they are special to the discipline and are products of its evolution."

Takeda, et al. (1990) have analyzed the reasoning that occurs in the course of a general design cycle illustrated in Figure 3. This diagram can be interpreted as an elaboration of the Knowledge Using Process arrow in Figure 2. In following the flow of creative effort through this diagram the types of new knowledge that arise from design activities and the reason that this knowledge is most readily found during a design effort will become apparent.


* An operational principle can be defined as “any technique or frame of reference about a class of artifacts or its
characteristics that facilitates creation, manipulation and modification of artifactual forms” (Dasgupta, 1996; Purao, 2002)


In this model all design begins with Awareness of a problem. Design science research is sometimes called “Improvement Research” and this designation emphasizes the problem- solving/performance-improving nature of the activity. Suggestions for a problem solution are abductively drawn from the existing knowledge/theory base for the problem area (Pierce, 1931). An attempt at implementing an artifact according to the suggested solution is performed next. This stage is shown as Development in the diagram. Partially or fully successful implementations are then Evaluated (according to the functional specification implicit or explicit in the suggestion). Development, Evaluation and further Suggestion are frequently iteratively performed in the course of the research (design) effort. The basis of the iteration, the flow from partial completion of the cycle back to Awareness of the Problem, is indicated by the Circumscription arrow. Conclusion indicates termination of a specific design project.


New knowledge production is indicated in Figure 3 by the arrows labeled Circumscription and Operation and Goal Knowledge.� The Circumscription process is especially important to understanding design science research because it generates understanding that could only be gained from the specific act of construction. Circumscription is a formal logical method (McCarthy, 1980) that assumes that every fragment of knowledge is valid only in certain situations. Further, the applicability of knowledge can only be determined through the detection and analysis of contradictions – in common language, the design science researcher learns or discovers when things don’t work “according to theory." This happens many times not due to a misunderstanding of the theory, but due to the necessarily incomplete nature of ANY knowledge base. The design process, when interrupted and forced back to Awareness of Problem in this way, contributes valuable constraint knowledge to the understanding of the always-incomplete-theories that abductively motivated the original design.��

References on Design and Design Science Research

  The Outputs of Design Science Research


Even within design science research communities there is lack of consensus as to the precise objective – and therefore the desired outputs - of design science research. In this page we present a broad perspective that explicates the types and levels of knowledge that can be derived from design science research while reserving judgment on whether a narrower goal of design science research should be held within any specific research community.


March and Smith (1995) in a widely cited paper contrasting design science research with natural science research, propose four general outputs for design science research: constructs, models, methods, and instantiations. Constructs are the conceptual vocabulary of a problem/solution domain. Constructs arise during the conceptualization of the problem and are refined throughout the design cycle. Since a working design (artifact) consists of a large number of entities and their relationships, the construct set for a design science research experiment may be larger than the equivalent set for a descriptive (empirical) experiment.�

A model  is “a set of propositions or statements expressing relationships among constructs.” March and Smith identify models with problem and solution statements. They are proposals for how things are or should be. Models differ from natural science theories primarily in intent: natural science has a traditional focus on truth whereas design science research focuses more on (situated) utility. Thus a model is presented in terms of what it does and a theory described in terms of construct relationships. However a theory can always be extrapolated to what can be done with the implicit knowledge and a set of entities and proposed relationships can always be expressed as a theoretical statement of how or why the output occurs.


A method is a set of steps (an algorithm or guideline) used to perform a task. "Methods are goal directed plans for manipulating constructs so that the solution statement model is realized." Implicit in a design science research method then is the problem and solution statement expressed in the construct vocabulary. In contrast to natural science research, a method may well be the object of the research program in design science research. Since the axiology of design science research stresses problem solving, a more effective way of accomplishing an end result – even or sometimes especially a familiar or previously achieved end result – is valued.

The final output from a design science research effort in March and Smith’s explication is an instantiation which “operationalizes constructs, models and methods.”� It is the realization of the artifact in an environment. Emphasizing the proactive nature of design science research, they point out that an instantiation sometimes precedes a complete articulation of the conceptual vocabulary and the models (or theories) that it embodies. We emphasize this further by referring to the aeronautical engineering example given earlier in this page: aircraft flew decades before a full understanding of how such flight was accomplished. And, it is unlikely the understanding would ever have occurred in the absence of the working artifacts.

Rossi and Sein (2003) and Purao (2002) in an ongoing collaborative effort to promote design science research have set forth their own list of design science research outputs. All but one of these can be mapped directly to March and Smith’s list. Their fifth output, better theories, is highly significant and merits inclusion in our general list of design science research outputs. Design science research can contribute to better theories (or theory building) in at least two distinct ways, both of which may be interpreted as analogous to experimental scientific investigation in the natural science sense. First, since the methodological construction of an artifact is an object of theorizing for many communities (e.g. how to build more maintainable software), the construction phase of a design science research effort can be an experimental proof of method or an experimental exploration of method or both.

Second, the artifact can expose relationships between its elements. It is tautological to say that an artifact functions as it does because the relationships between its elements enable certain behaviors and constrain others. However if the relationships between artifact (or system) elements are less than fully understood and if the relationship is made more visible than previously during either the construction or evaluation phase of the artifact, then the understanding of the elements has been increased, potentially falsifying or elaborating on previously theorized relationships. (Theoretical relationships enter the design effort during the abductive reasoning phase of Figure 3). For some types of research, artifact construction is highly valued precisely for its contribution to theory. Human-Computer Interface (HCI) researchers Carroll and Kellogg (1989) state that “. . . HCI artifacts themselves are perhaps the most effective medium for theory development in HCI.” Walls, Widmeyer and El Sawy (1992) elaborate the theory building potential of design and construction in the specific context of IS. Table 1 summarizes the outputs that can be obtained from a design science research effort.

 

Output

Description

1

Constructs

The conceptual vocabulary of a domain

2

Models

A set of propositions or statements expressing relationships between constructs

3

Methods

A set of steps used to perform a task – how-to knowledge

4

Instantiations

The operationalization of constructs, models and methods.

5

Better theories

Artifact construction as analogous to experimental natural science, coupled with reflection and abstraction.
             

           Table 1. The Outputs of Design Science Research



A different perspective on the output of design science research is developed in Purao (2002) following Gregg, et al. (2001). In Figure 4 the multiple outputs of design science research are classified by level of abstraction.�



Explicitly the upper level of Figure 4 and implicitly the middle level, knowledge about operational principles, are theories about the emergent properties of the inner environment of the artifact (Simon, 1996). However, in any complex artifact, at either level of abstraction, multiple principles may be invoked simultaneously to explain aspects of the artifacts behavior. In this sense, the behavior of the artifact in any single design science research project is overdetermined (Carroll and Kellogg, 1989). This inevitable aspect of design science research has consequences discussed further in the section on Philosophical Grounding of Design Science Research.


  An example of community determined outputs


Precisely what is obtained from a design science research effort is determined by (1) the phase of research on which reflection and analysis focuses (from Figure 3) and (2) the level of abstraction to which the reflection and analysis generalize (from Figure 4). These factors in turn are strongly influenced by the community performing the research.


To illustrate the different outputs that are commonly seen as the desired result for design science research, consider the same artifact development as carried out by different IS research sub-communities: database, software engineering, Human-Computer interface (HCI), decision sciences, and IS Cognitive Researchers (IS Cognitive Research Exchange -- IS CORE): the construction of a data visualization interface for complex queries against large relational databases. For all of the communities, the research is motivated by a common problem awareness: that a better interface can be developed that will allow users to more quickly and effectively obtain answers to questions about the performance of their business operations.

The theoretical impetus for the prospective improvement would vary between research communities. For the software engineering or database communities the motivation could be new knowledge of faster access techniques or visual rendering techniques.� For the decision sciences community and the HCI and cognitive research communities the impetus could be new research in reference disciplines on visual impacts on cognition and/or on decision-making. The resulting artifact would be quite similar for all communities, as would the construction mechanics – the computer languages used in development, the deployment platforms, etc. However the stages of development on which observation and reflection centered and the measures used to evaluate the resultant artifact (cf. Figure 3) would be considerably different for each community.� Table 2 lists the communities that might construct a data visualization artifact, the primary perspective with which they would view the artifact and the different knowledge that would emerge from the research effort as a result of the differing perspectives.

Community

Perspective

Knowledge Derived

HCI; IS CORE; Decision science

Artifact as experimental apparatus

What database visualization interfaces reveal about the cognition of complex data relationships

Database; Decision science Software engineering

Artifact as focused design principle exploration

Principles for the construction of data visualization interfaces

Database; Software engineering

Artifact as improved instance of tool.

A better data visualization interface for relational, business oriented databases.
           Table 2. Design Science Research Perspectives and Outputs by Community


Some explications of design science research in IS have stated that the primary focus is always on the finished artifact and how well it works rather than its component interactions i.e. why it works (Hevner, et al, 2004). Other writers and our example present a broader view. The apparent contradiction may simply be in how wide the net of IS Research is cast and the selection of sub-communities it is considered to contain.

         Theory Development in Design Science Research

Another example of the rapid evolution of DSRIS is the recent attention directed to theory - both design theory per Walls, et al. (1992, 2004) and traditional explanatory/predictive theory - as a potential output of a DSRIS project. One of the seminal DSRIS papers, Nunamaker et al. (1991) alludes to theory and refinement of theory as an output from what they termed the "engineering model" of IS research. Shortly thereafter, Walls, et al. (1992) presented their conception of design theory (ISDT), a prescriptive encoding of design knowledge abstracted from a DSRIS project, and a number of widely cited IS papers subsequently made use of ISDT, for example: Kasper (1996) and Markus, Majchrzak and Gasser (2002). However, two influential papers subsequent to Walls, et al. 1992: March and Smith (1995) and Hevner, et al. (2004) did not explicitly mention theory and this has been interpreted by some in the field as suggesting that theory is not an output to be sought from DSRIS. Yet more recent papers including Gregor and Jones (2007), Kuechler and Vaishnavi (2008) and Arazy, Kumar and Shapira (2010) explicitly mention theory, both prescriptive and explanatory, as DSR project outputs and present methods for developing such theory during the course of design science research. A link directly to a subset of recent papers concerned with theory in DSR is directly below.

References on Theory and Theory Development in DSR

General References on Design Science Research



  The Philosophical Grounding of Design Science Research


Ontology is the study that describes the nature of reality: for example, what is real and what is not, what is fundamental and what is derivative?

Epistemology is the study that explores the nature of knowledge: for example, on what does knowledge depend and how can we be certain of what we know?

Axiology is the study of values: what values does an individual or group hold and why?

The definitions of these terms are worth reviewing because although assumptions about reality, knowledge and value underlie any intellectual endeavor, they are implicit most of the time for most people, including researchers. Indeed, as historians and philosophers of science have noted, in “tightly” paradigmatic communities, people may conduct research for an entire career without considering the philosophical implications of their passively received areas of interest and research methods (Kuhn, 1996; first published in 1962). It is typically only in multi-paradigmatic or pre-paradigmatic communities - such as IS - that researchers are forced to consider the most fundamental bases of the socially constructed realities (Berger and Luckman, 1966; Searle, 1995) in which they operate.

The contrasting ontological and epistemological assumptions implicit in natural science and social science research approaches have been authoritatively explicated in a number of widely cited works (Bunge, 1984; Guba and Lincoln, 1994). Gregg et al. (2001) add the meta-level assumptions of design science research (which they term the Socio-technologist / developmentalist approach) to earlier work contrasting positivist and interpretive approaches to research. We have drawn from Gregg, et al. in compiling Table 3 which summarizes the philosophical assumptions of those three “ways of knowing” and have added several insights from our combined 40+ years of design science research experience. Our first addition is the stress on iterative circumscription (cf. Figure 3) and how this essential part of the design science research methodology iteratively determines (or reveals) the reality and the knowledge that emerge from the research effort. The second addition to Table 3 is the row labeled Axiology – the study of values. We believe it is the shared valuing of what researchers hope to find in the pursuit of their efforts that binds them into a community. Certainly the self and community valuation of their efforts and findings is a highly significant motivator for any researcher, and we were surprised to find how little stress this topic has received in the literature, especially given the significant differences in what each community values.

The metaphysical assumptions of design science research are unique.� First, neither the ontology, epistemology, nor axiology of the paradigm is derivable from any other. Second, ontological and epistemological viewpoints shift in design science research as the project runs through circumscription cycles depicted in Figure 3. This iteration is similar to but more radical than the hermeneutic processes used in some interpretive research.�

Design science research by definition changes the state-of-the-world through the introduction of novel artifacts. Thus, design science researchers are comfortable with alternative world-states. The obvious contrast is with positivist ontology where a single, given composite socio-technical system is the typical unit of analysis; even the problem statement is subject to revision as a design science research effort proceeds. However, the multiple world-states of the design science researcher are not the same as the multiple realities of the interpretive researcher: many if not most design science researchers believe in a single, stable underlying physical reality that constrains the multiplicity of world-states. The abductive phase of design science research (Figure 3) in which physical laws are tentatively composed into a configuration that will produce an artifact with the intended problem solving functionality virtually demands a natural-science-like belief in a single, fixed grounding reality.

Epistemologically, the design science researcher knows that a piece of information is factual and knows further what that information means through the process of construction/circumscription. An artifact is constructed. Its behavior is the result of interactions between components. Descriptions of the interactions are information and to the degree the artifact behaves predictably the information is true. Its meaning is precisely the functionality it enables in the composite system (artifact and user). What it means is what it does. The design science researcher is thus a pragmatist (Pierce, 1931). There is also a flavor of instrumentalism (Hendry, 2004) in design science research.� The dependence on a predictably functioning artifact (instrument) gives design science research an epistemology that resembles that of natural-science research more closely than that of either positivist or interpretive research.

Axiologically, the design science researcher values creative manipulation and control of the environment in addition to (if not over) more traditional research values such as the pursuit of truth or understanding. Certainly the design science researcher must have a far higher tolerance for ambiguity than is generally acceptable in the positivist research stance. As many authors have pointed out, the end result of a design science research effort may be very poorly understood and still be considered a success by the community (Hevner, et al, 2004). A practical or functional addition to an area body of knowledge, codified and transmitted to the community where it can provide the basis for further exploration, may be all that is required of a successful project. Indeed, it is precisely in the exploration of “wicked problems” for which conflicting or sparse theoretical bases exist that design science research excels (March and Smith, 1995; Carroll and Kellogg, 1989).�

Finally, the philosophical perspective of the design science researcher changes as progress is iteratively made through the phases of Figure 3. In some sense it is as if the design science researcher creates a reality through constructive intervention, then reflectively becomes a positivist observer, recording the behavior of the system and comparing it to the predictions (theory) set out during the abductive phase. The observations are interpreted, become the basis for new theorizing and a new abductive, interventionist cycle begins. In this sense design science research is very similar to the action research methodology of the interpretive paradigm, however, the time frame of design science research construction is enormously foreshortened relative to the social group interactions typical of action research.


Bunge (1984) implies that design science research is most effective when its practitioners shift between pragmatic and critical realist perspectives, guided by a pragmatic assessment of progress in the design cycle.� Purao (2002) presents a very rich elaboration on the perspective shifts that accompany any iterative design cycle. His analysis is grounded in semiotics and describes in detail how “the design science researcher arrives at an interpretation (understanding) of the phenomenon and the design of the artifact simultaneously.”

References on Philosophical Grounding of Design Science Research

  Design Science Research Methodology (by Example)


In this section of the page the general method underlying design science research in its multiplicity of as-practiced variants is described, followed by a discussion of the method as used in a published example of IS design science research.

The astute reader will recognize Figure 5, the general methodology for all design science research, as a variant on Figure 3, reasoning in the design cycle. This is a logical and inevitable result of the fact that in design science research knowing (Figure 3) is making (Figure 5). To better focus on the process as a research method, a column labeled Outputs has been substituted for the Logical Formalism column.

Note: there are many excellent descriptions (and diagrams) of the process of design science research in IS, cf. Hevner, et al., 2004; Purao, 2002;
Gregg, et al., 2001;. March and Smith, 1995; Nunamaker, et al., 1991. We chose this diagram because it emphasizes the knowledge generation
inherent in the method and because it originated in an analysis of the processes inherent in any design effort.

With reference to Figure 5 a typical design science research effort proceeds as follows.

Awareness of Problem: An awareness of an interesting problem may come from multiple sources: new developments in industry or in a reference discipline. Reading in an allied discipline may also provide the opportunity for application of new findings to the researcher’s field. The output of this phase is a Proposal, formal or informal, for a new research effort.

Suggestion: The Suggestion phase follows immediately behind the proposal and is intimately connected with it as the dotted line around Proposal and Tentative Design (the output of the Suggestion phase) indicates. Indeed, in any formal proposal for design science research such as one to be made to the NSF or an industry sponsor, a Tentative Design and likely the performance of a prototype based on that design would be an integral part of the Proposal. Moreover, if after consideration of an interesting problem a Tentative Design does not present itself to the researcher, the idea (Proposal) will be set aside. Suggestion is an essentially creative step wherein new functionality is envisioned based on a novel configuration of either existing or new and existing elements. The step has been criticized as introducing non-repeatability into the design science research method; human creativity is still a poorly understood cognitive process. However the step has necessary analogues in all research methods; for example, in positivist research creativity is inherent in the leap from curiosity about an organizational phenomena to the development of appropriate constructs that operationalize the phenomena and an appropriate research design for their measurement.

Development: The Tentative Design is implemented in this phase. The techniques for implementation will of course vary depending on the artifact to be constructed. An algorithm may require construction of a formal proof. An expert system embodying novel assumptions about human cognition in an area of interest will require software development, probably using a high-level package or tool. The implementation itself can be very pedestrian and need not involve novelty beyond the state-of-practice for the given artifact; the novelty is primarily in the design, not the construction of the artifact.

Evaluation: Once constructed, the artifact is evaluated according to criteria that are always implicit and frequently made explicit in the Proposal (Awareness of Problem phase). Deviations from expectations, both quantitative and qualitative are carefully noted and must be tentatively explained. That is, the evaluation phase contains an analytic sub-phase in which hypotheses are made about the behavior of the artifact. This phase exposes an epistemic fluidity that is in stark contrast to a strict interpretation of the positivist stance. At an equivalent point in positivist research, analysis either confirms or contradicts a hypothesis. Essentially, save for some consideration of future work as may be indicated by experimental results, the research effort is over. For the design science researcher, by contrast, things are just getting interesting!� Rarely, in design science research, are initial hypothesis concerning behavior completely borne out. Instead, the evaluation phase results and additional information gained in the construction and running of the artifact are brought together and fed back to another round of Suggestion (cf. the circumscription arrows of Figures 3 and 5). The explanatory hypotheses, which are quite broad, are rarely discarded, but rather are modified to be in accord with the new observations. This suggests a new design, frequently preceded by new library research in directions suggested by deviations from theoretical performance. (design science researchers seem to share Allen Newell’s conception (from Cognitive Science) of theories as complex, robust nomological networks. This conception has been observed by philosophers of science in many communities (Lakatos, 1978), and working from it Newell suggests that theories are not like clay pigeons, to be blasted to bits with the Popperian shotgun of falsification. Rather they should be treated like doctoral students. One corrects them when they err, and is hopeful they can emend their flawed behavior and go on to be ever more useful and productive (Newell, 1990).)

Conclusion: This phase is the finale of a specific research effort. Typically, it is the result of satisficing, that is, though there are still deviations in the behavior of the artifact from the (multiply) revised hypothetical predictions, the results are adjudged “good enough.” Not only are the results of the effort consolidated and “written up” at this phase, but the knowledge gained in the effort is frequently categorized as either “firm” - facts that have been learned and can be repeatably applied or behavior that can be repeatably invoked - or as “loose ends” – anomalous behavior that defies explanation and may well serve as the subject of further research.


  An Example of IS Design Science Research

The example we have chosen to add detail and concreteness to the discussion of design science research philosophy and method in Information Systems is one from the joint experience of the design science research page authors. We make only two claims for this research: (1) it is a reasonable example as it comfortably encompasses all the points of the preceding discussion (2) since it is our research we are privy to and able to present a multitude of details that are rarely written up and available in journal publications. We describe the research, from conception to the first publication to be drawn from it, in phases corresponding to those in diagrams 3 and 5.


Smart Objects: A Design Science Research Project

Awareness of Problem

In the mid-1980’s one of the senior project participants, Vijay, began actively seeking to extend his research from designing efficient data and file structures (a primarily computer science topic) to software engineering (an area with a significant IS component). In the course of a discussion with one of his colleagues at Georgia State University (GSU) he became aware of a situation that showed research promise:� development of a computerized decision support system for nuclear reactors. Three Mile Island had brought national awareness to the problems associated with safe operation of a nuclear power plant, rule based decision support systems were a current area of general IS interest, and the director of the research reactor at Georgia Tech was interested in developing a system to support its operations.

A doctoral student (Gary) was brought into the project to begin a preliminary support system development in the rule-based language Prolog. Within a few weeks it became apparent that a system to support the several thousand procedures found in a typical commercial power plant would be nearly impossible to develop in Prolog and if developed would be literally impossible to maintain. The higher-level expert system development packages available at the time (and currently) were more capable but still obviously inadequate. The difficulty of constructing and maintaining large expert systems was widely known at the time; however, the Prolog pilot project gave the research group significant insights they would not otherwise have had into the root causes of the problem: continuously changing requirements and the complexity inherent in several thousand rule-based interlocking procedures. Out of detailed analysis of the failed pilot system emerged the first awareness of the problem on which the research would focus: how to construct and continuously maintain a support system for the operation of a complex, hierarchical, procedure driven environment.�

Suggestion

There are many approaches to the problems of software system complexity and the research group discussed them over a period of months. Some of the alternatives that were discarded were: development of a new software development methodology specifically focused on operation support systems, automation of the maintenance function, and development of a high-level programming environment. New insights into the problem continued to emerge even as (and precisely because) potential solutions to the problem were considered. One key insight was that the system complexity resided primarily in control of the system, that is, although the individual procedures could be modeled straightforwardly, the procedure which should take precedence (control) over the others and where the results of that procedure should be routed depended in a highly complex fashion on past and present states of multiple procedures. Essential to the development of the system was the effective modeling of this complex control structure.

By this point Gary had decided to adopt the problem as his dissertation topic and under Vijay’s direction began extensive research into various mechanisms for modeling (describing in a precise, formal way) control. As the realization grew that they were in effect seeking to describe the semantics of the system, his reading began to focus especially on some of the techniques to emerge from the area of semantic modeling.

During the alternating cycles of discussion, reading and individual cogitation that characterize many design science research efforts, several software engineering concepts were brought together with a final key insight to yield the ultimately successful direction for the development. During one discussion Vijay realized that the control information for the system was knowledge, identical in form to the domain knowledge in the procedures and could be modeled with rules, in the same way. However, since the execution of the individual procedures was independent of the control knowledge, the two types of rules could execute in different cycles, partitioning and greatly reducing the complexity of the overall system. Finally, the then relatively new concept of object orientation seemed the ideal approach to partitioning the total system knowledge into individual procedures. And if each object were further partitioned into a domain knowledge and a control knowledge component, and the rules were stated in a high level English like syntax that was both executable and readable by domain experts leading to the concept of smart objects . . .

Awareness of Problem Redoux

As noted in the general discussion of the design science research method, any of its phases may be spontaneously revisited from any of the other phases. Especially in the early stages of a project, this results in a conceptual fluidity that can be disconcerting to practitioners of less dynamic paradigms. Though it is difficult in retrospect to pinpoint exactly where in the process the change occurred, by the inception of the development phase the problem statement had changed to a sub-goal implicit in the original problem statement: how to effectively model operations support systems for complex, hierarchical, procedure driven environments. [This sort of “drilling down” into the problem or re-scoping the research at a more basic level occurs frequently in all research, but is effectively part of the method in design science research.]

Development

Although development of a design science research artifact can be straightforward, that was not the case for smart objects. The construction was completely conceptual and involved the “discovery” through multiple thought and paper trials of the details of the novel entity that had been conceptualized at a high level in the Suggestion phase, the “smart object.”

For example: what (exactly) would the syntax be for the two types of rules, domain and control? How (exactly) should the two rule evaluation cycles for each type of knowledge interleave? Should the two types of knowledge be permitted to interact? If so, how? Should control rules have the ability to “write” or “rescind” domain rules, a la Lisp? Or vice versa?

In a conceptual development such as this, the suggestion and construction phases blur because a successful design decision is an output product. The final deliverable (from this initial development) was a conceptual model consisting of: (1) a set of meta-level rules for implementing domain knowledge and control knowledge separately, but within a single structure, the “smart object” and (2) another set of meta-rules that described how the domain and control knowledge, once “modeled” as smart objects, would be interpreted (a virtual machine for executing the smart objects.)

Evaluation

In a sense evaluation takes place continuously in a design process (research or otherwise) since a large number of “micro-evaluations” take place at every design detail decision. Each decision is followed by a “thought experiment” in which that part of the design is mentally exercised by the designer. However for the remainder of this section we will describe the “formal” evaluation that occurred after the design had stabilized.

In order to test the conceptual design, various operating environments were modeled and “hand-stepped” through the execution rules to determine that logically correct system behavior occurred at appropriate times in the simulation. The simulation that appeared in Gary’s dissertation, the first publication to result from the research, was a grocery bagging “robot.” This example had been popularized in a best-selling artificial intelligence textbook of the time and had the advantage of being a familiar logic test bed to many external evaluators of the artifact. Exponents of other IS research paradigms may find the evaluation criteria simplistic, and wonder why, for example, modeling of the nuclear power plant operating environment was not the obvious choice. The answer is: resources; the modeling and hand testing of even the grocery-bagging example occupied several man-months. During the evaluation minor redesign of the artifact (the smart object conceptual model) occurred on several occasions, a common occurrence in design science research. By the end of the evaluation phase the smart object model had successfully completed simulation of numerous bagging exercises that included complex control situations and was adjudged a success by the design team.

Conclusion

The finale for the first research effort involving Smart Objects was the codification of the problem development, design basis in prior work, the design itself, and the results of the evaluation effort in Gary’s dissertation (Buchanan, 1991) The successful defense of the dissertation at GSU required careful consideration and judgment of the artifact and its performance by a committee made up primarily of other design science researchers. The core concepts were considered to have substantial merit, and Gary and Vijay produced several conference papers based on smart objects.

  Epilogue

After Gary’s graduation Vijay and Gary collaborated on a paper based on the research project and submitted it to IEEE Transactions on Data and Knowledge Engineering (TDKE). The paper was returned for substantial revisions. At this point Gary’s interest in the project waned, however a recently admitted GSU CIS doctoral student (Bill) found the concepts interesting enough to enter into the research group and continue the development effort. After four years, four conference papers on smart objects and related topics and three major revisions the TKDE paper was finally published as “A Data/Knowledge Paradigm for the Modeling and Design of Operations Support Systems.” (Vaishnavi, Buchanan and Kuechler, 1997) By the time of acceptance, smart objects had been through several additional design science research cycles, each focusing on the refinement of a different aspect of the original design, or a critical support function for its use-in-practice such as the methodology developed for partitioning workflow information into smart objects.

  References on Design Science Research Methodology


  Design Science Research vs. Design

A significant and valid question posed frequently to design science researchers is: How is your research different from a design effort; what makes your work research and not simply state-of-practice design?

We propose that design science research is distinguished from design by the production of interesting (to a community) new knowledge. In a typical industry design effort a new product (artifact) is produced, but in most cases, the more successful the project is considered to be, the less is learned. That is, it is generally desirable to produce a new product using state-of-practice application of state-of-practice techniques and readily available components. In fact, most product design efforts in industry are preceded by many meetings designed to “engineer the risk out of” the design effort. The risks that are identified in such meetings are the “we don’t know how to do this yet” areas that are precisely the targets of design science research efforts. This is in no way meant to diminish the creativity that is essential to any design effort. We merely wish to point out that design is readily distinguished from design science research (within its community of interest) by the intellectual risk, the number of unknowns in the proposed design which when successfully surmounted provide the new information that makes the effort research and assures its value.

 

      Current Issues in Design [Science] Research

 

References on Understanding Design Science Research in the Context of Information Systems Research

  Citations on Design Science Research


  Resources for Design Science Researchers

Communities of Practice

  • Design and Science: The site "discusses the relationship between Design and Science."
  • Design Research Society (DRS) is a multi-disciplinary international learned society founded in 1967.� Members of DRS are drawn from diverse backgrounds ranging from fine art to engineering to computing.� The aims of DRS include advancing the theory and practice of design and understanding design research and its relationship with its education and practice.� DRS is involved with such activities as organizing and sponsoring conferences, sponsoring e-mail discussion groups and a monthly e-mailed newsletter.
  • Design-Based Research Collective: The design research paradigm appears to have been pursued as well in education research (especially educational software and systems design). Although many of the early core readings are the same (Schon, Simon, Alexander) the later traditions overlap a lot less. In education it has traditionally been called design experiments, although this term is falling out of favor. The Design-Based Research Collective has been helping define the theory and practice of this research paradigm.
  • AIS Systems Analysis and Design Special Interest Group (SIGSAND) "AIS SIGSAND provides a forum for AIS members: To discuss, debate, collaborate, develop, and promote issues pertaining to the history, reference disciplines, theories, ontologies, methodologies and techniques, principles, new developments, practice, evaluation, quality control, management and pedagogy of systems requirements, analysis, design, and implementation tasks and technologies In the business and organizational contexts. "
  • AIS Special Interest Group on Philosophy and Epistemology in IS (SIGPhilosophy) "Currently we can observe a growing methodological debate in IS research. This debate appears to focus on epistemic issues, especially research methods and techniques without relating to the broader issues of the philosophy of science, epistemology and theory of knowledge. To overcome too narrow focus, it will be necessary to link the debates in IS research to questions about the very nature of research and science and their societal role in general."
  • Association for Information Systems (AIS) is "the premier global organization for academics specializing in Information Systems."
  • Association for Computing Machinery (ACM), founded in 1947, is "a major force in advancing the skills of information technology professionals and students worldwide."
  • IEEE Computer Society (IEEECS) is the "world's leading organization of computer professionals with nearly 100,000 members.
  • IEEE Systems, Man, and Cybernetics Society (IEEESMC)
  • American Society for Information Science and Technology
  • INFORMS (Institute for Operations Research and the Management Sciences)
  • The Information Institute "is an academe-industry consortium founded to further understanding of intricate relationships between information science and technology." 
  • Informing Science Institute

Design And Design Science Research Centers / Labs

  • Center for Design Research Center for Design Research at Stanford University, established in 1984, focuses on engineering design, design tool development, and design process research and promotes collaboration between industry and academia.
  • IIT Institute of Design The Institute of Design (ID) at Illinois Institute of Technology teaches systemic, human-centered design and focuses on the development of design methods and theories and the practical demonstration of their utility.
  • MIT Media Lab The Media Laboratory at MIT, established in 1985, emphasizes interdisciplinary research that combines disciplines such as cognition, electronic music, graphic design, holography with research in computation and human-machine interfaces.
  • The Palo Alto Research Center The Palo Alto Research Center (PARC), a subsidiary of Xerox Corporation, conducts interdisciplinary design research in physical, computational, and social sciences.
  • Carnegie Mellon Software Engineering Institute (SEI) SEI is a federally funded research and development center sponsored by the US Department of Defense devoted to making measured improvements in software engineering capabilities.
  • ISEing - Information Systems Evaluation and Integration Group is a center located at Brunel University, West London (U.K.) that is devoted to the conducting of basic and applied research on designing organizations and systems for actual operational conditions.
        Other Research Centers / Labs

  Journals

  • Journal of Design Research an electronic journal established in 2001, is a general design research journal that is devoted to integrated studies of social sciences and design disciplines emphasizing human aspects as a central issue of design
���� The following journals tend to be receptive to design research in information systems:
        ACM Transactions on:       AIS Journals and Transactions:

      IEEE Magazines

        IEEE Transactions On

       Other Journals

�     Conferences

  Help Build This Page

You are invited to contribute links to design science research material. Additionally, we are soliciting critiques of the page and short abstracts of items in the references pages (maximum 100 words). Please contact the Section Editors by email Vijay Vaishnavi or Bill Kuechler to see how you can help

  Citation Information

The complete citation for this work is as follows:

Vaishnavi, V. and Kuechler, W. (2004). “Design Science Research in Information Systems” January 20, 2004, last updated November 11, 2012. URL: http://www.desrist.org/design-research-in-information-systems/

This page was last updated on November 11, 2012