Design Research in Information Systems
Design Science Research in Information Systems
Section Editors: Vijay Vaishnavi and Bill Kuechler
[Overview of Design Science Research] [Can
Design be Research] [Outputs of
Design Science Research]
[ An Example of Community
Determined Outputs] [The Philosophical
Grounding of Design Science Research]
[ Design Science Research Methodology] [A
Design Science Research Example ]
[Design Science Research vs. Design] [Citations
on Design Science Research] [Resources
For Design Science Research]
Welcome
Welcome to the page on Design Science Research in Information Systems (IS). The intent
of the page is to provide design science researchers in IS as well as others interested
in design science research with useful information on understanding, conducting, evaluating,
and publishing design science research.
Introduction
This page is dedicated to design science research in Information Systems (IS). Design science research is yet another "lens" or set of analytical techniques and
perspectives (complementing the Positivist and Interpretive perspectives) for
performing research in IS. Design science research involves the design of novel or innovative artifacts and the analysis of the use
and/or performance of such artifacts to improve and understand the behavior of aspects of Information Systems. Such artifacts
include - but certainly are not limited to - algorithms (e.g. for information
retrieval), human/computer interfaces and system design methodologies or languages.
design science researchers can be found in many disciplines and fields, notably Engineering
and Computer Science, using a variety of approaches, methods and techniques.
In Information Systems, following a number of years of a general shift in IS
research away from technological to managerial and organizational issues, an
increasing number of observers are calling for a return to an exploration of
the "IT" that underlies all IS research (Orlikowski
and Iacono, 2001).
The page is organized as follows. We begin with a general overview of design science research, provide its philosophical and epistemological underpinnings, and contrast
design science research in IS with traditional positivist and qualitative research in
IS. This is followed by a section on design science research methods that includes an
extended discussion of a published example of design science research in IS. Through
the example we hope to make concrete all phases of the design science research method:
artifact design, construction, analysis and evaluation. This is then followed
by a number of resource sub-sections that relate to design science research in general
as well as to design science research in IS: citation lists, links to resources on the
Internet for design science researchers, calls for papers and links to conferences,
workshops, journals and communities of practice for IS design science research.
The goal is to provide the IS community with useful information on design science research both in and outside of IS. The page contains numerous citations permitting
the interested reader to easily access original material on and examples of
this unique and dynamic IS research paradigm.
If you wish to cite this work, the complete citation
information is included below. Please send suggestions for improvement to
the Section Editors at: vvaishna@gsu.edu
or kuechler@unr.edu
Overview of Design Science Research
Research
Drawing heavily from Kuhn
(1996; first published in 1962) and Lakatos
(1978), research can be very generally defined as an activity that contributes
to the understanding of a phenomenon. In the case of design science research,
all or part of the phenomenon may be created as opposed to naturally
occurring. The phenomenon is typically a set of behaviors of some
entity(ies) that is found interesting by the researcher or by a group
- a research community. Understanding in most western research communities
is knowledge that allows prediction of the behavior of some aspect of
the phenomenon. The set of activities a research community considers appropriate
to the production of understanding (knowledge) are its research methods or techniques.
Historically, some research communities have been observed to have nearly universal
agreement on the phenomenon of interest and the research methods for investigating
it; in this page we term these paradigmatic communities. Other research
communities are bound into a nominal community by overlap in sets of phenomena
of interest and/or overlap in methods of investigation. We term these pre-paradigmatic
or multi-paradigmatic research communities. As of the writing of this
page Information Systems is an excellent example of a multi-paradigmatic
community.
Design
Design means "to invent and bring into being" [Webster's Dictionary
and Thesaurus, 1992]. Thus, design deals with creating something new that does
not exist in nature. The design of artifacts is an activity that has been carried
out for centuries. This activity is also what distinguishes the professions
from the sciences. "Schools of architecture, business, education,
law, and medicine, are all centrally concerned with the process of design"
[Simon, 1996;
first published in 1969]. However, in this century natural sciences almost drove
out the design from professional school curricula in all professions, including
business, with exceptions for management science, computer science, and chemical
engineering -- an activity that peaked two or three decades after the Second
World War [Simon,
1996].
Simon sets
out a prescription for schools of business (in which most IS departments are
housed) that has motivated this page to a considerable degree: ". . . The
professional schools will reassume their . . . responsibilities just to the
degree that they can discover a science of design, a body of intellectually
tough, analytic, partly formalizable, partly empirical teachable doctrine about
the design process . . . "
To bring the design activity into focus at an intellectual level, Simon
[1996] makes a clear distinction between "natural science" and "science
of the artificial" (also known as design science):
A natural science is a body of knowledge about some class of things
-- objects or phenomenon -- in the world (nature or society) that describes
and explains how they behave and interact with each other. A science of the
artificial, on the other hand, is a body of knowledge about artificial (man
made) objects and phenomena designed to meet certain desired goals.
Simon further
frames sciences of the artificial in terms of an inner environment, an
outer environment, and the interface between the two that meets
certain desired goals. The outer environment is the total set of external forces
and effects that act on the artifact. The inner environment is the set of components
that make up the artifact and their relationships - the organization -
of the artifact. The behavior of the artifact is constrained by both its organization
and its outer environment. The bringing-to-be of an artifact, components and
their organization, which interfaces in a desired manner with its outer environment,
is the design activity. The artifact is “structurally coupled” to
its environment and many of the concepts of structural coupling that Varela
(1988) and Maturana
(1987) have developed for biological entities are applicable to designed artifacts.
In a perspective analogous to considering design as the crafting of an interface
between inner and outer environment, design can be thought of as a mapping from
function space - a functional requirement constituting a point in this multidimensional
space - to attribute space, where an artifact satisfying the mapping constitutes
a point in that space (Takeda,
et al, 1990). Design then, is knowledge in the form of techniques and methods
for performing this mapping - the know-how for implementing an artifact
that satisfies a set of functional requirements.
Design Research vs Design Science Research
Design science research in IS is a rapidly evolving field. In the relatively short period from the time this web page was first initiated (late 2004) till now (9/2011) even the most commonly accepted name for the field has changed - from 'design research' (DR) to 'design science research' (DSR). As the DSR literature gained breadth and depth researchers came to understand the term 'design research' had a long prior history as the study of design itself and designers - their methods, cognition and education. DR is a much broader area than DSR spanning all design fields, and more importantly, does not have the defining feature of DSR: learning through building - artifact construction. IS Design Science researchers thus (in about 2005/6 as a scan of the literature will show) widely began to add the distinguishing word 'science' to the field designation. The distinction frequently expressed is that DR is research into or about design whereas DSR is research using design as a research method or technique.
DSR when defined as learning through building is not unique to IS. The fields of education, health care, computer science and engineering also make extensive use of DSR. DSR in education - where curriculula and learning programs are designed and empirically evaluated and in health care - where programs of treatment are designed and empirically evaluated - share the DSRIS concern with rigorous evaluation and especially the codification of design knowledge in design theories to a greater degree than do the technical disciplines of computer science and engineering (Kuechler and Vaishnavi, 2011). More information on the history of DSR, especially in North America is available at the link immediately below.
A
Short History of Design Science Research in Information Systems
�
Can Design Be Research
The question this page intends to answer in the affirmative is: can design
(i.e. artifact construction) ever be considered an appropriate technique for
conducting research into Information Systems? We will pursue the specific question
in the next section. For the remainder of this section we discuss the question
in the abstract - can design be research? - using as exemplars communities
other than IS where the question of whether or not design is a valid research
technique has for many years been a resounding Yes!
Owen (1997)
discusses the relation of design to research with reference to a conceptual
map of disciplines (Figure 1) with two axes: Symbolic/Real and Analytic/Synthetic.
The horizontal axis of the map position disciplines according to their defining
activities: disciplines on the left side of the map are more concerned with
exploration and discovery. Disciplines on the right side of the map are
characterized more by invention and making. The map’s vertical
division, the symbolic/real axis, characterizes the nature of the subjects of
interest to the disciplines - the nature of the phenomena that concerns
the research community. Both axes are continua and no discipline is exclusively
concerned with synthesis to the exclusion of analytic activities. Likewise,
no activity is exclusively concerned with the real to the exclusion of the symbolic
although the strong contrast along this axis between the physical science of
chemistry (real) and the abstract discipline of mathematics (symbolic) is strongly
and accurately indicated in the diagram.
The disciplines that lie predominantly on the synthetic side of the map are
either design disciplines or the design components of multi-paradigmatic disciplines.
Design disciplines have a long history of building their knowledge base through
making - the construction of artifacts and the evaluation of artifact
performance following construction. Architecture is a strongly construction-oriented
discipline with a history extending over thousands of years. The architectural
knowledge base consists of a pool of structural designs that effectively encourage
the wide variety of human activities and has been accumulated largely through
the post-hoc observation of successful constructions (Alexander,
1964). Aeronautical engineering provides a more recent example. From the Montigolfer
balloon through WWI, the aeronautical engineering knowledge base was built almost
exclusively by analyzing the results of intuitively guided designs - experimentation
at essentially full scale.��
Owen (1997)
further presents a general model for generating and accumulating knowledge (above)
that is helpful in understanding design disciplines and the design science research
process: “Knowledge is generated and accumulated through action. Doing
something and judging the results is the general model . . . the process is
shown as a cycle in which knowledge is used to create works, and works are evaluated
to build knowledge." While knowledge building through construction is sometimes
considered to lack rigor, the process is not unstructured. The channels
in the diagram of the general model are the “systems of conventions and
rules under which the discipline operates." They embody the measures and
values that have been empirically developed as “ways of knowing”
as the discipline has matured. They may borrow from or emulate aspects of other
disciplines channels, but, in the end, they are special to the discipline and
are products of its evolution."
Takeda,
et al. (1990) have analyzed the reasoning that occurs in the course of a general
design cycle illustrated in Figure 3. This diagram can be interpreted as an
elaboration of the Knowledge Using Process arrow in Figure 2. In following
the flow of creative effort through this diagram the types of new knowledge
that arise from design activities and the reason that this knowledge is most
readily found during a design effort will become apparent.
*
An operational principle can be defined as “any technique or frame of
reference about a class of artifacts or its
characteristics that facilitates creation, manipulation and modification of
artifactual forms” (Dasgupta,
1996; Purao,
2002)
In this model all design begins with Awareness of a problem. Design science
research is sometimes called “Improvement Research” and this designation
emphasizes the problem- solving/performance-improving nature of the activity.
Suggestions for a problem solution are abductively drawn from the existing
knowledge/theory base for the problem area (Pierce,
1931). An attempt at implementing an artifact according to the suggested solution
is performed next. This stage is shown as Development in the diagram.
Partially or fully successful implementations are then Evaluated (according
to the functional specification implicit or explicit in the suggestion). Development,
Evaluation and further Suggestion are frequently iteratively performed
in the course of the research (design) effort. The basis of the iteration, the
flow from partial completion of the cycle back to Awareness of the Problem,
is indicated by the Circumscription arrow. Conclusion indicates
termination of a specific design project.
New knowledge production is indicated in Figure 3 by the arrows labeled Circumscription
and Operation and Goal Knowledge.� The Circumscription process
is especially important to understanding design science research because it generates
understanding that could only be gained from the specific act of construction.
Circumscription is a formal logical method (McCarthy,
1980) that assumes that every fragment of knowledge is valid only in certain
situations. Further, the applicability of knowledge can only be determined through
the detection and analysis of contradictions - in common language, the
design science researcher learns or discovers when things don’t
work “according to theory." This happens many times not due to a
misunderstanding of the theory, but due to the necessarily incomplete nature
of ANY knowledge base. The design process, when interrupted and forced back
to Awareness of Problem in this way, contributes valuable constraint
knowledge to the understanding of the always-incomplete-theories that abductively
motivated the original design.��
References on Design and Design Science Research
The Outputs of Design Science Research
Even within design science research communities there is lack of consensus as to the
precise objective - and therefore the desired outputs - of design science research.
In this page we present a broad perspective that explicates the types and levels
of knowledge that can be derived from design science research while reserving
judgment on whether a narrower goal of design science research should be held within
any specific research community.
March and
Smith (1995) in a widely cited paper contrasting design science research with natural
science research, propose four general outputs for design science research: constructs,
models, methods, and instantiations. Constructs
are the conceptual vocabulary of a problem/solution domain. Constructs arise
during the conceptualization of the problem and are refined throughout the design
cycle. Since a working design (artifact) consists of a large number of entities
and their relationships, the construct set for a design science research experiment
may be larger than the equivalent set for a descriptive (empirical) experiment.�
A model is “a set of propositions or statements expressing
relationships among constructs.” March
and Smith identify models with problem and solution statements. They
are proposals for how things are or should be. Models differ from natural science theories
primarily in intent: natural science has a traditional focus on truth whereas
design science research focuses more on (situated) utility. Thus a model is presented
in terms of what it does and a theory described in terms of construct relationships.
However a theory can always be extrapolated to what can be done with the implicit
knowledge and a set of entities and proposed relationships can always be expressed
as a theoretical statement of how or why the output occurs.
A method is a set of steps (an algorithm or guideline) used to perform
a task. "Methods are goal directed plans for manipulating constructs so
that the solution statement model is realized." Implicit in a design science research
method then is the problem and solution statement expressed in the construct
vocabulary. In contrast to natural science research, a method may well be the
object of the research program in design science research. Since the axiology of design science research stresses problem solving, a more effective way of accomplishing an
end result - even or sometimes especially a familiar or previously achieved
end result - is valued.
The final output from a design science research effort in March
and Smith’s explication is an instantiation which “operationalizes
constructs, models and methods.”� It is the realization of the artifact
in an environment. Emphasizing the proactive nature of design science research, they
point out that an instantiation sometimes precedes a complete articulation of
the conceptual vocabulary and the models (or theories) that it embodies. We
emphasize this further by referring to the aeronautical engineering example
given earlier in this page: aircraft flew decades before a full understanding
of how such flight was accomplished. And, it is unlikely the understanding would
ever have occurred in the absence of the working artifacts.
Rossi and
Sein (2003) and Purao
(2002) in an ongoing collaborative effort to promote design science research have set
forth their own list of design science research outputs. All but one of these can be
mapped directly to March and Smith’s list. Their fifth output, better
theories, is highly significant and merits inclusion in our general list
of design science research outputs. Design science research can contribute to better theories
(or theory building) in at least two distinct ways, both of which may be interpreted
as analogous to experimental scientific investigation in the natural science
sense. First, since the methodological construction of an artifact is an object
of theorizing for many communities (e.g. how to build more maintainable software),
the construction phase of a design science research effort can be an experimental proof
of method or an experimental exploration of method or both.
Second, the artifact can expose relationships between its elements. It is
tautological to say that an artifact functions as it does because the relationships
between its elements enable certain behaviors and constrain others. However
if the relationships between artifact (or system) elements are less than fully
understood and if the relationship is made more visible than previously during
either the construction or evaluation phase of the artifact, then the understanding
of the elements has been increased, potentially falsifying or elaborating on
previously theorized relationships. (Theoretical relationships enter the design
effort during the abductive reasoning phase of Figure 3). For some types of
research, artifact construction is highly valued precisely for its contribution
to theory. Human-Computer Interface (HCI) researchers Carroll
and Kellogg (1989) state that “. . . HCI artifacts themselves are
perhaps the most effective medium for theory development in HCI.” Walls,
Widmeyer and El Sawy (1992) elaborate the theory building potential of design
and construction in the specific context of IS. Table 1 summarizes the outputs
that can be obtained from a design science research effort.
| |
Output |
Description |
| 1 |
Constructs |
The
conceptual vocabulary of a domain |
| 2 |
Models |
A
set of propositions or statements expressing relationships between constructs |
| 3 |
Methods |
A
set of steps used to perform a task - how-to knowledge |
| 4 |
Instantiations |
The
operationalization of constructs, models and methods. |
| 5 |
Better
theories |
Artifact
construction as analogous to experimental natural science, coupled with reflection and abstraction. |
| |
Table
1. The Outputs of Design Science Research |
A different perspective on the output of design science research is developed in Purao
(2002) following Gregg,
et al. (2001). In Figure 4 the multiple outputs of design science research are classified
by level of abstraction.�
Explicitly the upper level of Figure 4 and implicitly the middle level, knowledge
about operational principles, are theories about the emergent properties of
the inner environment of the artifact (Simon,
1996). However, in any complex artifact, at either level of abstraction, multiple
principles may be invoked simultaneously to explain aspects of the artifacts
behavior. In this sense, the behavior of the artifact in any single design science research
project is overdetermined (Carroll
and Kellogg, 1989). This inevitable aspect of design science research has consequences
discussed further in the section on Philosophical Grounding of Design Science Research.
An example of community determined outputs
Precisely what is obtained from a design science research effort is determined by
(1) the phase of research on which reflection and analysis focuses (from Figure
3) and (2) the level of abstraction to which the reflection and analysis generalize
(from Figure 4). These factors in turn are strongly influenced by the community
performing the research.
To illustrate the different outputs that are commonly seen as the desired
result for design science research, consider the same artifact development as
carried out by different IS research sub-communities: database, software engineering,
Human-Computer interface (HCI), decision sciences, and IS Cognitive Researchers
(IS Cognitive Research Exchange -- IS CORE): the construction of a data visualization
interface for complex queries against large relational databases. For all of
the communities, the research is motivated by a common problem awareness:
that a better interface can be developed that will allow users to more quickly
and effectively obtain answers to questions about the performance of their business
operations.
The theoretical impetus for the prospective improvement would vary between
research communities. For the software engineering or database communities the
motivation could be new knowledge of faster access techniques or visual rendering
techniques.� For the decision sciences community and the HCI and cognitive research
communities the impetus could be new research in reference disciplines on visual
impacts on cognition and/or on decision-making. The resulting artifact would
be quite similar for all communities, as would the construction mechanics -
the computer languages used in development, the deployment platforms, etc. However
the stages of development on which observation and reflection centered and the
measures used to evaluate the resultant artifact (cf. Figure 3) would be considerably
different for each community.� Table 2 lists the communities that might construct
a data visualization artifact, the primary perspective with which they would
view the artifact and the different knowledge that would emerge from the research
effort as a result of the differing perspectives.
| Community
|
Perspective |
Knowledge
Derived |
| HCI;
IS CORE; Decision science |
Artifact
as experimental apparatus |
What
database visualization interfaces reveal about the cognition of complex
data relationships |
| Database;
Decision science Software engineering |
Artifact
as focused design principle exploration |
Principles
for the construction of data visualization interfaces |
| Database;
Software engineering |
Artifact
as improved instance of tool. |
A
better data visualization interface for relational, business oriented
databases. |
| |
Table 2. Design Science Research Perspectives and Outputs by Community |
Some explications of design science research in IS have stated that the primary focus
is always on the finished artifact and how well it works rather than its component
interactions i.e. why it works (Hevner,
et al, 2004). Other writers and our example present a broader view. The apparent
contradiction may simply be in how wide the net of IS Research is cast
and the selection of sub-communities it is considered to contain.
Theory Development in Design Science Research
Another example of the rapid evolution of DSRIS is the recent attention directed to theory - both design theory per Walls, et al. (1992, 2004) and traditional explanatory/predictive theory - as a potential output of a DSRIS project. One of the seminal DSRIS papers, Nunamaker et al. (1991) alludes to theory and refinement of theory as an output from what they termed the "engineering model" of IS research. Shortly thereafter, Walls, et al. (1992) presented their conception of design theory (ISDT), a prescriptive encoding of design knowledge abstracted from a DSRIS project, and a number of widely cited IS papers subsequently made use of ISDT, for example: Kasper (1996) and Markus, Majchrzak and Gasser (2002). However, two influential papers subsequent to Walls, et al. 1992: March and Smith (1995) and Hevner, et al. (2004) did not explicitly mention theory and this has been interpreted by some in the field as suggesting that theory is not an output to be sought from DSRIS. Yet more recent papers including Gregor and Jones (2007), Kuechler and Vaishnavi (2008) and Arazy, Kumar and Shapira (2010) explicitly mention theory, both prescriptive and explanatory, as DSR project outputs and present methods for developing such theory during the course of design science research. A link directly to a subset of recent papers concerned with theory in DSR is directly below.
References on Theory and Theory Development in DSR
General References on Design Science Research
The Philosophical Grounding of Design Science Research
Ontology is the study that describes the nature of reality: for example,
what is real and what is not, what is fundamental and what is derivative?
Epistemology is the study that explores the nature of knowledge: for
example, on what does knowledge depend and how can we be certain of what we
know?
Axiology is the study of values: what values does an individual or
group hold and why?
The definitions of these terms are worth reviewing because although assumptions
about reality, knowledge and value underlie any intellectual endeavor, they
are implicit most of the time for most people, including researchers.
Indeed, as historians and philosophers of science have noted, in “tightly”
paradigmatic communities, people may conduct research for an entire career without
considering the philosophical implications of their passively received areas
of interest and research methods (Kuhn,
1996; first published in 1962). It is typically only in multi-paradigmatic or
pre-paradigmatic communities - such as IS - that researchers are forced to consider
the most fundamental bases of the socially constructed realities (Berger
and Luckman, 1966; Searle,
1995) in which they operate.
The contrasting ontological and epistemological assumptions implicit in natural
science and social science research approaches have been authoritatively explicated
in a number of widely cited works (Bunge,
1984; Guba
and Lincoln, 1994). Gregg
et al. (2001) add the meta-level assumptions of design science research (which they
term the Socio-technologist / developmentalist approach) to earlier work contrasting
positivist and interpretive approaches to research. We have drawn from Gregg,
et al. in compiling Table 3 which summarizes the philosophical assumptions of
those three “ways of knowing” and have added several insights from
our combined 40+ years of design science research experience. Our first addition is
the stress on iterative circumscription (cf. Figure 3) and how this essential
part of the design science research methodology iteratively determines (or reveals)
the reality and the knowledge that emerge from the research effort. The second
addition to Table 3 is the row labeled Axiology - the study of values.
We believe it is the shared valuing of what researchers hope to find in the
pursuit of their efforts that binds them into a community. Certainly the self
and community valuation of their efforts and findings is a highly significant
motivator for any researcher, and we were surprised to find how little stress
this topic has received in the literature, especially given the significant
differences in what each community values.
The metaphysical assumptions of design science research are unique.� First, neither
the ontology, epistemology, nor axiology of the paradigm is derivable from any
other. Second, ontological and epistemological viewpoints shift in design science research
as the project runs through circumscription cycles depicted in Figure 3. This
iteration is similar to but more radical than the hermeneutic processes used
in some interpretive research.�
Design science research by definition changes the state-of-the-world through the introduction
of novel artifacts. Thus, design science researchers are comfortable with alternative
world-states. The obvious contrast is with positivist ontology where a single,
given composite socio-technical system is the typical unit of analysis; even
the problem statement is subject to revision as a design science research effort proceeds.
However, the multiple world-states of the design science researcher are not the same
as the multiple realities of the interpretive researcher: many if not most design science researchers believe in a single, stable underlying physical reality that constrains
the multiplicity of world-states. The abductive phase of design science research (Figure
3) in which physical laws are tentatively composed into a configuration that
will produce an artifact with the intended problem solving functionality virtually
demands a natural-science-like belief in a single, fixed grounding reality.
Epistemologically, the design science researcher knows that a piece of information
is factual and knows further what that information means through the process
of construction/circumscription. An artifact is constructed. Its behavior is
the result of interactions between components. Descriptions of the interactions
are information and to the degree the artifact behaves predictably the information
is true. Its meaning is precisely the functionality it enables in the composite
system (artifact and user). What it means is what it does. The design science researcher
is thus a pragmatist (Pierce,
1931). There is also a flavor of instrumentalism (Hendry,
2004) in design science research.� The dependence on a predictably functioning artifact
(instrument) gives design science research an epistemology that resembles that of natural-science
research more closely than that of either positivist or interpretive research.
Axiologically, the design science researcher values creative manipulation and control
of the environment in addition to (if not over) more traditional research values
such as the pursuit of truth or understanding. Certainly the design science researcher
must have a far higher tolerance for ambiguity than is generally acceptable
in the positivist research stance. As many authors have pointed out, the end
result of a design science research effort may be very poorly understood and still be
considered a success by the community (Hevner,
et al, 2004). A practical or functional addition to an area body of knowledge,
codified and transmitted to the community where it can provide the basis for
further exploration, may be all that is required of a successful project. Indeed,
it is precisely in the exploration of “wicked problems” for which
conflicting or sparse theoretical bases exist that design science research excels (March
and Smith, 1995; Carroll
and Kellogg, 1989).�
Finally, the philosophical perspective of the design science researcher changes as
progress is iteratively made through the phases of Figure 3. In some sense it
is as if the design science researcher creates a reality through constructive intervention,
then reflectively becomes a positivist observer, recording the behavior of the
system and comparing it to the predictions (theory) set out during the abductive
phase. The observations are interpreted, become the basis for new theorizing
and a new abductive, interventionist cycle begins. In this sense design science research
is very similar to the action research methodology of the interpretive paradigm,
however, the time frame of design science research construction is enormously foreshortened
relative to the social group interactions typical of action research.
Bunge (1984)
implies that design science research is most effective when its practitioners shift
between pragmatic and critical realist perspectives, guided by a pragmatic assessment
of progress in the design cycle.� Purao
(2002) presents a very rich elaboration on the perspective shifts that accompany
any iterative design cycle. His analysis is grounded in semiotics and describes
in detail how “the design science researcher arrives at an interpretation (understanding)
of the phenomenon and the design of the artifact simultaneously.”
References
on Philosophical Grounding of Design Science Research
Design Science Research Methodology (by Example)
In this section of the page the general method underlying design science research
in its multiplicity of as-practiced variants is described, followed by a discussion
of the method as used in a published example of IS design science research.
The astute reader will recognize Figure 5, the general methodology for all
design science research, as a variant on Figure 3, reasoning in the design cycle. This
is a logical and inevitable result of the fact that in design science research knowing
(Figure 3) is making (Figure 5). To better focus on the process as a
research method, a column labeled Outputs has been substituted for the
Logical Formalism column.
Note: there are many excellent descriptions
(and diagrams) of the process of design science research in IS, cf. Hevner,
et al., 2004; Purao, 2002;
Gregg, et al., 2001;. March
and Smith, 1995; Nunamaker,
et al., 1991. We chose this diagram because it emphasizes the knowledge generation
inherent in the method and because it originated in an analysis of the processes
inherent in any design effort.
With reference to Figure 5 a typical design science research effort proceeds as follows.
Awareness of Problem: An awareness of an interesting problem may come
from multiple sources: new developments in industry or in a reference discipline.
Reading in an allied discipline may also provide the opportunity for application
of new findings to the researcher’s field. The output of this phase is
a Proposal, formal or informal, for a new research effort.
Suggestion: The Suggestion phase follows immediately behind the proposal
and is intimately connected with it as the dotted line around Proposal and Tentative
Design (the output of the Suggestion phase) indicates. Indeed, in any formal
proposal for design science research such as one to be made to the NSF or an industry
sponsor, a Tentative Design and likely the performance of a prototype based
on that design would be an integral part of the Proposal. Moreover, if after
consideration of an interesting problem a Tentative Design does not present
itself to the researcher, the idea (Proposal) will be set aside. Suggestion
is an essentially creative step wherein new functionality is envisioned based
on a novel configuration of either existing or new and existing elements. The
step has been criticized as introducing non-repeatability into the design science research
method; human creativity is still a poorly understood cognitive process. However
the step has necessary analogues in all research methods; for example, in positivist
research creativity is inherent in the leap from curiosity about an organizational
phenomena to the development of appropriate constructs that operationalize the
phenomena and an appropriate research design for their measurement.
Development: The Tentative Design is implemented in this phase. The
techniques for implementation will of course vary depending on the artifact
to be constructed. An algorithm may require construction of a formal proof.
An expert system embodying novel assumptions about human cognition in an area
of interest will require software development, probably using a high-level package
or tool. The implementation itself can be very pedestrian and need not involve
novelty beyond the state-of-practice for the given artifact; the novelty is
primarily in the design, not the construction of the artifact.
Evaluation: Once constructed, the artifact is evaluated according to
criteria that are always implicit and frequently made explicit in the Proposal
(Awareness of Problem phase). Deviations from expectations, both quantitative
and qualitative are carefully noted and must be tentatively explained.
That is, the evaluation phase contains an analytic sub-phase in which hypotheses
are made about the behavior of the artifact. This phase exposes an epistemic
fluidity that is in stark contrast to a strict interpretation of the positivist
stance. At an equivalent point in positivist research, analysis either confirms
or contradicts a hypothesis. Essentially, save for some consideration of future
work as may be indicated by experimental results, the research effort is over.
For the design science researcher, by contrast, things are just getting interesting!�
Rarely, in design science research, are initial hypothesis concerning behavior completely
borne out. Instead, the evaluation phase results and additional information
gained in the construction and running of the artifact are brought together
and fed back to another round of Suggestion (cf. the circumscription arrows
of Figures 3 and 5). The explanatory hypotheses, which are quite broad, are
rarely discarded, but rather are modified to be in accord with the new observations.
This suggests a new design, frequently preceded by new library research in directions
suggested by deviations from theoretical performance. (design science researchers seem
to share Allen Newell’s conception
(from Cognitive Science) of theories as complex, robust nomological networks.
This conception has been observed by philosophers of science in many communities
(Lakatos, 1978), and working from
it Newell suggests that theories are not like clay pigeons, to be blasted to
bits with the Popperian shotgun of falsification. Rather they should be treated
like doctoral students. One corrects them when they err, and is hopeful they
can emend their flawed behavior and go on to be ever more useful and productive
(Newell, 1990).)
Conclusion: This phase is the finale of a specific research effort.
Typically, it is the result of satisficing, that is, though there are still
deviations in the behavior of the artifact from the (multiply) revised hypothetical
predictions, the results are adjudged “good enough.” Not only are
the results of the effort consolidated and “written up” at this
phase, but the knowledge gained in the effort is frequently categorized as either
“firm” - facts that have been learned and can be repeatably applied
or behavior that can be repeatably invoked - or as “loose ends”
- anomalous behavior that defies explanation and may well serve as the
subject of further research.
An Example of IS Design Science Research
The example we have chosen to add detail and concreteness to the discussion
of design science research philosophy and method in Information Systems is one from
the joint experience of the design science research page authors. We make only two claims
for this research: (1) it is a reasonable example as it comfortably encompasses
all the points of the preceding discussion (2) since it is our research we are
privy to and able to present a multitude of details that are rarely written
up and available in journal publications. We describe the research, from conception
to the first publication to be drawn from it, in phases corresponding to those
in diagrams 3 and 5.
Smart Objects: A Design Science Research Project
Awareness of Problem
In the mid-1980’s one of the senior project participants, Vijay, began
actively seeking to extend his research from designing efficient data and file
structures (a primarily computer science topic) to software engineering (an
area with a significant IS component). In the course of a discussion with one
of his colleagues at Georgia State University (GSU) he became aware of a situation
that showed research promise:� development of a computerized decision support
system for nuclear reactors. Three Mile Island had brought national awareness
to the problems associated with safe operation of a nuclear power plant, rule
based decision support systems were a current area of general IS interest, and
the director of the research reactor at Georgia Tech was interested in developing
a system to support its operations.
A doctoral student (Gary) was brought into the project to begin a preliminary
support system development in the rule-based language Prolog. Within a few weeks
it became apparent that a system to support the several thousand procedures
found in a typical commercial power plant would be nearly impossible to develop
in Prolog and if developed would be literally impossible to maintain. The higher-level
expert system development packages available at the time (and currently) were
more capable but still obviously inadequate. The difficulty of constructing
and maintaining large expert systems was widely known at the time; however, the
Prolog pilot project gave the research group significant insights they would
not otherwise have had into the root causes of the problem: continuously changing
requirements and the complexity inherent in several thousand rule-based interlocking
procedures. Out of detailed analysis of the failed pilot system emerged the
first awareness of the problem on which the research would focus: how
to construct and continuously maintain a support system for the operation of
a complex, hierarchical, procedure driven environment.�
Suggestion
There are many approaches to the problems of software system complexity and
the research group discussed them over a period of months. Some of the alternatives
that were discarded were: development of a new software development methodology
specifically focused on operation support systems, automation of the maintenance
function, and development of a high-level programming environment. New insights
into the problem continued to emerge even as (and precisely because) potential
solutions to the problem were considered. One key insight was that the system
complexity resided primarily in control of the system, that is, although the
individual procedures could be modeled straightforwardly, the procedure which
should take precedence (control) over the others and where the results of that
procedure should be routed depended in a highly complex fashion on past and
present states of multiple procedures. Essential to the development of the system
was the effective modeling of this complex control structure.
By this point Gary had decided to adopt the problem as his dissertation topic
and under Vijay’s direction began extensive research into various mechanisms
for modeling (describing in a precise, formal way) control. As the realization
grew that they were in effect seeking to describe the semantics of the
system, his reading began to focus especially on some of the techniques to emerge
from the area of semantic modeling.
During the alternating cycles of discussion, reading and individual cogitation
that characterize many design science research efforts, several software engineering
concepts were brought together with a final key insight to yield the ultimately
successful direction for the development. During one discussion Vijay realized
that the control information for the system was knowledge, identical in form
to the domain knowledge in the procedures and could be modeled with rules, in
the same way. However, since the execution of the individual procedures was
independent of the control knowledge, the two types of rules could execute in
different cycles, partitioning and greatly reducing the complexity of the overall
system. Finally, the then relatively new concept of object orientation seemed
the ideal approach to partitioning the total system knowledge into individual
procedures. And if each object were further partitioned into a domain knowledge
and a control knowledge component, and the rules were stated in a high level
English like syntax that was both executable and readable by domain experts
leading to the concept of smart objects . . .
Awareness of Problem Redoux
As noted in the general discussion of the design science research method, any of its
phases may be spontaneously revisited from any of the other phases. Especially
in the early stages of a project, this results in a conceptual fluidity that
can be disconcerting to practitioners of less dynamic paradigms. Though it is
difficult in retrospect to pinpoint exactly where in the process the change
occurred, by the inception of the development phase the problem statement had
changed to a sub-goal implicit in the original problem statement: how to
effectively model operations support systems for complex, hierarchical, procedure
driven environments. [This sort of “drilling down” into the
problem or re-scoping the research at a more basic level occurs frequently in
all research, but is effectively part of the method in design science research.]
Development
Although development of a design science research artifact can be straightforward,
that was not the case for smart objects. The construction was completely conceptual
and involved the “discovery” through multiple thought and paper
trials of the details of the novel entity that had been conceptualized at a
high level in the Suggestion phase, the “smart object.”
For example: what (exactly) would the syntax be for the two types of rules,
domain and control? How (exactly) should the two rule evaluation cycles for
each type of knowledge interleave? Should the two types of knowledge be permitted
to interact? If so, how? Should control rules have the ability to “write”
or “rescind” domain rules, a la Lisp? Or vice versa?
In a conceptual development such as this, the suggestion and construction phases
blur because a successful design decision is an output product. The final
deliverable (from this initial development) was a conceptual model consisting
of: (1) a set of meta-level rules for implementing domain knowledge and control
knowledge separately, but within a single structure, the “smart object”
and (2) another set of meta-rules that described how the domain and control
knowledge, once “modeled” as smart objects, would be interpreted
(a virtual machine for executing the smart objects.)
Evaluation
In a sense evaluation takes place continuously in a design process (research
or otherwise) since a large number of “micro-evaluations” take place
at every design detail decision. Each decision is followed by a “thought
experiment” in which that part of the design is mentally exercised by
the designer. However for the remainder of this section we will describe the
“formal” evaluation that occurred after the design had stabilized.
In order to test the conceptual design, various operating environments were
modeled and “hand-stepped” through the execution rules to determine
that logically correct system behavior occurred at appropriate times in the
simulation. The simulation that appeared in Gary’s dissertation, the first
publication to result from the research, was a grocery bagging “robot.”
This example had been popularized in a best-selling artificial intelligence
textbook of the time and had the advantage of being a familiar logic test bed
to many external evaluators of the artifact. Exponents of other IS research
paradigms may find the evaluation criteria simplistic, and wonder why, for example,
modeling of the nuclear power plant operating environment was not the obvious
choice. The answer is: resources; the modeling and hand testing of even the
grocery-bagging example occupied several man-months. During the evaluation minor
redesign of the artifact (the smart object conceptual model) occurred on several
occasions, a common occurrence in design science research. By the end of the evaluation
phase the smart object model had successfully completed simulation of numerous
bagging exercises that included complex control situations and was adjudged a success by the design team.
Conclusion
The finale for the first research effort involving Smart Objects was the codification
of the problem development, design basis in prior work, the design itself, and
the results of the evaluation effort in Gary’s dissertation (Buchanan,
1991) The successful defense of the dissertation at GSU required careful consideration
and judgment of the artifact and its performance by a committee made up primarily
of other design science researchers. The core concepts were considered to have substantial
merit, and Gary and Vijay produced several conference papers based on smart
objects.
Epilogue
After Gary’s graduation Vijay and Gary collaborated on a paper based
on the research project and submitted it to IEEE Transactions on Data and Knowledge
Engineering (TDKE). The paper was returned for substantial revisions. At this
point Gary’s interest in the project waned, however a recently admitted
GSU CIS doctoral student (Bill) found the concepts interesting enough to enter
into the research group and continue the development effort. After four years,
four conference papers on smart objects and related topics and three major revisions
the TKDE paper was finally published as “A Data/Knowledge Paradigm for
the Modeling and Design of Operations Support Systems.” (Vaishnavi,
Buchanan and Kuechler, 1997) By the time of acceptance, smart objects had
been through several additional design science research cycles, each focusing on the
refinement of a different aspect of the original design, or a critical support
function for its use-in-practice such as the methodology developed for partitioning
workflow information into smart objects.
References on Design Science Research Methodology
Design Science Research vs. Design
A significant and valid question posed frequently to design science researchers is:
How is your research different from a design effort; what makes your work research
and not simply state-of-practice design?
We propose that design science research is distinguished from design by the production
of interesting (to a community) new knowledge. In a typical industry
design effort a new product (artifact) is produced, but in most cases, the more
successful the project is considered to be, the less is learned. That is, it
is generally desirable to produce a new product using state-of-practice application
of state-of-practice techniques and readily available components. In fact, most
product design efforts in industry are preceded by many meetings designed to
“engineer the risk out of” the design effort. The risks that are
identified in such meetings are the “we don’t know how to do this
yet” areas that are precisely the targets of design science research efforts.
This is in no way meant to diminish the creativity that is essential to any
design effort. We merely wish to point out that design is readily distinguished
from design science research (within its community of interest) by the intellectual
risk, the number of unknowns in the proposed design which when successfully
surmounted provide the new information that makes the effort research and assures
its value.
Current
Issues in Design [Science] Research
References
on Understanding Design Science Research in the Context of Information Systems Research
Citations on Design Science Research
Resources for Design Science Researchers
Communities of Practice
- Design and Science: The site
"discusses the relationship between Design and Science."
- Design Research Society
(DRS) is a multi-disciplinary international learned society founded in 1967.�
Members of DRS are drawn from diverse backgrounds ranging from fine art to
engineering to computing.� The aims of DRS include advancing the theory and
practice of design and understanding design research and its relationship
with its education and practice.� DRS is involved with such activities as
organizing and sponsoring conferences, sponsoring e-mail discussion groups
and a monthly e-mailed newsletter.
- Design-Based Research Collective:
The design research paradigm appears to have been pursued as well in education
research (especially educational software and systems design). Although many
of the early core readings are the same (Schon, Simon, Alexander) the later
traditions overlap a lot less. In education it has traditionally been called
design experiments, although this term is falling out of favor. The
Design-Based Research Collective has been helping define the theory and practice
of this research paradigm.
- AIS Systems Analysis and Design
Special Interest Group (SIGSAND) "AIS SIGSAND provides a forum for
AIS members: To discuss, debate, collaborate, develop, and promote issues
pertaining to the history, reference disciplines, theories, ontologies, methodologies
and techniques, principles, new developments, practice, evaluation, quality
control, management and pedagogy of systems requirements, analysis, design,
and implementation tasks and technologies In the business and organizational
contexts. "
- AIS Special Interest
Group on Philosophy and Epistemology in IS (SIGPhilosophy) "Currently
we can observe a growing methodological debate in IS research. This debate
appears to focus on epistemic issues, especially research methods and techniques
without relating to the broader issues of the philosophy of science, epistemology
and theory of knowledge. To overcome too narrow focus, it will be necessary
to link the debates in IS research to questions about the very nature of research
and science and their societal role in general."
- Association for Information Systems (AIS)
is "the premier global organization for academics specializing in Information
Systems."
- Association for Computing Machinery (ACM),
founded in 1947, is "a major force in advancing the skills of information
technology professionals and students worldwide."
- IEEE Computer Society (IEEECS) is
the "world's leading organization of computer professionals with nearly
100,000 members.
- IEEE Systems, Man, and Cybernetics Society
(IEEESMC)
- American Society for Information Science
and Technology
- INFORMS (Institute for Operations
Research and the Management Sciences)
- The Information
Institute "is an academe-industry consortium
founded to further understanding of intricate relationships between information
science and technology."
- Informing
Science Institute
Design And Design Science Research Centers
/ Labs
- Center for Design Research Center
for Design Research at Stanford University, established in 1984, focuses on
engineering design, design tool development, and design process research and
promotes collaboration between industry and academia.
- IIT Institute of Design
The Institute of Design (ID) at Illinois Institute of Technology teaches systemic,
human-centered design and focuses on the development of design methods and
theories and the practical demonstration of their utility.
- MIT Media Lab The Media Laboratory
at MIT, established in 1985, emphasizes interdisciplinary research that combines
disciplines such as cognition, electronic music, graphic design, holography
with research in computation and human-machine interfaces.
- The Palo Alto Research Center The Palo
Alto Research Center (PARC), a subsidiary of Xerox Corporation, conducts interdisciplinary
design research in physical, computational, and social sciences.
- Carnegie Mellon Software Engineering Institute
(SEI) SEI is a federally funded research and development center sponsored
by the US Department of Defense devoted to making measured improvements in
software engineering capabilities.
- ISEing
- Information
Systems Evaluation and Integration Group is a center located at Brunel
University, West London (U.K.) that is devoted to the conducting of basic
and applied research on designing organizations and systems for actual operational
conditions.
Other
Research Centers / Labs
Journals
- Journal
of Design Research an electronic journal established in 2001, is a general
design research journal that is devoted to integrated studies of social sciences
and design disciplines emphasizing human aspects as a central issue of design
���� The following journals tend to be
receptive to design research in information systems:
ACM
Transactions on:
AIS
Journals and Transactions:
IEEE
Magazines
IEEE
Transactions On
Other
Journals
�
Conferences
Help Build This Page
You are invited to contribute links to design science research material.
Additionally, we are soliciting critiques of the page and short abstracts of
items in the references pages (maximum 100 words). Please contact the Section
Editors by email Vijay Vaishnavi or Bill
Kuechler to see how you can help
Citation Information
The complete citation for this work is as follows:
Vaishnavi, V. and Kuechler, W. (2004). “Design Science Research
in Information Systems” January 20, 2004, last updated November 11, 2012.
URL: https://desrist.org/design-research-in-information-systems/
This page was last updated on November 11, 2012