Information Systems design science research
-
Upload
raimo-halinen -
Category
Technology
-
view
104 -
download
0
description
Transcript of Information Systems design science research
ICT-DESIGN SCIENCE RESEACRHRaimo Hälinen (2012)
Lahti 11.10.2012
Dimensions of Innovation
The theory creation process
Source: Christensen and Sundahl (2001)
Paradigm
Theory
Classification
Observe, describe & measure phenomena
Dec
lara
tive
proc
ess
Pred
ict
Inductive process
Confirm
Theory is a statement of what causes why, what, and under what circumstances
Anomaly
An Activity Framework for DSR
Source: Venable (2006)
TechnologyInvention/DesignEnhancement creation of a method, product, system, practice or
techniques
Theory Building
Solution space and Problem theoriesUtility theories or
hypothesesTechnology Evaluation
Field studiesExperiments
Action researchSimulation
Problem DiagnosisProblem space understandingProblem causes
and consequences
Information Systems Design theory (ISDT) (Walls et al. (1992)1. Design theories must
deal with goals as contingencies
2. A design theory can never involve pure explanation or prediction
3. Desing theories are prescriptive
4. Design theories are composite theories with encompass kernel theories from natural, social science and mathematics.
Applying design theoryComponents Software development Managing risks
Tarkoitus ja alaPurpose and scope
Tavoitteena on kehittää vaiheittain rakentuvan tietosysteemin kehittämis- ja testauspolitiikka.
Sovellusten kehitystyöhön sisältyvien riskien hallinnan lähestymistavan kehittäminen.
Käytetyt ilmaisutConstructs
Vaiheittaisen kehittämisprosessin virheiden tunnistaminen ja testattavuus
Riskitekijöiden tunnistaminen, riskin aiheuttajien tunnistaminen ja riskin eliminoiminen
Muodon ja toiminnan periaatteetPrinsciples of form and function
Dynaamisten ohjeiden kehittäminen ja järjestelmäintegraation tunnistaminen
Riskien tunnistamisprosessin ja sen aktiviteettien kehittäminen
Artefaktin kehitysluonneArtefact mutability
Tiimioppisiseen liittyvien vaikutusten tunnistaminen kehitystyön erivaihessa.
Sovellusten ja tietosysteemien käytettävyys ja itse-ohjautuvuuden huomioon ottaminen
Testattavat väittämätTestable propositions
Sovelluksen tai tietosysteemien tuotosten (tulosten) arviointi ja simulointi perustuen suunnitteluteorian tosilauseisiin.
Suunniteltu riskien lähestymistapa on sovellettavissa yleisesti sovellusten kehittämistyöhön
Todistava tietämysJustifactory knowledge
Luonnon- ja sosiaalitieteiden antama tietämys, johon suunnittelutyö perustuu
Yleisestä riskienhallinnan johtamismalleista johdetut periaatteet.
ToteutusperiaatteetPrinciples of implementation
Sovelluksen tai tietosysteemin käyttöönottamisen periaatteet
Käyttöönottoon liittyvien riskien hallinta ja osaaminen
Konkreettinen toteutusExpository instantiation
Varsinaisen käyttöönottoprosessin vaiheistus ja ohjeistus
Riskitekijöiden tunnistaminen ja arviointi käyttöönottovaiheessa.
The General Methodology of DSR
Awareness of problem
Suggestion
Development
Evaluation
Conclusion
Proposal
Tentative desing
Artifact
Performance measures
Results
Knowledge Flows Process Steps Outputs
Circumscription
Operation and Goal knowledge
Source: Vaishnavi & Kuehler, (2004,2011)
Rigor CycleGrounding
Additions to KB
Relevance Cycle
RequirementsField testing
Design Science Research Cycle
Application Domain People Orgnizational Systems Technical Systems
Problems and Opportunities
Environment Design Science Research Knowledge Base
Build DesignArtifactsProcesses
Evaluate
FoundationsScientific Theoriesand Methods
ExperienceExpertise
Meta-ArtifactsDesign Products andDesign Processes
Design Cycle
Source: Hevner (2007)
Hevner: ” Design science research is essentially pragmatic in nature.” It emphasizes relevance.It makes clear contribution into the application environment.
Design science research cycle
Theory and models of ISIt-artefact
Outcomes(Reflection)
Hypotheses(Technological
rules)
Observations(Testing)
why, what, how, who,when
What might workfor whom and why
Context-mechanism-outcomes (CMO)
Source: Pawson and Tilley(1997) and Kazi (2003), Carlsson (2006)
Outputs Description
Constructs The conceptual vocabulary of a domain
Models A set of propositions or statement expressing relationships between constructs
Methods A set of steps to perform a task – how-to knowledge
Instantiation
The operational of constructs, models and methods.
Better theories
Artifact construction as analogous to experimental natural science, coupled with reflection and abstraction
Engaged Scholarship
Source: van de Ven (2006)
Engaged Scholarship (ES)...
Researchers
Consultants
Managers
Executives
Users
ES-Dia
mond
model b
y Van
de Ven
New theory
New practical solution
Design research guidelinesGuideline Descriptions
Design as an artifact
Tutkimuksen tavoitteena on sovelluksen, menetelmän tai tietosysteemin suunnittelu. Tarkoituksena ratkaista jokin tietosysteemiin tai sen käyttöön liittyvä ongelma.
Problem relevance Suunnittelu perustuu organisaation saamaan hyötyyn (taloudellinen ja toiminnallinen)
Desing evaluation Suunnittele, miten artifaktin hyöty ja tarkoitus voidaan arvioida suunnitteluprosessin aikana ja sen jälkeen.
Research contribution
Tutkimuksen tavoitteena on tuottaa suunittelutieteen käyttöön uutta tietoa: 1) uusi artifakti, 2) suunnittelutieteen perusteita, 3) arviointiin ja arviointimetodeihin liittyvää tietoa.
Research rigor Tutkimusote perustuu täsämälliseen metodiin, jota käytetään suunnittelussa ja /tai arvioinnissa. (Tutkimuksen toistettavuus).
Design as a search process
Suunnitteluprosessi on etsintää, missä pyritään löytämään parempia tapoja tuottaa artifakteja. (iteratiivinen prosessi)
Communication of research
Tutkimuksen tulokset julkaistaan ja välitetään sekä tutkijoille että käytännön toimijoille (johto, suunnittelijat, käyttäjät).
Source: Hevner et al. (2004)
Research Process ModelResearch Process
Focus Contribution
Problem identification
Relevant research problems
Formulation of research problem and questions
Research Proposal development
Context and scopeResearch stragegy
Answer the questions:why, what, how, who and when
Literature Review
Current state of theory and application
Analysis of focal theory, application and supporting technology
Conceptualization
Concept discovery and potential empirical generalization
Conceptual model, variables and relationships and constraints
Experimentation Design the approach and selecting methods and how to demonstrate
Confirmation of theoretical conjecture, propositions and hypothesesSource: Steenkamp and McCord (2007)
Stages of design processNeeds for
new artifact
Assump-tions
Initiative(First hunch)
Requirements and
assumptions
Structural specifications
Protype
Implementation
Evaluation
Evaluation
Verschuren and Hartog (2005, p.739Plan of evaluation1. Evaluation of
goals2. Evaluation of
means3. Evaluation of
relationship between goals and means.
Types of evaluation4. summative5. formative
Possible Research Entry Points
Design Science Research Process
Source: Peffers et al. (2008), Design Science Methodology Process Model (DSRM)
Identify problem and motivate
Show importance
Define objectives of a solutionWhat would be a better artifact to accomplish?
Design and development
Artifact
Demonstration
Find suitable context
Use artefact to solve problem
Evaluation
Obserce how
effective, efficient Iterate back to desing
Communication
Scholarly publications
Professional publications
Problem-centered initiation
Objective-
centered solution
Design &
development-
centered initiation
Client/ context initiated
Process iteerationNominal process sequence
Inference Theory How to Knowledge Metrics, Analysis Knowledge
Disiplinary knowledge
The design science research process
Design science research approach
(Educational) approach
Defining features of an application or
a process
Defining stakeholders’ preferences
and requirements
Designing research settings
and selecting methods
Data gathering
and analysis
Results and discussion
Feedback and possible contirubutions
An Incremental software process
First version project
Change projects
Requirementsand Features
Desining and programming
Requirements and Features
Implementation Usage of apps
New version
Use Maintenance
Source: modified from Järvinen P. (2011)
Design research and evaluation processes
The searech process
The design process
Ex-ante evaluatio
n
The construction process
The artefact
Ex-post evaluatio
n
Design science
Desing research
Source: Pries-Heje and Baskerville, (2008)
An innovation evaluation research process
Identify evaluation problem and
select evaluation method
Defining evaluation criteria
Develop and specify an evaluation
method
Data gathering and
demonstrating evaluation
ResultsNew methodImprovement
Failure
Time
Knowledge flows
Feedback and knowldge flows
User centred design activities
1. Plan the human centred
process
2. Specify the context of use
3. Specify user and
organizational requirements
4. Produce design
solutions
5. Evaluate design against
user requirements
Feedback
Research proposalsStages Questions of the research and -
paper
1. Title of research proposal What is the main results?
2. Problem area How precise are the claims?
3. Research question How could the outcomes be used?
4. Answer of the research question
What is evidence?
5. Reason for this answer How was the evidence gathered?
6. Research situation How were measurements taken?
7. Key concepts that are investigated
How carefylly are the algoritms and experiments described?
8. The research design Why is the paper (research) trustworthy?
9. The research results will address the research problem
Has the background literature been discussed?What would reproduction of the results involve?
Source: Van de Ven (2006)
Design research elements for product
Design product
Definition Description
Meta-requirement
Goals and main aims of the designed product
Describe the class of problems.
Meta-design A plan for system to meet meta-requirements
Describe the class of artefacts that will meet the meta-requirements.
Kernel theories Theories from natural or social sciences that provides a conceptual bridge between meta-requirements and meta-design
Are applied to describe conceptual connections between meta-requirements and meta-design. IT-artefacts are used IT-related work systems (Alter).
Testable design product hypotheses
Propositions to be examined regarding satisfaction of meta-requirements and meta-design
How developed or planned artefact can be tested to see if hypotheses can be verified.How meta-design satisfies the meta-requirements.
Design research elements for desiging processes
Design process
Definition Description
Design method Procedures to be used for construction of system
Design method defines the procedures, how artefact is constructed.
Kernel theories Theries that support the design method
Applied theories from natural or social sciences that used to develop the design process.
Testable design process hypotheses
Propositions to be examined regarding whether the design method produces an acceptable system
Defined hypotheses, which are used to verify, if the designed process is consistent with meta-desing.
Two research methodsVariance Method (Poole et al.(2000)
The Process Approach
1. The world is made up of fixed entities with varying attributes
1. The world is made up of entities that participate in events
2. Necessary and sufficient causality is requisite for explanation
2. Necessary causality is requisite for explanation
3. Efficient causalisity is the basis of explanation
3. Final and formal causality, supplemented by efficient causality is the basis for explanation
4. The generality of explanation depends on their ability to apply uniformly
4. The generality of explanation depends on their versality
5. The temporal sequince in which independent variables influence the dependent variables is immaterial to the outcome
5. The temporal sequince of event is critical
6. Explanations should emphasize immediate causation.
6. Explanation should incorporate layers of explantion ranging from immediate to distal
7. Attributes have one and only one causal meaning over the course of time.
7. An entity, attribute, or event may change in meaning over time.
Variance method as a research plan Stages Decision Description
1. The research question and perspective
What is the causal conditional proposal or question?
Try to take potential users, your own interest and organization’s goal into account. Reflective viewpoint may help to define research questions.
2. Unit of analysis
What individual or collective properties are being studied ?
Individual, group, organization, technology, it-artefact, process, data. Global, relational or analytical properties.
3. Causal model What is the variance research model?(see e.g. Whetten (2002, Modeling-as-theoryzing: A systematic methodology for theory development, Thousand Oaks, Sage)
List variables, draw arrows, list assumptions and boundary conditions. List alternative factors that may be rival explanations.
4. Experimental design
Is this a randomized, quasi or non-experimental design?
Randomized, quasi-experiment (survey), or non-experiment study (no control-group).
5. Sampe selection and size
What criteria are used to select units, constructs, observations & settings?
construct validity, external validity. Sample size must meet statistical significance.
6. Measurement How to measure variables? systematic and unsystematic measurement biases
7. Data analysis What data gathering technique will be used?
Select technique that fit the research guestions and model.
8. Validity What are the threats to validity of study findings?
Statistical conclusion, internal, construct and external validity.
Van de Ven: ”A variance research model represents the theory as a causal relationship among variables of units are sampled, measured and analyzed in accordance with expremimental design procedures.”
Process method as research planPhases Decision Research ideas and proposals
1. Meaning of process
A category of concepts or developmental sequence?
Are researchers interested in concepts or how a process itself can be described and analyzed (narrative, longitudinal).
2.Theories of process
Examine one or more models?
Select a process model or two models that can be compared.
3.Reflexity Whose viewpoint is featured?
Researchers, practitioner’s or user’s viewpoint to consider research outcomes.
4.Mode of inquiry
Deductive, inductive or retrodeductive?
Is the research based on the model of reality?Is the research based on how it-artefact process is going on at the real time?
5.Observational method
Real-time or historical observations?
Is a research process exploring active it-artefact development process?Is a research process observing how it-artefact process was carried out?
6.Source of change
Age, cohort or transient sources?
Try to develop parallel, synchronic research design.
7.Sample diversity
Homogenous or hetereogenous?
Try to gather different types of events.
8.Sample size
Number of events and cases?
Try to find out more than one case or more events inside the one case.
9.Process research designs
What data analysis methods to use?
Match data analysis methods to number of cases and events.
Ethics methodTask Description
Objective setting
Traditionally, system objectives have been set by top management and computer technologists. But increasingly other groups are asking or demanding to be involved in this process, in particular, line management, white and blue collar workers and the trade unions.
Adaptation The process of adaptation is concerned with moving from one kind of organizational structure and state to another and the means by which this change is assisted to take place smoothly and successfully. It is what normally happens during the implementation phase of a new system. Rapid adaptation does not easily come about of its own accord and there is a need for philosophies, facilities and strategies to assist the process.
IntegrationTechnologyPeopleTasks
Integration is the action taken, once the system has been designed and is being implemented, to ensure a new situation reaches a state of equilibrium.
Stabilization Once integration has been achieved it has to be maintained. To avoid "one man's job enrichment becoming another man's job impoverishment" imaginative solutions or even compromises may be required. The maintenance of a state of equilibrium into the future requires processes for socializing and educating new group members.
Source: Mumford (1983)
A Representational framework for generic IS artifact
Category Structure Behaviour Motivation Instantiation
Analytical Salient properties and features
Process supported
Major generic business/ organizational motivations
Mock-up prototypes
Synthetic Overall logical organization
The dynamic behaviour of IS; its working principles
Motivations for the type of IS solution as related to requirements
Working prototypes that illustrated the concept
Technological Refined structural design for the type of IS
Refined methods, algoritms, interactions employed by IS
Objectives for the design
Generig packages, frameworks, shells.
Implementation
Architecture design of implemented systems
Dynamic aspects of implemented systems
Specific business motivation
Implemented functionin systems
Source: Vahidov (2006)
The role of researchers and practitioners
Phases the researcher the practitioner
Agreement develop an agreement agree agreement
Start of research process
create a research plan agree research plan
During the real process
Develop a new artifact based on requirements and needed features of the artifact.
Participate in development process and agree the requirements and features.
Demonstration Carry out laboratory test for artifact simulating a reality.
Carry out simulation and lab-test for new artifact at the real environment.
Evaluation Develop evaluation plan and define needed criteria and combine results to the practitioner’s evaluation results.
Carry out evaluation process and give feedback to the researcher.
Conclusion Publish what is learned during the research process.
Discuss with the researcher concerning the research process.Source: Hevner et al. (2004), March and Smith (1995), Järvinen,(2012)
Selecting Design research approachResearch item
Questions Design science approach
The artefact
What is the role of the artefact?Is design of the artefact or improving organizational practice the primary goal of the intervetion?
The artefact is central of the research.Collaborators will focus on the artefacts, but success may need some changes to process and organizational practice
The process and research cycles
How the research process problem going to determined and agreed?Is a predetermined cycle of activity to be followed?Is softwarte development method a necessary part of the activities?
Problem must be identified and design criteria may be used to establish potential solutions.
The focus of evaluation
Can evaluation be a by-product of the research cycle, or must explicit evaluation activites be used?Does user acceptance testing and similar activities have a role to play?Does the artefact require the generation of use cases and some level of validation? (verification?)
Evaluation of the designed artefacts is crucial.Evaluation steps are included to the approach. Software testing is vital andis part of the research process.Design as a serch process suggest that use cases and testing are included to the research process.
Selecting Design research approach ...
Research item
Questions Design science approach
The role of knowledge
Is actionable knowledge a realistic goal of the research?To what extent is informing theory and objective?
To some extent knowledge emerges from the design.Explicit theory is less prevalent in DS. Design theory is employed as a framework and a reference point for outcomes.
The role of learning
What expectations are there with regards to learning?Will learning emerge from the design of the artefact or the actions undertaken by the researcher and collaborators?
Expectations for learning will exist.Expectations of learning from the artefact will perdominate.Researchers might be surprised by what they learn.
Source: Papas et al. (2012)
Design research evaluation framework
Semiotic level
Evaluation criteriaHelfert and Donnellan
Pragmatic Relevance, usability, completeness, timeliness, actuality, efficiency.
Semantic Precise definitions and terminology, easy to understand, interpreditability, accuracy (free-of error), consistent content.
Syntax Consistent and adequate syntax, syntactical correctness, consistent representation, accessability.
Source: Helfert M. and Donnellan B. (2012)
Ex-ante Ex-post
Naturalistic Design process/product
Design process / product
Artificial Design process / product
Design process / product
Source: Pries-Heje and Baskerville (2008)
Method Evaluation process
Observantional Case or field study
Analytical Static analysis, Architecture analysis,Optimization, Dynamic
Experimental Controlled experiment
Testing Functional (black box)Structural (white box)
Descriptive Informed argumentScenarious
Souce: Hevner et al. (2004)
Evaluation of Information systems
Evaluation type
Information system in use
Invormation system as such
Goal-based
Data source
Evaluator’s roleParticipators
Has the IT-system fulfilled the desired goals?IT-system, goal definitions, requirements, , interaction between users and the it-system.DeductiveEvaluator and users, managers
Has the IT-system fulfilled the desired goals? What is the contribution?IT-system, goal definitions, requirements.
DeductiveEvaluator expert
Goal-free
Data sourceEvaluator’s roleParticipators
Try to gain a deeper and broader understanding of the IT-systems.IT-system, description of IT-system.InductiveEvaluator expert
Try to gain a broader understanding of the IT-systems.IT-system, description of IT-system.InductiveEvaluator expert
Criteria-based
Data sourceEvaluator’s roleParticipators
Try to gain a deeper and broader understanding of the IT-system.IT-system, observation, users’ perceptions.InductiveEvaluator expert and user, manager
Try to evaluate quality of the IT-system. IT-system, description of IT-system, defined evaluation criteria.DeductiveEvaluator expert
Source: modified from Cronholm and Goldkuhl (2003)
Evaluation criteria for research papersEvaluation object
Relevant evalution questions
1. Design of an artefact
Produce the research an it-artefact, that includes a construct, a model or an instantiation.
2. Problem relevance
Is the research problem and objective of the it-artefact important? Is the it-artefact or solution relevant business problem?
3. Design evaluation
Could the results be verified?Are the evaluation method rigorous and well-defined?Is the utility, quality and efficacy rigorously demonstrated?
4. Research contribution
Is there a contribution?Is it significant?Is the contribution timely interest?
5. Research rigor
Are the results correct?Are the all technical detail correct? Are they sensible?
6. Design as a search process
Are the researchers utilized available means?Do the results satisfy laws on the problem environment?
7. Communication of research
Are the appropriate conclusions drawn from the results?Can the paper be understand?Is it clearly written?Is the results presented effectively both to the technology-oriented and management oriented audiences?
References Carlsson S.A. (2006), Towards and Information Systems Design Research Framework: A Critical Realist
Perspective, Destrist 2006, February 24-25 2006, Claremont CA. Crossan M.M. and M. Apaydin (2010), A Multi-Dimensional Framework of Organizational Innovation: A
Systematic Review of the Literature, Journal of Management Studies 47, No 6, 1154-1191. Hevner A.R. (2007), Three Cycle View of Design Science Research, Scandinavian Journal of Information Systems, Vol.
19, No.2, pp. 87-92. Hevner A.R., March S.T., Park J, and Ram S. (2004) Design science in information systems research, MIS Quarterly, Vol.
28, Issue 1, March 2004. Järvinen A & P. (2011) Tutkimustyön metodeista, Opinpajan kirja, Tampere. Järvinen P. (2012), On boundaries between field experiment, action research and design research, University of
Tampere, Shcool of Infomation Sciences, Reports in Information Sciences 14, Tampere. Iivari J. (2005), Information Systems as a design science, Information Systems Development: Adnvances in Theory,
Practice and Education, Edited bgy O. Vasilegas et atl. Springer. Papas N., O´Keefe R.M. and Seltsikas P. (2012), The action research vs design research science depate: reflections
from an intervention in eGovernment, European Journal of Information Systems, Vol. 21, pp- 147-159. Peffers K., Tuunanen T., Gengler C.E., Rossi M., Hui W., Virtanen V. and Bragge J.(2006), The Design Science Research
Process: A model for producing and presenting information systems research, Destrist 2006, February 24-25 2006, Claremont, CA.
Vahidov R. (2006), Design Researcher’s IS Artifact: a Representational Framework, Destrit 2006, February 24-25 2006 Clamenont CA (CGU2006).
Vaishnavi V. and Kuechler W. (2004), Design Science Research in Information Systems, http://desrist.org/design-research-in-information-systems/, last updated September 2011
van Aken Joan E. (2001), Management Research based on the paradigm of the Design Sciences; The Quest for tested and grounded technological rules.
Van de Ven (2007), Engaged Scolarship: A Guide for Organizational and Social Research, Oxword University Press Inc., New York
Venable J.R. (2006), The Role of Theory and Theorising in Design Science Research, Destrist 2006, Proceeding/2A_1pdf.
Verschuren P. and Hartog R. (2005), Evaluation in Design-Oriented Research, Quality & Quantitity, Vol. 39, pp. 733-762.
Zobel J. (2004), Writing for computer science second edition, Springer,-Verlag, London Limited.