Skip to Main Content

Section 2

Overview of Design and Development Research (DDR) For Applied Doctorate Students in the Instructional Design Program

The purpose of this quick guide is to assist Applied Doctorate students in the Instructional Design Program in determining the best methodology and design for their Applied Doctorate Experience (ADE) dissertation. The guide covers intended target audience, an overview of Design and Development Research (DDR), types of DDR research including product, program, tool research, and model research, 3 stages providing alignment of DDR with NUs Applied Doctoral Record (DDR) deliverables, examples of problem, purpose, and research questions for DDR research, and suggested references. 

Target Audience: Doctoral Students in Instructional Design in the ADE program

This quick reference guide will aid doctoral students in instructional design challenged with deciding on what type of applied research study they want to do for their dissertation.

Overview of Design and Development Research (DDR) Methods

At the core of the instructional design and instructional technology and media field, is the design, development, implementation and evaluation of instructional products, tools, programs, models, and frameworks.  In many ways DDR is like Action Research (Goldkuhl, 2012), however, there are many differences. DDR research allows instructional designers a pathway to test theory, models, and frameworks and to authenticate practice. The focus of DDR is on the use, design, development, implementation, and evaluation of products, tools, programs, and models using instructional design models and frameworks. Richey and Klein (2007) defined DDR as “the systematic study of design, development, and evaluation processes with the aim of establishing an empirical basis for the creation of instructional and non-instructional products and tools and new or enhanced models that govern their development” (p. xv). Often the models and frameworks are validated and/or further developed and enhanced through the DDR. DDR is applied research. An area of DDR research that is particularly applicable to ADE students is the creation, implementation, and evaluation of one or more artifacts, such as products, tools, models, new technologies, and learning objects that will aid in solving a complex problem in practice that can be addressed through human imagination, creativity, engagement, and interaction (Ellis & Levy, 2010). These types of problems are found in K-12 education, higher education, corporations, not-for-profits, healthcare, and the military. 

Example

Types of Design and Development Research

Types of Design and Development Research

The field of DDR is constantly evolving and expanding as technology and media are changing at exponential rates.  Richey and Klein (2007) in their seminal work divided DDR into two major categories:

  1. Product and Tool Research and
  2. Model Research.

Table 1 provides a summary of common designs used in DDR. Most DDR work falls under the qualitative research category of qualitative case study, however, methodologies such as quantitative and mixed method have been used as well as other qualitative designs, including Delphi.

Table 1: Types of DDR Research, Focus, Data Collection Methods, and Artifacts
Type of DDR Research DDR Focus Data Collection Methods Researcher/Designer Artifact Examples
Product, Program, and Tool Development Research Full Life Cycle Design and Development Projects Needs Assessment, Content Analysis, Surveys/Questionnaires, Artifact Development, In-depth Interviews, Observations, Evaluation Methods (Kirkpatrick Level 1-4) Researcher/Designer Artifact Examples
Needs Assessment
Design Document
Story Boards
Program Materials or
Product Prototype or
Tool Prototype
Formative Evaluation: Pilot, Alpha/Beta Test
Evaluation Report
Product, Program, and Tool Development Research One or More Phases of the Life Cycle Needs Assessment, Content Analysis, Surveys/Questionnaires, Artifact Development, In-depth Interviews, Observations, Evaluation Methods (Kirkpatrick Level 1-4) Needs Assessment
Design Document
Story Boards
Program Materials or
Product Prototype or 
Tool Prototype
Formative Evaluation: Pilot, Alpha/Beta Test
Evaluation Report
Product and Tool Development Research Tool Development Needs Assessment, Expert Interviews, Artifact/Tool Development, Expert Validation (NGT), Participant Interviews, Focus Group Interviews, Evaluation Methods (Kirkpatrick Level 1-4) Needs Assessment
Design Document
Story Boards
Tool Prototype
Formative Tool Evaluation: Pilot, Alpha/Beta Test
Formative Tool Evaluation Report
Product and Tool Research Tool Use Participant Interviews, Participant Think Aloud/Talk Aloud Methods, Focus Group Interviews,  Evaluation Methods (Kirkpatrick Level 1-4) Needs Assessment
Tool Use Evaluation Report
Model Research Model Development Expert Interviews, Expert Review, Expert Evaluation - Nominal Group Technique (NGT) and Focus Group Interviews Needs Assessment
Design Document
Story Boards
Model Built
Formative Model 
Expert Evaluation Report
Model Research Model Use Participant Interviews, Participant Think Aloud/Talk Aloud Methods, Evaluation Methods (Kirkpatrick Level 1-4) Needs Assessment
Design Document
Model Use
Participant Evaluation Report
Model Research Model Validation Expert Individual Interviews, Expert Review, Expert Evaluation - Nominal Group Technique (NGT) 
Focus Group Interviews		 Needs Assessment
Model Validation Plan
Expert Evaluation Report

 

Product, Program, and Tool Research

Ellis and Levy (2010) asserted that DDR must go beyond commercial product development by determining a research problem, based on existing research literature and gaps in the literature that researchers assert must be studied to add to the instructional design knowledgebase.

Product and Tool Research can be further divided into:

  1. Comprehensive Design and Development Projects covering all phases of the instructional design process,
  2. Specific Project Phases (such as those in the ADDIE model: Analysis, Design, Development, Implementation, and Evaluation), and
  3. Design, Development, and Use of tools (Richey & Klein, 2007).
Model Research

Instructional designers and instructional technologists have focused on model research since the emergence of the field.

Model research can be broken into three types:

  1. Model Development,
  2. Model Validation and
  3. Model Use.

Model development can focus on a comprehensive model design or on part of a process. Model validation research uses empirical processes to prove the effectiveness of a model in practice. Finally, model use research addresses usability typically from the perspective of instructional designers and stakeholder experts.

3 Stages in Design and Development Research

3 Stages in Design and Development Research for the ADE Doctoral Student’s Dissertation

NU doctoral students in the Instructional Design Program can use one of the various types of DDR research to complete their doctoral dissertation using the NU ADE template. There will be three stages in this process and in each stage the student will have one or more deliverables using the NU template and posting in the ADR on NU One.

Stage 1: Design and Development Research aligned with the NU ADE Template Process
  1. Identify a research worthy problem which is expressed by researchers in peer reviewed research literature. Ask yourself, what is going wrong? What do researchers say is known about the problem? And what is needed to be known to address the problem?
  2. Describe the purpose of your research ensuring that it aligns with your problem statement. In the description state your methodology and design and which DDR type of research you will do. Be sure to include a description of your target population (audience), the size of your sample and the sampling strategy you will use to access your sample. What permissions do you need? Site permission? Other IRB permission?
  3. Write your research questions to align with your problem and purpose statements.
  4. Complete Section 1 of your Applied Doctoral Experience (ADE) template securing all necessary approvals in the Applied Doctoral Record (ADR).
  5. Create Design Document including:
    1. Needs Assessment
    2. Measurable Goals and Objectives
    3. Target audience (target population) definition
      1. Sample size and Access to the sample
      2. Sampling strategy
    4. Content analysis (course, program, product, or tool descriptions)
    5. Technology and media analysis/selection
    6. Delivery platform(s):
      1. Learning management system(s)
      2. Asynchronous
      3. Synchronous
      4. Blended
    7. Evaluation Plan
  6. Complete Section 2, Proposal Draft, Proposal for AR, and Final Proposal of the ADE securing all necessary approvals in the ADR.
  7. Submit Proposal and IRB Application to secure IRB approval.
Stage 2: Design and Development Research aligned with the NU ADE Template Process

After receiving IRB approval of your ADE Proposal, it is time to design, develop, test, validate, and/or evaluate your artifacts. Below are example steps:

  1. Review and Finalize Design Document
  2. Recruit Expert Participants, if required
  3. Recruit Artifact User/Participants, if required
  4. Develop the artifact(s), which are the end products, if required
    1. Lesson plan or syllabus
    2. Instructional strategies and activities
    3. Participant materials
    4. Trainer materials
    5. Storyboards and scripts
    6. Other media
    7. Create model, tool, product, or program.
    8. Validate model, if required
    9. Evaluation plan (Kirkpatrick Levels 1, 2, 3, 4)
  5. Pilot (alpha/beta) test the artifact(s) as a formative evaluation.
    1. Alpha test, Beta test, Pilots.
    2. Rapid Prototyping
  6. Evaluate test results.
    1. Participant reaction
    2. Trainer/facilitator reaction
    3. Learning
      1. Mastery
      1. Retention
    2. Behavior transfer to the job or task performance Results
      1. Were Design Goals met?
      1. Were Design Objectives met?
  2. Revise artifact(s), Retest, if necessary.
Stage 3: Design and Development Research aligned the NU ADE Template Process

Complete Section 3 of the ADE template presenting the study findings, conclusions, and implications. Next pull all three sections into a dissertation manuscript for approval in the ADR.

Data Collection Methods and Sources of Data in DDR

While DDR covers a wide variety of approaches, most doctoral students in the ADE program will find case study to be the preferred design. To strengthen trustworthiness of the data, multiple sources of data will typically be used.  Using multiple sources of data is called triangulation in research. Figure 1 shows examples of sources of data for DDR.

The goal is to create, use, and/or validate New Artifacts by collecting and analyzing various sources of data including:

  1. Existing artifacts,
  2. Expert individual and focus group interviews,
  3. Participant/user individual interviews, talk aloud-think aloud interviews, focus group interviews,
  4. Research observation and participant observation,
  5. Evaluation, Kirkpatrick Levels 1-4, and
  6. Needs assessment and design documents.

The new artifacts may be lesson plans, student guides, facilitator/teacher guides, learning objects, tools, models, programs, and/or products.

Figure: Sources of Data in DDR

Sources of data in DDR graphic.

References

Ellis, T.J. & Levy, Y. (2010). A guide for novice researchers: Design and development research methods. Proceedings of Informing Science & IT Education Conference (InSITE) 2010, pp. 108-118. http://proceedings.informingscience.org/InSITE2010/InSITE10p107-118Ellis725.pdf

Goldkuhl, G. (2012). From Action Research to Practice Research. Australasian Journal of Information Systems, 17(2). https://doi.org/10.3127/ajis.v17i2.688

Richey, R. C. & Klein, J. D. (2007). Design and Development Research. Routledge