AERA: Monday 1

April 10, 2006: Developing Methodological Rigor in Design Research in education

www.rochester.edu/warner/aera
John Beck, George Mason "Handbook Design Research Methods"
Sasha Barab, IU "Illuminating "ILF"

• • Web-supported community
• • (Barab & Duffy) (Barab & Squire 2004 p3)
• • traditional DBR: theory, design, and problem it addresses
• • iteration #2 changes all three
• • historical methods : central subject problem (boundaries); colligation problems (case endures); Kernels (complex causality)
• • what is community? a persistent, sustained social network of individuals...

takeaway: Interesting how the visual design of ILF really affected the use of it.

Paul Cobb, Vanderbilt University "Experimetning to Support and Understand Learning Processes"

• • mature methodologies are underpinned by distinct argumentative grammars... (Kelly, 2004)
• • different methodologies fit different questions, but also produce different forms of knowing (and representation)
• • design-based pedagogies produce
• • (Brown, 1992, pp. 163-164) "I have never taken the Hawthorne criticisms of my work seriously..."
• • Brown makes a close coupling between the learning and the means of support of learning (the learning environment)
• • How do we explicate our interpretive framework?
• • Basic claim of DBR: DBR embraces the full complexity of learning
• • Inferences must be open to public scrutiny
• • What constructs do we use? More specifically than Sociocultural or Activity Theory.
• • Use explanatory constructs to make sense of data, and also tease out criteria for how the constructs make sense of the data
• • **** (Maxwell in Ed researcher 2004 p. 4) types of causality can be discerned based on a single case rather than on many cases. *****
• • although repeatability further strengthens the case

takeaway: There's a lot in this one, but it's thick and hard to get to. Love the Brown quote...

Joanne "Transfer"

• • "unnatural, libratory game" (Lave 1988)
• • classical transfer: The application of one knowledge learned in one situation to another situation. (same structure; different surface features)
• • PFL Preparation for Future Learning (Bransford & Schwarz) "sequestered problem solving ignores real-world conditions that people exploit
• • AOT Action Oriented Transfer (observer -> Actor)
• • Point 1: methods are not isolated beliefs

1. o Classic: Did transfer occur? Prove it.
2. o PFL: which method prepares students to benefit from a learning opportunity?
3. o AOT: What are the images by which learners construct two situations as similar?

• • To establish AOT: 1. Identify themes 2. Analyze and loko for influence of instruction
• • Stage approach of Transfer Approach: Context of discovery: 1. Informed exploration 2. Enactment. context of verification: 3. Local Impact. 4. Broader impact.

takeaway: I should find out more about AOT (Action Oriented Transfer)...

Finbarr Sloane, ASU "Modeling emergence in Design Settings"

• • Emergence is "bottom up and interactive" shaped and constrained and influence by higher level contextual levels
• • The iterative process in DBR (Cobb et all 2003; Barab & Kirshner, 2001)
• • Multilievel modeling tools, (Raudenbush & Bryk, 2003; Sloane 2005)

takeaway: This just further reinforces the importance and validity of design-based pedagogy (and, of course DBR).

Summary by --- ??
"What do you do if you think control groups are silly? Design research." Used to be called teaching design

• • Have a design and theory and test it
• • Have an artifact
• • The most important thing in evaluating them is to look at the underlying assumptions
• • Do your assumptions fit your research?
• • Is design research leading to something else?
• • Not everything we know reduces to a tested hypothesis, or answered question -- in fact not much that we know does.
• • Once people interpret something, their interpretations change -- each iteration of this is further modified, so "classic" transference is nonsense
• • Solutions for real problems don't' come from single disciplines -- you need to draw on multiple theories to answer them. DBR cannot use just one theory and be meaningful
• • We live in designed worlds, as soon as you understand them, you change them
• • Understand systems then change them -- Quest Atlantis: two worlds, with one destiny
• • We draw on multiple theories to create a model.
• • Theories are good if they are falsifiable. Models cannot be falsifiable
• • What makes something DBR?
• ♣ Multi-theoretical

takeaway: One theory is not enough -- legs to support a stool.

Questions

• Is transference a Learning process or an application process? If pure transfer occurs, no learning takes place -- we want dynamic reconstruction of knowledge to the specific iteration of the "new" problem situation.
• Stochastic vs. Process
• Design in the making vs. ready-made design
• Dewey's useful vs. not useful
• Plausibility vs. Verifiability
• DBR is the wrong level to think about rigor (Paul Cobb)