University of Illinois, Urbana-Champaign
Educational Psychology 387
Computer Uses in Education
Multiple methodologies for evaluating computer uses in education
But first ...
.
.
.
.
.
.
"war stories" in learning
- studies by sociologists at Xerox PARC
- Julian Orr Talking About Machines : An Ethnography of a Modern Job
.
.
.
.
The Range of Methodologies
- Naturalistic observation
- Participant observation
- Interview
- Survey/Questionaire
- Quasi-experimental
- Experimental
.
.
.
.
Kinds of evaluation
- Summative evaluation
- Formative evaluation
.
.
.
.
Choosing a methodology
- What are your goals?
- How much do you know about what is happening?
.
.
.
.
Examples of effective observation
- The "space bar" example
- The "DoIt" example
.
.
.
.
The "space bar" example
- Push the space bar to continue.
- The students expected...
.
.
.
.
The "DoIt" example
Do you want to save your file?
.
.
.
.
.
.
.
.
And the user read "DoIt" (DoIt = DO IT) as
"Dolt" (Dolt = DOLT)
dolt: a stupid person
Process data
Some methodological suggestions
- survey or questionaires
- rank order and then reduce the number of questions
- "debug" your questions - test them on a number of people before using them
- end with some open ended questions
- What is the best thing about this?
- What is the worst thing?
- What improvements would you suggest?
- observations
- don't make suggestions, only respond to help questions, not volunteer help
- whenever the user asks for help, note that down after giving help
- write notes, both during and after
- can followup observations with a survey at the end
- for preliminary studies, use audio tape or video tape as a backup source of data to written notes
.
.
.
.
Spring break next week
.
.
.
.
For the following week
.
.
.
.
