Case Study 2 Michael Bishop

Implementing Gaming Technologies in Traditional K–12 Contexts
by Susan Pedersen
Michael Bishop was at Oakdale Middle School last April when the results of the recent statewide proficiency tests were released. He’d been meeting with middle school teachers involved in pilot testing the educational games his team had developed for middle school science classes. As he left the training session he found Nancy Levin, the district-level science curriculum specialist, in the hallway, talking with Paul Russell, the Oakdale principal. Michael, a researcher at the university and the director of the project developing the science games, had been working with Nancy for two years now, and they had an easy relationship. So he approached her smiling, not yet realizing how serious the conversation was.
The news was grim. The previous year, the district average on the eighth-grade proficiency tests trailed the state average by 8%. This had prompted the district to implement new approaches in their middle schools designed to boost test scores in reading, math, and science. The new approaches included heavy use of software that offered individualized tutorials, as well as after-school programs for students who fell just short of proficiency in any of these subjects. Committees of teachers had convened over the summer to assess alignment of the curriculum to state standards and to select (and sometimes write) test items to be used for practice throughout the school year. Confidence in these approaches ran high in the district office, and school officials had waited eagerly for the confirmation of their effectiveness that the proficiency test results would provide. But the results showed exactly the opposite. The district average for eighth grade had fallen by 8% while the state average had risen by 2%. This left the district trailing the state average by a whopping 18%. The results of tests at the fifth-grade level were slightly better, but they still trailed state averages by 11%. Michael knew that in this state, such a drop typically had consequences for district personnel, and he wondered if some of this would fall on Nancy’s shoulders.
So Michael was not entirely surprised six weeks later, after school was out for the summer, to get an e-mail from Nancy announcing that she was returning to the classroom to teach biology, and that her replacement would be Tara Jones. She did not mention Michael’s project. Michael was surprised at the formality of the e-mail, so he tried calling Nancy, both then and two days later. She returned neither call. Michael then contacted Tara to set up a time to meet to discuss the science game project.
Tara called Michael two days later. “I’ve had a chance to sit down with our superintendent to discuss your project, and I’m afraid that we’ve decided to withdraw,” Tara told Michael. “We’ve decided that we just need to double down on essential skills and knowledge and that we can’t afford to have our kids spend time playing games, even ones that are supposed to be educational. Each of your games takes over a week to play, and that’s just too much time. The cohort you’re working with will be eighth grade students next year, and since that’s the grade in which kids take the science proficiency test, we’ve really got to work on preparing them. Now, I understand the problem we’re causing for your project and that it will probably be an issue with your funding agency, but I’m sorry, the decision has been made.”
Finding a New Pilot Site
Michael was frustrated. Now in the middle of the project, he needed to recruit another school district to pilot test the games. So after taking a few days to consider his pitch, Michael called three different district offices and followed up with e-mails; one never responded and the other two formally declined to participate without even meeting with Michael. Michael decided to take a different approach and asked a colleague who had some contacts with district-level science coordinators to make some introductions. This led to conversations with administrators from four districts.
Michael felt as if he had to pitch the games to a skeptical audience, something he didn’t like doing. He also knew that these administrators had full schedules, so he needed to give them enough detail to grab their interest but not overwhelm them, address issues head on, and make sure they could see the potential these games had for both learning and motivation. He sent a handout to each of the administrators before their conversation (see Figure 2–1). He began these conversations by explaining, “Each game addresses specific grade-level science concept standards, and does it in a way that makes it more likely that kids will understand and remember them. But more importantly, these games address inquiry standards. Kids ponder complex problems, ask questions, look for existing information, design investigations to gather data to answer their questions, support their decisions with evidence, and communicate their reasoning to classmates.” Michael then walked the coordinator through the game model using the handout, answering questions as they arose. He made sure to emphasize how motivated students were during the game: “Even kids who normally seem unmotivated in science class stay on task and are enthusiastic. We really don’t see much off-task behavior.” Then he discussed the potential impact on standardized test results, citing results from the National Assessment of Educational Progress (NAEP) that suggested how games and other technology-based approaches that engage students in higher order thinking were correlated with higher outcomes on standardized tests than traditional approaches and tutorial software.
Bailey Richards, the science curriculum specialist in the Weyman independent school district (ISD), seemed genuinely interested in games as a means to engage students in
Figure 2–1 Game Description Sent to District Personnel Describing the Game Model, with Examples from One of the Games,Rigglefish.
Rigglefish
Game Description: Rigglefish is a game designed to address middle school standards related to genetics and scientific inquiry. In Rigglefish, learners take on the role of Dr. Waters, a geneticist tasked by the government with developing a source for Omega X, a fatty acid that can be used as a protectant against a deadly bioweapon. That source is the rigglefish, a recently discovered species of fish rich in Omega X. Rigglefish can be red, orange, or yellow, but only the yellow ones produce high concentrations of Omega X. Rigglefish also possess some traits that make them difficult to breed in captivity, including a sensitivity to low pressure environments, sharp spikes, and their distinctive wiggle. Players must breed a mating pair of rigglefish that can be farmed to provide a source for the needed protectant.
Game Component
Example: Rigglefish
A Complex Task: Each game presents a complex, ill-structured task which requires students to engage in student-directed inquiry
Students must breed a mating pair of fish that are purebred for four key traits. To accomplish this task, students determine, through observation and testing, the phenotypes and genotypes of fish they collect, then breed these fish to obtain the target fish.
Opening Scenario: Video introduction to problem; provides a compelling backstory, but does not tell learners what to do
Student is cast in the role of Dr. Waters, a geneticist who receives an urgent request from the government to develop a source for Omega X, a protectant against a deadly new bioweapon. Task must be completed before enemy agents discover the lab where players are working.
A Virtual Environment: A confined space that contains all the tools and resources learners need; players spend very little time on navigation
Students work in a top-secret underwater lab with four rooms: bathysphere, sample room, pressure room, and breeding room.
Virtual Scientific Instruments: Virtual models of real-world scientific instruments, simplified to emphasize key characteristics relevant to student learning; students must interpret the data the instruments return
Sample Instruments:Bathysphere: Used to collect rigglefish for testing and breeding
Sample tanks: Used to observe rigglefish and determine phenotypes and genotypes of each
Gel electrophoresis and PCR: Determine a fish’s genotype for wiggle trait
Breeding tanks: Breed rigglefish and select offspring for further testing and breeding
Information Resources: All information needed to handle task is provided within game so that learners do not need to search online; information is divided among resources, which discourages reading without a purpose
Sample Resources:Genetics Guide: Information on topics such as dominant, recessive, co-dominant, and incompletely dominant alleles
Punnett square: Interactive square that players can use to determine possible offspring; connects genotypes with phenotypes
Expert Modeling Videos: An expert thinks aloud about how he or she would handle different tasks within game, making scientific thinking overt; videos available on demand
Menu allows students to ask questions such as:What should I do first?
How can I tell the genotypes and phenotypes of different fish?
How do I breed a fish without a wiggle?
Tool Demonstration Videos: Shows how each tool within program functions
Videos on how to use:Bathysphere
Sample tanks
Pressure tank
Gel electrophoresis
PCR
Punnett square
Breeding tanks

scientific inquiry and agreed to meet with Michael and let him present the games to her. But, like Tara Jones, she balked at the amount of time required. “We hit a lot of topics in those middle grades and yes, we want depth, but we have to go deep quickly. So we want inquiry, but we really have to guide them through it so they don’t spend a lot of time just trying to figure out what to do. Your games look great, but these kids aren’t used to having to figure so much out for themselves. I think you would find a lot of kids wasting time not knowing what to do. Maybe advanced learners would be able to handle something like this, but I don’t think this would be an efficient use of time for the average learner.”
Laura Kenner and Daniel Brown, the science coordinators in two neighboring districts that had received ratings in the satisfactory range on the proficiency tests in the previous two years, raised other issues as well. Both districts were moving to a common curriculum in which lessons were developed by a group of teachers and then implemented in every classroom on the same day. The approach was new and many teachers were protesting. Laura explained, “At this point I just don’t see how we could allow a few teachers to do something different. If we did, everyone would be asking to be allowed to bring in their pet projects.” Daniel explained further, “We’re trying to limit how much time our science classes use computers so that we can save computer lab time for math and language arts. If we had one or two science teachers getting to use computers for two weeks in a row, the other teachers and students would complain. I just don’t think we could make that work.”
Jim Harrington, the assistant superintendent for curriculum in Mason ISD, a large district where the middle school ratings fell into the excellent to exemplary categories, met with Michael for over an hour and seemed to really enjoy playing one of the games Michael’s team had developed, even though he hit a couple of bugs. He was less concerned than the others about the length of the games themselves, but raised another issue about the time required. “I know some of these approaches have great potential, and I think that’scertainly true for games. And I also know you have to pilot test them somewhere, but I feel like we really need to protect our kids from spending too much time on that sort of thing. We’ve had university folks come into our schools before, and they want to have kids complete a lot of surveys and tests and try out new materials and approaches. But not all of them are ready for prime time. Even in your game, there were some bugs; those could bring a class to a screeching halt and end up wasting time. And you want them to complete pretests and surveys and interviews. We just can’t spend that sort of time on research.”
Interestingly, none of the school personnel Michael spoke with argued that games were inappropriate in science education. In fact, many of them seemed enthusiastic about the use of games in education. In the end, Jim offered to let Michael implement the games in pre-AP classes in two middle schools and a magnet laptop program in another. Laura asked if he would like to work with the after-school science clubs at two of the schools in her district, and Daniel suggested he consider summer programs. They liked the games; they just didn’t want them in their regular science classes.
A major purpose of the games Michael’s team was developing, and the purpose for which they had received funding, was to hone an innovative model to use technology to increase the engagement of all students in scientific inquiry in their science classes. Restricting use of the games to gifted students, after-school programs, and science summer camps seemed like an admission that they weren’t appropriate for regular kids in regular classes. Michael wondered if there were ways to make them more appealing to school districts. Though Nancy had given him some good advice, such as trying to keep the games to a maximum of eight days and including tool demonstration videos, and he had adjusted the original model for the games accordingly, it had been insufficient to make the games appealing to anyone who wasn’t already an advocate of this type of approach. In considering what the options might be, Michael sought advice from people in diverse fields.
Getting Advice
Michael recruited a small group of people with expertise in different areas to participate in an advisory session with him. Craig Dawson was the director of science education for the state education agency. He had 20 years of teaching experience in middle and high school science classes and was a strong advocate for inquiry-based learning. Michael had attended a couple of talks he had given, but did not know him well. Bob Blanchard was a game designer who had spent nine years in California working on some triple-A game titles, then moved two years ago to lead game design in a midsize company located in the same town as the university. Michael had met him the previous year when he was conducting interviews with gaming professionals as part of a research project led by a colleague at another university. Antonia Fisher was a professor of science education at Michael’s university. She had projects of her own and had pleaded lack of time when Michael had invited her to be a co–principal investigator on his project. However, Michael had known her for years and valued her opinion, so he was glad she’d agreed to participate in the advisory session. Despite their busy schedules, they were able to meet for a four-hour block in early August.
Figure 2–2 Players Use the Bathysphere in Rigglefish to Collect Fish They Can Study and Breed.
Michael began the meeting with introductions followed by a presentation of the game model, similar to the one he had given district administrators. He then showed them the opening scenario of Rigglefish and let them play for about 40 minutes (See Figure 2–2). This was enough time for everyone to capture fish and try out the different tools, including the breeding tank (see Figure 2–3). They all chatted as they played, commenting mostly on features they liked and asking questions about how to accomplish specific tasks. It was obvious they enjoyed the game and wanted to figure out how to breed the target fish.
Michael began transitioning the group from playing Rigglefish to discussing the project. “What I’d like from you is your advice on moving forward with the design of these games.” He then explained about the withdrawal of one district from the pilot test and the responses from other districts. “We’d like to see these games implemented in middle school science classes, not only with pre-AP or gifted students, but with regular education classes as well. Is that reasonable? Do we need to design these games differently in order to make them attractive to teachers and districts? Or should we give up on that and design for a different audience?”
Craig Dawson shifted in his seat and at the same time seemed to shift roles from enthusiastic player to concerned educator. “That’s a tough one, Michael. Teachers and administrators are concerned about those tests, but really what we’re all concerned about is making the best use of instructional time. Games are highly engaging, but do students
Figure 2–3 The Breeding Tank in Rigglefish Allows Players to Cross Fish to Breed Offspring with Desired Traits.
learn as fast or as deeply as they do through other approaches? Unless you can build a case that these games engage students in real science and that this causes them to learn better and faster, you’re going to continue to have the sort of pushback you’ve gotten so far. Now I know that standardized tests have a bad reputation among academics, and perhaps they’re imperfect, but they’re the best tool we have for identifying schools that are failing and students who need help. So you’ve got to be able to show that your games lead to learning that shows up on those tests.”
Antonia Fisher responded, “I think that’s a somewhat unrealistic expectation, Craig. Performance on those tests depends on a lot of factors. I would say that if you can show that you are successful in getting kids to engage in scientific practice within the games, that’s enough to justify their use. The science education community has been trying for decades to incorporate better opportunities for kids to engage in authentic scientific practices and from what I noticed playing your game, I think it’s pretty clear that players ‘do science’ in it. In particular what resonated with me was that I could see kids collaborating and debating the best ways to tackle various parts of this problem, just like the three of us did as we played. That type of critical thinking just doesn’t happen often enough in our science classes.”
“I agree that the game encourages higher level thinking,” Craig replied. “I think that’s great. All I’m saying is that if you want buy-in from schools, you’re going to have to show an impact on performance. I’d recommend creating a bank of test items that teachers can use either as warm-up activities or homework, or perhaps even embed them in your games. Kids reach certain checkpoints and they answer a few questions to test their understanding. That would allow both the teachers and the students to see how much they understand.”
Bob Blanchard objected, “Games are great at gathering data about players. There’s no need to incorporate multiple-choice questions because you can tell through gameplay what a player does and doesn’t understand. Now it’s difficult to pull that data out of games and make it really usable for teachers, but it can give a much better picture of what players understand. Besides, incorporating multiple-choice or short-answer questions in a game will break up gameplay, distract the player, and kill motivation. You might be able to do some of that in a homework exercise, but I suspect that as soon as those worksheets come out, the type of critical thinking you’re trying to get at here will disappear.”
Antonia replied, “I think that’s a real danger. You know, Michael, schools need alternatives. If the only types of learning materials out there are ones that prep students for tests, where are the models of alternative approaches? Where are the materials those innovative teachers can use? You have a vision here, and it’s a pretty good one, so I say stick with it. These games won’t be for everyone, but if you can find a few partners who believe in this work and show impact on the types of things that really matter, then you’ve done something important.”
“What do you mean by ‘really matter’?” Michael asked.
Antonia was growing passionate. “By the time a student finishes high school, he or she should have adopted the habits of mind that good scientists have. For example, they should bring the type of healthy skepticism that scientists have to making decisions in all areas of their lives. By that I mean that they should expect, no, demand, evidence for claims that others make. And they should think critically about whether that evidence is unbiased and valid. I think games like this can support that type of thinking. I think kids are likely to challenge each other about what claims they make and that they are going to want to defend what they are doing. In other words, they’re going to ask each other for evidence and offer it themselves when challenged. That’s incredible. That’s what we should want out of our educational system. Achieving a goal like that—that should be the measure of success.”
Craig smiled. “That’s all fine and good, but you have to consider the realities of the classrooms where you want your games played. In addition, you have to think the teachers and students who play them.”
Antonia nodded and said, “What I question about these games is whether you really support teachers in using them well. It takes time to learn to use an approach as different as this effectively.”
Bob agreed. “I worked on an educational game and we did some testing in schools. As soon as the teacher was the one managing the implementation, we had problems. That’s not a slam at the teachers; they were quite professional and enthusiastic, but they started telling the kids to do things that we’d never anticipated. In particular, they would show them specific strategies and tell them to play that way, even though there were multiple effective strategies. It undermined the whole gameplay experience in that the kids were not really figuring things out for themselves. You really have to provide good teacher training if you want them to support, rather than direct, the inquiry experience.”
“But teachers are going to need to hold students accountable for learning while they are playing,” Craig said. “Otherwise, what are you going to do about kids who get off-task?”
Michael responded, “Actually, we don’t see a lot of off-task behavior.”
“Perhaps you haven’t seen it because you’re there leading the class and the approach is novel,” said Craig. “But when you’ve got a class with kids with diverse needs and a teacher who isn’t entirely comfortable with the approach, you’re going to have some kids get confused and off track. The teacher has got to have some control to bring them back on task. There have got to be checkpoints and at least a few definite assignments with due dates. Otherwise there’s a danger of a lot of time being wasted.”
“Michael,” said Antonia, “if you are really looking for a pure implementation of your games, then you might seriously consider summer camps and homeschool markets. That would allow you to implement your vision of the games while at the same time learning about effective implementation strategies. Once you know how to optimize the player experience, you might be able to try that out in a school setting. There have been a variety of innovations that caught on outside of classrooms, and once they were really polished, started being adopted by schools.”
Michael tried to get the group to identify areas of consensus about how he should move forward with the project, but didn’t get very far. Beyond agreeing that the games had merit, their opinions diverged too much to coalesce into an action plan he could walk away with. The lofty principles Antonia espoused resonated with Michael’s own beliefs about the goals of science education, but he also recognized that sticking with his vision might mean that the games were doomed to irrelevance. Even if he could find isolated teachers or schools willing to pilot test them, they would probably never achieve broad dissemination in middle schools under their current design. Michael left the meeting realizing he had a lot of thinking to do.
Preliminary Analysis Questions
Identify the different barriers Michael encountered when he tried to convince school district personnel to implement the games in middle school classes. What questions do you think educational game designers must consider when designing a game for K–12 contexts?
What arguments could Michael make to convince school administrators and teachers about the potential benefits of educational games?
Why does Michael feel so strongly about notputting the game in an after-school program? Discuss the pros and cons of Michael’s decision.
Implications for ID Practice
What characteristics of middle school learners must designers consider when planning educational games? Provide specific examples.
Identify the different contexts in which an educational game might be played and how those contexts affect design decisions.
How can teachers assess student learning through educational games?
How are the factors affecting the adoption of a game in this case similar to or different from efforts to introduce other innovations in schools (e.g., problem-based learning, mobile devices)?