Susan E. Schultz, PhD
Learning through interactive, inquiry-based experiences engages learners in their education and provides opportunities to master skills that are essential in achieving quality science education: critical thinking, problem solving, and investigative. Increasingly, teachers looking for innovative ways to engage their students turn to the informal community to learn more about place-based and inquiry-based teaching and learning. To meet the needs of teachers, the National Park Service (NPS) and other informal learning institutions are striving to produce curriculum-based programs aligned with state and national science standards. The Parks as Resources for Knowledge (PARK) program aims to create a professional development model that bridges the gap between formal and informal learning communities and institutionalizes best practices among pre-service science teachers by strengthening partnerships between schools of education and national parks
Through an array of programs, the NPS has provided educational experiences for thousands of K through 12 teachers and students. While these programs were highly successful, evaluations revealed a major gap: for their students to succeed and flourish, teachers need significant support and professional development opportunities in both science content and innovative science teaching methods. Responding to this need, the PARK program seeks to develop innovative ways to improve teachers’ access to and use of national parks’ educational resources.
My role in the project was to design and conduct an external evaluation of the PARK program. The evaluation plan consists of formative and summative components. The formative evaluation components include documenting the planning process, designing assessment instruments, and monitoring the implementation of the program by providing early and ongoing feedback to the project team. The summative evaluation focuses on assessing the program’s overall performance and effectiveness, as well as lessons learned. Qualitative and quantitative indicators, based on direct observation, surveys, and interviews, were used throughout the evaluation.
Goals and Outcomes
The overarching goal of the PARK program is to combine the best education techniques and methodology from the formal and informal education communities with the powerful science content of the national parks. Specific outcomes include:
Beginning April 2005, Golden Gate National Recreational Area staff members Lynn Fonfa (NPS program director), Roxi Farwell (NPS educator), and Will Elder (NPS interpretive ranger and park geologist) began working with Dr. Kathleen O’Sullivan of San Francisco State University (SFSU) to develop a park-based module on geology for the university’s “Curriculum and Instruction in Science” course. The team selected geology as the key content area because the park has natural resources uniquely suited to teaching that subject, and because many new teachers are less confident in their content knowledge and teaching skills in geology.
As with any project, the most difficult task is the development of the essential question: “What do we want participants to know and be able to do at the end of the program?” For the evaluator, this is the fundamental piece that guides the evaluation design, dictates the formation of the evaluation matrix, influences the development of the assessment instruments, and provides the framework for reporting on program findings. (Unfortunately, too many projects develop curriculum materials in a linear fashion, starting with goals, deciding on learning activities, and adding the assessment component after the program is already designed.)
Using the principles of “backward design” described in Understanding by Design (Wiggins and McTight), the planning team made a deliberate effort to consistently integrate the planning components of developing the essential question, identifying desired outcomes for the participants, determining acceptable evidence that the participants had achieved the results, and planning the learning experiences. To visualize this process, imagine four interconnected circles, each circle containing one of the key components listed above. This interconnection meant that, when making decisions about one component of the process, the planning team would also examine the impact the decision might have on other elements of the program.
One of my major roles as evaluator in this program was to ask, “What do we want the participants to know and be able to do at the end of the program?” After numerous meetings, the team agreed on the essential question, “How do I recognize evidence of geological change in my environment?” This essential question encompassed the need for participants to understand the processes involved in geological change, to realize that they need to back up their conclusions with valid evidence, and to be able to apply this knowledge to their local communities as well as to the selected Golden Gate National Recreation Area site.
Another key decision involved determining the location for the field investigation. Though the park staff pushed for a more “natural” site in the Marin Headlands, a consideration of various logistical issues eventually resulted in a compromise site, the Presidio’s Crissy Field. Working in the Crissy Field site’s favor was an easily accessible (though “unnatural placement”) of riprap rock; reduced travel time for the pre-service teachers from the university; and proximity to the Crissy Field Center, where participants could complete the “classroom” portion of the program.
When the group’s energy turned to the learning experiences, everyone agreed that the program should focus on geology content and provide participants with inquiry-based learning experiences. Once these two curriculum elements had been agreed upon, there were lengthy discussions regarding the depth of content, the amount of content to be taught prior to the field investigation, and the level of inquiry (teacher directed vs. student directed) during the investigative phase of the program. Finally, the group decided to provide limited content prior to the field investigation, to design a more student-centered investigation at the riprap site, and to provide handouts and increasing amounts of resource materials as the participants moved from the outdoor experience to the classroom experience at the Crissy Field Center. (See Planning Place-based Inquiry).
After months of planning, the group produced a six-hour program (two three-hour sessions administered one week apart) in which a balance of content and investigative experiences were included. The program module, titled Rockin’ in the Riprap, provides participants with an opportunity to study global geologic changes that formed the San Francisco Bay Area’s landscape, to learn how to conduct a geologic trail exploration that examines global forces recorded in local rocks, and to learn strategies that challenge learners to use observation and critical-thinking skills to speculate about possible future geological developments.
Program Instructional Components
The first stage of the program, an in-classroom activity in Dr. Kathleen Sullivan’s class, provided the teacher candidates with an introduction to where they would be going in the park, what they would be looking at, and how to make geologic observations. A general investigatory question was posed regarding geological change in the environment, and the teacher candidates also participated in a brief exercise in which they used a grid to analyze sand samples; this familiarized them with a technique they would use when they went to the park for the field study the following week. In later versions of the program, the participants used the “magic windows” that were developed later in the program’s evolution.
During the field study component, park staff showed the teacher candidates how to take an outdoor area and create an investigation for students. Working in teams, the group identified rocks in a riprap area and discussed where they came from and why they were there. The same day, they participated in a lab activity, during which park educators modeled how to integrate outdoor and indoor activities. Both the park staff and Kathleen Sullivan observed that the activities and materials developed were sufficiently compelling to inspire the students to start working spontaneously, with little instruction. This demonstrated that carefully developed materials can lead to student-centered learning. The park educators also demonstrated how a similar investigation could be created in any urban area, making it applicable to teachers who may not be able to take their students to a national park.
After numerous planning meetings and clarification of the essential question for the geology module, I constructed an evaluation matrix. The evaluation matrix consists of three columns: (1) educational outcomes based on the stated goals of the program, (2) assessment tasks and instruments needed to evaluate the outcomes, and (3) how the assessments will be used to measure the achievement of the goals. In creating the matrix, the program goals were restated as educational outcomes that could be more easily measured.
The next step was to develop a list of all the tasks to be completed throughout the evaluation process. For example, when designing a geology curriculum module, it is important to make sure that the curriculum has explicitly stated goals that are in alignment with content standards; teaching strategies that represent best practices; transitions between learning activities to reinforce the essential question; opportunities to check for understanding throughout the lessons to ensure that participants grasp key concepts; and connections to the participants’ local environment. Then the evaluator systematically determines whether the curriculum includes each component and provides continual feedback to the program developers.
Using the evaluation matrix, the evaluation plan focused on three types of assessment instruments: pre- and post-program surveys, concept maps, and interviews with focus groups. Each assessment type provides a different piece of the overall picture of the effectiveness of the program. Throughout numerous implementations of the program, the assessments were modified based on participants’ feedback and changes in the goals of the program.
The focus of the pre- and post-program survey assessments was to measure three primary factors: (1) pre-service teachers’ knowledge about the availability of teaching and/or educational resources in the national parks, (2) pre-service teachers’ content knowledge in geology, and (3) pre-service teachers’ confidence in teaching geology to students. These factors were included in a survey instrument that pre-service teachers completed at the beginning and end of the program. By comparing participants’ pre- and post-program survey responses, the team obtained valuable feedback on the whether the program was successful in achieving its stated goals.
In addition to survey items asking pre-service teachers about their level of content knowledge in geology and their confidence level in teaching geological concepts, I designed a concept map assessment as a tool to measure changes in the participants’ geology content knowledge during the program. After a brief training on how to construct concept maps, participants are then provided with a list of questions pertaining to geology; as part of the assessment, participants are asked to identify key geology concepts and to construct a web, a visual representation of how the concepts are connected to one another, including explicit statements explaining these relationships. Each concept is inside a circle with lines connecting the circles, and phrases are written on the lines to describe the relationship between the concepts. Ideally, concept maps depict a maze of interconnections between the concepts and reflect accurate statements about the relationship between them. An expert in a content area will produce a much more detailed concept map with more connections per concept than will a novice. The goal of this assessment was to compare participants’ pre- and post-program concept maps to gather evidence of changes in content knowledge.
At the conclusion of the program, local teachers gathered for a reunion meeting, during which they were asked questions about how the program helped them in specific content areas, whether they used any of the program components with students, and if they modified the materials to fit the needs of the students or the content being taught. While in-person classroom visits during which the teacher’s methodology can be observed result in a more accurate measurement, arranging a meeting for a focus group discussion enables the evaluator to gauge how much of the program is actually being used in classrooms.
Developing assessment instruments for this project presented two main challenges. First, measuring changes in participants’ content knowledge is always tricky. This is a difficult task when evaluating a week-long professional development workshop, and a program with just two three-hour sessions adds an additional complication. The second challenge is determining whether or not the PARK program makes a difference in the ways pre-service teachers use teaching strategies and investigative skills in their classrooms to promote scientific literacy and engage students in environmental issues within their communities.
Assessments and Preliminary Findings for Pre-Service Teachers
Eight statements comprised the survey assessment tool. For each statement, participants circled whether they strongly agreed, agreed, disagreed, or strongly disagreed with the statement. The second part of the assessment asked participants to think about some probing questions, to identify key concepts in response to the questions, and to construct a concept map. An identical version of the assessment was administered to the participants at the end of the program. See the initial pre- and post-program assessments.
After carefully examining the participants’ responses on the pilot assessments, I revised some items on the survey and eliminated the construction of a concept map. Although concept maps have proven to be a useful tool for assessing participants content knowledge (Ruiz-Primo, M.A., et al. 2001), their use for this project encountered two main problems: (1) the participants were not able to complete the concept map in the limited time provided, and (2) it was difficult to attribute changes between the pre- and post-program concept maps to what participants had learned in the PARK experience. Because most of the participants had very limited geology background and were uncertain of the content, there were no observable consistencies between the pre- and post-program concept maps.
Knowing that a key goal for the project was to measure changes in the participants’ geology content, I revised the survey to include a number of open-ended content questions. The new assessment focused on whether pre-service teachers could identify characteristics used by a geologist to identify rocks, explain the types of rocks they would expect to find in the San Francisco Bay Area, describe the type of tools and sampling techniques used to identify rocks, and provide a general description of geological processes. (Assessment 2). The same assessment was administered as a pre- and post-program survey. This assessment tool proved to be too open-ended and did not provide valuable information.
Working with park staff, we developed new items to try to measure participants’ geology content knowledge. The items consisted of multiple-choice, matching, and selecting the correct answer. The content questions were very specific and were intended to focus the participants on the geology of the San Francisco Bay Area. The modified pre-program survey contained eight statements to which the participants indicated their agreement level, an open-ended question to learn more about the participants’ geology background, five geology content questions, and one open-ended question to determine the participants’ previous knowledge about the origin of rocks in the San Francisco Bay Area. The post-program survey asked participants to respond to: (1) eleven statements assessing the PARK program, including whether their understanding of geology and their confidence in teaching geology concepts had improved, (2) a request to rate the educational effectiveness and the inquiry level of the program, (3) an open-ended question relating to the program’s essential question, (4) geology content questions identical to the pre-program survey, and (5) an open-ended question that asked participants to identify which components of the PARK program they might use in their teaching.
The post-program survey data completed by 13 participants in November 2005 revealed that the PARK program was well designed, improved the participants’ geology content knowledge, and increased their confidence in teaching geology. Most of the participants (92 percent) selected “agree” or “strongly agree” on the eleven statements, with two clear exceptions. On the statement, “The field experience seemed artificial,” 11 participants disagreed and 2 strongly disagreed, relieving the project managers’ concerns about using the riprap site. Four participants disagreed with the statement, “I felt successful in completing the tasks and worksheets.” Based on this feedback, the program team revised the task and simplified the directions on the worksheet. Eight-five percent of the participants rated the educational effectiveness and level of inquiry “high” for the project and the balance gave a middle rating. All participants indicated that they would be able to use or modify components of the PARK program in the future. The most evident finding was that the program was providing many of the participants with their first exposure to geological concepts.
The final pre-program survey assessment version retained the eight statements to which participants indicated whether they strongly agreed, agreed, disagreed, or strongly disagreed and dropped all the content-specific questions. Instead, the survey asked participants, guided by the essential question, to respond to the following prompts: “What does geologic change in your environment mean to you? What are some examples of evidence of geologic change in your environment?” The final version of the post-program survey retained the items of the earlier version, with the exception of the geology-content questions, which were deleted.
Project Findings for Pre-Service Teachers
A total of 36 SFSU pre-service teachers have participated in the Rockin’ in the Riprap program and completed the survey. The breakdown of participants by gender is 22 males and 14 females; by ethnicity, 1 African American, 8 Asian, 16 Caucasian, 3 Latino, 6 Mixed, and 2 who declined to identify themselves. During the implementation of the program, 20 of the participants were working in their student teaching assignments, 14 were teaching courses without having yet completed their credential, and 2 did not file a report.
The assessment of the PARK program by pre-service teachers focuses on curriculum, inquiry-based field experience, pre-service teacher learning, national parks as resources for teachers, and strategies to facilitate teaching geology to students. In terms of curriculum, all the participants agreed or strongly agreed with the statements that the on-campus introduction prepared them for the field experience, the essential question for the geology module was clear, the curriculum and instructional ideas were excellent, and the materials (props, rocks, handouts, etc.) were informative.
Many of the pre-service teachers indicated that they had limited first-hand experiences with inquiry-based learning. All of the pre-service teachers agreed that the program’s field component increased their level of interest and motivated them to learn the material. One participant summarized it well: “Being in the field, touching, feeling the sun, the wind, being near the water made me feel more engaged than sitting in the classroom.” Other participants stated the importance of “using what we see or experience in our environment to understand specific geological principles.” Most of the participants commented on the program’s sequence —they “liked the combination of field work and then coming into the lab to examine results.” A few participants expressed appreciation that the program “allowed us to pose our own explanations for what we were observing without the pressure of being right.” Finally, some of the pre-service teachers stated that “engaging in an inquiry-learning experience helped me realize for the first time how important it is to let students try to figure out how the rocks formed and how they got to where they are.” This statement alone indicated to the evaluator that the project was a tremendous success. All too often, due to time constraints and the pressure for students to achieve on standardized tests, teachers decide to directly teach science concepts instead of developing learning situations in which students can experience the scientific process. When teachers have an opportunity to engage an inquiry-based learning and themselves experience the learning gains, they are more likely to expose their students to these types of learning environments as well.
When asked to comment on the effect of the program on their own understanding of geology, most of the participants indicated that this was their first real exposure to geology content, that they had adequate opportunities to present ideas, and that they had to provide evidence to support their conclusions during the discussion of findings. All but two of the participants stated that their understanding of evidence for geological change improved after participating in the PARK program. All except three of the participants were more confident in their ability to teach students about plate tectonics and geological processes after participating in the program. In each case, the participants who did not agree with the statements above indicated “no change” in their understanding of geological change or in their confidence in their ability to teach students about geological change. Six pre-service teachers had an extensive geology background prior to attending the program, so the fact that some of these teachers felt that their understanding of evidence for geological change had improved as a result of the program, and they were more confident in their ability to teach students geological concepts, was a positive outcome for the project.
All the pre-service teachers agreed or strongly agreed that they had learned newthings about the educational resources available at a national park after participating in the PARK program. This experience also enhanced their interest in using field trips as educational supplements to their classroom teaching. They expressed interest in finding out more about how to bring students on this field trip, and the park staff offered pre-service teachers the opportunity to apply for funding required for buses to bring their students on a field trip. One participant summarized it well with the statement, “I think it was effective for my own education to know how much students can get out of a well-planned and -executed field experience in terms of appreciation for environment and others.” Another participant concluded, “I believe the PARKProgram would be useful for all teachers as a resource to teach students about science, history, and sustainability.” The pre-service teachers indicated that participation in the PARK program increased their awareness of and interest in informal science education and modeled new and effective teaching strategies in the classroom and in the field.
Although only a few of the pre-service teachers expected to teach geology, many indicated they planned to use and/or modify instructional materials from the project. All the teachers liked the “Cookie Tectonics” activity, in which an Oreo cookie is used to demonstrate subduction along continental plates, and “Edible Geology,” which uses three different types of candy (jelly beans, Jolly Ranchers, and gum drops) to describe the formation of various rock types.(See Teaching Techniques and Tools section). Most of the teachers commented that these activities helped strengthen their understanding of these geological concepts. Likewise, all of the teachers, regardless of subject area, were impressed with the vocabulary review activity called “I have tectonics.” All the pre-service teachers thought this was an inventive technique for students to either learn and/or review scientific vocabulary. The teachers expressed interest in using this activity, but expressed a need to modify the vocabulary terms.
Assessments and Findings for In-Service Teachers
A total of 46 in-service teachers (8 males, 38 females) completed the Rockin’ in the Riprap program in a six-hour block during the ASSET workshop hosted by SFSU in the summer of 2005 and 2006. A breakdown of teachers by ethnicity was 1 African American, 8 Asian, 16 Caucasian, 9 Latino, 6 Mixed, and 6 who declined to report. The teachers were grouped into six categories based on the number of years of teaching. There were 15 teachers with one to five years, 8 with six to ten years, 5 with eleven to fifteen years, 6 with sixteen to twenty years, 5 with more than twenty years, and 6 who declined to report.
These experienced teachers provided the program team with an opportunity to receive extensive feedback and to gather information that would help improve the program. I used the eleven statements from the pre-service teachers’ post-program survey, again asking teachers to respond with strongly agree, agree, disagree, or strongly disagree. Teachers were also asked to describe how we might improve or modify the program overall as well as to provide suggestions for improving each section of the program (i.e., introduction, field experience, lab activity, etc.).
Interestingly, when we asked the experienced teachers to agree, strongly agree, disagree, or strongly disagree with the list of statements, they agreed or strongly agreed with all of the statements except the last three. On the statement, “This experience has enhanced my interest in using field trips as an educational supplement to my classroom teaching,” four teachers disagreed. When asked about their responses later in the day, they indicated that at their particular schools, field trips were absolutely forbidden. Three teachers also disagreed with the statement, “I would like to know more about how to bring my students on this field trip.” The project team anticipated that more of the in-service teachers would disagree with this statement because most of the teachers were from other states. Finally, more of the experienced teachers (10) agreed that “The field experience seemed artificial.”
In the open-ended responses, I anticipated significant constructive criticism from these seasoned teachers but was surprised to see that many of their comments focused on how the program enriched their knowledge or how they would use the curriculum materials in their course. The following quote is typical of the overall response related to an increase in their geology concept knowledge: “This is my first field experience with rocks, and it helped me learn about classification and physical landscape features.” Responses related to the inquiry-based portion of the lessons included: “Other than a very brief introduction, the program was based on exploration and observation with explanation reserved for the end of the lesson”; “This program provided lots of opportunity to ask my own questions and was teacher guided but student centered”; “We were given plenty of opportunity to engage when we observed the rocks in the quadrant, and we were given a chance to think about our own explanation”; “I like the plan of starting from observations in the field and then moving to classification and explanation”; and “Hands-on exposure followed by discussion and further inquiry followed by the vocabulary activity and interest of educators made me give this activity a +5 on educational effectiveness and +5 ranking inquiry.” Other quotes referred to how the experienced teachers planned to modify various elements of the program, such as “I plan to use the vocabulary review activity and the ‘cookie simulation’ in the future. I would modify them slightly to fit local geology in Colorado. I also plan to modify the lesson on rock types. Thank you for putting this all together!” and “The PARK program is a very good example of an inquiry-based lesson. I’m going to modify my teaching strategy using this model.”
On June 23, 2007, the project team hosted a Marin Headlands reunion day for thirteen former pre-service teachers; Carlos Ayala, a science faculty member from Sonoma State; and Larry Hovath, a new SFSU faculty member. The beginning teachers talked with Kathleen about their current teaching positions, provided feedback to the evaluator on how they used or modified the materials from the program, and engaged in a new inquiry-based investigation focusing on Kirby Cove pillow basalt, in the Marin Headlands, an authentic location for the Franciscan Complex.
Two of the thirteen teachers attending the reunion day have not yet completed their teaching credential, and one explained that she is teaching biology.
Of the remaining ten teachers, two indicated that they had a degree in geology prior to attending the program and said that as a result of the program, they had increased their “knowledge of the local geology and now feel more confident to teach geology to my students.” These teachers also indicated that they used the “I have tectonics” and “Cookie Tectonics” as culminating activities with their students. One teacher noted that “the cookie activity was a big hit with my kids.” The other wrote, “Although the program did not greatly impact my own content knowledge, it greatly impacted my perspectives on how to teach the content.”
The remaining eight teachers talked about how the program “helped me think about creative ways to teach the geology content.” A few indicated that they did not use the program materials last year, but hope to use them in their second year of teaching, as they now have a better idea of how to pace the teaching of the curriculum over the entire year. One of these teachers observed that when he took Dr. O’Sullivan’s course and attended the Rockin’ in the Riprap program, he had no idea that he would be teaching geology the following year. He expressed regret that he didn’t pay more attention to the program and also indicated that he had arranged with park staff to bring his students to experience the program last year, but had to cancel the trip due to time constraints with other faculty members at his school.
All the teachers stated that they would or already had recommended this program to other teachers, typically, “I would definitely recommend this program. I believe students can learn a lot by actually going out there and studying the rocks,” Another participant said, “Yes, I would recommend the program to any teacher with geology as a part of their curriculum because it covers the content with local and accessible features.” Still another participant wrote, “Yes, it’s a fun and informative experience. Provides useful resources, sensitive to participants’ needs and feedback, and is well organized.” Finally, a teacher commented that the program connects to the California geology standards and provides opportunities for teachers to network with other teachers.
When asked to indicate the resources that would help them as teachers, they responded in each of three categories: information, resource materials, and additional curriculum. The teachers specifically asked for information on how to plan a field trip and asked how to go about having NPS rangers make presentations in their classrooms. They wanted resource materials, including transparencies to make “magic windows” and a supply of rocks to use with their students. (Park staff let them know that traveling trunks—containing local rocks with permits, geological maps of the area, and an assortment of other resources that can be loaned to teachers during the academic year—are available.) Teachers also requested a three-day curriculum that could stand alone without a field trip to the park and with suggestions for extending the program. When park staff described the resources that were scheduled to be made available online, all the teachers were excited and wanted to know the site’s URL.
Much of the success of this project can be attributed to the development of a strong partnership between the National Park Service staff and Kathleen O’Sullivan at SFSU. Clear communication, a good sense of humor, and a commitment to make the project work were key ingredients to strengthening the partnership. Another essential component for the project was the work of the project director, Lynn Fonfa. Under her guidance, the development team continually communicated via meetings, the telephone, and e-mail to confirm recent decisions and discuss next-steps. The success of the project can be attributed to Lynn’s ability to know when to push and when to give the group some thinking time. She did a tremendous job of keeping everyone on task, understanding the political constraints of each institution, summarizing decisions, asking questions, and gently pushing the group to stick to deadlines. After the retirement of the primary faculty member, Kathleen O’Sullivan, Lynn worked with the university to continue thepartnership with the help of an interim faculty member, Dakota Swett and is currently developing a relationship with the newly appointed faculty member for the course. Any group who wants to develop a lasting partnership will need to focus on selecting a project director who can devote a considerable amount of time to fostering the implementation and growth of the partnership.
Everyone was generous with their expertise in the development of the curriculum module. Roxi Farwell, an exceptional park educator, shared her experience of developing educational materials in the park and provided a number of interactive teaching strategies. Will Elder, PhD, NPS interpretive ranger and park geologist, helped ensure the accuracy of the content and developed a Powerpoint presentation to introduce pre-service teachers to the geology of the San Francisco Bay Area. Kathleen O’Sullivan, PhD, SFSU science methods course faculty, focused the team on the needs of pre-service teachers and the importance of inquiry-based learning. Lynn Fonfa, NPS Education Specialist, M.Ed and project director, kept the group moving forward by asking us to make decisions and to avoid getting bogged down on a specific aspect of the curriculum. Susan Schultz, PhD, project evaluator, pushed the group to clearly state the essential question for themodule, developed assessment instruments, and provided the group with formative feedback from the participants throughout the project.
Use of the “backward design” approach was key to this module’s success. This approach includes explicitly stating the essential question; determining sufficient evidence that a participant has achieved the goals; and creating alignment between the goals, the content, the teaching strategies, and the assessments. Other important components included developing an inquiry-based investigation; piloting activities and being willing to revise, modify, and/or delete an activity; listening to the feedback of the participants; and focusing on a few key concepts instead to trying to cover too much material in a short period of time.
During the original development of the module, the team focused on content and the specific rock formations of the San Francisco Bay Area. Unfortunately, the majority of the pre-service and in-service teachers participating in this program had either limited or no background in geology. Those few with geology backgrounds were grouped together in an attempt to keep them from “giving away” answers to the other participants. It would be interesting to deliver the program to a group of geology teachers and to gather feedback from them about the effectiveness of the program.
For those planning to develop a curriculum module, our first suggestion is to bring together a group of educators with different perspectives, as the variety will enrich the final product. Using the “backward” approach will facilitate the process of developing the curriculum and maintain focus on the goals and what the participants need to know and be able to do at the completion of the program. An essential ingredient is to ensure that project goals, the content to be taught, and the strategies used to teach it are in alignment.
Whenever possible, we encourage those doing the planning to think about how to make the activities more student-centered, giving the participants opportunities to pose their own questions and find evidence to support their conclusions. As previously stated, pre-service teachers found the program’s inquiry-based learning element beneficial for their own content learning, and it also helped them understand the importance of allowing students to figure things out for themselves. Finally, plan to pilot and revise the curriculum numerous times, using feedback from participants.
As described earlier, participant feedback was critical to the evolution of the assessment instruments throughout the project. The formative assessments guided the development team in improving the curriculum materials rather than simply summarizing the findings at the end of the program. The project also benefited from eliciting feedback from experienced teachers as well as from the pre-service teachers, as each group focused on different aspects of the program. Pre-service teachers were most interested in collecting information and gathering the tools to improve their teaching, whereas the experienced teachers’ comments focused on what they learned from the experience and how they would modify the materials to match their instructional style.
When developing assessment instruments, focus on the primary goals of the project and make sure there is a match between what kinds of answers are needed and what information needs to be gathered. In order to identify potential problems, it is essential to pilot the instruments with a sample of people similar to the potential study group. Don’t be afraid to change the instruments if people are not responding as anticipated. When this type of situation is encountered, it is always a good idea to interview a few people to try to identify the problem with the item. And finally, whenever possible, share the findings from the assessments with the development team so they can use the information to modify the curriculum or other aspects of the program as appropriate.
Last updated: February 28, 2015