This research is supported in part by funding from the National Science Foundation (NSF) project number 9731306. The views expressed here do not necessarily represent those of the NSF.

This report is part of an extensive evaluation of the first year of a five-year Local Systemic Change (LSC) project called TAPESTRIES that is funded by the National Science Foundation (NSF). The purpose of this project is to develop comprehensive school science programs through the sustained professional development of a critical mass of K-6 teachers in a large, urban school district and in an adjoining suburban school district. Teacher-based leadership and other support structures are implemented as teachers use inquiry-based science curriculum and instructional strategies. Teachers are involved in professional development activities during intense summer institutes and academic year sessions.

The general philosophy of LSC project partners is that teachers need and desire sustained professional development to implement newly adopted curriculum materials, and staff development action plans are needed to reach all teachers and students in the districts. Each school district made a commitment to adopt new K-6 science activity-based curriculum to support their course of study.

Fullan and Stiegelbauer (1991) observed that systemic change involves many complex, interconnected factors. However, they believe that reformers must begin with one major factor and assume that other related components will be changed as a result of the impact made on that major factor. It is becoming increasingly clear that teachers are important to the success of science education reforms as experts contend that teachers, among others, play a key role in system-wide school change processes (Fullan & Miles, 1992). According to the National Science Education Standards, "The most important resource is professional teachers" when evaluating science education programs (National Research Council, p. 218).

Furthermore, Bybee (1993) is convinced that...

the decisive component in reforming science education is the classroom teacher....unless classroom teachers move beyond the status quo in science teaching, the reform will falter and eventually fail. (p. 144)

In a review of the literature regarding effective teacher-based school change, Haney & Lumpe (1995) noted that teachers' beliefs regarding curriculum, instruction, and assessment must be identified and clarified prior to, and during professional development activities. In order to ensure successful classroom implementation of innovative materials, support structures should be implemented (Valencia & Killion, 1988; Shroyer, 1990). In describing the LSC program, Weiss, Montgomery, Ridgway, and Bond (1998) stated,

The literature on effective staff development emphasizes the importance of establishing a professional development culture where teachers can explore content and pedagogy in a collegial, risk-free environment. (p. 6)

The project examined in this report was designed upon the premise that teachers should be the central focus of a local reform effort and that they should be provided with a supportive context for reform.

Luft (1999) recently noted that a large sum of government money is being spent in the United States to provide inservice for science teachers and that many of these funded programs rely upon possibly ineffective professional development strategies. Because of this great expenditure, there exists a need to examine current teacher professional development programs.

The primary goal of this report is to describe the context and support structures involved in the implementation of the first year of a five-year, professional development program designed to train elementary teachers to use exemplary science curriculum materials. With respect to this primary goal, and as part of a larger evaluation plan described later, the purpose for this study was to examine support structures and teacher beliefs during the first year implementation of the professional development program.

This report focuses on a Local Systemic Change project that involves a partnership between two regional universities and two public school districts with the purpose of achieving a system-wide comprehensive reform of K-6 science education. One district is a large, urban district in a Midwestern city and the other district is a suburban district in the surrounding metropolitan area. Both districts have relatively large proportions of students below the poverty level and their students tend to score at or below state averages on the state administered science achievement exams.

Prior to developing the grant proposal, teacher needs assessments were conducted within the participating school districts. Five hundred K-12 teachers were randomly selected to complete a questionnaire focusing on the strands of current science education reform. Teachers were asked to indicate their perceived need for staff development within these strands. The teachers indicated that their greatest need for training were within the following strands: teaching the nature of science, incorporating science/technology/society applications, assessing students, using hands-on activities, and teaching specific science content (Czerniak & Lumpe, 1996).

The results of these needs assessments led to the development of several Title II/Eisenhower funded projects designed to assist teams of teachers develop a new science curriculum scope and sequence that was aligned with the state curriculum model. While developing the new scope and sequence, the teacher teams were exposed to a variety of kit-based science curriculum materials. Materials that were aligned with the new scope and sequence were selected by the teacher teams and forwarded to the board of education. The local boards of education from both participating districts officially approved the science scope and sequence and the curriculum materials for use throughout their entire district. The above activities set the stage for comprehensive teacher professional development.

Based upon the perceived needs of the teachers and following-up on previous externally funded activities, the following five goals were delineated for the project:

1) To develop, support, and utilize a cadre of Support Teachers along with other sufficient support structures in order to provide local leadership for the implementation of effective science programs.

2) To provide effective and sustained professional development for all K-6 teachers of science in the participating school districts.

3) To implement quality inquiry-based science curriculum and instruction in classrooms that is consistent with local, state, and national recommendations so that all students receive opportunities to become scientifically literate.

4) To coordinate curriculum, classroom practice, and student assessment with the district adopted science courses of study and statewide assessments.

5) To enhance teachers' science content knowledge.

These goals were specifically chosen to be aligned with national and state standards for science education.

Over the history of the project, the majority of elementary teachers in the two districts will attend two-week summer sessions staffed by scientists and science educators from both universities. Scientists will also be involved in classroom activities and monthly meetings with teachers during the academic year. Each teacher receives at least 100 hours of professional development to enhance the implementation of science curriculum materials that have been selected by both districts. Principals participate in a one-day retreat with follow up sessions throughout the year. Each school will develop an action plan that involves the parents and the community.

Support Teachers from the local schools, who were jointly selected by school administrators and project staff for their knowledge of science content, are given full time release from teaching. They receive graduate level leadership training in action planning, mentoring other teachers, and inquiry methods. The Support Teachers are assigned to work in teams with scientists to provide assistance to the classroom teachers.

Four K-6 science curriculum programs are the focus of the professional development: Full Option Science System (FOSS), Science and Technology for Children (STC), Insights, and Scholastic. These materials have been officially adopted by the school districts and have been aligned with the state science standards. The first three programs were developed using National Science Foundation funds. They use ideas matching current reform trends in science education such as integrated content chosen around life, physical, and earth/space science; a hands-on, inquiry approach based on learning cycles; a focus on broad themes in science; and holistic student assessment. Scholastic was developed and published by a private corporation.

The two-week summer professional development sessions are primarily geared toward providing the participating teachers with first hand experiences with the content and pedagogy of the curriculum materials. Each teacher attends sessions for the specific curriculum materials they are assigned to teach based on their district adopted materials. The teachers then implement these materials in their classrooms during the academic year following their participation in the summer sessions.

The core evaluation plan developed by Horizon Research, Inc., which was designed to evaluate all Local Systemic Initiatives funded by the National Science Foundation, is incorporated into the evaluation plan of this project (Horizon, 1997; Weiss, Montgomery, Ridgway, & Bond, 1998). Horizon developed the following core evaluation questions:

1) What is the overall quality of the professional development activities?

2) What is the extent of school and teacher involvement in the activities?

3) What is the impact of the professional development in teacher preparedness, attitudes, and beliefs about science teaching and learning?

4) What is the impact of the professional development on classroom practices in science?

5) To what extent are the district and school contexts becoming more supportive of the vision for exemplary science education?

6) What is the extent of institutionalization of high quality professional development systems in the districts? (Weiss, et al., 1998, p. 2).

This report focuses primarily upon Research Questions 3 and 5.

Most of the evaluation questions are applied to each cohort of teachers as they participate in summer institutes and academic year activities. With many of the evaluation strategies, baseline information was collected before project activities began. This information was used to assist in designing project activities and for comparison purposes. A variety of data types, both qualitative and quantitative, were used to triangulate the validity of the data. Data sources included teacher and principal questionnaires, academic year classroom observations by trained observers, summer institute observations, teacher interviews, project team and Project Director interviews, student interviews, support teacher interviews, reflective journals, district action plans, and teacher belief instruments. Evaluation team members working for the project received extensive training from Horizon in use of the observation protocol. The teacher questionnaires, principal questionnaires, and the interview protocols were all developed by Horizon Research, Inc. specifically for the NSF Local Systemic Initiative projects (Horizon, 1997). Items from the teacher questionnaires were combined by Horizon into composites using factor analysis. The belief instruments included the Science Teacher Efficacy Belief Instrument (STEBI) (Riggs & Enochs, 1990) and the Context Beliefs About Teaching Science (CBATS) instrument (Lumpe & Haney, 1998). The CBATS instrument results in scores for two constructs, enabling and likelihood beliefs. A variety of sampling techniques including purposeful and random selection were employed. Horizon randomly selects subjects, classrooms, and professional development sessions for data collection.

The program evaluation analysis flowchart outlined by Collins and Spiegel (1998) was utilized in data analysis. This analysis technique, which was designed to assist evaluators analyze data from large teacher professional development programs, represents a type of analytic induction (Goetz & LeCompte, 1984). In this flowchart strategy, raw data sources receive some action such as building lists, organizing data, and confirming or removing components. These actions result in intermediate products such as refined lists or essential components. The use of multiple data sources, triangulation of data from the multiple sources, and searches for counterexamples to the assertions provided a level of trustworthiness to the analysis (Mathison, 1988, Maxwell, 1996). As a result of the initial analysis, four themes (see below) emerged from the intermediate products. These themes were then tested against the data sources to determine their legitimacy. This technique results in grounded theory (Strauss & Corbin, 1994).

Four themes related to effective reform processes emerged from the analysis. These themes included 1) support structures; 2) science curriculum materials; 3) science content; and 4) teachers' beliefs. For each theme, background information is provided and results supporting each theme are offered.

Throughout the initial implementation of this project, it became increasingly clear that support structures would be critical to its success. The NSF requires that support structures be built into the project. Horizon included some of these structures in their questionnaires and they developed composite constructs for parental support, resource availability, administrative support, and collegiality. Figure 1 displays baseline composite scores for this project.

Prior to receiving any professional development from the Project, a large number of teachers scored low on the Resource Availability Composite possibly indicating potential problems with time to plan, work with other teachers, and participate in professional development experiences. On the Principal Questionnaire, school administrators agreed with the teachers about the lack of support. Formal classroom observations revealed that teachers in the two districts lack resources, space, and room to teach science.

Perhaps the most alarming composite score in the Teacher Questionnaire was Parent Support. A majority of teachers scored fairly low on this composite. This lack of parent support was evident in a remark made by one school administrator,

This may not be the best NSF answer, but it is the truth. The [name] Local Schools parent community is somewhat passive; they really don't show a lot of trust or apathy. We have had some school science nights with good attendance. They're not really positive or negative, they're just there.

Prior to the start of the project activities, it became clear that the lack of focused support for the teaching of science in the two school districts should be a prime focus. For this project, three primary avenues designed to achieve the necessary support were included, the use of Support Teachers (STs), science materials management, and administrative support.

Most of the LSC projects funded by NSF include the use of teacher leaders similar to the Support Teachers used in this project. The STs are assigned to work with a group of teachers from one or more elementary school buildings. As planned in the original proposal, the Support Teachers are carrying out the following tasks during the academic year:

1) Visiting their cohort of teachers on a biweekly basis;

2) Discussing needs-based issues with the classroom teacher;

3) Providing assistance for obtaining and scheduling curriculum materials;

4) Providing strategies for teaching science (such as inquiry, integration, and cooperative learning);

5) Providing science content background information (if necessary, with the help of the scientists);

6) Assisting with classroom and district science performance-based assessments;

7) Team teaching with classroom teachers in order to model strategies; and

8) Peer coaching the classroom teacher.

During an interview, the Support Teachers viewed their role as the following:

1) Help teachers use the science curriculum materials;

2) Provide peer support;

3) Help teachers get started using the science curriculum materials;

4) Promote enthusiasm, interest, and comfort in teaching science;

5) Help teachers find resources and materials; and

6) Provide public relations for the program.

The STs received over 200 hours of training regarding their leadership roles in the project. This training is part of a 2 year graduate level course offered at the two participating universities. The course, meeting every other week, focuses on the following topics: peer coaching, interpersonal skills, state and performance testing, national and state standards, systemic change processes including stages of concern, science curriculum kit use and maintenence, and grant writing. Interviews of the Support Teachers revealed that they believe the training courses are very beneficial and provide important leadership tools. One Support Teacher expressed enthusiasm about the benefits of this course by stating,

It was really neat over the last several weeks that we've met with the (leadership) class. We've gone through some team building programs and initially, we've said if somebody's standing in front of you complaining, my old self would have thought 'that's too bad.' But now I respond with 'I understand.' Hopefully, the rest of us are using the kind of techniques that (the instructors) are attempting to teach us.

This kind of training has the potential to prepare the Support Teachers to be effective in their roles as science teacher leaders in their respective buildings and appears to be critical in assisting the STs in their new roles.

As the project unfolded, several essential components regarding effective support structures emerged. These components were purposeful interactions among stakeholders and peer mentoring. It should be noted that these two components are not mutually exclusive but each will be discussed individually.

Support Structures: Purposeful Interactions Among Stakeholders

The primary stakeholders in this project include classroom teachers, Support Teachers, administrators, teacher union officials, and university faculty (both education and science faculty). This project has ensured that all of these stakeholders are involved in the planning and delivery of the program. Steps have been established to ensure effective and meaningful interactions among all involved. Some of these steps are discussed below.

Each Support Teacher has scheduled regular building level sessions with classroom teachers at the primary support buildings who participated in the summer institutes. These academic year sessions are occurring at least once a month and are lasting approximately two hours. Teachers receive stipends and/or teaching release from their districts for participation in these sessions, and they are receiving two semester hours of graduate credit for these sessions. During the academic year sessions, classroom teachers are being encouraged to try curriculum activities and seek immediate help, if necessary, from their Support Teachers. However, to supply additional support, one science educator and one scientist are assigned to assist each Support Teacher. Support Teachers are encouraged to use the science educator and scientist assigned to them as a "first call for help" whenever they cannot address a teacher's question or concern. Support Teachers are also being asked to invite the science educator or scientist to attend several monthly meetings in individual or groups of buildings. Support Teachers are receiving continuous assistance from these science educators/scientists through e-mail and telephone contact. In addition, the project has established a section of their web page for questions and answers where Support Teachers can submit questions.

Some evidence from people directly involved in the project (participants, Support Teachers, Principal Investigators) exists to indicate that there is a wide degree of administrative support. Some administrators attended summer workshops, recruited a large portion of their teachers for the summer workshops, provided extra funds and space for materials and Support Teachers, and advertised the program to their local school communities. Others show little interest in the program. Nonetheless, teachers' union has been supportive of the project from the beginning.

Signs of administrative support for the project is evident in a comment made by one of the Support Teachers:

It makes you feel like the administration is going over and above what they have to do. It's not just another science curriculum being thrown at us and we don't know what to do with it, but actually, it's like somebody is putting it out there and actually giving us resources and everybody's involved with it.

One school administrator highlighted the importance of the collaboration with the universities in the following statement:

The universities are the strongest advocates, the prime movers. They're not only providing the summer institutes but also graduate courses for the support teachers.

Evidence exists to demonstrate that school districts are making commitments to ensure the institutionalization of the program. Both districts have previously adopted a science curriculum scope and sequence that is aligned with state standards and testing. They have also adopted and purchased science curriculum materials to match their officially adopted curriculum scope and sequence. These purchases alone have amounted to over $1 million in addition to resources spent on scope and sequence preparation. The commitment by the districts to hire full time Support Teachers after federal funding ends demonstrates a willingness to ensure program success.

One school district, the larger urban district, recently hired a full time person to work with the project. His impact is already being felt as evidenced by a comment made by one of the Support Teachers,

[name] is probably the biggest help on needs and wants, when classroom teachers are having problems I think he's their psychologist, mentor, and go getter.

And another Support Teacher reinforced this view in the following comment,

We have access to [name] every day. He is available to nurture us in our jobs, to provide resources and information. He helps us to get any materials we need for the teachers. He gives us ideas for lessons, how to expand our lessons, and clarification. I really appreciate the encouragement he gives.

Support Teachers and Project Staff worked with teachers to develop district action plans. The Support Teachers have been collaborating with school administrators, parents, and community representatives on the action plans. These actions plans may include, but are not limited to the following:

1) Plans to address local, needs-based issues;

2) Details regarding evaluation of the districts' new science program;

3) Future staff development strategies beyond the scope of this project;

4) Assessments, the district course of study, and statewide assessments;

5) Maintenance and replacement of science materials using the districts' procedures;

6) Strategies to sustain current and additional stakeholders in the district science program; and

7) Pursuit of additional resources to maintain or expand the program.

These action plans have helped schools address issues relevant to their own situations. For example, at one elementary building, the teachers work with high school students from a neighboring school building to help elementary students improve their cooperation skills in order to work effectively on science lessons.

During interviews, the Support Teachers indicated that there are a variety of people who are helping them in their roles including the principal investigators, scientists, other Support Teachers, teacher's union officials, and school administrators. All of these people appear to be critical to the effective functioning of the Support Teachers.

An example of the purposeful interactions among these stakeholders is noted in the following narrative written by a Support Teacher:

A sixth grade teacher was having difficulty explaining the concept of chemical change to her class. She requested help from her Support Teacher. The Support Teacher contacted a scientist at the university who was a biology professor and project scientist. The professor and the teacher agreed on a classroom presentation and demonstrated chemical change to the students. The teacher indicated that this was a worthwhile experience for the students and noted that this endeavor was productive and demonstrates the effectiveness of the program.

Response to the Support Teachers has been overwhelmingly positive. For example, this quote from one teacher exemplifies the teachers' reactions:

Salutations, Science Support Teachers! Just to let you know that many teachers are grateful for your presence in their buildings and even the kind words that you share with them are making a difference in their lives. You have made teachers happy.

In spite of some of the positive support structures, some weaknesses have been discovered. In addition to the data sources specified by Horizon, the project team developed and administered an interview protocol with the first year Support Teachers. The Support Teachers identified the following issues that should be addressed during the course of the project:

1) Refine the materials management systems;

2) Teachers need more class time to teach science;

3) There is teacher apathy in some buildings;

4) Some teachers still use traditional teaching methods;

5) Teachers need more planning time; and

6) The focus on mathematics and reading is taking energy away from science.

One Support Teacher expressed the following concern about the culture of the district:

Our biggest problem right now is the frustration as science teachers because of the disorganization at the schools right now with science. In fact, there's so much emphasis on the student reading program that people are saying they can't teach science because they don't have time.

These concerns are being addressed in most of the actions plans described earlier. For example, both districts have materials distribution and replacement programs in place, but these are new for the teachers, and they need assistance learning the procedures for replacing their materials and obtaining their kits.

Support Structures: Peer Mentoring

One of the most positive aspects of this project has been the peer mentoring conducted by the Support Teachers. The STs have full release time to work entirely on the science program. As they spend time in the buildings, they answer questions, obtain materials, peer and co-teach science lessons, and provide general assistance to the classroom teachers. One Support Teacher stated,

[we are to]. . .help the teachers who have never taught the kits, those who have gotten the kits that maybe skipped part of it because of the unknown. For example, one teacher, the very first day came up and started complaining and thought that the kids wouldn't get anything from it. I told him that I would help him through an earth materials kit for 3rd grade. The day we were supposed to begin, he was absent and asked that I wait until his return. I replied that I could start with his sub there, then he stated that he wanted to be there. I commented to him that it sounded like he was becoming interested. So I think this was a major thing. Also, when the kids see me in the halls, they'll say, "There's the science lady, are we going to have science today?" They ask about it and I think that's a neat thing.

Regarding their mentoring, another Support Teacher mentioned,

I'm enjoying this. At first I just had one person asking questions and basically just understanding the manual. Now I've had five people who've approached me and asked for help when they're ready to start science. So just my presence there is getting people thinking about science and being more into it. The teachers know me and feel freer to ask me for help, or to get this or that, to set up or to come in and teach. Being that this is the first year, I feel it's coming along.

And another Support Teacher provided an excellent example of the impact of her efforts at mentoring,

There's a teacher in one of my schools who was convinced she could not use the kits because it meant having her students work in cooperative groups. I offered to model the kit, went into the classroom and got the students divided up and spoke about working together in pairs, started on the activity, and it went very smoothly, and she was surprised. Later in the day the same students were having some problems with some things, she decided to try pairing them up again and they were so involved from working in pairs that the discipline and management problems she was experiencing were kind of disappearing, so I think I've already made an impact.

Prior to the project, the two school districts purchased and implemented new science curriculum materials including FOSS, STC, Insights, and Scholastic. These kits are used concurrently in the districts depending on the official curriculum adoption. For example, a 4th grade teacher may be assigned two FOSS kits and two STC kits. All of these materials are kit-based in that they include hands-on materials for each lesson. Although the teachers have had some of the kits for two years, there has been little professional development for the teachers. The formal classroom observations revealed that many of the teachers were not even using the kits prior to the beginning of this project. In spite of the adoption of these kits, there was not a concerted effort to organize the delivery or use of these kits. One Support Teachers noted,

A lot of the kits are still in the boxes they came in. I think that may be the fault of the school situation because we had one little in-service. I think they need to promote it differently. I'd say a good 80% of the kits were never opened, ever.

One PI noted,

A system for purchasing/replacing materials does not currently work well. Teachers are not knowledgeable of the system for replacement materials.

Before teachers participated in professional development activities associated with the project, 63% indicated that they were quite uncomfortable with the science kits and only 10% responded with a positive comfort level. Therefore, these weaknesses became a primary emphasis in this project.

The program evaluation revealed three essential components for successful program implementation: 1) Teachers should have positive experiences with the curriculum materials prior to using them in their classrooms. 2) Interdisciplinary connections are critical for busy elementary teachers who are preoccupied with teaching other subjects that are often perceived as more important than science. 3) The quality of the science curriculum materials used impacts professional development and classroom implementation.

Science Curriculum Materials: Positive Experiences for Teachers

The summer sessions for classroom teachers focused on teaching the teachers the science content of the science curriculum kits by having the teachers participate in the activities of the kits. Just having a practical experience with the activities provided enthusiasm to use the materials. The comments from classroom teachers listed below are representative of their thoughts about how the summer sessions helped prepare them for using the science curriculum materials.

Now I understand how to do the experiments, so I'm much more likely to have the students do them!

I think the areas in which we got to do the experiments and talk about what worked and didn't work was valuable because it gave us real life experience.

I'm really glad I was able to participate. I felt so inadequate at teaching these kits. Now I feel prepared and know who to go to for help.

The kit-based approach to teaching science was new to many of the teachers yet the summer sessions appeared to help prepare them for using the kits. After attending the summer sessions, 86% of the teachers indicated that they felt comfortable with the kits, a remarkable improvement from 10% before the sessions.

Science Curriculum Materials: Need for Interdisciplinary Connections

There also appears to be a fear that teaching science will take away from teaching the basics of reading and mathematics. The state proficiency exams seem to play a large role in teachers' views about the importance of various subjects. This fear is clear in the comment from a second grade teacher:

I had never really looked at these materials before coming to this workshop. I don't have time to teach science so I never took the time. Teaching these units is really going to be time consuming and I'm still not sure if I can squeeze it into the year. I know we are supposed to but the proficiency test is not in my grade level. I think doing this would be fun for the kids but I'm still not sure how much of it I can do next year.

In an effort to alleviate these fears, the adopted curriculum materials have interdisciplinary connections and the emphasis on reading and mathematics in the two districts have caused some teachers to look for ways to use the science kits as thematic units. One first grade teacher stated,

I always thought I didn't have time to do science with my students. I was mad when we were told we had to use these kits to teach science and they tried to take away our books. I even hid some of the books that we were supposed to turn in or throw away. The kits looked like a lot of work and I just didn't have the time. Now I know that I can teach a lot of things other than just science with the kit. Science kits can also help teach math and reading.

Science Curriculum Materials: Quality of the Materials

Some of the science curriculum materials were noted as being of high quality in terms of helping students learn science concepts and involving students in science inquiry. When teachers were observed in their classrooms using the adopted materials, the quality of the teaching and learning was much higher. Students were more engaged, on task, excited about learning, and actively involved in developing deep understandings of science concepts. In fact, the new science curriculum materials have caused some teachers to change their views about the focus of science teaching from having students find the "right answer" to fostering science inquiry. One third grade teacher realized this while using the kits:

I've used this kit a couple times and this is the first time I figured out that kids aren't necessarily trying to find answers but they are supposed to be asking questions and making observations.

Nevertheless, some evidence exists to support the claim that some of the purchased science curriculum materials do not meet high standards for excellence. When teachers used the Scholastic kits, the formal classroom observations revealed that scientific inquiry was not fostered and teachers tended to follow the directions like a cookbook. One PI noted that,

The FOSS and STC kits are excellent, the Scholastic Kits are poor.

A Support Teacher reinforced this view by stating,

Thank God we adopted FOSS and STC.

The above issues cause some teachers to become frustrated with the kit-based approach. These issues will more than likely be addressed through the individual school action plans described earlier.

Prior to participating in professional development activities, the teachers indicated that they do not feel prepared to teach science (See Figure 2). This is fairly typical of elementary teachers throughout the United States. In order to address this content inadequacy, the summer professional development sessions for classroom teachers were designed so teachers learned science content as they participated in the activities of the curriculum materials they are to use in the following academic year. Scientists from the universities assisted in the delivery of these summer sessions. One of their primary roles is to assist the teachers as they have difficulty understanding the science content. A Support Teacher indicated the value of this assistance:

It was kind of nice talking with a scientist. They went a little bit more in depth so that my content knowledge was a little bit broader so that I can explain it better to the teachers who ask questions about it.

And another Support Teacher stated that it is was helpful...

just having the scientists there to ask questions and clear things up.

As a result of the program evaluation, one essential component regarding science content emerged during the first year of this project; well-planned professional development activities can help teachers' adopt broader views of the nature of science.

As the teachers were involved in the summer sessions, they began to view science from a broader viewpoint moving away from science simply as information to a more holistic image where science is viewed as a way of knowing. This shift may be a result of their experiences with the quality science curriculum materials that portray science as a dynamic, process oriented endeavor. In addition, interactions with the university scientists seemed to impact the classroom teachers' views. Several quotes from classroom teachers demonstrate the dramatic shifts in their thinking.

I was really surprised to learn that science isn't just definitions. During the last two weeks I've learned that science is a process. I never saw it that way before.

The last two weeks have been grueling. But I've learned so much science. I also learned that it is OK not to have answers. Even the scientists and instructors didn't know the answers to all of our questions. What really surprised me was that they didn't always know the answers either, and they were comfortable with it. They always turned it into another question like "How could we find out?" I learned that science isn't always answers but it is questions, too.

In the last two weeks I learned that science is all around us. I always thought science was just done in labs, or when people went out and studied a particular plant or animal. Like in those nature shows. During the last two weeks I learned that science is everywhere! Even in my kitchen and my kids' toys.

The following statements illustrate how the teachers came to view scientists:

I was shocked how friendly the scientist and the science educators were. I kind of expected that the educators might be friendly. But I thought the Scientists would be really dry and just want to lecture all the time.

I thought scientists always knew all the answers. I was surprised when they would ask each other questions. A couple times the scientist from the other room would stick his head in and they asked each other question. I thought that was weird.

For the last 15 years I've been carrying this image of scientists around in my head. The image came from my freshman science courses in college. Scientists were always kind of cold, knew all the answers, and asked questions of people to trick them and make us look dumb. The scientists we've been working with aren't like that. They are real people.

The professional development session observations and Support Teacher interviews revealed that the session facilitators possessed good content knowledge that helped the teachers learn science content relevant to their assigned kits. It was also noted that the content being learned was often connected to other disciplines. Many teachers commented that they felt more confident because they understood the subject matter content of their kits more than ever before. For example:

It has been many years since I have done anything with this subject matter and knowing the content material has greatly increased my comfort level with using this kit.

This project was designed to help enhance teachers' beliefs that their environmental context can be supportive of their science teaching (context belief) and that they have the ability to teach science effectively (self-efficacy). Examination of the data thus far revealed one essential component: Positive professional development experiences are needed to impact teachers' beliefs.

Based on results from the Context Beliefs About Teaching Science (CBATS) instrument, it was found that the teachers involved in this project generally possessed positive beliefs and attitudes about teaching science. While they believe that certain support structures enable them to be better teachers, they don't perceive full administrative support in terms of providing planning time, funding, and resource management. The professional development activities appear to have a positive impact on the teachers but coupling these activities with classroom implementation is critical for success.

Overall, these teachers believe that the environmental context can enable them to be a better teacher of science. Their positive enabling beliefs are a good sign for the project; these teachers may respond well to quality professional development programming. On the other hand, the teachers who participated in the 1998 summer sessions do not necessarily believe that these factors are likely to become a reality. They are specifically concerned about community involvement, funding, class period length, science equipment availability, planning time, their classrooms, teaching loads, class size, and parent involvement. Some of these concerns are addressed in the action plans.

Findings from journals kept by classroom teachers provide evidence that the classroom teachers are teaching science and enjoying it immensely. For example, quotes from several teachers illustrate how their confidence in teaching science has been positively impacted by the program activities.

This year I can teach science and work with these kits and know it is OK not to know all the answers. I can ask other teachers who've used the kits, my support teacher and the folks at the university. This makes teaching science a lot less scary.

I'm looking forward to teaching science this year. My husband and kids will be glad this workshop is over. They are getting tired of me coming home every night talking about what we did each day. I want my students to go home with that same kind of excitement.

They [the summer sessions] were excellent. I loved the aspect of taking the kits apart because I had attempted to teach them before and sort of float through them and do what you can. But when you take them apart step-by-step or lesson or groups of lessons, it really helps a lot and you feel more confident. It was great help.

Based on preliminary results and previous studies (Ramey-Gassert, Shroyer, & Staver, 1996), it was hypothesized that teachers' beliefs may be related to certain key background variables. These variables included years of teaching experience, number of graduate level science education courses, grade level assignment, how often science is taught, and the variety of science teaching strategies teachers use in their classrooms. In order to this hypothesis, spearman correlations were calculated between the teachers' context beliefs (CBATS instrument), self-efficacy (STEBI), and the background variables listed above. See Table 1 for these results.

It was found that science teaching self-efficacy was positively related to the number of graduate courses the teacher had taken, how many minutes per week they teach science, the variety of science teaching strategies used, and the number of science teaching methods courses taken. It was disturbing to discover a negative correlation between science teaching self-efficacy and years of teaching experience. Positive correlations between the teachers' enable beliefs and several of the background variables were found but it was also found that likelihood beliefs were not correlated with any background variables except grade level.

As a result of the evaluation of the first year of this professional development program, the following essential components were identified for the successful implementation of the systemic reform efforts:

1) Purposeful interactions among all important stakeholders in the project;

2) Peer mentoring by teacher leaders;

3) Purposeful experiences with the science curriculum materials;

4) Interdisciplinary connections to other key subjects such as reading and mathematics;

5) Adoption of quality science curriculum materials;

6) Professional development experiences that promote the nature of science;

7) Professional development on science content related to the curriculum materials; and

8) Experiences that lead to positive teacher beliefs.

The essential components listed above have been previously addressed in literature on professional development. In reviewing the literature on effective professional development, Haney and Lumpe (1995) proposed a framework that included three primary components: Planning, Training, and Follow-up.

In the Planning Phase, training should be planned to be concrete, teacher specific, and extended over a long period of time (Valencia & Killion, 1990). Leadership teams should be selected who can support the professional development efforts for a critical mass of teachers (Glickman, Hayes, & Hensley, 1992) and all stakeholders should be involved in both the planning and delivery of the program (Fullan & Miles, 1992). These components appear to be present in the project described in this report. All stakeholders including teacher's unions, community members, scientists, university faculty, and school administrators must philosophically agree and actively participate in the reform process. Any lack of involvement by any of these groups will drastically hinder the professional development.

Also in the Planning Phase, teachers' beliefs should be identified and addressed (Haney, Czerniak, & Lumpe, 1996). Positive teacher beliefs and attitudes toward science teaching provide a strong foundation for the reform process. However, these beliefs must be maintained and nurtured during classroom implementation through the use of support structures. By learning about the images of science they bring to professional development activities, we can shape those experiences to meet their needs. The results from this report collectively would lend support to Bandura's (1997) theory that experiences impact one's belief systems. As we continue to track teachers' beliefs throughout the length of this project, we expect to see Bandura's theory in action. It would also be interesting to examine Bandura's concept of collective self-efficacy when groups of people collectively develop beliefs about their abilities. Perhaps entire school buildings go through changes in beliefs simultaneously. However teachers' beliefs are addressed in professional development, it is becoming increasingly clearer that teachers' beliefs and attitudes are important in effective reform efforts. In fact, a recent report by Walberg and Lai (1999) indicated that the most effective type of professional development programs were those that involved changing teachers' attitudes and beliefs.

In the Training Phase, teachers need opportunities to engage in hands-on learning experiences, pilot activities, observe successes of others, and reflect upon their progress (Valencia and Killion, 1988; Etchberger & Shaw, 1992; Bandura, 1997). Teachers in this project were able to experience the science curriculum materials before trying them in their classrooms. They can also observe success though peers and their Support Teachers. Reflection is ongoing and fostered through building level meetings, interactions with Support Teachers and scientists, and journals.

Finally, the Follow-up Phase should include classroom assistance from teacher leaders, evaluative feedback, and revisions of the program (Valencia and Killion, 1988; Glickman, Hayes, & Hensley, 1992). While the teachers in the project described in this report are receiving extensive classroom assistance, it is too early to determine if necessary revisions are occurring.

Horizon Research, Inc. publishes an annual report that overviews the evaluation of all Local Systemic Change projects funded by NSF (Weiss, Montgomery, Ridgway, & Bond, 1998). There are currently 60 LSC projects being evaluated using the Horizon model. The results from this report can be used to compare with the overall effectiveness of the LSC projects.

Many of the LSC projects do not actively involve parents, non-LSC teachers, and teacher unions. Parents are not yet a strong emphasis in the project described in this report but this project is designed to impact all, or at least a majority of the teachers in the two districts. And the teacher unions have been very supportive and involved from the planning stages. More community involvement should occur in this project.

Weiss, et al (1998) reported that many districts are too dependent on NSF funding and there is a lack of commitment to continuing leadership after NSF funding ends. The project described in this report has extensive, tangible support both financially and administratively. Support Teachers should continue their work after funding ends as the districts have agreed to pay their salaries after NSF funding ends.

In many of the LSC projects, teachers don't have time to plan and prepare for teaching science. This finding seems to also hold true for this project. Mechanisms should be established whereby teachers could work with other teachers to plan their science lessons and reflect upon the results of their science instruction.

As seen in this report, the school district must provide ample investment in order to bring about reform. Such investment includes policy realignment, administrative support, planning time, and other incentives for teachers to use the innovative materials. The Support Teachers appear to be the glue that holds this project together. They will serve as the focal point for classroom teachers and project staff. Therefore, the professional development community really revolves around them and the classroom teachers. Successful support of classroom implementation includes time to work with other teachers, peer mentoring by Support Teachers, an understanding of the inquiry process, and the involvement of parents and other community members.

The evaluation of this project has identified several emerging keys necessary for systemic reform including support structures, opportunities to experience success, quality materials, and a strong science content focus. Professional development facilitators should contextualize these factors for their particular situation when planning systemic change.

Finally, evaluators should continue to examine the effectiveness of systemic reform efforts in science education. In addition to using student achievement as an indicator of effectiveness, other indicators such as those examined in this report should be pursued. For large, systemic projects, a variety of data sources are needed to develop clear images of the programs. Evaluators should move beyond looking just at the professional development activities and to observations of actual classroom practice. The Horizon evaluation plan for LSC projects has the capabilities of accomplishing this goal. Longitudinal studies should be conducted to examine patterns of support and sustainability over long periods of time. Factors that either inhibit or reinforce the program goals should be delineated. Hopefully, as more thorough evaluations are conducted of teacher professional development programs, we can develop more effective activities to implement and sustain systemic change in science teaching and learning.

Bandura, A. (1997). Self-efficacy: The exercise of control. New York, NY: W. H. Freeman.

Bybee, R. W. (1993). Reforming science Education. New York, NY: Teachers College Press.

Collins, A., & Spiegel, S. A. (April, 1998). A Successful Teacher Enhancement Program: The Essential Components. A paper presented at the annual meeting of the National Association for Research in Science Teaching. San Diego, California.

Czerniak, C. M., & Lumpe, A. T. (1996). Relationship between teacher beliefs and science education reform. Journal of Science Teacher Education. 7, 247-266.

Etchberger, M. L., & Shaw, K. L. (1992). Teacher change as a progression of transitional images: A chronology of a developing constructivist teacher. School Science and Mathematics, 92, 411-417.

Fullan, M. G. & Stiegelbauer, S. (1991). The new meaning of educational change (2nd edition). New York: The Falmer Press.

Fullan, M. G., & Miles, M. B. (1992). Getting reform right: What works and what doesn't. Phi Delta Kappan, 73, 745-752.

Glickman, C. D., Hayes, R., & Hensley, F. (1992). Site-based facilitation of empowered schools: Complexities and issues for staff developers. Journal of Staff Development, 13(4), 22-26.

Goetz, J. & LeCompte, M. (1984). Ethnography and qualitative design in educational research. New York: Academic Press.

Haney, J. J., & Lumpe, A. T. (1995). A teacher professional development framework guided by science education reform policies, teachers' needs, and research. Journal of Science Teacher Education, 6, 187-196.

Haney, J. J., Czerniak, C. M., & Lumpe, A. T. (1996). Teacher beliefs and intentions regarding the implementation of science education reform strands. Journal of Research in Science Teaching, 33, 971-993.

Horizon Research, Inc. (1997). Local Systemic Change 1997-98 Core Evaluation Data Collection Manual. Chapel Hill, NC: Author.

Luft, J. A. (1999). Teachers' salient beliefs about a problem-solving demonstration classroom inservice program. Journal of Research in Science Teaching, 36, 141-158.

Lumpe, A. T., & Haney, J. J. (April, 1998). Development of the Context Beliefs About Teaching Science (CBATS) Instrument. A paper presented at the annual meeting of the National Association for Research in Science Teaching. San Diego, California.

Mathison, S. (1988). Why triangulate? Educational Researcher, 17, 13-17.

Maxwell, J. A. (1996). Qualitative research design: An interactive approach. Thousand Oaks, CA: Sage.

National Research Council (1996). National science education standards. Washington, D.C.: National Academy Press.

Riggs, I. M., & Enochs, L G. (1990). Toward the development of an elementary teachers' science teaching efficacy belief instrument. Science Education, 74, 625-637.

Shroyer, M. G. (1990). Effective staff development for effective organization development. Journal of Staff Development, 11(1), 2-6.

Ramey-Gassert, L, Shroyer, M. G., & Staver, J. R. (1996). A qualitative study of factors influencing science teaching self-efficacy of elementary level teachers. Science Education, 80, 283-315.

Strauss, A., & Corbin, J. (1994). Grounded theory methodology: An overview. In N. K. Denzin & Y. S. Lincoln (Eds.), Handbook of qualitative research (pp. 273-285). Thousand Oaks, CA: Sage.

Valencia, S. W., & Killion, J. P. (1988). Overcoming obstacles to teacher change: Direction from school-based efforts. Journal of Staff Development, 9(1), 2-8.

Walberg, H. J., & Lai, J. (1999). Meta-analytic effects for policy.In G. J. Cizek (Ed.), Handbook of educational policy (pp. 419-453. San Diego, CA: Academic Press.

Weiss, I. R., Montgomery, D. L, Ridgway, C. J., & Bond, S. L. (1998). Year three cross-site report. Chapel Hill, NC: Horizon Research, Inc.

About the authors...

Andrew T. Lumpe is an Associate Professor in the Department of Curriculum and Instruction at Southern Illinois University at Carbondale, Illinois.

Charlene M. Czerniak is Professor and Interim Dean of the College of Education at the University of Toledo in Toledo, Ohio.

Jodi J. Haney is an Assistant Professor in the Deparment of Educational Curriculum and Instruction at Bowling Green State University in Bowling Green, Ohio.

Figure 1. Baseline Questionnaire Composite Results

Figure 2. Subject Preparedness Prior to Project

Table 1. Spearman Correlations between belief and background measures

	EB	LB	CB	SE
Enable Beliefs (EB)	1.00
Likelihood Beliefs (LB)	.11	1.00
Context Beliefs (CB) (EB + LB)	.66*	.79*	1.00
Science Teaching Self-Efficacy (SE)	.12*	.13*	.18*	1.00
Graduate Courses Taken	.08	.03	.08	.40*
Grade Level Assignment	.02	-.15*	-.10	-.05
Minutes Per Week Teaching Science	.06	-.03	.05	.15*
Science Teaching Methods Taken	.15*	.05	.15*	..16*
Use of 10 Common Science Teaching Strategies	.14*	.11	.21*	.26*
Years of Teaching Experience	.19*	.09	.19*	-.20

* p < .05

n = 229

Back to the top

Back to the EJSE