An Analysis of the Reliability and Validity of Personal Internet

Teaching Efficacy Beliefs Scale

by

Ravinder Koul
Pennsylvania State University- Great Valley
Graduate Center for Professional Studies

and

Pete Rubba
Pennsylvannia  State University

Note: The research for this article was funded by the National Science Foundation (ESI-9550017), The West Virginia Research Corporation and the West Virginia University. Any findings, conclusions and recommendations expressed here-in are those of the authors and do not necessarily reflect the views of funding agencies.

Introduction, Rationale and Purpose

Self-efficacy can be defined as belief about one’s own capabilities to organize and execute a certain task (Bandura, 1977, 1982, 1993,1997). Self-efficacy beliefs influence our thought patterns and emotions which, in turn, enable actions. We make efforts in pursuit of goals, persist in the face of obstacles and rebound from temporary setbacks to exercise some control over events that affect our lives (Bandura, 1997). Bandura has postulated four sources of efficacy expectations: mastery experiences, physiological and emotional states, vicarious experiences, and social persuasion (Bandura, 1997; Tschannen-Moran, Woolfolk, & Hoy, 1998). Perceiving one’s performance as successful may raise efficacy expectations for future successes, just as perceiving one’s performance as failure may lower them. Along with degrees of anxiety and excitement, social persuasion and peer-feedback contribute to self-efficacy.

A teacher’s self-efficacy is curriculum specific; a teacher may have a high level of personal efficaciousness in the science domain yet feel inefficacious in instructional technology. Efficacy is context specific in other ways also, and may vary with the nature of the individual classroom and student population (e.g. Tschannen-Moran, Woolfolk, & Hoy, 1998). Researchers have conducted studies on efficacy in the context of classroom management (Emmer & Hickman, 1990; Raudenbush, Rowen, & Cheong, 1992.), special education (Coladarci & Breton, 1997), institutional climate and behavior of the principal (Hoy & Woolfolk, 1993), and the decision-making structures at school (Moore & Esselman, 1992). Scales have been devised to assess efficacy beliefs toward science teaching, namely the Science Teaching Efficacy Beliefs Instrument (STEBI) (Enoch & Riggs, 1990) and Chemistry Teaching Self-Efficacy (Rubeck & Enoch, 1991), etc.

Efficacy beliefs are known to affect the receptivity to technological innovations (Jorde-Bloom & Ford, 1988; Moore & Esselman, 1992; Shroyer & Borchers, 1996). One’s ability to effectively integrate technology into curriculum is not automatic and does not follow providently from knowledge and skill with technology; teachers need systematic, research-based guidance in the development of ways to better integrate technology, such as the Internet, into instruction (OTA, 1995; Panel on Educational Technology, 1997; Chrisholm & Wetzel, 1997). A scale that assesses teacher self-efficacy beliefs on information technology will contribute significantly to the ability of teacher educators and supervisors to help teachers make effective instructional use of the Internet.

This study, designed to validate a scale to assess self-efficacy beliefs toward teaching with the Internet, was conducted in two phases.

Methodology

In the first phase, we used a sample of 389 inservice teachers from a professional development project to establish the reliability and validity of a task specific Personal Internet Teaching Efficacy Beliefs Scale (PITEBS). In the second phase, we used the PITEBS to assess changes in Internet teaching efficacy beliefs among 155 different participants in the same professional development project.

Efficacy expectations have been proposed to consist of two components: General Teaching Efficacy and Personal Teaching Efficacy (Gibson & Dembo, 1988). General Teaching Efficacy is a teacher’s personal beliefs about the general relationship between teaching and learning, and corresponds to what Bandura calls Outcome Expectancy (1997). Personal Teaching Efficacy is a teacher’s general sense of his/her own effectiveness. A teacher may be convinced of his/her own ability to teach (Personal Teaching Efficacy) but doubtful about his/her students’ ability to learn (General Teaching Efficacy).

Gibson and Dembo (1984) devised their self-efficacy instrument with two distinct dimensions: Personal Teaching Efficacy (PTE) and General Teaching Efficacy (GTE). We added a third dimension, Personal Internet Teaching Efficacy (PITE), to our instrument. The Personal Internet Teaching Efficacy Beliefs Scale (PITEBS) had 41 items, 16 of which were borrowed from Gibson and Dembo (1984) and STEBI (Enoch & Riggs, 1990) instruments, appropriately modified to refer to the Internet (eight items each for PTE and GTE). We added 12 items to the PTE and GTE dimensions (to increase the potential reliability of the items associated with these dimensions), and devised 13 items for the added PITE dimension. Our items use a five choice Likert-Scale, which asks respondents to select a number (5 = strongly agree to 1 = strongly disagree) to indicate their level of agreement with each statement.

Data Analysis

Phase 1

We asked teachers participating in a National Science Foundation funded teacher development project on teaching and learning with the Internet to respond to our instrument. The project provides training to public school teachers on the use of Internet resources in a manner consistent with the state level science curriculum framework and the National Science Education Standards (National Research Council, 1996). The project anticipates the eventual training of over twelve hundred teachers of science and mathematics in grades K-12 throughout the state. Teachers begin their training at a five-day summer workshop that provides introduction to the Internet and its uses in the classroom. Following the summer workshops, teachers participate in fall and spring on-line courses offered through two major state universities. The fall semester course serves to review and enhance Internet and telecommunication skills learned during the summer, and the spring semester course focuses on the integration of Internet-based resources into the curriculum, and classroom implementation.

Of 389 respondents, 51% were elementary, 21% middle and 28% high school teachers in a response rate of 91%. Teaching experience ranged from 1 to 28 years. Approximately 80% of the respondents were females, and a majority of them were white. A majority (approximately 80%) of the respondents was drawn from the summer workshops and nearly 20% had completed the spring course.

We used SPSS Varimax Rotation with Kaiser Normalization for Principal Component factor analysis to identify factors (number of factors not predetermined) that could be used to represent relationships between the items. When we used all 41 items, we found that the factors for GTE and PTE were moderately correlated (r = .529). Notably, some of the items that were intended to correspond to the General and Personal dimensions loaded onto the same factor. For example, items "When a student does better than usual, it is often because the teacher exerted a little extra effort," and "When a student does better than usual, many times it is because I exerted a little extra effort," loaded onto the same factor. Substituting "teacher" for "I" to measure the potential impact of teaching in general (and the Outcome Expectancy dimension) is inappropriate (see also Woolfolk & Hoy, 1990).

We found that not all the 16 items borrowed from Gibson and Dembo (1984) fell into two distinct dimensions. Gusky and Passaro (1993) have suggested that the two factors (PTE & GTE) actually represent internal and external locus of control (for example, "I can" and "teacher cannot", respectively). Gusky and Passaro (1993) used a modified version of Gibson and Dembo’s instrument containing 21 items to reflect internal and external locus of control with a sample of 342 preservice and inservice teachers. They named "Personal Efficacy" items that use the referent "I can" as Personal-Internal (P-I) and those items that use the referent "I can’t" as Personal-External (P-E). General Efficacy items that use the referent "teachers can" were renamed as "Teacher-Internal" (T-I), while items with the referent "teacher can’t" were renamed Teacher-External (T-E). Gusky and Passaro concluded that the internal versus external locus of control distinctions were measures of causal attributions (Gusky & Passaro, 1993).

When we followed the procedures used by Gusky and Passaro (1993) and conducted factor analysis using generalized least squares, our findings indicate that not all items under the "external" or "internal" influences correspond to the same dimension, respectively. For example, we found that the following items do not represent a common "external" dimension and load onto more than one factor: "the hours in my class have little influence on students compared to the influence of their home environment", "the amount that a student can learn is primarily related to family background and environment", "the influences of a student’s home experiences can be overcome by good teaching," and "even a teacher with good teaching abilities may not reach many students". We found that the 16 items corresponding to PTE and GTE dimensions of Gibson and Dembo (1984) and the 11 items corresponding to Internal and External dimensions of Gusky and Passaro (1993) do not load onto two distinct dimensions.

The fact that the responses to items did not fall neatly into distinct dimensions suggests that teachers do not clearly distinguish between their personal ability to affect student learning and the potential influence of teachers in general. Even though a teacher’s personal influence and power can be distinguished from the impact of elements that lie outside the control of teachers (e.g., the influence of social demographics or economic conditions), external items cannot be lumped into a single distinct dimension. It is inappropriate to assume that a particular level of self-efficacy for the external home influence domain will accurately reflect all external influences (see also Rich & Lev, 1996).

Table 1

Pearson Correlation Coefficients Results (N=389)

  General Teaching Efficacy Beliefs Personal Teaching Efficacy Beliefs Personal Internet Teaching Efficacy Beliefs
General Teaching Efficacy Beliefs 1.000 .529** .112*
Personal Teaching Efficacy Beliefs .529** 1.000 .161*
Personal Internet Teaching Efficacy Beliefs .112* .161* 1.000

* p<.05

** p<.01

Analysis of the items found three factors with eigenvalues greater than 1.0 (1.49, 2.32, and 7.05, based on the factor order in Table 2). A prior criteria of eigenvalues greater than 1.0 and factor loadings greater than .32 (as recommended by Stevens (1996)) for retaining the items were satisfied for each of the three dimensions. Our results (see Table 2) showed three items corresponded to the External dimension (alpha = .4972), 4 items corresponded to the Internal dimension (alpha = .7399) and 13 items represented a Personal Internet Teaching Efficacy Beliefs dimension (PITEBS) (alpha = .9259).

Table 2

Principal Component Factor Analysis

Factor/

Item

Stem

Factor 1 Factor 2 Factor 3
Factor 1 General Teaching Efficacy/GTE      
 
  • A teacher is very limited in what he/she can achieve because a student’s home environment is a large influence on his/her achievement.
    .700
 
  • If students are not disciplined at home, they are not likely to accept discipline in school.
    .762
 
  • If parents would do more with their children, I could do more.
    .643
Factor 2 Personal Teaching Efficacy/PTE      
 
  • When a student gets a better grade than he/she usually gets, it is frequently because I found a better way of teaching that student.
  .789  
 
  • When the grades of my students improve it is usually because I found more effective teaching approaches.
  .793  
 
  • When a student does better than usual, many times it is because I exerted a little extra effort
  .765  
 
  • If a student masters a new concept quickly, it most probably is because I taught the concept well
  .631  
Factor 3 Personal Internet Teaching Efficacy/PITE      
 
  • I am continually finding better ways to teach with the INTERNET.
.633    
 
  • Even when I try very hard, I do not teach as well using the INTERNET as I teach in other ways.
.600    
 
  • I know how to teach effectively using the INTERNET.
.829    
 
  • I am not very effective in monitoring activities that involve using the INTERNET.
.702    
 
  • I generally teach ineffectively when using the INTERNET.
.589    
 
  • I understand how to use the INTERNET well enough to be effective in teaching with it.
.805    
 
  • I find it difficult to explain to students how the INTERNET works.
.825    
 
  • I am typically able to answer students' INTERNET questions.
.817    
 
  • I wonder if I have the necessary skill to teach using the INTERNET.
.746    
 
  • Given a choice, I would not invite the principal to evaluate my teaching when I use the INTERNET in a lesson.
.576    
 
  • When teaching using the INTERNET, I usually welcome student questions.
.727    
 
  • I don't know what to do to turn students on to using the INTERNET.
.765    
 
  • When a student has difficulty understanding how to use the INTERNET, I am usually at a loss as to how to help the student understand it better.
.796    

 

A low reliability for the measure of External Dimension suggests the influence of additional dimensions on self-efficacy beliefs. For example, factors such as "teachers’ value of responsibility" may affect measures of teacher efficacy. Teachers who place a high value on teaching may take greater responsibility for their own actions and the performance of their students, while teachers who place a lower value on teaching are more likely to attribute students’ performance to the students themselves or to situational factors. Most researchers agree that "the nature of the student performance outcome, specifically whether that outcome is positive (success) or negative (failure), does influence teacher perceptions of efficacy and the measure of those perceptions" (Guskey, 1987, pg. 42). A teacher’s self-efficacy may be a multidimensional construct, related to context variables such as student ability (high or low), performance outcome (positive or negative) and scope of influence (single student versus a group of students) (see Guskey, 1987; Ashton, 1984).

The 13 items of factor 3 in Table 2 constitute the Personal Internet Teaching Efficacy Beliefs Scale. In our analysis we found consistently that this PITE dimension does not correlate with the internal, external, personal or general efficacy dimensions. The lack of interaction between the Personal Internet Teaching Efficacy Beliefs Scale and other efficacy dimensions (i.e., PTE and GTE) implies that teachers do not relate their efficacy in the use of Internet technology with their effectiveness as teachers.

Phase II

In the second phase of this study, we reported changes in the efficacy of 155 public school teachers enrolled in five-day summer workshop sponsored by the same professional development project on the use of Internet resources. Prior to the workshops, most participants had very little Internet experience; fewer than 10% rated themselves as having a moderate to high level of expertise.

Instruction on the first day of the workshop involved an overview of the computer’s operating system and file management, e-mail fundamentals, telneting, and netiquette. "Surfing the Web" was the focus of day two. Teachers were introduced to web browsers and learned about Netscape, search engines, and search strategies. On day three, strategies for curriculum integration were explored, including watershed education using the Internet, on-line science collaboration activities, and classroom resources. Day four was spent with on-line discussion formats (e.g., listservs, newsgroups, Web Board) and "advanced" Internet topics with Netscape (e.g., downloading text/graphics/software, plug-ins, Win/Zip, and PKUnZip). During the morning of the fifth day, teachers worked individually and in small groups to search the web for classroom resources. Findings from these searches were presented to their peers in the afternoon.

Teachers were given the three-scaled instrument on the morning of Day 1 and the afternoon of Day 5 of the workshop. Table 3 shows dependent sample t-test results. The test-retest comparison showed significant gains in teacher’s self-efficacy beliefs levels in the use of Internet resources for teaching purposes (t =13.48, p < .001), providing further evidence of PITEB scale construct validity (Cronbach & Meehl, 1966).

Table 3

t-Value for dependent samples (N=155)

Pair Pre-test Mean Post-test Mean Standard Deviation t- value df p
Pair for Factor 1 10.18 9.90 1.73 1.96 154 .052
Pair for Factor 2 9.85 9.57 2.07 1.63 154 .105
Pair for Factor 3 29.70 37.56 7.23 13.48* 154 .000

 

Conclusion

Personal Internet Teaching Efficacy Beliefs Scale (PITEBS) is a valid and reliable instrument that can be used to assess self-efficacy beliefs in teaching with the Internet. The PITEBS contributes to existing tools for teacher assessment in teaching and learning with technology. The PITEBS is a generic instrument, not related to teaching any particular content area with the Internet, which can be used with K-12 teachers. Still, we recognize that the PITEB scale has some limitations. The rapidly changing technological demands of Internet usage make it difficult to compare results over time whether the PITEBS or another instrument is used. If one is interested in assessing teaching efficacy for specific Internet tools, it is advisable to supplement the PITEBS with additional items that target specific Internet tools (e.g., email, www). Also, it is likely that more task specific applications, and conceptualizations and operationalizations of the PITEBS scale will yield different information (see also Rich & Lev 1996; Woolfolk & Hoy, 1990).

Needless to say, higher PITEB scores following the workshops are no guarantee for future accomplishments in the use of information technology. Technological choices available within a school largely determine which potentialities are cultivated and which remain underdeveloped. Teachers are the products as well as the producers of their school environments---organizational factors such as resource availability, peer support, encouragement and collegiality in the form of technical assistance, all contribute to enhancing the use of technology in schools (Shroyer & Borchers, 1996). These are crucial issues, particularly in the early phases of skill development when people may distrust their efficacy (Bandura, 1997). Further research is needed to examine efficacy beliefs for specific technology usage in order to illuminate the possible differences in teachers’ perceptions of a variety of Internet tools. Research also is needed to examine the role of context variables like student ability (high or low), performance outcome (positive or negative) and scope of influence (single student versus a group of students) on teachers’ perceptions of teaching with the Internet, and the measure of those perceptions. The PITEBS can play an important role in that research and also in program evaluation.

 

References

Ashton, P. (1984). Teacher efficacy: A motivational paradigm for effective teacher education. Journal of Teacher Education, 35(5), 28-32.

Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 84 (2), 191-215.

Bandura, A. (1982). Self-efficacy mechanism in human agency. American Psychologist, 37, 122-147.

Bandura, A. (1993). Perceived self-efficacy in cognitive development and functioning. Educational Psychologist, 28(2), 117-148.

Bandura, A. (1996). Self-efficacy in changing societies. New York: Cambridge University Press.

Bandura, A. (1997). Self-efficacy: The exercise of control. New York: W. H. Freeman.

Chisholm, I. M. , & Wetzel, K. (1997). Lessons learned from a technology-integrated curriculum for multicultural classrooms. Journal of Technology and Teacher Education (1997), 5(4), 293-317.

Coladarci, T., & Breton, W. (1997). Teacher efficacy, supervision, and the special education resource-room teacher. Journal of Educational Research, 90(4), 230-239.

Cronbach, L. J. & Meehl, P. E. (1966). Construct validity in psychological texts. In C. Chase, & H.

Ludlow (Ed.). Readings in educational and psychological measurement. Boston:Houghton-Mifflin.

Emmer, E., & Hickman, J. (1990, April). Teacher decision making as a function of efficacy, attribution, and reasoned action. Paper presented at the Annual Meeting of the American Educational Research Association, Boston.

Enoch, L G., & Riggs, I. M. (1990). Further development of an Elementary Science Teaching Efficacy Belief Instrument: A Preservice Elementary Scale. School Science and Mathematics, 90(8), 694-706.

Gibson, S., & Dembo, M.H. (1984). Teacher Efficacy: a construct validation. Journal of Educational Psychology, 76(4), 569-582.

Guskey, T. R. (1987). Context Variables That Affect Measures of Teacher Efficacy. Journal of Educational Research., 81(1), 41-47.

Guskey, T. R., & Passaro, P. D. (1994). Teacher efficacy: A study of construct dimensions. American Educational Research Journal, 31(3), 627-643.

Jorde-Bloom, P., & Ford, M. (1988). Factors influencing early childhood administrators' decisions regarding the adoption of computer technology. Journal of Educational Computing Research, 4 (1), 31-47.

Moore, W., & Esselman, M. (1992, April). Teacher efficacy, power, school climate and achievement: A desegregating district's experience. Paper presented at the Annual Meeting of the American Educational Research Association, San Francisco.

Office of Technology Assessment (OTA) (1995). Teachers and technology: making the connection. Washington, DC: Office of Technology and Assessment Press.

Panel on Educational Technology (1997). Report to the President on the use of technology to strengthen k-12 education in the United States. New York, NY: D E Shaw & Co.

Raudenbush, S., Rowen, B., & Cheong, Y. ( 1992). Contextual effects on the self-perceived efficacy of high school teachers. Sociology of Education, 65(2), 150-167.

Rich, Y., & Lev, S., Fischer, S. (1996). Extending the concept and assessment of teacher efficacy. Educational and Psychological Measurement, 56(6), 1015-1030.

Rubeck, M. L., & Enochs, L. G. (1991, April). A path analytic model of variables that influence science and chemistry teaching self-efficacy and outcome expectancy in middle school science teachers. Paper presented at the annual meeting of the National Association for Research in Science Teaching, Lake Geneva, WI.

Shroyer, M. G., & Borchers, C. A. (1996). Factors that support school change to enhance the use of microcomputers in rural schools. School Science and Mathematics, 96(8), 419-431.

Soska, M. (1994). An introduction to educational technology. Directions in Language and Education, 1(1), 1-7.

Stevens, J. (1996). Applied multivariate statistics for social sciences (3rd ed.). Mahwah, NJ: Lawrence Erlbaum.

Tschannen-Moran, M., Woolfolk, A., & Hoy, W. (1998). Teacher Efficacy: its meaning and measure. Review of Educational Research, 68(2), 202-248.

Woolfolk, A. E., & Hoy, W. K. (1990). Prospective teachers’ sense of efficacy and beliefs about control. Journal of Educational Psychology, 82(1), 81-91.


About the authors…

Ravinder Koul is an Assistant Professor of Curriculum and Instruction, College of Education, Pennsylvania State University- Great Valley Graduate Center for Professional Studies, 30 Swedesford Road, Malvern, PA 19355
rxk141@psu.edu

Pete Rubba is a Professor of Science Education and C&I Department Head at Penn State University, 140 Chambers Building, University Park, PA 16802 par4@psu.edu.

Back to the EJSE