Lake Bedilu
lakeymca@gmail.com
Department of Psychology
Faculty of Educational & Behavioral Sciences
Bahir Dar University
Abstract: The purpose of this study was to assess the competence of secondary school teachers in the educational assessment of students among secondary schools in Bahir Dar town. To do so, a sample of 60 secondary school teachers in Bahir Dar town was taken. A questionnaire adapted from Plake, Impara and Fager’s (1993) was used to collect data. The questionnaire was composed of 31 multiple-choice items used to assess teachers' competence in the educational assessment of students. Descriptive statistics and t-test was used to determine whether there exists a significant difference in teachers' competence based on differences in the school type where teachers taught. It was found out that teachers participating in this study demonstrated knowledge rated below average in the educational assessment of students. Across the seven competency areas, teachers showed low level of competency with minor differences. Finally, teachers in the nongovernmental schools had a significantly higher level of competence than teachers who taught at governmental schools at a < 0.05. The findings suggest that, it is time for the education community to recognize that teachers are incompetent to successfully undertake student assessment. This is especially most important due to the current trend in student assessment, involving continuous assessment strategies that require more knowledge and skill about assessment. Unless corrective measures are taken to improve this poor competence of teachers in the educational assessment of students, the poor quality of education that exists currently will get much worse and the whole educational process might ‘collapse’.
Keywords: assessment literacy, secondary schools, Ethiopia
Assessment is one of the most powerful educational tools for promoting effective learning (Assessment Reform Group, 1999). In a review of research on assessment and classroom learning, Black and Wiliam (1998) synthesized evidence from over 250 studies linking assessment and learning. The outcome was a clear and indisputable message: that initiatives designed to enhance effectiveness of the way assessment is used in the classroom to promote learning can raise student achievement. Thus, student success is largely dependent on teacher practice.
In support of this relationship between teacher practice and student success, Rice as cited in Greenstein (2010, p.1) asserts that “Teacher quality matters: It is the most important school- related factor influencing student achievement”. Black and Wiliam (1998) also concluded that teacher’s skill in classroom assessment enhances student achievement. To better understand this relationship, consider a teacher who begins at the 50th percentile in terms of his/her skill at using classroom assessments and a student in his/her class who begins at the 50th percentile in terms of his/her achievement. If the teacher increases his/her effectiveness at using classroom assessment to the 84th percentile over time, one would predict that the student’s achievement would increase to the 63rd percentile given Black and Wiliam’s findings. Similarly, if the teacher increases from the 50th to the 99th percentile in terms of skill at using classroom assessments, one would predict the student’s achievement to increase to the 78th percentile (Marzano, 2006).
Assessing student performance is one of the most critical aspects of the job of a classroom teacher. It impacts nearly everything that teachers do. In their role in the classroom, Stiggins as cited in Plake and Impara (1997) estimated that teachers spend up to 50% of their instructional time in assessment-related activities such as quizzes, tests, questions, and projects. The percentage of time spent on assessment or assessment preparation is growing (Quilter, 1999). In the Ethiopian Educational System, a good portion of the budget also goes into formal testing. With so much time and money devoted to assessment, it's worth critically understanding the knowledge and skills teachers possess in the educational assessment of students.
Further, student assessment is an essential part of teaching and good teaching
cannot exist without good student assessment (Eckhout et al., cited in Kiomrs,
Abdolmehdi, & Naser,
2011). However, studies show that teachers have consistently used a variety
of factors in their assessment practices and consequently make erroneous decisions.
Even more disturbing is that most teachers lack effective assessment knowledge
and skills; that is, when evaluating student academic achievement, teachers
exhibited misconceptions about assessment practices (Cizek, Fitzgerald, &
Rachor, 1996; McMillan, 2001; cited in Chen, 2005). Thus, the continuing need
to develop the potential of classroom assessment to support learning has recently
been stressed by a number of researchers in the field (Assessment Reform Group,
1999). In particular, Black & Wiliam (1998) have called for research which
supports teachers in trying to establish new practices in formative assessment.
Professional organizations have also acknowledged the need for assessment literacy within the teaching profession. Teachers, organizations, and the educational measurement community have come together to promote assessment literacy through seven assessment standards (American Federation of Teachers, National Council on Measurement in Education, & National Education Association [AFT, NCME, & NEA], 1990). Among the skills espoused in the Standards for Teacher Competence in Educational Assessment of Students (AFT, NCME, & NEA, 1990) are the following: (a) choosing appropriate assessment methods; (b) developing appropriate assessment methods; (c) administering, scoring, and interpreting assessment results; (d) using assessment results to make instructional or curricular decisions; (e) developing appropriate grading practices; (f) communicating assessment results; and (g) recognizing unethical, illegal, and otherwise inappropriate uses of assessment information. According to AFT, NCME & NEA (1990), some of these standards focus on classroom-based competencies while the other standards address assessment competencies underlying teacher participation in decisions related to assessment at the school, district, state, and national levels because of teachers' growing roles in education and policy decisions beyond the classroom. These competencies are the knowledge and skills critical to a teacher's role as an educator.
Research studies have been conducted over the past 20 years that have addressed one or more of the seven standards. However, very few studies (Impara, Plake, & Fager, 1993; Plake & Impara, 1997; Campbell, C., Murphy & Holt, 2002; Mertler, 2005) have specifically examined inservice teachers' knowledge of assessment to meet the seven standards. Thus, taking into consideration the lack of adequate information in the area internationally and in the local context specifically, and the emphasis it was given in the Education and Training Policy (FDRE, 1994) of the government of the Federal Democratic Republic of Ethiopia, the current study assessed the competence of secondary school teachers in Bahir Dar town in the educational assessment of students and addressed the following leading questions.
This study is believed to provide information to schools, education offices, and teacher training institutions about the level of assessment literacy so that they can improve teachers’ competence in the educational assessment of students. Further, the study will serve as a foundation for consecutive studies on the area. However, this study has limitations. The national standards for teacher competence in the educational assessment of students are not comprehensive enough to cover all issues concerning assessment. Thus, it drove the researcher to adopt the American Teachers Association standards the questionnaire. Thus, the results of the study may not reflect standards which are relevant to the local context. Teachers were allowed to take the questionnaire home, filled in/completed it, and returned it the next day. As a result, they might have an opportunity to consult resources and/or discuss with colleagues the questions answered. Thus, the lack of a controlled testing situation might result in a response that did not reflect the respondents' actual knowledge and skill in the area and might complicate the interpretation of the results.
Assessment Literacy refers to the level of competence of teachers as evidenced from their responses from the assessment literacy questionnaire. As a result higher score on the questionnaire reflects good literacy on the educational assessment of students.
High School refers to the first cycle (Grades 9 – 10) of the secondary education.
Non-Governmental School refers to schools owned by either private investors or nongovernmental organizations.
A total of 60 grade 9 teachers from four high schools in Bahir Dar town were randomly selected to serve as the sample of the study. The sample was randomly selected from the total Bahir Dar town high school teacher population. The selection of the sample was done as follows. First, from the total of 8 high schools (four governmental schools namely Tana Haik, Fasilo, Ghion, and Bahir Dar Zuria and four non-governmental schools namely Bahir Dar Academy, ADM (Haile), SOS Herman Gmeiner (SOS), and Aba G/Michael (Catholic)), four schools (two governmental namely Tana Haik and Fasilo and two non-governmental namely Bahir Dar Academy and SOS) were randomly selected using a lottery system. Second, from the selected four schools, on average twenty teachers were selected from each of the four schools mentioned above using accidental sampling based on their number and availability of teachers on that specific day and time of data collection.
To collect data from the respondents, a questionnaire was used. The questionnaire had two components: background information and the Teacher Assessment Literacy Questionnaire. The background part was composed of questions about the participants’ sex, qualification, years of experience, educational level, and course status on educational measurement and evaluation. On the other hand, the Teacher Assessment Literacy Questionnaire was composed of 31multiple-choice items with four response options. This questionnaire was adopted from Plake, Impara and Fager’s (1993) Teacher Assessment Literacy Questionnaire. Plake, Impara and Fager (1993) in cooperation with The National Council on Measurement in Education & the W.K. Kellogg Foundation developed an instrument (Teacher Assessment Literacy Questionnaire) to measure teacher knowledge in the seven competency areas based on the “Standards for Teacher Competence in the Educational Assessment of Students” developed by the collaborative effort between the American Federation of Teachers (AFT), the National Education Association (NEA), and the National Council on Measurement in Education (NCME) in 1990. This questionnaire was composed of 35 multiple-choice items (5 multiple- choice test questions for each of the 7 competency areas) designed to assess teacher assessment literacy. Plake, Impara, & Fager (1993) found a reliability of 0.54 for the entire test using the KR–20 method. However, four of the 35 items (item numbers 14, 20, 28, and 31) were found to be irrelevant to our context because either they refer to interpretation of standardized tests, rules, or community based curriculum issues (see Table 1) that did not exist in the Ethiopian Education system. Thus, the questionnaire used in this study had only 31 of these 35 multiple-choice questions and its reliability was found to be 0.44 for the entire 31-item test using the Spearman-Brown method.
Data were collected from the respondents using the questionnaire introduced above. The questionnaire was presented in English. The questionnaire was distributed to each respondent in person in their school and collected in the same way the next day. During the delivery of the questionnaire, the objectives of the questionnaire and the study, and instructions on how to fill the questionnaire were clearly communicated. In addition, respondents were informed that their responses would be confidential and it would be used only for a research purpose.
First, the responses of the 31 multiple-choice questions were coded as 0 if it is incorrectly responded and 1 if it is correctly responded. Following this coding the data was entered into a computer by the researcher, cleaned and analyzed using SPSS version 16.0 for windows. In this process, cases that had more than three missing values were discarded and not considered in the computation. Thus, the final computations were carried out only on valid data excluding missing data. Means and standard deviations were used to describe the results of each item for all the respondents and the level of competence in the 7 areas. In addition, independent samples t- test was used to investigate whether there exists a statistically significant difference among teachers who taught in governmental and non-governmental schools in their level of competence in the educational assessment of students.
Eighty and fifteen percent of the respondents were male and female respectively. The majority of them (93 %) had Bachelor’s degrees and the rest (7 %) had Masters degrees. Fifty eight percent of them had less than ten years of experience; whereas, the rest (forty two percent) had experiences exceeding ten years teachers. Table 1 below shows the socio- demographic characteristics of the respondents.
Table 1: Socio-Demographic Characteristics
of the Respondents
Seventy eight percent of the teachers took at least one course on educational measurement and evaluation in their undergraduate preparation as teachers; whereas, the rest twenty two percent said that they did not take a separate course in their undergraduate preparation.
Overall, the mean performance on the 31-item instrument was 12.08 (standard deviation [SD] –2.82) or near 39% correct. Given that the Education and Training Policy of the FDRE (1994) set 50% achievement as a minimum score in order to get promoted from one level to the next, most teachers participating in this study would receive a failing grade based on their demonstrated knowledge of educational assessment of students. Across the seven competency areas, teachers showed the highest level of competency in the area of Using Assessment for Grading (average performance on the five item subset was 2.10 [SD - 0.93] and the lowest level of competency in the area of Communicating Assessment Results (mean -1.00, standard deviation - 0.71). Teacher performance across the seven competency areas is summarized in Table 2.
Table 2: Average Performance by Teachers across the Seven Competency Areas
To better understand the performance of teachers across these seven competency standards, one may refer to Table 3 which presents the proportion of teachers who correctly answered each of the items, within each competency area.
An examination of Table 3 reveals several items that were very difficult for these teachers (items with the proportion correct less than .30). There were a total of nine very difficult items, two each from competency area 1 (Choosing Assessments Methods), area 2 (Developing Assessment Methods), area 6 (Communicating Assessment Results) and area 7 (Recognizing Unethical Practices) and one from competency area 4 (Using Assessments for Decision Making). These items are presented in Table 4.
Table 3: Item Performance by Competency
Area
Similarly table 3 reveals that there were no items that were very easy for these teachers (those items with p values of .90 or greater). However, one item from competency area 4 (Using Assessments for Decision Making) seemed relatively easy for these teachers.
A t-test was conducted to test the differences on the overall mean scores
on the teacher assessment literacy questionnaire between teachers who taught
in the governmental schools and those in the non-governmental schools (Table
5). Equality of variances was supported by the non significant Levene’s
test result. As a result, the t-test indicated that in comparison with teachers
who taught in governmental schools, teachers in the nongovernmental schools
had a significantly higher level of competence in the educational assessment
of students at a
< .05.
Table 5: t-test for Assessment Competence
between Governmental and Nongovernmental School Teachers
The purpose of this study was to assess secondary school teachers’ competence in the educational assessment of students in Bahir Dar town. The discussion of the findings of this study is presented as follows.
As indicated in table 2, it was found out that the overall average score of the sample secondary school teachers’ competence in the educational assessment of students was 12.08 (39 % correct). This indicates that secondary school teachers in Bahir Dar lack the competence or ability in the educational assessment of their students. The findings also confirmed that teachers also lacked the competence in each of the seven competency areas. This result is by far below the average teachers' competence in the educational assessment of students demonstrated in Plake, Impara and Fager’s (1993) national survey. Plake, Impara and Fager (1993) conducted a national assessment of teachers' competence in the educational assessment of students all over the United States of America on 555 teachers from elementary, middle, and high school levels and they found out nearly 66 % correct which is nearly twice the competence level found in this study.
Seen from the point of view of the extent to which teachers met the seven standards, one of the results of this study paralleled both Mertler (2005) and Plake and Impara (1997). The present participants had the most difficulty with Standard 6 (M = 1.00), which is communicating assessment results. Plake and Impara (1997) also found out that inservice teachers scored the lowest (M = 2.70) on this standard. Even though participants in Mertler's (2005) study likewise did not score high on this standard (M = 2.48), the scores were very similar to those obtained in the present study as well as in Plake and Impara (1997). Despite a few similarities among these studies in identifying the degree of assessment literacy for inservice teachers, the present study showed that communicating assessment results was the most difficult standard to meet.
The study revealed that teachers who taught in nongovernmental schools had a higher competence in the educational assessment of students than teachers in the governmental schools. It needs further research to explore the reasons for this difference; the researcher, however, thought the following three reasons might contribute to this difference. First, usually nongovernmental schools recruit teachers who are more competent using criteria such as who scored good cumulative grade point average (CGPA) during their undergraduate teaching preparation and who demonstrated good teaching skills in the practical examinations. However, this was not observed among governmental schools. Second, relatively speaking, one can find more number of high achiever students in the nongovernmental schools than that of governmental schools. Thus, these high achieving students demanded more in terms of knowledge of subject matter and skill in teaching from their teachers that might in turn lead teachers to work hard and become more competent. Finally, the presence of better monitoring and supervision system in the nongovernmental school administration than the governmental ones might lead teachers to work hard and meet the standards of the supervisors and hence demonstrate their competence as a teacher.
The evidence gathered from this study suggests that it is time for the education community to recognize that teachers are incompetent to successfully undertake one of the most prevalent activities of their instructional program: student assessment. This is especially most important due to the current trend in student assessment, involving continuous assessment strategies that require even more knowledge about assessment as they more directly involve the teacher in the administration and scoring or the results than do multiple-choice assessments.
Thus, unless corrective measures are taken to improve this poor competence of teachers on the educational assessment of students, the poor quality of education (Negash, 2006) that exists currently will get much worse and the whole educational process might ‘collapse’.
Based on the findings of this study, the researcher would like to recommend the following in order to improve the overall poor competence of teachers in student assessment.