Innovative Strategy for Measuring Skill Performance of Students of Vocational Agricultural Education for Sustainable Livelihood in Nigeria

Juliana Adimonye Ukonze
1 Department of Agricultural Education, Faculty of Vocational and Technical Education University of Nigeria, Nsukka, Nigeria
(*Author for correspondence: juliana.ukonze @gmail.com)

Abstract: Agricultural trades have been introduced for learning in Secondary Schools to complement Agricultural Science. Success or failure in Agricultural Science is being determined by West African Examination Council through essay test, multiple choice tests based on bloom taxonomy on knowledge, comprehension, application and alternative to practical. The observed effect is that most students who offered Agricultural Science, pass with Credit, but couldn’t practice farming because of lack of skill, instead they prefer office jobs that are difficult to find. Even those from parents in Agrarian Communities migrate to cities with their West African School certificate to hunt for jobs. Based on this unsatisfactory effect or effort of the Government to diversify the economy through Agriculture, Agricultural trades are now being introduced for learning in Senior Secondary Schools across the country. Performance in these trades cannot be measured fairly through essay cognitive test and alternative to practice. Hence there is need for new assessment instrument. Psycho-productive multiple choice or criterion reference test has been developed and tried and found successful as contained in this paper. It is therefore recommended that WAEC, other examination boards and other stakeholders should adopt Psychomotor or criterion reference test for measuring students in trades in Agriculture in Senior Secondary Schools across the nation to ensure a sustainable livelihood.

Keywords: Psycho-productive, Psychometric properties and Rating Scale

1. INTRODUCTION

Students in secondary schools are usually assessed by National Examination Boards like West African Examination Council (WAEC) through standard Examination which are judged valid. Presently, the relevance of West African School Certificate Examinations (WASCE) for secondary school students in certain vocational and technical subjects are being challenged by individuals or groups who could have used WASC Examination result to recruit some of these secondary school graduates for intermediate jobs in vocational and technical areas like agriculture, home management, business among others. The relevance or suitability of these examinations are vividly attested to by most of these students with credits in WASC examinations in Agriculture, Home Management, Business, metal work, technical drawing among others but could not perform any skill in jobs in these areas when expected to do so with their background credit as awarded by WAEC. West African Examination Council could have been exonerated from this hidden facts but a clear examination of WAEC strategies of measuring skill in her final examination revealed as follows:

1. Multiple choice test items administered on student in vocational and technical subjects are based on the cognitive domain of knowledge, comprehension and application which is regarded as cognitive reasoning instead of criterion reference test items that should be in Psycho-motor domain of perception, set, guided response, mechanism, complex overt response and adaptation.
2. Basing test of practice on the job on alternative to practical, a good understanding of grammar will make some individuals to understand that alternative to practical means opposite to practical or substitute which implies that practical is absent. As it is practiced in secondary school both by the teacher and WAEC. Alternative to practical appears to consolidate the cognitive domain especially in the area of application where students are asked certain questions about their knowledge in agriculture and certain areas where they can be applied instead of testing how they can do work practically.

This paper assumes that confidence should be built in WASC examination which is verifiable in other subjects like English, Physics, Chemistry, and Biology among others. What the paper is saying is that there is high reliability in WASC results in these areas because individuals with credit or above in these subjects could be ascertained as excelling in the subject.

The researcher observation has also been confirmed by other examining bodies like Joint Admission and Matriculation Board (JAMB) i.e. current research between performance in JAMB and WASC results with some of the subjects mention above in English, Physics among others show a high correlation (rho) of 0.81 i.e. This cannot be true if credit or above in Agricultural Science, Home Management, Business Management, Industrial Technical subjects when it comes to performance of the job. The correlation between performance on the job especially Agric Science and credit and above in Agric Science and practice on the job has a low correlation (rho) of 0.21 ( Elom 2016). This shows that alternative to practical did not measure practical in Agric Science in West African School Certificate Examination, therefore, there is need to design other strategies that should be alternative to practical which means a strategy that will measure practical.

2. Rating Scale and its Limitations

Many scholars may wonder why the examining bodies have not been using other instrument like rating scale or observation to measure practical. Rating scale according to Ali (2016) is an instrument for measuring personality traits such as feelings, attitudes and preferences, among others. Okpala, Onocha and Oyedeji (1993) stated that rating scale is an instrument for measuring expressed behaviors in order to determine how good or bad such expressed behaviors are. It is a useful instrument for judging attributes to be appraised in a person or group of persons.

The researcher like to support the non-use of rating scale by WAEC and other agencies for measuring practical in secondary school subjects like agricultural science and others because research efforts by most of the researchers and colleagues such as Elom (2016) and Okpala, Onocha and Oyedeji (1993) among others have revealed some limitations of rating scale that could render obtained result serviceably invalid as follows: (1) rating scale as reported by Iupui.ed (2017) has the following limitations ; (a) Limited reliability. (b)May be substantial variations among informants. (c) Do not assess sources of behaviour problems (d) Unqualified users may use and interpret these scales, and (f) Not suitable for sophisticated treatment planning.

Rating scale as observed by Okpara, Onocha and Onyedeji (1993) has the following limtations; (a) Hallow effects (b) Sympathetic rating and (c) Highly expensive for parents and examining bodies to conduct. Equally, Elom (2016) found out that rating scale has; (a) block loading effect. It exaggerates scores when a four point rating scale is based on 100%. Each block therefore has 25% for example, if a student nearly present himself/herself for practical with little or low performance that student obtains 25% which is an exaggeration based on the activity acerbated, the other student with slight performance will move to 50%. Such scores are very invalid.

The other faults associated with rating scale by the author especially in Agricultural Science is that the production process of crops like maize, rice or animals from breeding to maturity cannot be examine practically through rating scale with a time limit of three hours for practical examination, hence it has to adopt piece meal approach which may be time consuming, too expensive and boredom to the examiners and examinees. Probably these limitations scared away WAEC and other agencies from using rating scale for subject like agriculture, home management among others to examine student since WAEC and other agencies have been very proud of the validity and reliability of their measuring instrument such as norm reference test or cognitive driven objective test.

2.1. Introduction of New Trades in Secondary School

In 2014, the West African Examination Council introduced 39 additional subjects to the curriculum of secondary schools to prepare the students for challenges in order to meet up with the modern-day realities. Eguridu (2014) also stated that the introduction of additional 39 subjects will prepare the students for skill acquisition and entrepreneurial studies. The author further stated that the review was done by the Nigerian Educational Research and Development Council (NERDC) which is the body responsible for the review of primary and secondary school curriculum in the country. The NERDC introduced certain trades in these vocational areas into the school for students to learn for the following reasons.

• The nation is expanding in population while the needs of the youth are increasing in the area of occupation, work and maintenance, while adults in the field are growing old in meeting up with the needs, because of the use of primitive technology. The use of this technology could only produce little for the farmers family and very little for the teaming population. Youths seems not to be interested in imbibing of the primitive technology of their parents because of their education. The education acquired by them are knowledge based and theoretical in nature but could not meet their occupational and production needs, therefore, the NERDC has introduced change into learning in order to equip youths with productive skills for employment.
• Also, the nation has not been able to produce enough so as to export to other country that need them because of inadequate human power in the production cite, had it been that the majority of these youths who are in schools are empowered with technological skills and facilities to produce, the nation’s needs from her soil and environment, therefore NERDC has started these revolution through the introduction of trades and teaching of these trades in the schools.
• Many other African countries are looking up to this nation to help meet some of their needs that can be produced within the tropics and for the people in the tropics. The nation must rise to these challenges and hence, the challenge should begin with the trades in secondary school, so that by the time these youths reached adulthood production technology could have been part of their way of life which they could easily transfer to those coming behind.

The revolution or the challenges so to say could not be accomplished by NERDC alone without active participation of other agency such as WAEC. West African Examination Council is now being challenged to look further in words for innovative strategies that could be utilized by teachers to measure performance skills through technology like computer based test examination (CBTE), this innovative strategy is invoke for measuring skills in many developed countries. The major problems we have in this country at the moment are;

1. How to develop these tests in various occupational areas?
2. How to make others learn how to develop it.

2.2. Psycho-Motor Multiple Choice Test

Development of CBTE in skill areas involved what is known as psycho productive multiple choice test which is known as criterion reference test. edglossary.org (2014) stated that criterion referenced test and assessments are designed to measure student performance against a fixed set of predetermined criteria or learning standards. In any skill – oriented occupation, learners are expected to acquire psycho productive skills during training for effective performance on the job. To ascertain the level of acquisition of skills, learners can be accessed through certain recognized assessment procedures. One of such assessment procedures in skill acquisition is the psycho-productive multiple choice test.

Psycho-productive multiple choice test in the view of William, Ombugus, Umara (2016) is an instrument for determining the extent to which students can demonstrate their practical competence in Agriculture using production process skills of psycho-multiple choice test item. According to the authors, psycho-productive multiple choice test is a device with process skills to be responded to by the learners. Psycho-productive multiple choice test was introduced by Simpson in (1972). Simpson (1972) utilized psycho-productive multiple choice test to measure the performance of students in home economics based on the seven levels of psycho-motor taxonomy. These levels are perception, set, guided response, mechanism, complex overt response, adaptation and organization.

In the opinion of Olaitan, Nwachukwu, Igbo, Onyemachi and Ekong (1999), some vital steps that should be followed before writing psycho-motor multiple choice test items are: be followed before writing psycho-motor multiple choice test items are; the identification of occupational areas within a specific skill, establishment of a table of specification, construction of test items; about two or more items on each objective listed. assemble the test items, write clear and concise direction for each type of questions, construct key, validation of the test items (have another teachers to criticize the test), conduct a pilot study, and make versions based on result of pilot study, that is, carry out standardization.

The authors stated further that psycho production multiple choice test can be constructed in form of multiple choice test based on Simpson’s (1972) taxonomy of the psychomotor domain, which includes physical movement, coordination and use of the motor –skill. Development of these skills requires practice and is measured in term of speed, precision, distance, procedure or technique in execution. This psychomotor skills range from manual tasks, such as digging a ditch or washing a car, to more complex task such as operating a complex piece of machinery or dancing (Clark 2015) This psycho motor domain according to Simpson are in seven levels as follows: perception, set, guided response mechanism, complex overt response, adaptation and origination.

Perception: This is the ability to use sensory cues to guide motor activity. This ranges from sensory stimulation through cue selection to translation. In test development, illustrative verbs in perception include choose, describe, detect, differentiate, distinguish, identify, isolate, relate, separate and recognize.

Example: One of the following facilities is not very important in sitting a poultry house.

• Electricity
• Access road
• Land
• River *

Set: This is the readiness to act; it includes mental, physical and emotional sets. These three sets are disposition that predetermine a person’s response to different situations sometimes called mindsets. It requires the learners to demonstrate awareness (mindset) or knowledge of the behaviour needed to carry out the skill. The illustrative verbs associated with set include: begins, display, explain, move, proceed, states react, respond, demonstrate, show and volunteer.

Example: The particular time a doe is ready to accept a buck for mating is called

• Ovulation period
• Parturition period
• Heat period *
• Lactation period

Guided response: This is the early stage of learning complex skills. It involves imitation, trial and error. At this level of psychomotor domain, adequacy of performance is achieved by constant practice. Common illustrative verbs used in developing a test in guided response include: copy, trace, follow, react, respond, assemble, build, calibrate, construct, dismantle, display, dissect, fasten, fix, grind, heat, manipulate, measure, mend, mix and organize.

Example: The incubator hatches better when the eggs are in one of the following positions.

• Large part facing operator
• Small part facing operator
• Large part standing downward
• Large part standing upward *

Mechanism: This is the intermediate stage in learning a complex skill. At this stage, learned responses have become habitual and the movements can be performed with some confidence and proficiency. In test development, the following illustrative verbs are used in mechanism: mixes, organize sketches, assemble, build, calibrate, construct, dismantle, display, dissect, fasten, grind, heat, manipulate, measure and mend.

Example: Preparation to receive day old chicks for rearing will include the following steps except

• Wash brooder house
• Provide heat source
• Prepare foot dip
• Formulate grower ration *

Complex overt response: This is the stage of skilled performance of motor acts that involves complex movement patterns. It expresses ‘doing’ in a continuous movement pattern of increasing complexity. Proficiency is indicated by a quick, accurate and highly coordinated performance without hesitation. Illustrative verbs used in developing tests at this stage include: assemble, build, calibrate, construct, display, dismantle; dissect, fasten, fix, grind, heat, manipulate, measure, mend, mix, organize and sketch.

Example: The production of a calf follows one of the following orders below

• Mating – fertilizing – gestating – calving *
• Fertilizing – mating – calving – gestating
• Mating – gestating – fertilizing – calving
• Calving – gestating – fertilizer – mating

Adaption: This is the ability of the individual to modify movement patterns to fit special requirement or a new situation. At this level, skills are well developed for action.

• Common illustrative verbs for developing tests at this level include adapt, alter change, rearrange, revise, organized and vary.

Example: A battery cage can be made of one of the following instead of metals

• Iron sheets
• Plastic and wire
• Wood
• Wood and wire*

Organization: This is the ability to develop original skills that replaces the skill as initially learned. It involved creating new movement patterns to fit in particular situations for specific problems. Learning outcomes emphasize creativity based on highly developed skills. Illustrative verbs used in developing tests at this level include: arrange, combine, compose, construct, create, design and originate.

In the view of Fatusin (1996), the psycho-productive multiple choice test (usually called criterion reference assessment) has the following merits:

• It can examine all activities carried out in an occupation from growing to processing.
• It measures those activities that cannot be rated by rating scale in one shut examination such as land clearing to harvesting in a single practical examination of about 3 hours without interruptions due to seasons during the growth cycle of the crop.
• It follows finite taxonomy like perception, set, guided response, mechanism, complex overt response, adaptation and origination.
• The items can be stored and re-used for subsequent examinations.
• It can measure students’ acquisition of process skills in large classes in agricultural occupations conveniently without stress.

3. Psychometric properties for developing Psycho-Motor Multiple Choice Test Items

A good psycho-motor multiple choice test items have to be guided by the psychometric properties: such as: Table of Specification, Item difficulty Index, Item discrimination Index and Item distracter Index.

3.1. Table of Specification

Table of specification, sometimes referred to as test blue print, is a table that helps teachers align objectives, instruction and assessment .Table of specifications as stated by Kansas University (2017) is a two-way chart which describes the topics to be covered by a test and the number of items or points which will be associated with each topic. Akem and Agbe in Alade and Omoruyi(2014) viewed table of specification as a guide to assist a teacher or examiner in the evaluation system. The table shows the total number of items to be allocated to each instructional objectives, it also suggest what might be covered under each item, take decision on what types of items to be used. In fact the blue-print stage” is the last and crucial stage in an evaluation plan since it enables the teacher to combine properly the objective and the content areas, bearing in mind the importance and the weight attached to each areas. Okpala, Onocha and Oyedeji in Alade and Omoruji (2014) noted that table of specification enables the test developers to complete the cells in the table and decide the percentage of the total number of items that will go to each of the cells. According to Quijano (2014) steps in preparing table of specification are as follows;

1. List down the topics covered for inclusion in the test
2. Determine the objectives to be assessed by the test
3. Specify the number of days/hours spent for teaching a particular topic
4. Determine percentage allocation of the test items for each of the topics covered
5. Determine the number of the items for each topic
6. Distribute the numbers to the objectives

Equally Gareis and Grant in Okeme (2011) stated the steps that will guide the teacher in developing a table of specification to include;

• Develop learning objectives based on the taxonomy of educational objectives.
• Identify instructional activities that target the learning objectives
• Implement the instructional activities
• Reflect on instructional activities and identify relevant learning objectives that will be assessed based on the instructional experience.
• Determine relative importance and weightings of each objective.
• Generate test items based on the designed learning objectives and weightings.

Simpson (1972) indicated relative weight to be attached to the test items so as to guide the test developer;

• Perception (5-10%)
• Set (5-10%)
• Guided response (20-30%)
• Mechanism (20-30%)
• Complex overt response (20-25%)
• Adaptation (5-10%)
• Originality (5-10%)

The study therefore adopted the above weightings of Simpson’s taxonomy to develop 46 test items on animal husbandry for Senior Secondary School Students, to be considered by West African Examination Council in administrating their multiple-choice test questions in Animal Husbandry. Based on the weighting indicated above, the researcher came up with the following distribution of the items according to the table of specification as follow:

• Perception 4 items
• Set 4 items
• Guided response 12 items
• Mechanism 12 items
• Complex overt response 10 items
• Adaptation 4 item, making a total of 46 items (Appendix A)

3.2. Item Analysis

The item analysis is an important phase in the development of an exam program. In this phase statistical methods are used to identify any test items that are not working well. If an item is too easy, too difficult, failing to show a difference between skilled and unskilled examinees, or even scored incorrectly, an item analysis will reveal it. The two most common statistics reported in an item analysis are the item difficulty, which is a measure of the proportion of examinees who responded to an item correctly, and the item discrimination, which is a measure of how well the item discriminates between examinees who are knowledgeable in the content area and those who are not. An additional analysis that is often reported is the distracter analysis. The distracter analysis provides a measure of how well each of the incorrect options contributes to the quality of a multiple choice item. Once the item analysis information is available, an item review is often conducted. (Professional Testing Inc. 2015)

Item Difficulty

The item difficulty index is one of the most useful, and most frequently reported, item analysis statistics. It is a measure of the proportion of examinees who answered the item correctly; for this reason it is frequently called the p-value. As the proportion of examinees who got the item right, the p-value might more properly be called the item easiness index, rather than the item difficulty. It can range between 0.0 and 1.0, with a higher value indicating that a greater proportion of examinees responded to the item correctly, and it was thus an easier item.(Professional Testing Inc. 2015) According to wood in Matlock-Hetzel (1997), Item difficulty is simply the percentage of students taking the test who answered the item correctly. The larger the percentage getting an item right, the easier the item. The higher the difficulty index, the easier the item is understood to be To compute the item difficulty, divide the number of people answering the item correctly by the total number of people answering item. The proportion for the item is usually denoted as p and is called item difficulty (Crocker & Algina, in Matlock-Hetzel 1997). An item answered correctly by 85% of the examinees would have an item difficulty, or p value, of .85, whereas an item answered correctly by 50% of the examinees would have a lower item difficulty, or p value, of .50.. The authors concluded that criterion group usually consists of the upper and lower achievers in the test.

It was recommended by experts that the average level of the difficulty of a test for a four options multiple choice test should be between 60% and 80%. An average difficulty within this range can be obtained when the difficulty individual items fall outside of this range.

3.3. Item Discrimination

Item discrimination is an index that states how well the item serves to discriminate between students with higher and lower level of knowledge. The item discrimination index is a measure of how well an item is able to distinguish between examinees who are knowledgeable and those who are not, or between masters and non-masters. There are actually several ways to compute an item discrimination, but one of the most common is the point-biserial correlation. This statistic looks at the relationship between an examinee's performance on the given item (correct or incorrect) and the examinee's score on the overall test. For an item that is highly discriminating, in general the examinees who responded to the item correctly also did well on the test, while in general the examinees who responded to the item incorrectly also tended to do poorly on the overall test.(Professional Testing Inc. 2015).

The possible range of the discrimination index is -1.0 to 1.0; however, if an item has a discrimination below 0.0, it suggests a problem. When an item is discriminating negatively, overall the most knowledgeable examinees are getting the item wrong and the least knowledgeable examinees are getting the item right. A negative discrimination index may indicate that the item is measuring something other than what the rest of the test is measuring. More often, it is a sign that the item has been mis-keyed. (Professional Testing Inc. 2015)

Items discrimination is greatly influenced by item difficulty. In order to determine the discrimination index, you must use guideline. Shadish and William (2002) said the guideline include;

• 0.30 to 0.45 – Moderate positive discrimination
• 0.20 to 0.29 – Borderline positive discrimination
• To 0.19 – Low to zero positive discrimination and below – Zero to negative discrimination.

The literature reviewed above on Simpson’s (1972) taxonomy of psychomotor domain enable the researcher to develop psycho-productive multiple choice test items in animal husbandry using similar illustrative verbs for the assessment of students’ performance in the occupation. It is of the opinion of the researcher to advocate that to WAEC, of which if adopted will help to give an adequate assessment to student, and to determine who has actually acquired the skills required in the occupation, hence this study.

4. CONCLUSION AND RECOMMENDATION

New trades have been introduced for learning in secondary schools, to equip students with skills for work, but there is paucity of skill measuring instrument for determining skill performance of students instead of using alternative to practices.

This paper has reported the suitability of , or adequacy of psychomotor productive multiple choice test or criterion reference test for measuring skill performance in Agriculture, to ascertain the quality of skills possessed by students for work after graduation, if this Psycho-productive tests are developed for various trades or occupation in Agriculture, such as Fishery, Animal husbandry, Crop Production,, Processing among others, it will help teachers to teach the students towards mastering of skills for success in West African School Certificate Examination, thereby indirectly preparing students for work i.e not to look for success in examination, It is therefore recommended that West African Examination Council should:

• Use Psycho productive or psychomotor test items to assess students’ performance in Agricultural trades, such as Fishing and Animal husbandry in Senior Secondary Schools.
• Organize training workshops for teachers in the development and validation of Psycho-productive test for assessing students’ skill performance in the different trades they teach in Senior Secondary School.
• Help to mobilize stakeholders in Senior Secondary education for students such as Federal and State Government, Religious organizations, Proprietors of Private Schools, Community Leaders, Parents Teachers Association, among others, to accept this new initiative and commit themselves maturely into the success of adoption of Psychomotor production or criterion reference measurements for measuring skill performance of students in Agricultural trades for success in WAEC and for work after graduation.

REFERENCES