StudentShare
Contact Us
Sign In / Sign Up for FREE
Search
Go to advanced search...
Free

Reading, Test Design, and Evaluation - Assignment Example

Cite this document
Summary
This assignment "Reading, Test Design, and Evaluation" presents the pilot test that was necessary for that it would assess the draft provision and make corrections based on the stakeholders’ realistic experience and recommendations. Upon drafting these specifications, sample materials were produced…
Download full paper File format: .doc, available for editing
GRAB THE BEST PAPER97.3% of users find it useful

Extract of sample "Reading, Test Design, and Evaluation"

Heading: Reading, Test Design & Evaluation Your name: Course name: Professors’ name: Date Final draft of the prototype (has been edited) Complete the dialogue between two friends: John and Mary. What does John reply his friend Mary? In questions 1-5, tick the right answer on the answer sheet provided. Samples Sample 1: Mary: Hey, John. It is nice to see you again. How did your vacation go? John: 0……….B 0 A B C D E F G H Mary: How is your family? John: 0…………….D 0 A B C D E F G H In the above question, the correct answer is D Mary: Where did you visit this year? Your uncle’s place again? John: I……… Mary: How was the weather? John: II……………. Mary: Fantastic! Did you take photographs? John: III………… Note: I. B; II.E; III.G; IV. H; V.C Mary: Please do! You stayed in a hotel, right? John: IV……………… Mary: That sounds like you had a great time. So, will visit there again? John: V………. Statement of rationale for the design The choice for design of the above prototype test is determined by a number of factors discussed below. Primarily, a Criterion-Referenced Test (CRT) was adopted because an individual test had been designed for each student. Apart from determining whether or not each student achieved the specific skills or concepts, CRTs measured the specific ski;;s that make up a curriculum. Similarly, the test did not consider the performance of other examinees and instead compared each individual with a preset standard of acceptable achievement. The individual test for each student will aid in effectively measuring the intended language skill as the students will be required to write down their answers on their answer sheets (Harmer, 2007). Further, as Harmer, (2007) says, the prototype test assumes a multiple choice testing design. Here, the students are provided with complete and incomplete statements in form of a conversation. They are then required to complete the conversation by choosing the most appropriate answer or completion firm from the alternatives provided consisting of the destructors and key. The test consists of five-question dialogue (one answered question in sample 1 and another question in sample 2) between two friends. The students have to each answer the questions on the provided answer sheet, choosing the correct answers from the multiple choices. In terms of scoring, design adopts the objective test method. Under objective test, the scoring procedure is fully specified to enable agreement among various scorers. It implies that the tester makes the answer key, and the students should select the best or appropriate answers should be similar to the one in the answer key (Hughes, 2003). It is also a language skill test as it enables the tester or teacher to gain information regarding the ability levels of students to use language whenever in a particular situation. This information indicates the real ability level at an actual time. There are three forms of language skill test including writing skill, reading skill, and speaking skill tests. The above prototype test involves a reading and speaking skill since it is about an incomplete conversation. The students are required to read the conversation and complete it with the best answer in the available answer key (Harmer, 2007). The design has also met vital aspects of a suitable prototype for a specific class or students. Firstly, the test considered validity, which entails test measuring what was intended to measure. The test ought to aid inference to certain domain of the intended language use. This implies that for the tester to determine the validity of the test outcomes, it is imperative to state the expectations of a tester on the students’ capacity to use language in real life situations. The tester should then find out whether the test given produces suitable evidence of their potential to do so. Therefore, in this test, validity is achieved from the fact that the test intends to measure students with skills beyond classroom, but in practical life situations (Brown, Hudson, Norris, & Bonk, 2002). Secondly, it is designed to meet reliability requirement. Reliability entails consistency, which requires the test to have reliable scores in order to yield similar results if repeatedly used. This implies that the test would often rank-order a number of candidates in almost the similar way. This does not imply that the same candidates will fail or pass, as the pass mark will be put separately. Dependability has been used in test because of the need for accuracy and consistency. The design of the test also meets ethics and fairness. Normally, test givers ought to ensure that pertinent principles are broadly spread and comprehended in their organizations (Hughes, 2003). In terms of fairness, all tests should fair in terms of the three dimensions: unbiased, equitable handling of the testing process, and fair results. Test givers should try to reduce bias during test designing, for instance avoiding local issues that are unfamiliar to a section of the test takers. In relation to this design, the test considered both ethics and fairness as the questions are applicable to all the target candidates (Brown, 2005). Description of the pilot trial of the test The pilot test was necessary in that it would assess the draft provision and make corrections based on the stakeholders’ realistic experience and recommendations. Upon drafting these specifications, sample materials were produced. Piloting occurred through test takers similar to the intended candidates. This was also important in determining the effectiveness, feasibility, and adequacy of the test techniques. It was also vital in determining whether test technique was realistic and practical. It would also help in the identification of logistic issues that could happen while using the proposed techniques. In addition, pilot testing was essential in the determination of the necessary resources in the main test. It also helps in the development of questions and test plan. It is also indispensable in convincing the stakeholders on the feasibility and effectiveness of the test. It is essential in convincing the stakeholders that the test is knowledgeable and competent; hence the need to support it (Taylor, 2005). According to Hughes (2003), the pilot test was conducted using the following procedures: 1. Finding five students from the same group of candidates who were to be actually measured (target students). 2. Sample students were arranged to do the test under similar conditions as they would in actual test. Some of the factors considered here include location, time, and the test method. In this case, the method was writing, and thus, students were required to complete the conversation with suitable answer in the alternatives given. 3. After all respondents had completed the test, time was taken with each one of them to discuss their experiences. Some of the questions asked concerned the time taken to complete the test; the purpose of the test; clarity of directions; clarity of instructions; and challenges encountered during the test. 4. The completed tests were collected, and responses were read to determine whether students answered correctly. 5. The data was analyzed and the pilot test results were presented as it would be in the real test. 6. The pilot test results were shared with the stakeholders who would the use the data. The test was them modified using the information gathered from the students (Alderson, 2000). Results The test was trailed on a group of five students from the target group. In the reporting of the performance test outcomes, the pilot test conducted was conducted in order to give readers more confidence in the reported results. As per the test results, three out of five students scored high marks because they seemed to follow instructions and understood the questions clearly. Some found problems completing the conversation because they did could not follow the conversation. The results indicated that the pilot test was not very knowledgeable, effective, feasible, and competent (Hughes, 2003). Recommendations As Hughes (2003) maintains, to achieve the best results, the pilot test should be modified to enable all students understand the questions. More needs to be done to equip learners with reading and writing skills, especially through several exercises on conversations. This will also enable them to relate whatever they learn in class to the realities of life. Generally, there is need for slight modification of the test in order to measure the intended abilities of the target candidates and achieve the objectives. ASSESSMENT CRITERIA 1. Quality of the designed test and rationale (15%) a. Test type, purpose, duration, conditions and test taker characteristics are clearly defined (4) √ b. Clear task instructions and formatting (2) c. Scoring method/weighting of tasks are defined (2) √ d. Results of the pilot test are present (1) √ e. Recommendations for improvement have been made (2) √ f. Formal requirements of the assignment are met (as per Unit Outline section 6h) (2) √ g. References are appropriate and presented in the required manner (APA style) (2) √ 2. Meets task requirements (10%) a. Test includes two sample items (1) b. Test focuses on both language points and functional skills (2) c. Test is performance-based and practical (2) d. Test has been revised according to the recommendations (2) √ e. Original and revised test have been appended to the assignment (1) f. Assignment appendix includes all testing instructions, marking schemes and descriptors to enable the test’s administration by a third party (1) √ g. Signed statements of participatory consent have been provided by all pilot test participants (1) √ 3. Clear, substantial, critical, systematic presentation and discussion (15%) a. All stages of test development are present and the procedure is described in a logical, critical and systematic manner (4) √ b. Decisions made during each stage of the test development procedure are justified by references to the literature (2) c. Demonstrated understanding of the relationship between teaching methodology/language acquisition and testing (2) √ d. Evaluation of test results demonstrates an appropriate understanding of item analysis techniques (3) √ e. Recommendations are strongly associated with or justified by pilot test results and evaluation (3) √ f. Clear discussion outlining how the chosen pilot test participants conform to the test sample requirements (1) 4. Demonstrated understanding of language testing principles and practices (15%) a. Test rationale describes how the test adheres to the six qualities of test usefulness (e.g. reliability, validity, authenticity, interactiveness, impact, practicality) (6) √ b. All test development choices and task specifications (format, language focus, scoring method) are described and justified (9) √ 5. Appropriate use of testing terminology (5%) References Alderson, C. (2000). Assessing Reading. Cambridge: Cambridge University Press. Pp. 1-30. Brown, J.D., Hudson, T., Norris, J. & Bonk L. (2002). An investigation of Second Language Task-based Performance Assessments. Second Language Teaching and Curriculum Center, University of Hawaii.Pp. 20-50. Brown, J.D. (2005). Testing in Language Programs: A comprehensive Guide to English Language Assessment. NY: The McGraw-Hill Companies Inc. Pp. 10-40. Harmer, J. (2007). The Practice of English Language Teaching. (4th ed.). Harlow: Longman. Pp. 10-30. Hughes, A. (2003). Testing for Language Teachers. 2nd Ed. Cambridge: Cambridge University Press. Pp. 5-25. Taylor, L. (2005). Key concepts in ELT: Washback and impact. ELT Journal, 59(2), 154- 155. Appendices Appendix 1 First draft of the prototype (to be submitted for editing) Instructions: Complete the dialogue between two friends: John and Mary. What does John reply his friend Mary? In questions 1-5, tick the right answer on the answer sheet provided. Samples Sample 1: Mary: Hey, John. It is nice to see you again. How did your vacation go? John: 0……….B 0 A B C D E F G H In the above question, the answer is B Sample 2: Mary: How is your family? John: 0………D In the above question, the answer is D Mary: Where did visit this year? John: I……… Mary: How was the weather? John: II……………. Mary: Nice! Did you take photographs? Note: I. B; II.E; III.G; IV. H; V.C John: III………… Mary: You stayed in a hotel, right? John: IV……………… Mary: That sounds like you had a great time. So, will visit there again? John: V………. Mary: That will be more wonderful. Appendix 2 Final draft of the prototype (has been edited) Complete the dialogue between two friends: John and Mary. What does John reply his friend Mary? In questions 1-5, tick the right answer on the answer sheet provided. Samples Sample 1: Mary: Hey, John. It is nice to see you again. How did your vacation go? John: 0……….B 0 A B C D E F G H Mary: How is your family? John: 0…………….D 0 A B C D E F G H In the above question, the correct answer is D Mary: Where did you visit this year? Your uncle’s place again? John: I……… Mary: How was the weather? John: II……………. Mary: Fantastic! Did you take photographs? Note: I. B; II.E; III.G; IV. H; V.C John: III………… Mary: Please do! You stayed in a hotel, right? John: IV……………… Mary: That sounds like you had a great time. So, will visit there again? John: V………. Read More
Cite this document
  • APA
  • MLA
  • CHICAGO
(Reading, Test Design, and Evaluation Assignment, n.d.)
Reading, Test Design, and Evaluation Assignment. https://studentshare.org/english/2061104-reading-test-design-evaluation
(Reading, Test Design, and Evaluation Assignment)
Reading, Test Design, and Evaluation Assignment. https://studentshare.org/english/2061104-reading-test-design-evaluation.
“Reading, Test Design, and Evaluation Assignment”. https://studentshare.org/english/2061104-reading-test-design-evaluation.
  • Cited: 0 times

CHECK THESE SAMPLES OF Reading, Test Design, and Evaluation

Literature Evaluation1 Transmission of Aggression Through Imitation Of Aggrressive Models

Based on the cited resources, however, the authors found that there was a need for more effective tests and evaluation mechanism.... LITERATURE evaluation: Transmission of Aggression Through Imitation of Aggressive Models Mary Sue Andreozzi Your School February 10, 2013 1.... LITERATURE evaluation: Transmission of Aggression Through Imitation of Aggressive Models Mary Sue Andreozzi Your School February 10 Describe the background knowledge and what unanswered questions led the authors to conduct the current study....
3 Pages (750 words) Dissertation

The different methods of research design

Failure to differentiate between method and design elicits poor evaluation of a research design.... Name Instructor Course Date The Different Methods of Research design Response to Post 1 Research design differs from the methods by which a researcher collects the data.... Some researchers treat research design as a way of data collection instead of a logical framework of an inquiry.... hellip; Nothing is intrinsic about a research design that necessitates a particular data collection method....
3 Pages (750 words) Essay

Experimental Design

roblems: The study was conducted only for two month period in all patients admitted into the intensive care unit, which led to getting an evaluation of only of 103 patients.... The article which has been chosen for study of experimental design is the Lim, et al, 2008, Detecting myocardial infarction in critical illness using screening troponin measurements and ECG recordings article. There are several factors that make the clinical diagnosis of… This study evaluates the impact of a screening intervention using assays of cardiac troponin and electrocardiograms (ECGs) in the diagnosis of critically ill patients in comparison to clinical diagnosis alone. Assays of Experimental design Introduction: The article which has been chosen for study of experimental design is the Lim, et al, 2008, Detecting myocardial infarction in critical illness using screening troponin measurements and ECG recordings article....
2 Pages (500 words) Essay

Practice papers

They plan the course of action for the project.... Knowing the project demands and resources available, they control the activities. Pre-assembly is the manufacture prior to or fabrications of… It can be considered an initial gathering process of materials and other requirements or the project. Contractor concurrent with the time extension claim puts loss and expense claims forth....
4 Pages (1000 words) Assignment

Actual Tips for Interior Design

It is therefore important that the goal of an interior designer be to make sure that the fundamental design and the materials are classic, long lasting, and appropriate to a space in the building.... om/Interior-Design-Materials-Specifications there are various books on interior design and they usually cover all the information that is important as far as planning and execution of interior design projects of all sizes and shapes are concerned.... The following are examples of materials used in interior design and finishes....
4 Pages (1000 words) Essay

Government Policies in a Population Survey about Good Government

This is an example of a classic controlled experiment design (Campbell & Stanley 1963).... aximum internal and external validity may be obtained by the use research study designs that involve group studies rather than case studies or same subject design, subject randomization, and statistical analyses that are appropriate for the type of research study to which they are applied (Bulpitt 1996)....  The purpose of this essay is to test whether one's perception of standards of good government may be easily manipulated by exposing the test subject to highly favorable (or unfavorable) commentary immediately prior to being given a survey on personal attitudes towards government policies....
5 Pages (1250 words) Essay

West Music Recording Corporation

nbsp;… The procedure of how the test was carried out the two users were brought and told what to do.... The main objective of the questionnaire was to test if navigation, visibility and provision of what clients needed were right.... From the paper "West Music Recording Corporation" it is clear that the co-discovery learning method was chosen because it required a minimum of two users, which would be cheap....
7 Pages (1750 words) Case Study

Soils and Highways Laboratory: California Bearing Ratio

In order to design a road, it's important, to begin with, a sense of their sub-grade strength to support the load and an idea of which options are possible.... Most successful and sustainable highway designs are usually built on the California Bearing Ratio (CBR) test, which is an empirical test relating to the performance of the subgrade under test compared with a standard curve.... The particular test is conducted by measurement of the plunger forces needed to penetrate a soil sample mold at penetrations of 2....
6 Pages (1500 words) Term Paper
sponsored ads
We use cookies to create the best experience for you. Keep on browsing if you are OK with that, or find out how to manage cookies.
Contact Us