Relationship between Test Item Arrangements and Testee's Performance and Test Usefulness Criteria

Abstract

Test method, test content and test type are supposed to affect test taker's performance and have close connections
with test characteristics. However, these issues have not been subject to much in-depth investigations. To shed some
lights on the issues like test item arrangements in relation to test taker's performance, test usefulness criteria
including test Validity, Reliability, Impact, Interactiveness, Authenticity, and Practicality, and also test characteristics
such as Item Facility (IF), Item Discrimination (ID), and Choice Distribution (CD), this ex post facto design study
was conducted with a group of Iranian EFL learners. For this purpose, some university randomly selected students
majoring in English Language Teaching, English Translation Studies, and English Literature and learners from
different language institutes received a version of the Nelson proficiency test. Then, two different versions of a same
researcher-made test in terms of item arrangements were administered to 116 students obtaining an acceptable score
in the Nelson proficiency test. Simultaneously, a Test Usefulness Criteria Questionnaire developed and validated by
Abbasian and Nassirian (2015) based on Bachman and Palmer’s (1996) framework, was also attempted by the
participants. Respective statistical analyses revealed contradictorily that test item arrangement did not have any
significant effect on the performance of the test takers. However, test usefulness criteria, though with varying extent,
proved to be subject to test item arrangement. In the same vein, IF and ID were also affected in the light of test
method facet. The first contradictory finding leaves room open for further research, while the reaming findings offer
insights to test developers, classroom teachers and all practitioners to pay due attention to test method facets in their
educational assessment decisions as the learners' performance, nature of the measurement devices as well as the
nature of the construct itself are all affected test method factors.