Main research problems at present
Date: 0000-00-00
The statement that the theoretical test in the Swedish Driving License Examination is a criterion-referenced test is the main basis and starting point for research. This implies that all problems connected with criterion-referenced testing also apply to the Swedish test. This includes questions about how to clarify and specify the objectives, how to define the cutoff score, and how to evaluate the quality of the items as well as the whole test. The focus is also on the process of judgment in the practical test as well as on the basis for making these judgments.
We are dealing with the following questions in particular:
A model for description of the objectives in the theoretical test
We are currently working on a model distinguishing between stationary and non-stationary conditions. The model also includes three factors; the driver, the vehicle and the environment and these factors are considered in isolation as well as in combinations with each other.
Methods for describing the quality of the items
One method for judging the reliability, validity and appropriateness of new items is to compare both the proportion of correct answers and the difference in answering patterns between beginners and educated pupils. On the basis of these results it is possible to judge the quality of the items and get information about required improvements. Another method is to include try-out items in the real test. These items will not be included in the test takers' score. Practical considerations will probably be the main reason for selection among methods in the future. At present the quality of the items is based only on judgments from experts.
Parallel versions of the theoretical test
Developing parallel forms is always a problem for a test constructor. One has to consider content, the mix of different cognitive levels, type of illustrations, kinds of questions asked, the mix of the number of correct alternatives, difficulty (both average level and the variation in difficulty in total and between the different parts) etc. This requires much pretest information. Larger groups, many new items, practical working routines, proper handling and appropriate methods for analyzing the results will then be of great importance. The answer-sheets must also be changed to a lay-out that can be used for optical reading. In summary, we are dealing with strategies for trying new items on a large-scale basis, with new methods for validation, calibration and selection.
Item banking
The basic requirement for constructing parallel versions of the theoretical test is that a selection of items can be made from a great amount of items. This implies methods and models for selection of items and we are working with the many problems associated with this question.
The item format
Another problem in the evaluation of the test and the test results is the tricky item format. It is not just a problem for the examinee who, for every item, in fact has to give four different answers (True-False for every single alternative), but it is also a problem for the evaluator to find out what is crucial in an item where there are so many possible combinations of wrong answers. This has also resulted in many comments on justice. "Why should I have no credit for the three correct answers that I gave to this item, just like those who marked all four wrongly". This is still an unsolved problem and research is done in order to find out, for example, what difference it would make to score each single alternative as true-false instead of treating them as four choices to one single question.
A change in item format must be considered, to make the questions more clear to the examinees and easier to deal with from a measurement point of view. There are many ways to deal with this problem. Either clearly ask the examinees to answer each alternative as a separate question, even if they deal with the same problem (in which case the items also have to be scored as separate items), or change to items with only one correct alternative out of four. In the latter case a larger number of items can be used since they can be answered more quickly.
A model for description of the objectives of the theoretical test
Our intention is, as compared with our intention for the theoretical test, to develop a model for the objectives.
The judgment process in the practical test
Our intention is to develop a theoretical model for the judgment process. We hope that this model can also be used as a basis for discussions of consensus, as well as a guide for the examiner in the practical driving situation and as general information to the examinee.
Return to the main article: "The driving license examination in Sweden"
Comments to this article have been disabled.
