Background
Traffic accidents are a leading cause of death among young people, partly due to underdeveloped executive functions like planning and impulse control. Young drivers often underestimate risks and overestimate their abilities, highlighting the need for more accurate driving tests. One solution is to use driving simulators alongside Sweden's current on-road and theoretical exams. While simulator tests show promise, further development is needed. A screening test has been created to identify risky drivers, using challenging situations to assess hazard perception and risk awareness. To make it viable, multiple versions of these scenarios must be developed and validated.
Aim
This study aimed to assess how small variations in the simulator screening test affect the results. This knowledge is essential for creating numerous unpredictable, fair, and reliable variations of the screening test. The following research questions were formulated:
- How do small changes in the road environment or traffic impact test outcomes
- To what degree do the different test versions maintain consistent difficulty levels?
Method
A between-group design was used, with participants divided into three groups, each completing one of three versions of the same simulator test. All three versions included the same 14 situations, with small variations in the road environment or surrounding traffic. The independent variable was the version (1-3) and the situations (S1-S14), while the dependent variable was participants' overall performance and performance on individual situations (pass/ fail).
Participants
Data were collected at five automotive high schools, recruiting 126 students—92 men and 14 women. All participants were 17 years old and had extensive prior experience with the simulator used (Figure 1).
Equipment
A fixed-base high-fidelity driving simulator with Skillster software was used (skillster.se). Three tests versions were administered, each containing 14 challenging and potentially dangerous situations, set in urban traffic, on country roads, and highways. For instance, one scenario tested merging onto a highway, another handling a left turn on a country road with oncoming traffic, and a third avoiding a hidden pedestrian. The 14 scenarios had slight variations across the versions, such as different colored cars and houses or buses replaced by trucks (Figure 2). The goal was to introduce small changes between version1 and 2, and slightly more changes in version 3. In one situation, version 3 was made even more complex to test the extent of possible variations.
Procedure
Before starting the simulator test, participants were informed of their anonymity and right to withdraw. After giving consent, they were randomly assigned to one of three versions, each taking approximately24 minutes. Each situation had specific evaluation criteria, such as giving way, driving at a safe speed, and maintaining distance from other vehicles. To pass the entire test, participants had to pass all 14situations. Participants were not informed of their results.
Analysis
Data was compiled in Excel, with separate files for each version, and then analyzed in Jamovi. Rasch analysis was used to assess the difficulty of the 14 situations across the three versions, producing three corresponding result tables (Table 1-3). Situation difficulty was evaluated using the Measure parameter from the Rasch analysis, where higher values indicate more difficult situations and lower values indicate easier ones.
Results
Results show similarities between situations across versions. For instance, situation 13 was the most difficult in all versions, though slightly more so in version 3. Situation 11 was consistently the easiest. The pass rate for each situation was also similar across versions. However, some variations in difficulty warrant further examination. For example, the difficulty of situation 8 increased from version 1 to 2, and more so from version 2 to 3. Situations 3 and 9 were more difficult in version 3 than in the others.
Conclusion
Small variations in simulator test situations do not significantly affect the outcome. However, adding more changes and increasing complexity can alter the difficulty level. These insights are crucial for creating multiple unpredictable simulator screening tests with consistent difficulty. Further analysis will be conducted, and additional results will be presented at the conference.