A Turing-inspired test of the i4Driving driver models in driving simulators
The i4Driving project has built a modular simulation library that captures the complexity of road traffic and the uncertainties of human behaviour and driving scenarios. The aim is to help stakeholders develop, test and validate automated driving systems that can operate safely in mixed traffic. By grounding validation in realistic human behaviour, i4Driving supports robust CCAM systems and contributes evidence toward future AV licensing approaches.
Why this matters?
Automated vehicles will have to coexist with human drivers for many years. This means they must not only be safe but also behave in ways that people recognise as “humanlike”, accepting gaps naturally, signalling clearly, and responding smoothly to others. Traditional validation tells us how different a model is, but not why. To close that gap, we designed a Turing-inspired driving simulator method to investigate if people can distinguish between a human-driven vehicle and an automated vehicle controlled by the i4Driving model when they interact with it.
How we tested it
Participants drove an ego vehicle and repeatedly interacted with a single, clearly coloured target vehicle controlled either by a human confederate (in a second simulator) or by an i4Driving model connected through the Open Traffic Simulator. We used three familiar motorway situations (A: On‑ramp, B: Truck overtaking, C: Weaving) and repeated each situation so participants could vary their interaction strategy (e.g., help or hinder a merge) and observe how the target vehicle responded. Throughout the experiment, participants continuously verbalised their thoughts, describing what they observed, what they intended to do and how they expected the target vehicle to respond. They were also asked after each situation to rate the behaviour of the target vehicle in the specific situation. The experiments were conducted at two test sites:
• TUM (Germany): 32 general‑public drivers (30 analysed) interacted with either a human or the i4Driving model with default settings. They rated realism, predictability, safety, and aggressiveness after each scenario and used a think‑aloud protocol while driving.
• VTI (Sweden): 20 driving school instructors (plus 4 RDW vehicle inspectors for qualitative feedback) interacted with three agents: Human, i4Driving Careful & Competent, and i4Driving Less Cautious. Participants provided post‑scenario ratings (predictable, safe, “like a human”) and post‑agent ratings (recognition of traffic cues, speed, smooth/decisive, road‑space use, defensive), with think‑aloud throughout.
The experimental design was slightly refined between the two experiments. At VTI, the i4Driving model parameters were updated to resemble a Careful and Competent driver or a Less Cautious driver, as opposed to the default parameters representing the average driver that were used at TUM. Further, at TUM the agent × scenario order was fully randomised while at VTI, participants completed all scenarios per agent, enabling post‑agent evaluations and clearer qualitative contrasts across agents.
Figure 1 Overview of the Turing-test-inspired driving simulator experimental set up, including the three scenarios experienced in which the white ego vehicles are controlled by the participants, the pink agent vehicles are controlled by either the i4Driving model or by a human confederate driver
What we found
In routine conditions, models can be hard to tell from humans
At TUM, a classification based on realism ratings showed only moderate ability to distinguish human from model (accuracy ≈ 0.67). In the VTI study, participants often favoured the human agent on human‑likeness/predictability, but equivalence tests generally did not support practical equivalence between the models and the human driver. In essence, models can appear human‑like when interactions are simple.
Differences show up in more complex interactions
In on‑ramp and weaving situations, participants noticed non‑human‑like cues: the model sometimes hesitated to accept an obvious gap, braked right after merging instead of accelerating away, or was unresponsive to social pressure from vehicles behind. In the TUM experiment (Error! Reference source not found.) the participants tended to judge the Human agent as more predictable, safer, more realistic but less aggressive than the i4Driving agents. A similar pattern was observed in the VTI experiment (Error! Reference source not found.) where participants tended to judge the Human agent as more predictable, safer and more human-like than the automated agents, particularly in scenarios A (on-ramp merge) and C (weaving).
Figure 2. Participants ratings per scenario and agent type in the TUM experiment
Figure 3. Participants ratings per scenario and agent type in VTI experiment.
Human‑likeness depends on timing and social responsiveness.
Beyond general predictability and safety, participants focused on these aspects when determining if the models behaved human-like:
• recognising traffic cues in time,
• natural gap acceptance and decisive lane changes,
• post‑merge pace management (accelerate away after merging),
• cooperative/defensive spacing under pressure, and
• clear, timely signalling (indicators, brake lights).
Experts pinpointed where to improve the models.
Driving school instructors and vehicle inspectors pointed to tactical decision‑making and anticipation: take the offered gap when it’s clearly safe; don’t brake after merging; respond better to social pressure; and signal reliably. From a methodological point of view, they also suggested longer interaction windows and a more structured plan‑ahead phase so they could probe each agent consistently.
What it means for i4Driving model development
The i4Driving models already look competent in many situations, but they need refinement where timing and social negotiation matter most. Concretely, we should:
• increase sensitivity to indicator/brake cues and relative decelerations,
• tune gap acceptance and pre-merge initiative to avoid standoffs,
• enforce post-merge acceleration (clear the lane, restore flow), and
• ensure complete signal integration in co-simulation so signalling is seen and understood.
Main takeaways for future Turing-inspired driving simulator tests
• Allow longer interactions and more repetitions, and where time allows, let participants repeat a scenario until they are satisfied before giving their final ratings.
• Extend the practice phase for each scenario so participants can develop a clear testing plan that they then apply consistently across all agent types.
• Start with simple interactions and progressively increase difficulty, moving toward challenging boundary scenarios.
• Combine quantitative ratings with qualitative insights,
Summary
The interactive Turing-inspired method, especially when paired with think aloud, complements objective metrics by explaining why behaviour is (or is not) perceived as humanlike and by pinpointing tactical and social gaps. This aligns with prior work showing that perceived human-likeness depends more on interaction complexity and timing than on kinematic similarity alone. These results will guide further development of the i4Driving models and inform refinements to the interactive study protocol.