The developers at DrivingBench have integrated several advanced language models with a real 2022 Toyota Corolla, allowing these systems to manage the car's steering, acceleration, and braking. Among the models tested, GPT-6 Astra excelled, completing a 134.7-meter cone course in just 5 minutes and 22 seconds on its second attempt. During the trial, the model received visual input from the vehicle’s cameras along with telemetry data, enabling it to instruct the car on steering adjustments, speed, and when to halt. Initially, Astra navigated only 49% of the course but learned from its previous attempt and successfully finished the course on the second try. In contrast, Claude Fable 5.1 managed to cover just 45% of the route, while Grok 4.6 and GPT-5.6 Sol were unable to progress beyond the first turn. The testing occurred in an empty parking lot, where the vehicle traveled at a slow walking pace, with a human driver present to intervene if necessary. The DrivingBench team aimed to explore whether general-purpose language models could leverage their planning and tool-use capabilities in real-world scenarios. The findings suggest that these language models could have promising applications as versatile AI systems. OpenAI's Head of Applied Research, Boris Power, remarked that the results signify a significant setback for specialized models designed for narrow tasks.
Informational material. 18+.
