Foundation Models for Trajectory Planning in Autonomous Driving

This repository provides an up-to-date list of approaches that utilise foundation models (FM) in trajectory planning for autonomous driving. It follows the taxonomy provided in the following paper:

Kemal Oksuz, Alexandru Buburuzan, Anthony Knittel, Yuhan Yao, Puneet K. Dokania

"Foundation Models for Trajectory Planning in Autonomous Driving: A Review of Progress and Open Challenges"

Transactions on Machine Learning Research, 2026.

Table of Contents

Organised according to the taxonomy presented in the paper:

1. Foundation Models Tailored for Trajectory Planning
   1.1 Methods Focused Solely on Trajectory Planning
   1.2 Methods Providing Additional Capabilities
2. Foundation Models Guiding Trajectory Planning
   2.1 Knowledge Distillation Only During Training
   2.2 Knowledge Transfer During Inference

1.Foundation Models Tailored for Trajectory Planning

1.1. Methods Focused Solely on Trajectory Planning

• Methods without Chain-of-Thought Reasoning:

  • CarLlava, CARLA challenge winner in 2024. [paper] [code]
  • DriveGPT4v2, CVPR 2025. [paper]
  • V2X-VLM, preprint in 2024. [paper]

• Methods using text output of the FM for the Chain-of-Thought Reasoning:

  • GPT-Driver, NeurIPS 2024 Workshops. [paper] [code]
  • DriveVLM, CoRL 2024. [paper]
  • Auto-VLA, NeurIPS 2025. [paper] [code]
  • RAG Driver, RSS 2024. [paper] [code]
  • S4 Driver, CVPR 2025. [paper]

• Methods using an initial trajectory prediction for the Chain-of-Thought Reasoning (with/without text output of the FM):

  • Agent-driver, CoLM 2024. [paper] [code]
  • FeD, CVPR 2024. [paper]
  • Solve-VLM, CVPR 2025. [paper]

1.2. Methods Providing Additional Capabilities

• Methods Providing Language Interaction Capability Only:

  • DriveGPT4, IEEE Robotics and Automation Letters in 2024. [paper] [code]
  • DriveLM-Agent, ECCV 2024. [paper] [code]
  • Emma, TMLR in 2025. [paper] [code]
  • OpenDriveVLA, AAAI 2026. [paper] [code]
  • DiMA-MLLM, CVPR 2025. [paper]
  • Omni-Q/L, CVPR 2025. [paper] [code]
  • Orion, ICCV 2025. [paper] [code]
  • ReCogDrive, ICLR 2026. [paper] [code]

• Methods Providing Action Interaction Capability Only:

  • DriveMLM, Visual Intelligence in 2025. [paper] [code]
  • LMDrive, CVPR 2024. [paper] [code]

• Methods Providing Language and Action Interaction Capabilities:

  • SimLingo, CVPR 2025. [paper] [code]

2. Foundation Models Guiding Trajectory Planning

2.1. Knowledge Distillation Only During Training

• Methods:

  • VLP, CVPR 2024. [paper]
  • VLM-AD, CoRL 2025. [paper]
  • DiMA, CVPR 2025. [paper]
  • Solve-E2E, CVPR 2025. [paper]

2.2. Knowledge Transfer During Inference

• Methods:

  • VLM-E2E, preprint in 2025. [paper]
  • DME-Driver, AAAI 2025. [paper]
  • Senna-E2E, preprint in 2024. [paper] [code]
  • DiffVLA, Runner Up in Autonomous Grand Challenge 2025. [paper]
  • DriveVLM-Dual, CoRL 2024. [paper]
  • Solve-E2E-Async, CVPR 2025. [paper]
  • DiMA-Dual, CVPR 2025. [paper]
  • HE-Drive, preprint in 2024. [paper]
  • VDT-Auto, preprint in 2025. [paper] [code]
  • FasionAD++, preprint in 2025. [paper]
  • AsyncDriver, ECCV 2024. [paper] [code]

How to request addition of a paper?

If you know of an approach built on a FM (VLM/LLM etc.) or utilizes a foundation model for knowledge transfer, and is not in this repository, you are welcome to request the addition of that paper. For such an addition, please open a pull request by briefly explaining which section of the taxonomy that the paper fits into.

How to cite?

@article{
    oksuz2026foundation,
    title={Foundation Models for Trajectory Planning in Autonomous Driving: A Review of Progress and Open Challenges},
    author={Kemal Oksuz and Alexandru Buburuzan and Anthony Knittel and Yuhan Yao and Puneet K. Dokania},
    journal={Transactions on Machine Learning Research},
    issn={2835-8856},
    year={2026},
    url={https://openreview.net/forum?id=E2L5J2O2Bk},
    note={Survey Certification}
}

Contact

Please contact Kemal Öksüz (kemal.oksuz@bosch.com) for your questions about this webpage.