Learning to Drive is a Free Gift: Large-Scale Label-Free Autonomy Pretraining from Unposed In-The-Wild Videos

Training self-driving cars today requires enormous amounts of specialized data: calibrated multi-sensor rigs, millions of miles of driving, and painstaking labeling. This is expensive and hard to scale to new cities and conditions.

Yet millions of hours of dashcam footage are freely available online, people driving across the world in all kinds of scenarios. Just as humans build road intuition long before getting behind the wheel, such video contains rich information about how roads look, how objects move, and how scenes unfold. The problem? It comes with no labels at all, which is why it has remained largely untapped.

The research described here, nicknamed LFG or "Learning to Drive is a Free Gift," bridges this gap. It uses a set of existing AI models as teachers, each one labeling a different aspect of unlabeled dashcam footage, and trains a new model to reconstruct the current scene and predict what comes next. When fine-tuned for driving, this single front-camera model outperforms systems built with far richer sensor setups. It resulted in a paper accepted to CVPR 2026.

A Data Bottleneck, And Millions of Hours of Untapped Footage

Building self-driving systems today requires vast amounts of expensive labeled data: 3D lidar scans, high-definition maps, hand-annotated bounding boxes, and expert driver trajectories. Collecting this data is slow, costly, and hard to scale across geographies and driving conditions.

Meanwhile, the internet is overflowing with dashcam footage. Millions of hours of dashcam footage are uploaded every year, covering an enormous diversity of roads, weather, traffic, and driving styles around the world. This data is free, abundant, and captures the real complexity of driving, but it comes with no labels at all. No depth, no poses, no semantics. Just raw pixels.

Large-scale pretraining on unlabeled data has already transformed language and vision. LFG asks: Can we do the same for autonomous driving? Can we learn geometry, motion, and scene understanding directly from unposed video, and use that knowledge to drive better?

How does a Model Learn from Unlabelled Videos?

LFG takes a few consecutive dashcam frames as input and produces a rich set of outputs for both the current and upcoming moments:

• 3D structure: A dense reconstruction of the scene from each viewpoint
• Camera motion: Where the car is heading and how it's moving
• Scene semantics: What's in the scene: road, vehicles, pedestrians, sky, buildings
• Confidence: How reliable each prediction is
• Motion masks: Which parts of the scene are moving independently (other cars, pedestrians) vs. the static world

None of this requires ground-truth labels. Instead, the system uses a “teacher-student” training setup: Several existing models, each strong at one specific task, provide pseudo-supervision on unlabeled video. One teacher handles 3D geometry, another provides semantic understanding, and a third detects and tracks moving objects. The student sees only the first few frames of each clip, while the teachers see the full sequence. This forces our model to learn to anticipate what happens next.

The entire system is trained on roughly two million clips from freely available driving videos, spanning varied roads, conditions, and traffic situations.

A Better Understanding of the Future

• Seeing into the future. LFG learns to anticipate how a scene evolves over the next few frames. On semantic segmentation, it actually surpasses its teacher on future frames—even though the teacher was given the real images and LFG had to predict without seeing them. On depth, it closely matches its teacher, even on predicted future frames.

• Understanding motion. LFG learns to separate moving objects from the static background — a critical capability for safe driving. It correctly identifies vehicles and pedestrians in motion, even in cases where the automated labeling pipeline itself gets it wrong.

• Planning with less. The ultimate test of a driving representation is whether it helps a car plan safe trajectories. On the NAVSIM planning challenge, LFG with only 10% of labeled data (81.4 PDMS) already matches what other pretrained encoders achieve with the full dataset. With all labels, it reaches 85.2 PDMS—outperforming multi-camera and lidar-equipped systems like UniAD and Hydra-MDP, despite using only a single front-facing camera.

LFG shows that large-scale pretraining on raw, unlabeled driving video is a viable path toward more capable and data-efficient self-driving systems. By learning from freely available internet footage, it sidesteps the data bottleneck that has constrained previous approaches.

This publication is provided for informational purposes only. Nothing in this publication should be interpreted as Applied Intuition, Inc. ("Applied") endorsing, granting, or representing any license, right, or permission to use any footage or other content made available online, including any footage referenced or depicted herein. Any person seeking to use such footage is solely responsible for confirming the applicable license terms and obtaining all necessary rights, permissions, and approvals from the appropriate rights holders. Applied expressly disclaims any and all liability arising from or relating to the use of such footage or reliance on this publication.