Neural Sim at Applied Intuition: Powering Evaluation of Our End-to-End SDS

Sensor simulation at scale remains a core challenge in ADAS and AV development. Manual replays and hand-built virtual worlds lack the realism and throughput necessary to validate next-generation self-driving systems, leaving teams struggling to efficiently test systems across millions of real-world scenarios.

Neural Sim translates raw fleet drive logs into dynamic, photo-realistic 3D environments for scalable closed-loop sensor simulation—delivering robust evaluations for both end-to-end and traditional autonomy programs.

This post details Neural Sim’s evolution as a tool and product, its technical foundation, and its impact on Applied Intuition’s own validation and safety practices.

‍

Above: Original drive log (front camera) vs. reconstruction in Neural Sim

‍Challenge

Modern autonomy stacks require validation across millions of sensor-level scenarios—a scale legacy simulation tools cannot deliver.

Previous solutions—whether raw log replay or manual scene building—fail to generate scalable closed-loop simulations. Integrated end-to-end stacks require simultaneous sensor-to-control evaluation; therefore mass-produced object-level scenarios are no longer sufficient for comprehensive testing.

Neural Sim’s Evolution

Neural Sim’s ongoing development has progressed the product from a series of promising technical demos into a sophisticated platform built on Applied Intuition’s battle-tested core simulation foundation.

Early experiments provided glimpses of what was possible—scene reconstructions from raw logs and basic actor playback—but the real market gap was the lack of a scalable, automated, and robust tool which engineering teams could trust for production evaluation workflows. 

Neural Sim addressed these demands in several key ways:

• Foundation and platform maturity: Neural Sim is not an isolated offering; it is built atop Applied Intuition’s modular simulation tooling ecosystem. This enables unified workflows for data management, scenario creation, and sensor-accurate simulation—both on-premises and in the cloud.
• Dynamic scene reconstruction: Neural Sim reconstructs detailed, actor-rich 3D environments from any set of drive logs. It not only models static background features (e.g., trees and buildings) but also tracks and animates dynamic actors such as vehicles, cyclists, and pedestrians, while preserving their original observed behaviors. State-of-the-art machine learning techniques like radiance fields and Gaussian Splatting drive realism in rendering, with sensor simulations reflecting true-to-life camera, lidar, and radar data.
• Integration with Applied’s Data Engine: Neural Sim leverages Applied Intuition’s Data Engine to transform raw fleet data into curated scenarios spanning hundreds of thousands of real-world edge cases. Any drive log collected by the fleet is immediately eligible for reconstruction, rapidly generating sensor-level scenario libraries over the Operational Design Domains (ODDs) which the fleet traverses every day.
• Scalability: Where manual scenario building was a bottleneck in traditional workflows, Neural Sim’s automated pipelines routinely transform newly collected drive logs into unique, varied sensor and object-level scenarios. Engineers can queue entire datasets of old logs and simulate over edge cases at a volume impossible just a year ago.
• Realism evaluation framework: Engineers now have qualitative and quantitative tools to evaluate neural reconstruction fidelity. Neural Sim generates metrics for scene reconstruction accuracy, lighting consistency, and dynamic agent realism. These metrics are automatically computed and reported in a fashion which supports both iterative development and regulatory requirements.
• Seamless V&V integration: Outputs from Neural Sim plug directly into Applied Intuition’s verification and validation (V&V) environment, making it straightforward for teams to progress from simulation work to formal test coverage analysis, failure triage, and compliance documentation.
• User experience and workflow: Significant investment in the tool’s UI/UX has resulted in a product designed not just for neural rendering experts, but for broad deployment across engineering and QA organizations. Visual workflows enable intuitive review and filtering of reconstructions, scenario-to-log comparisons, and seamless handoff to downstream validation.

Internal SDS Use Case

Neural Sim’s vertical integration—from fleet operations and scenario curation to simulation—enables daily, real-world validation for Applied Intuition’s Self-Driving System (SDS) program across L2++ to L4 domains.

Applied’s engineering teams have used Neural Sim to reconstruct thousands of drive log events into varied batches of sensor-level scenarios. These scenarios are used in simulation to evaluate the performance of SDS at scale, with thousands of neural simulations running daily.

Neural Sim unlocks transformative benefits for the SDS program:

• Accelerated internal validation: Instead of slow, fragmented on-road testing, bugs and performance gaps are surfaced in simulation months earlier, enabling rapid loop closure and robust regression analysis as new features and model improvements are deployed.
• Efficient workflow: Automated scenario extraction means simulation is no longer bottlenecked by manual technical artist labor or fragmented toolchains. Engineers trigger batch scenario creation, triage outputs by coverage or risk profile, and feed results directly into modular validation pipelines.
• Credibility for external users: Continuous, documented use across this breadth and depth establishes Neural Sim as a platform trusted for demanding safety-critical programs, and as a backbone for the regulatory, production, and compliance needs of global OEMs.

Leadership in Realism Evaluation

Neural Sim is built around a dozen consistent upstream and downstream metrics which measure how closely simulations mirror real-world logs and behaviors.

• Upstream reconstruction metrics quantify how well each virtual environment matches its source drive log: pixel-level scene overlap, semantic accuracy of actors, lighting, and road details. Neural Sim reconstructs dynamic, interactive scenes—enabling meaningful evaluation of both environments and behaviors.
• Open-loop downstream metrics measure how an integrated stack processes reconstructions that are unaltered from original log behavior, independent of scenario evolution. Standard metrics such as intersection over union (IoU) for object detection, perception precision, recall, and driving policy agreement provide fast, repeatable triage and benchmarking.
• Closed-loop downstream metrics represent the gold standard for evaluation, as they measure how the actors and scenarios adapt to stack actions during simulations. Fleet-derived scenarios ensure closed-loop tests accurately reproduce real behaviors, enabling direct comparisons with real world drive data and robust outcome-based validation.

Setting the industry standard for evaluation

Neural Sim is reshaping how the industry measures realism and safety. By delivering reproducible metrics for simulation and reconstruction, Neural Sim provides a regulatory-grade validation foundation that scales across programs and ODDs.

Upcoming features—like auto-extracted actor libraries, multi-traversal scene reconstructions, and expanded sensor modalities—will further strengthen closed-loop validation capabilities. These innovations will keep Neural Sim at the forefront of scalable, data-driven safety for autonomy across automotive, trucking, and defense applications.

Contact Applied Intuition to see how Neural Sim can accelerate safe, scalable autonomy development for your self-driving programs.