Closing the Gaps: Smarter ODD Coverage for AV Safety

The Operational Design Domain (ODD) defines the exact set of conditions in which an autonomous vehicle is meant to drive safely. These include weather, lighting, road type, speed limits, and more. ODDs are a central concept in the validation and verification of autonomous vehicle technology, because they give engineers clear boundaries for system capability and for safety assurance.

In a previous blog post, Applied Intuition explained how teams can create and refine ODDs—using insights from standards like ASAM OpenODD and Pegasus, and by learning from real-world testing. That post emphasized that ODDs aren’t static, that as new data is collected, and as unexpected events occur, an ODD must evolve to keep pace with the real world. Each update brings it closer to capturing the complexity of daily driving and the operational realities an AV will face.

However, defining an ODD is just the beginning. As autonomous vehicles are tested, critical challenges arise: ensuring that all important scenarios within the ODD are truly covered, and knowing when testing is sufficient. This prompts two important questions:

• “Where do I need to test more?” 
• “When have I tested enough?”

These questions are at the heart of ODD coverage analysis, and will be addressed in this post.

Understanding ODD Coverage Analysis

ODD coverage analysis is the process of evaluating how thoroughly autonomous vehicle testing has explored the conditions and situations defined in the ODD. It goes beyond simply listing where the vehicle is supposed to operate; it asks whether the system has actually been validated in all those circumstances. This analysis is crucial for safety—it helps ensure that vehicles perform as expected not only in common cases, but also in the edge cases and corner cases of the ODD.

Effective ODD coverage analysis is not something that happens once and is forgotten. It is a cyclical process. Each round of real-world driving, simulation, or closed-track testing generates data, which engineers review to spot coverage gaps or untested combinations of attributes and conditions. As the ODD evolves, so does the focus of coverage analysis, prompting new test campaigns and refinements. This ongoing loop allows teams to adapt as environments, regulations, or system capabilities change.

At the heart of this process are drive logs—digital records of real-world trips—and simulated trips. By tagging these logs with ODD attributes (such as weather, lighting, road geometry, and surrounding actors), teams can map out which parts of the ODD have been covered and which have not. This data-driven approach provides the foundation for making smart, targeted decisions about where to prioritize future testing and analysis, ensuring comprehensive and defensible safety cases.

Expanding the ODD Coverage

Identifying Gaps in Coverage

Once ODD coverage analysis begins, the first task is to uncover where testing falls short. Gaps appear in the “combinatorial space” of ODD attributes—combinations like rainy-night-urban intersections or clear-day-rural highways with heavy truck traffic. These gaps become visible by cross-referencing tagged drive logs and simulation results against the total ODD definition. Dashboard tools and customizable reports make it easy to see which slices of the ODD are missing data or have only sparse examples.

‍

Statistical and data-driven reviews are critical. When drive data fails to cover rare or unusual combinations, teams can use synthetic scenarios—simulated situations built to intentionally fill “holes” in test coverage. This approach ensures the system is robust not only in common settings, but also in the ‘long tail’ of rare events. Typical ODD attributes analyzed include weather, time of day, and actor types (such as trucks, cyclists, or pedestrians), but many programs also track road geometry, traffic density, and temporary events like construction.

Statistical Methods for Expanding Coverage

For attributes that are continuous (such as vehicle speed, lighting intensity, or temperature), it’s not possible to test every value. Instead, values are binned into representative groups. For instance, one might discretize vehicle speed as low speed, medium speed, and high speed. Teams can then employ statistical confidence intervals—mathematical tools that estimate overall safety and performance based on a sampling of tested cases. This allows engineers to use limited test data to infer how the system will behave across untested but similar scenarios, and to cover infinitely many scenarios in a finite amount of time.

Continuous improvement means revisiting and refining these gaps regularly. As new road data and test results flow in, the analysis loop repeats, gradually strengthening confidence that all relevant areas of the ODD have been examined.

Deciding When Enough is Enough

Determining when testing is sufficient is both a technical and strategic challenge. It is rarely possible—or necessary—to achieve 100% coverage of the ODD, given real-world constraints on time, cost, and resources. Instead, teams balance the completeness of testing with practical feasibility, guided by risk assessments and the criticality of different ODD segments.

Regulatory benchmarks, such as NCAP standards and scenario-based safety frameworks, play a major role in shaping what is considered “enough.” These benchmarks define minimum expectations for coverage and performance, providing structure for internal safety cases and regulatory approval. However, true completeness often goes beyond checking boxes; it demands thoughtful justification of how coverage decisions map to real-world risk and operational needs.

Industry Practices vs. Standards

There is a real gap between industry practices and formal standards. Frameworks like Pegasus offer structured layers for ODD definition and scenario selection, but often do not specify how deep to drill into certain variables, or whether behaviors—such as how a pedestrian moves—should be included in the ODD itself. In practice, many organizations adapt standards, adding attributes and behavioral considerations that align with their business model, geography, or fleet design. For example, some projects treat actor behaviors as core ODD attributes, while others focus primarily on static environmental conditions.

This disconnect highlights the need for tools and workflows that are flexible. The industry is still evolving toward best practices, and safety is an adaptable, context-driven goal—not a fixed checklist.

Applied Intuition’s Approach

Applied Intuition’s solutions are built for this landscape of diversity and change. The Validation Toolset module offers teams the ability to define ODDs in arbitrary detail, import or update ODD taxonomies, and analyze coverage with both user-friendly dashboards and advanced data science notebooks.

Drive data and simulation artifacts can be automatically or manually tagged with precise ODD attributes, including environmental details, road features, and even actor behaviors—if the program calls for it. Coverage analysis is customizable: teams can run simple matrix checks, deep statistical analyses, or run targeted reports on high-risk combinations.

Based on broad industry experience, Applied Intuition recommends several best practices:

• Start with the most safety-critical attributes—such as weather, lighting, and vulnerable road users.
• Regularly re-analyze and expand coverage as both the system and ODD definition evolve.
• Where feasible, leverage synthetic scenarios to fill in rare combinations or edge cases that are unlikely to appear in fleet logs.
• Prioritize transparency: document how coverage decisions were made, and link them directly to regulatory or business requirements.
• Use confidence intervals and statistical sampling wisely, reserving fine-grained exploration for areas of greatest risk or regulatory scrutiny.

ODD coverage analysis sits at the core of safe, credible autonomous vehicle development. By identifying gaps, leveraging statistical and synthetic methods, and continuously refining test programs, teams can build robust safety cases that stand up to real-world complexity and regulatory requirements. Applied Intuition’s flexible tools are designed to support every step of this journey—helping teams answer not just “are we finished testing?” but “is the system truly ready?”

Explore the Validation Toolset and Insights modules for a smarter, sharper approach to ODD coverage analysis.