When an autonomous haul truck moves through a construction or quarry site, it's operating in an extremely unstructured environment. There are no lane lines to follow, no curbs to define the edge of the road, and no directional signs to interpret. Instead, there are berms, continuously changing dump and load zones, unmaintained haul roads, and obstacles of every size, shape, and material composition.
Navigating a haul truck safely through that kind of environment comes down to one core challenge: perception. And because no two sites, operational conditions, or environments are alike, there's no universal sensor solution—which is a core reason many existing systems sidestep the problem entirely, opting instead for blind GPS-based waypoint following rather than true onboard perception.
A human operator in the cab can see the road ahead, the berm to the left, the haul road markings fading in the dust. But a well-instrumented autonomous truck sees more: centimeter-accurate position, 360-degree point clouds that don't degrade in rain or dark, inertial measurements updated hundreds of times per second.
Here’s how Applied Intuition’s sensor stack works.
Applied Intuition’s perception stack uses three sensor modalities in parallel: lidar, camera, and radar. None would be enough on their own, but together they fill in each other’s gaps.
Lidar is the foundational sensing modality. It provides high-resolution, 360-degree coverage of the vehicle's surroundings, with each return carrying an intensity value tied to the reflectance of whatever the beam strikes. A haul truck or excavator in the path returns a strong signal. But certain materials absorb rather than reflect incoming light, producing returns so weak the system can fail to register them as obstacles at all.
Cameras address part of that gap. The visual data they capture feeds into the perception stack, enriching the model’s understanding of the environment beyond what lidar intensity alone can convey. This matters because not all obstacles carry the same risk. The consequences of a collision with a person are categorically different from a collision with a rock, and a system that can’t distinguish between them has to treat everything as the worst case. Cameras can struggle in low-light conditions, which is part of why camera and lidar are used together, each compensating for the other’s blind spots.
Radar handles what neither lidar nor camera does well: detecting and characterizing dynamic obstacles. Radar can measure, through the Doppler effect, the radial velocity of a moving object, giving the system the data it needs to assess whether a collision course is developing. The hardware is already present on current-generation mining vehicles, but software fusion of radar into the perception model is still catching up to the lidar and camera stack. Full radar integration is a later-milestone item across several programs. Radar has its own limitations. Objects with small radar cross-sections or certain geometries can return weak or unreliable signals. But for moving obstacle detection, no other sensor in the stack provides velocity data directly.
Rounding out the suite are localization sensors: a GNSS/INS unit, an Inertial Measurement Unit, and wheel encoders. These don't detect obstacles. They tell the system where the vehicle is relative to everything it’s detecting.
Applied Intuition’s development philosophy is to begin with a high-quality reference sensor because starting with the highest-fidelity inputs removes the sensor as a variable during algorithm development. This means that every engineer building localization, perception, planning, and controls algorithms has a clean input to work with. Once those algorithms are proven, the hardware cost gets optimized for each deployment.
The same logic that governs lidar selection governs localization hardware: start with the most accurate measurement available, perfect the algorithms against that standard, then evaluate what lower-cost hardware can meet the performance requirements the application actually demands.
Most existing autonomous systems respond to uncertainty by stopping. When the perception stack lacks sufficient confidence, the system defaults to the most conservative action available: a hard stop.
Applied Intuition’s stack is built to a different standard. Rather than treating uncertainty as a binary pass/fail, the system uses learned models trained across diverse real-world scenarios to produce a probability distribution over possible actions. By fusing outputs across multiple sensor modalities and running them through ML inference pipelines, the planner can evaluate thousands of candidate trajectories against learned cost functions that weigh safety, progress, and dynamic feasibility, ultimately planning a path around an obstacle rather than halting in front of it.
The result is a system that generalizes. By training on enough edge cases, the models can handle novel scenarios at inference time without falling back to a stop. The goal is to solve 99% of scenarios encountered in deployment, with well-defined mitigation strategies for the remaining 1% where conditions exceed what current sensing and model coverage can reliably handle.
