Add a maximum ray-casting distance for `rangefinder` sensors
The feature, motivation and pitch
Request to add a way for a rangefinder sensor to limit ray casting to a maximum detection distance.
Current Implementation
Currently, the rangefinder's cutoff parameter only performs output post-processing. MuJoCo first casts the ray with no maximum distance, finds the nearest intersection, then clips the result.
The clipping occurs as follows, where $d$ is intersection distance and $C$ is the cutoff distance.
cutoff |
Condition | rangefinder Sensor Output |
|---|---|---|
C == 0 (default) |
Intersection found | d |
C == 0 (default) |
No intersection found | -1 |
C > 0 |
Intersection found at d <= C |
d |
C > 0 |
Intersection found at d > C |
C |
C > 0 |
No intersection found | -1 |
I.e., for a positive cutoff,
$$ f \left( d, C \right) = \begin{cases} \min\left( d, C \right), & \text{intersection found} \ -1, & \text{no intersection found} \end{cases}. $$
Desired Behavior
I am requesting that the behavior of the sensor when a positive cutoff is provided be modified as follows.
$$ f \left( d, C \right) = \begin{cases} d, & 0 \leq d \leq C \ -1, & \text{otherwise} \end{cases} $$
| Condition | rangefinder Sensor Output |
|---|---|
Intersection found at d <= C |
d |
Intersection found at d > C |
-1 |
| No intersection found | -1 |
The behavior should not change for the default cutoff == 0.
Site, perspective-camera, and orthographic-camera rangefinders should all use their existing miss representations for their respective output fields for out-of-range intersections. The cutoff should apply to ray distance rather than camera-plane depth for perspective cameras.
Rationale
Real-wold relevance
The current behavior does not accurately represent many real-world distance sensors.
Time-of-flight, LIDAR, and ultrasonic distance sensors all have a maximum effective range, and generally will not report a value beyond that cutoff distance. A sensor with a maximum detection range of 2-meters generally cannot detect an object at 5-meters. Reporting a clipped measurement of 2-meters incorrectly implies a detection when there was not one. Reporting `-1`` represents the missing measurement more clearly.
Stereo depth cameras are a little bit more complicated. The current approach better models a stereo camera that clamps far-away estimates to a maximum distance. However, the requested behavior is more apt for a stereo algorithm that marks far-away pixels as invalid.
The requested behavior is therefore a better model for sensors with a hard detection limit, but not for every distance sensor.
Simulation performance
Without a maximum distance, MuJoCo still checks geometry beyond the sensor's useful range. These checks can be expensive, especially for complex shapes.
A maximum distance would let MuJoCo skip geometry beyond the cutoff. After finding a closer hit, geoms/intersections beyond that distance could also be skipped.
mj_multiRay() already uses its cutoff to skip obviously distant geoms, but a final check is still needed to reject hits beyond the exact limit. Site and orthographic-camera rangefinders use mj_ray(), which does not currently support a maximum distance.
Arguments against
There are valid reasons to preserve the current behavior:
- Some physical sensors or software pipelines saturate their output while retaining a valid detection.
- Simulation-only consumers may need to distinguish a distant object from no object.
- Some consumers may prefer the current finite, capped semantics.
- The current output is continuous at the cutoff, while the requested output changes abruptly from
Cto-1.
Users could reproduce the current distance behavior with an unlimited rangefinder and clipping in their application.
For these reasons, it may be worthwhile to support maximum detection distance and output clipping as separate behaviors.
Alternatives
Add a separate maxdist attribute
Add a rangefinder-specific maxdist attribute for maximum detection distance and retain cutoff as output clipping.
This would preserve existing behavior and distinguish the two different physical concepts:
maxdist- maximum detection distance, beyond which there is no valid returncutoff- output saturation, which clips a successfully measured value.
Create a user or plugin sensor with the mj_multiRay() cutoff parameter
This would address individual models/applications, but is duplicating the functionality of the built-in rangefinder. Also, the existing mj_multiRay() function does not implement all possible optimizations and the mj_ray() function has no equivalent cutoff parameter.
Additional context
- MuJoCo already gives the
cutoffattribute sensor-specific semantics for collision sensors, so interpreting it differently for a particular sensor is not outlandish.
Source: google-deepmind/mujoco