LiDAR#
The LiDAR model traces the beams of a spinning sensor through the rendered scene and computes the returns with a radiometric model. This page gives the scan pattern, the two trace paths, the equations of the returns, the Polaris sensor and the LiDAR of ACRES Core.
Scope and Assumptions#
The model has three parts.
- Geometry. Each beam finds the surfaces on its path. The GPU path uses the ray-tracing scene of the renderer. The CPU path uses the collision geometry and proxy shapes.
- Radiometry.
AcresLidar::ProcessBeamchanges the surface hits of one beam into echoes, detects them and selects the returns. - Timing. Each column of the scan fires at its own time from the pose of the sensor at that time.
The model makes these assumptions.
- A beam is a bundle of 1 or 7 sub-rays. A sub-ray is a line.
- A surface has a diffuse reflectance, an optional specular lobe and, for foliage, a transmission for each layer.
- The receiver detects an echo when the peak signal with shot noise is above a threshold.
- The vehicle that carries the sensor is not in the trace. A measured table can add the returns of its body.
- The trace sees the world of the game frame that runs the scan. Only the sensor pose changes during the sweep.
The model does not compute the waveform of the pulse, crosstalk between sensors, sun glare or a water film on the window.
The default LiDAR is a generic 905 nm sensor with placeholder values. The Polaris LiDAR has a calibration against recorded scans. Sensors and Rigs gives the procedures to record the LiDAR.
Symbols#
| Symbol | Quantity | Unit |
|---|---|---|
| \(N_r, N_c\) | Number of rings and number of columns | - |
| \(\epsilon_r\) | Elevation of ring \(r\) | rad |
| \(\alpha_c\) | Block azimuth of column \(c\), counter-clockwise from forward | rad |
| \(\delta_r\) | Azimuth offset of ring \(r\) | rad |
| \(l\) | Distance of the lens centre from the spin axis | m |
| \(T\) | Sweep period, \(1/f\) | s |
| \(\Delta t_c\) | Firing time of column \(c\) minus the scan stamp | s |
| \(\beta, \beta_v\) | Full-angle beam divergence, horizontal and vertical | rad |
| \(w_j\) | Energy weight of sub-ray \(j\) | - |
| \(R\) | Range from the lens centre | m |
| \(\theta\) | Incidence angle between the ray and the surface normal | rad |
| \(\rho\) | Diffuse reflectance of a material at normal incidence | - |
| \(\tau\) | Transmission of one foliage layer | - |
| \(\rho_{bs}\) | Backscatter of a surface as an equivalent Lambertian reflectance | - |
| \(a\) | Albedo factor of a ground cell | - |
| \(K\) | Maximum number of hits of one sub-ray | - |
| \(P\) | Contribution of one surface hit | - |
| \(g\) | Peak factor of the echo stretch | - |
| \(\rho_e, R_e, g_e\) | Reflectance, range and peak factor of an echo | -, m, - |
| \(S_{10}\) | Mean signal of a 10 % target at the range \(R_{10}\) | counts |
| \(R_{10}\) | Range of the specification point | m |
| \(R_{ov}\) | Overlap range of the transmitter and the receiver | m |
| \(B\) | Ambient signal | counts |
| \(S, S'\) | Mean signal and measured signal of an echo | counts |
| \(S_T\) | Detection threshold | counts |
| \(L\) | Pulse length in range | m |
| \(\sigma_0, \sigma_{low}\) | Range noise floor and range noise at the specification SNR | m |
| \(\alpha\) | Extinction coefficient of rain and fog | 1/m |
| \(R_r, V\) | Rain rate and meteorological visibility | mm/h, m |
| \(n, z\) | Standard normal numbers | - |
The sensor frame is FLU: x forward, y to the left, z up. Azimuth is positive counter-clockwise when you look from above.
Scan Pattern#
Source: FAcresSensorRecorder::BeginScan in AcresSensors.cpp.
A scan has \(N_r \times N_c\) beams. The scan rate is lidar.hz.
Rings#
The key lidar.elevations_deg gives the elevation of each ring in the ring sequence of the sensor.
Without the table, the rings have equal steps between elevation_min_deg and elevation_max_deg:
With one ring, the elevation is the middle of the two limits.
Columns#
The key lidar.azimuth_order gives the direction of the column index.
Column 0 points forward in the two cases. The azimuth of a beam adds the offset of its ring: \(\alpha = \alpha_c + \delta_r\).
Beam Direction and Origin#
The direction of the beam and its origin in the sensor frame are
The origin is the lens centre. It is on the block azimuth, not on the azimuth of the ring.
The mount rotation \(M = R_z(\text{yaw})\,R_y(\text{pitch})\,R_x(\text{roll})\) of lidar.rpy_deg turns the two vectors into the body frame.
The chassis rotation at the firing time then turns them into the world.
A return at the range \(R\) gives the point
The point is in the sensor frame at the firing time of its column.
Sub-Rays#
Source: AcresLidar::SubRayPattern.
A beam has the full-angle divergence \(\beta\) (lidar.beam_divergence_mrad).
With lidar.sub_rays = 7, the model samples the beam with seven sub-rays.
| Sub-Ray | Angular Offset | Weight \(w_j\) |
|---|---|---|
| 0 | 0 (the beam centre) | 0.25 |
| 1 to 6 | \(0.35\,\beta\) at the angles \(k\pi/3\), \(k = 0 \ldots 5\) | 0.125 each |
With sub_rays = 1 or \(\beta = 0\), the beam is one ray with the weight 1.
For the offset \((o_1, o_2)\), the direction of a sub-ray is
\(\mathbf{a}\) is the up axis. When \(\lvert d_z \rvert \ge 0.9\), \(\mathbf{a}\) is the forward axis.
Scan Timing#
Source: AcresLidar::ColumnTimeOffset, FAcresSensorRecorder::PhysicsSample, FAcresSensorRecorder::SweepPoseAt.
Scan Stamp#
The physics step of 1/120 s is the clock. A scan is due at \(t = 0\) and then each \(T = 1/f\). The physics thread stamps the scan in the first step at or after the due time and puts a request in a queue. The stamp is the end of the sweep.
The game thread starts one scan in each rendered frame. Two scans at most are in work.
When a scan is due and two requests are open, the recorder drops the scan and increments lidar_busy in episode.json.
Firing Time of a Column#
With lidar.motion_distortion true, the sensor turns one time in the period \(T\).
The sweep starts at the azimuth \(\alpha_s\) (lidar.scan_start_azimuth_deg). The angle that the sensor turned to column \(c\) is
All rings of a column fire at the same time. The column at the start azimuth fires first, one period before the stamp.
Pose at the Firing Time#
The physics thread keeps the sensor poses of the last period plus two steps. For the time \(t_s + \Delta t_c\) between two physics poses \(a\) and \(b\), with \(\lambda = (t - t_a)/(t_b - t_a)\):
Before the first pose of the history, the model holds the first pose. This occurs at the start of an episode and after a reset of the vehicle.
The model does not correct the output points for the motion. A real driver also gives the points in this form.
To remove the distortion, transform each point with the sensor pose at the time time_s + time_offset_s.
With motion_distortion false, all columns use the pose at the stamp and \(\Delta t_c = 0\).
Delay#
A row of lidar.jsonl gets available_time_s = sample_time_s + lidar.delay_s. Without that key, the delay is delay_s.
The recorder releases the row when the files of the scan exist. The delay applies only to the session log.
The sensor stream sends a cloud when the thread pool completes it.
GPU Path#
Source: AcresLidarGpu.cpp, Acres/Shaders/AcresLidar.usf, FAcresSensorRecorder::MergeGpuScan.
The renderer builds an acceleration structure of the rendered scene in each frame for its own lighting.
The GPU path traces the LiDAR rays against that structure. It thus sees the Nanite meshes, the crop instances and the rut tiles.
The path needs hardware ray tracing with inline ray queries. The log line ACRES_SENSOR_START shows gpu_lidar=1 when the path is active.
Dispatch#
- The game thread computes the direction of each sub-ray and the origin of each ray relative to the scan origin.
FAcresLidarViewExtensionputs a compute pass into the next frame of the main view, after the acceleration structure is complete.- The shader
MainCSruns one GPU thread for each ray, in groups of 64. - An asynchronous readback copies the hits to the CPU. The game thread examines the state in each frame.
One scan is on the GPU at a time. Scene captures do not run the pass.
The Shader#
For each ray the shader records \(K\) surfaces at most, where \(K\) is lidar.max_hits_per_sub_ray.
- The ray starts 0.5 cm from its origin. Its maximum distance is
lidar.range_m. - The any-hit function ignores the components of the vehicle that carries the sensor.
- The shader writes the hit: the distance, the component number, the packed surface normal and the flags.
- If the component is not in the list of foliage components, the ray stops.
- If it is foliage, the ray continues
lidar.continue_step_mbehind the hit.
The shader does no alpha test. A leaf card with an alpha mask is a solid triangle.
| Buffer | Element | Contents |
|---|---|---|
Rays |
float4 |
Unit direction in the world, maximum distance in cm |
RayOrigins |
float4 |
Ray origin minus the scan origin in cm |
SelfIds |
uint |
Component numbers that the rays ignore |
PassIds |
uint |
Sorted component numbers of foliage, crops and ground cover |
Hits |
uint4 |
Distance in cm as float bits, component number, normal (octahedral, 2 x 16 bits), flags (1 = hit, 2 = normal valid) |
Merge with the CPU Trace#
The CPU traces the centre ray of each beam against the collision channel WheelGround.
That trace has two functions.
- Ground snap. The collision mesh of the survey ground is exact. A GPU ground hit within 0.5 m of the CPU ground hit takes the CPU range and the CPU normal. Rut tiles and water tiles keep their GPU range.
- Missing ground. A sub-ray can find no solid surface in the rendered scene. If the CPU ground is more than
continue_step_mbehind its last hit, the model adds the CPU ground hit.
The class of a GPU hit comes from the actor tags of the component. The incidence is \(\lvert\cos\theta\rvert = \lvert \mathbf{n} \cdot \mathbf{d}_j \rvert\). When the platform gives no normal, the model uses \(\lvert\cos\theta\rvert = 0.5\).
If the GPU result does not come in 3 s of episode time, the recorder keeps the CPU result of the centre rays.
It records an error and increments gpu_scan_failures in episode.json.
CPU Path#
Source: FAcresSensorRecorder::BeginScan, FAcresSensorRecorder::BuildStaticProxies.
The CPU path is active without hardware ray tracing or with the option -SensorCpuLidar.
It traces each sub-ray against the collision channel WheelGround: the survey ground, the buildings, the grain bins and the tree trunks.
Objects without collision get proxy shapes. A proxy is a foliage hit in front of the solid hit.
| Object | Proxy |
|---|---|
| Tree crown | A sphere with the radius \(0.33 H\) and the centre at \(0.62 H\) above the base. \(H\) is the height of the tree instance. |
| Shrub | A sphere with the radius \(0.5 H\) and the centre at \(0.5 H\). |
| Crop patch | A box of 1.52 m x 1.0 m with 4 x 4 cells. |
The height of a crop box is \(h_{max} = h_{crop}(0.03 + 0.97\,G)\), with the growth \(G\) from 0 to 1. A crushed cell has 12 % of that height and a harvested cell has 3 %. The model moves along the ray in steps of 0.05 m. The first point below the top of its cell is the hit, with \(\lvert\cos\theta\rvert = 0.5\).
The model finds the proxies with a walk through a grid of 4 m cells below the ray. It keeps the nearest \(K\) foliage hits and then the solid hit, if the sub-ray has fewer than \(K\) hits.
Materials#
Source: FAcresSensorRecorder::FinishScan, MaterialForClass.
Each hit has a semantic class and a material. The class of an object gives its material.
| Class Number | Class | Material |
|---|---|---|
| 0 | other |
other |
| 1 | ground |
From the ground rules below |
| 2 | tree_trunk |
bark |
| 3 | building |
building |
| 4 | silo |
metal_bin |
| 5 | tree |
tree_foliage |
| 6 | shrub |
shrub |
| 7, 8, 11 | corn, soybean, potato |
corn_leaf, soybean_leaf, potato_leaf |
| 9 | mud_clod |
mud, wetness 1 |
| 10 | ground_cover |
grass |
| 12 | vehicle |
vehicle |
| 13 | precipitation |
other (clutter from rain and fog) |
The classes tree, shrub, corn, soybean, potato and ground_cover are foliage: a sub-ray continues behind them.
Ground#
A ground hit takes its material from the map of ACRE.
- The cover grid of 4 m cells gives
grass,asphalt,concrete,gravelorsoil. - The surface polygons of roads, yards, grass lanes and verges give the material where a polygon exists. Without a polygon, the material stays
grassor becomessoil. - If the water model has more than
lidar.pond_threshold_mof water in the cell, the material iswater. - The wetness \(w\) of soil is the wetness of the water model. For other ground, \(w = \operatorname{clamp}(d_{pond} / 0.002\ \text{m},\, 0,\, 1)\).
Ground Relief and Texture#
A mesh is smooth at the scale of the beam footprint. Three terms give the ground its real variation. They apply only to ground hits, and each term is off when its parameter is 0.
Micro-slope. The key micro_slope_deg \(= \sigma_t\) tilts the surface in the plane of incidence.
\(z_{0.25}\) is a fixed normal number for each ground cell of 0.25 m.
Albedo texture. The key reflectivity_sd \(= s_\rho\) multiplies the reflectance by a lognormal factor with the mean 1.
\(z_{0.1}\) is a fixed normal number for each ground cell of 0.1 m.
Roughness. The key roughness_m \(= \sigma_h\) moves the range of the hit.
\(z\) is a new normal number for each scan, beam and sub-ray. The exponent 1.5 is \(1 + H\) for a self-affine surface with the Hurst exponent \(H = 0.5\).
Numbers from a Hash#
The three terms and the self returns use a hash, not a random generator. The noise stream of the LiDAR thus does not change.
The hash is the finaliser of SplitMix64 (AcresLidarPhysics::Mix64), with 64-bit arithmetic:
A uniform number is \(U(h) = ((h \gg 11) + 0.5) / 2^{53}\).
For a point \((X, Y)\) in metres, a cell size \(s\) and a seed \(k\), PlaceNormal gives
The micro-slope uses \(k\) = seed and the albedo texture uses \(k\) = seed + 1.
Backscatter#
Source: AcresLidar::Backscatter, AcresLidarPhysics::MinnaertBackscatter.
The backscatter of a surface is a diffuse term and a specular lobe.
| Symbol | Key of the Material | Meaning |
|---|---|---|
| \(\rho\) | reflectivity |
Diffuse reflectance at normal incidence |
| \(a_w\) | wet_darkening |
Loss of reflectance at the wetness 1 |
| \(k\) | incidence_exponent |
Minnaert exponent. 1 is a Lambertian surface. 0 gives no change with the incidence. |
| \(s\) | specular |
Peak of the specular lobe as an equivalent Lambertian reflectance |
| \(\theta_s\) | specular_width_deg |
Width of the lobe |
The diffuse term limits \(\lvert\cos\theta\rvert\) to \(10^{-4}\) or more. Water has a low \(\rho\) and a high \(s\). A pond thus gives a return only near normal incidence.
Contributions and Echoes#
Source: AcresLidar::ProcessBeam.
Contribution of a Hit#
The model sorts the hits of each sub-ray by range. \(T\) is the transmission of the foliage layers in front of the hit, with \(T = 1\) at the start.
For a foliage hit, \(\tau\) is the transmission of the material and the model then sets \(T \leftarrow T\,\tau\).
For a solid hit, \(\tau = 0\) and the sub-ray stops. The square of \(T\) is the path to the surface and back.
Echo Stretch#
A beam that meets a surface at a low angle has a footprint with a range extent.
The echo is the emitted pulse of the range length \(L\) (lidar.pulse_length_m), spread over that extent.
The energy stays constant, the width increases and the peak decreases by the factor
\(\beta_p\) is the divergence in the plane of incidence. The game uses the vertical divergence \(\beta_v\) for ground and the horizontal divergence \(\beta\) for other solids. Foliage has \(g = 1\). With \(L = 0\), all hits have \(g = 1\).
Echo Formation#
The model sorts all contributions of the beam by range, clutter included.
An echo starts at the nearest contribution that is not in an echo. It takes all contributions within lidar.min_return_separation_m of that first contribution.
The class and the material of the echo are those of its largest contribution.
Signal, Detection and Range Noise#
Source: AcresLidar::SignalCounts, FAcresLidarModelConfig::DetectionThreshold, AcresLidar::ProcessBeam.
Range Equation#
The signal follows the LiDAR equation for a target that is larger than the beam. The specification point sets the scale.
- \(S_{10}\) is the mean signal of a Lambertian target of 10 % at normal incidence at the range \(R_{10}\).
- The key
radiometry.ring_range_at_10pct_mgives \(R_{10}\) for each ring. Without it, all rings userange_at_10pct_m. - The third factor is the overlap of the transmitter and the receiver. It decreases the signal of targets nearer than approximately \(R_{ov}\).
- The last factor is the two-way loss in rain and fog. In clear air \(\alpha = 0\).
Detection Threshold#
The threshold \(S_T\) makes the specification target detectable with the probability \(p_{spec}\) (detection_probability_at_spec).
\(\Phi\) is the standard normal distribution function. The code finds \(z_p\) by bisection. The shipped values \(S_{10} = 100\), \(B = 20\) and \(p_{spec} = 0.9\) give \(z_p = 1.2816\) and \(S_T = 85.96\) counts.
Detection#
The measured signal has shot noise. The model uses a normal number with the variance \(S + B\).
The model detects the echo when all these conditions are true.
Dark targets, far targets and targets at a low angle are thus lost more frequently. The model has no fixed dropout rate.
Range Noise#
\(\sigma_0\) is lidar.std_m and \(\sigma_{low}\) is radiometry.range_std_low_snr_m.
Random Numbers#
The LiDAR noise uses one PCG32 generator with the state seed + 2018 and the sequence 3.
The model draws \(n_1\) and \(n_2\) for each echo in range order. The beams are in ring-major sequence.
With -SensorNoNoise the model draws the numbers but does not use them, thus the stream stays aligned.
The stream repeats only when the geometry repeats, because the number of echoes sets the number of draws.
Returns and Intensity#
Return Mode#
The key lidar.return_mode selects the echoes that become points.
| Mode | Points of One Beam |
|---|---|
strongest |
The echo with the largest \(S'\) |
first |
The nearest echo |
last |
The farthest echo |
dual_strongest_last |
The strongest and the last. If they are the same echo, the model adds the second strongest. |
dual_first_last |
The first and the last |
triple_first_strongest_last |
The first, the strongest and the last |
The points of a beam are in range order. The byte return_flags gives the function of a point: 1 = first, 2 = strongest, 4 = last, 8 = second strongest.
Reflectivity, Signal and Intensity#
A point has three brightness values.
reflectivityis \(\rho_{rep}\). \(S_c\) is the signal that the sensor expects in clear air. Rain and fog thus make the reflectivity darker. The echo stretch does not change it.signalis \(\min(S', S_{sat})\), rounded to an integer of 16 bits.intensityis the reflectivity as one byte:
The values 0 to 100 are diffuse per cent. The values 101 to 255 are a logarithmic scale for specular returns, with the minimum 101.
Rain and Fog#
Source: AcresLidar::Atmosphere, AcresLidar::RainExtinction, AcresLidar::FogExtinction, AcresLidar::ProcessBeam.
The physics thread reads the rain rate \(R_r\) and the visibility \(V\) from the weather model at the scan stamp.
The options -SensorLidarRain= and -SensorLidarVisibility= replace them. Refer to Weather.
Extinction#
The rain extinction follows the power law of Carbonneau and Wisely (1998), with \(a\) = rain_extinction_db_per_km and \(b\) = rain_extinction_exponent.
The visibility of the weather model includes rain. The model removes that part and applies the Koschmieder relation.
The model of Kim, McArthur and Korevaar (2001) scales the fog and haze extinction to the laser wavelength \(\lambda\).
| \(V_f\) | \(q\) |
|---|---|
| more than 50 km | 1.6 |
| 6 km to 50 km | 1.3 |
| 1 km to 6 km | \(0.16\,V_f + 0.34\), \(V_f\) in km |
| 0.5 km to 1 km | \(V_f - 0.5\), \(V_f\) in km |
| less than 0.5 km | 0 |
The total extinction is \(\alpha = \alpha_{rain} + \alpha_{fog}\). The detection threshold does not change.
Clutter#
Droplets send light back to the sensor. The backscatter coefficient is the extinction divided by a LiDAR ratio.
The model cuts the beam into slabs of the length \(\Delta R\) = min_return_separation_m.
The slabs start at min_range_m and stop at the smallest of clutter_max_range_m, the maximum range and the nearest solid hit.
A slab with the centre \(R_c\) has the equivalent reflectance
Rain is a set of drops. \(N\) is a Poisson number with the mean \(\Lambda\), and \(E_i\) is an exponential number with the mean 1.
\(n_d\) is the drop density of Marshall and Palmer (1948). The model limits \(N\) to 64.
The model ignores a slab when \(S + 5\sqrt{S + B} < S_T\).
Other slabs become contributions with the class precipitation and go through echo formation and detection.
The rain numbers come from a second generator (state seed + 4036, sequence 5).
Self Returns#
Source: FAcresSensorRecorder::FinishScan.
The trace ignores the vehicle that carries the sensor. The key lidar.self_returns names a table that puts the body back.
The file has four planes of \(N_c \times N_r\) values of the type float32. The cell index is \(c\,N_r + r\).
| Plane | Contents |
|---|---|
| 0 | \(p_{self}\): probability of a return from the body |
| 1 | Range of the body |
| 2 | Intensity of the body |
| 3 | \(p_{block}\): probability that the body stops the beam without a return |
For each scan and cell the model computes a uniform number \(u\) from a hash of the scan number, the seed and the cell.
| Condition | Result |
|---|---|
| \(u < p_{self}\) | One return at the table range plus \(\sigma_0\,z\), with the table intensity and the class vehicle. The minimum range is 0.2 m. |
| \(p_{self} \le u < p_{self} + p_{block}\) | No return |
| \(u \ge p_{self} + p_{block}\) | The beam sees the scene |
The Polaris LiDAR#
The Polaris has a RoboSense Helios with 32 channels (RS-Helios-5515, 905 nm).
The overlay sensors_polaris.json gives the sensor as the driver rslidar_sdk of the vehicle publishes it.
| Property | Value |
|---|---|
| Cloud | Organised, 1800 columns x 32 rings, one return (strongest) for each beam |
| Rate | 10 Hz, stamp at the last point of the sweep |
| Column \(c\) | Block azimuth \(-0.2\,c\) degrees, clockwise from forward |
| Sweep | Starts at 0°, spin clockwise |
| Range | 0.2 m to 161.3 m |
| Lens centre \(l\) | 0.035 m |
Mount from base_footprint |
(2.5, 0, 2.039) m, roll -1.518°, pitch 5.317°, yaw 0° |
| Divergence | 1.6 mrad horizontal, 6.9 mrad vertical, one sub-ray |
| Pulse length \(L\) | 1.0 m |
| Range noise | \(\sigma_0\) = 2.2 mm, \(\sigma_{low}\) = 39.3 mm |
| Delay | 0.012 s |
| \(R_{10}\) | Rings 0 to 15: 87.0 m. Rings 16 to 27: 38.7 m. Rings 28 to 30: 19.3 m. Ring 31: 9.7 m. |
Vertical Angles#
The ring index is the index of the driver. The elevations are not in a monotonic sequence between ring 16 and ring 23.
| Rings | Elevation in Degrees |
|---|---|
| 0 to 7 | 14.94, 13.03, 10.92, 8.90, 6.96, 5.46, 3.98, 2.67 |
| 8 to 15 | 1.33, 0.00, -1.33, -2.67, -3.96, -5.23, -6.63, -8.01 |
| 16 to 23 | -9.94, -15.97, -12.90, -18.98, -21.92, -27.87, -24.96, -30.95 |
| 24 to 31 | -33.891, -37.09, -39.951, -43.111, -46.10, -49.124, -51.904, -53.696 |
The even rings have azimuth offsets from +3.88° to +5.11°. The odd rings have offsets from -3.63° to -5.03°. These are the two laser groups of the sensor.
Self-Return Mask#
The file Acres/Content/Simulation/polaris_lidar_self.f32 is the table of the Polaris. Its size is 4 x 1800 x 32 x 4 = 921 600 bytes.
Calibration/Polaris/lidar_stats.py export makes it from the recorded scans.
- 20 711 cells give a return from the roof rack, the mast and the hood at 0.26 m to 1.58 m.
- 11 522 cells have a beam that the body stops without a return in some scans.
- Approximately 15 900 of the 34 500 points of a scan are the vehicle.
- A return nearer than 1.6 m is always the body.
Materials of the Polaris LiDAR#
The overlay replaces the reflectances with the medians of the recorded intensity. All materials of the overlay have the Minnaert exponent \(k = 0\).
| Material | \(\rho\) | \(\sigma_t\) (deg) | \(s_\rho\) | \(\sigma_h\) (mm) |
|---|---|---|---|---|
asphalt |
0.10 | 3.3 | 0.07 | 0 |
concrete |
0.10 | 2.7 | 0.09 | 0 |
gravel |
0.16 | 2.0 | 0.19 | 0.7 |
soil |
0.13 | 9.4 | 0.29 | 1.7 |
grass |
0.17 | 4.35 | 0.24 | 2.6 |
Other values: building 0.31, metal_bin 0.23, bark 0.23, vehicle 0.10, mud 0.08, other 0.20.
The leaves of the three crops have 0.30, shrub has 0.36 and tree_foliage has 0.45.
Calibration against Recorded Scans#
Calibration/Polaris/lidar_results.md gives the full results. The procedure has three parts.
lidar_stats.pymeasures the real sensor in 47 recordings: beam table, body mask, return rates, intensity and range noise.lidar_model.pyfits the radiometry without the game to 640 000 ground cells.lidar_sim.pyrenders the simulated sensor at the poses of 289 recorded scans from 42 runs. Each model gives 447 simulated scans.
The comparison scores 11 metrics. Before the calibration 1 metric was in its limit. With the calibration 9 metrics are in their limits.
| Metric (Worst Case) | Limit | Before | Calibrated |
|---|---|---|---|
| Beam elevation, worst ring | 0.05° | 6.16° | 0.00° |
| Ring azimuth offset, worst ring | 0.05° | 5.11° | 0.00° |
| Body mask IoU | 0.90 minimum | 0.00 | 0.99 |
| Return rate for each ring, mean absolute error | 5 points | 10.8 | 4.9 |
| Points in a scan, relative error | 0.10 | 0.195 | 0.077 |
| Ground detection against range, mean absolute error | 0.05 | 0.026 | 0.016 |
| Ground detection against range, worst bin | 0.15 | 0.411 | 0.152 (not in limit) |
| Range histogram, Jensen-Shannon distance | 0.15 | 0.262 | 0.124 |
| Ground intensity median, worst surface | 3 | 6 | 2 |
| Ground intensity histogram, worst surface | 0.20 | 0.90 | 0.47 (not in limit) |
| Range noise below 10 m, worst surface | 0.41 | 1.40 | 0.32 |
The two metrics that are not in their limits are asphalt between 25 m and 30 m and the intensity histogram of gravel.
The LiDAR of ACRES Core#
Source: Core/Source/AcresCoreLidar.cpp.
ACRES Core has a ray-cast LiDAR on the CPU. It reads the same lidar block of sensors_polaris.json and the same self-return table.
It traces the terrain height grid, the obstacle scene of Core/Data/acre_scene.json and the boxes of other vehicles.
| Topic | Game | ACRES Core |
|---|---|---|
| Geometry | Rendered scene on the GPU, or collision and proxies | Height grid, triangles, cylinders, crown ellipsoids, vehicle boxes |
| Sub-rays | 1 or 7 | 1 |
| Returns for each beam | 1 to 3 | 1 |
| Foliage | Layers with transmission, many echoes | A crown lets the beam through with the probability \(\tau\) |
| Crops and ground cover | Yes | No |
| Micro-slope, albedo texture, roughness, wetness | Yes | No |
| Divergence in the plane of incidence | \(\beta_v\) for ground, \(\beta\) for other solids | Computed for each hit |
| Rain and fog | Yes | No |
| Motion distortion | Yes | No |
| Noise | PCG32 stream | Hash of the seed and the cell, optional |
The backscatter, the range equation, the echo stretch, the detection threshold, the range noise and the intensity byte use the equations of this page. ACRES Core computes the divergence in the plane of incidence for each hit. It uses the hit normal \(\mathbf{n}\), the ray \(\mathbf{d}\) and the up axis \(\mathbf{k}\) of the sensor.
ACRES Core also has a planar pattern with 360 beams, no mount tilt and no radiometry (FAcresLidarPattern::Planar).
Comparison with the Game#
Core/Scripts/compare_lidar.py runs Core/Tools/lidar_compare.cpp at the poses of the calibration renders and compares the two scans beam by beam.
The script reads the work folder of the LiDAR calibration, thus it needs those renders.
Expected Result
A run on 2 October 2026 used 58 scans and gave these values. The body of the vehicle is not in the comparison.
| Measure | Game | ACRES Core |
|---|---|---|
| Beams with a return, rings at or below -8° | 0.995 | 1.000 |
| Beams with a return, rings above -8° | 0.456 | 0.353 |
| Range Difference, Core Minus Game | Median | Median Absolute | Within 0.1 m |
|---|---|---|---|
| Ground, lower rings (475 237 beams) | -0.1 mm | 5.4 mm | 89.9 % |
| Ground, upper rings (161 215 beams) | 3.2 mm | 0.18 m | 46.7 % |
| Buildings, lower rings (12 816 beams) | 0.13 m | 0.13 m | 28.7 % |
| Crop, lower rings (139 944 beams) | 0.51 m | 0.51 m | 2.0 % |
79.5 % of the beams agree: the two ranges are within 15 %, or the two scans have no return.
The ground agrees. The crop beams are different because ACRES Core has no crops: its beam continues to the ground.
Output Formats#
Point File#
With lidar.output = ply or both, each scan writes lidar/scan_<time>.ply. The time is the scan stamp in microseconds.
The file is a binary little-endian PLY with 31 bytes for each point. It contains all returns of the scan.
| Field | Type | Unit | Contents |
|---|---|---|---|
x, y, z |
float | m | The point \(\mathbf{p}\) in the sensor frame at its firing time |
reflectivity |
float | \(\rho_{rep}\). The value 1 is a Lambertian target of 100 %. | |
signal |
ushort | counts | The measured signal |
intensity |
uchar | The intensity byte \(I\) | |
ring |
uchar | Ring index \(r\) | |
column |
ushort | Column index \(c\) | |
class |
uchar | Class number | |
material |
uchar | Material number, in the sequence of the materials table | |
return_index |
uchar | Index of the point in its beam, from 1, in range order | |
num_returns |
uchar | Number of points of the beam | |
return_flags |
uchar | 1 = first, 2 = strongest, 4 = last, 8 = second strongest | |
time_offset_s |
float | s | \(\Delta t_c\) |
Organised Cloud#
With lidar.output = pcd or both, each scan writes lidar/scan_<time>.pcd: a binary PCD file of version 0.7.
WIDTH is \(N_r\) and HEIGHT is \(N_c\). The cell \(c\,N_r + r\) has the fields x, y, z and intensity as float32.
A beam without a return has NaN in x, y and z and the intensity 0.
Each beam gives one return: the strongest, or the first when no return has the strongest flag.
The sensor stream sends the same grid. Refer to Sensor Stream. ROS 2 Topics gives the layout of the ROS 2 cloud.
Scan Row and Preview#
Each scan adds one row to lidar.jsonl and writes a top view of 384 x 384 px as lidar/scan_<time>.png.
The row contains the point counts, the counts for each class and material, the pose of the sensor, the sweep timing and the weather.
episode.json contains the class table, the material table and a text description of each model part.
Parameters#
The LiDAR Block of sensors.json#
The names are relative to the lidar block of Acres/Content/Simulation/sensors.json.
| Name | Type | Unit | Default | Description |
|---|---|---|---|---|
enabled |
boolean | true | Records the LiDAR. | |
hz |
number | Hz | 10 | The scan rate, 1 to 120. |
columns |
integer | 180 | \(N_c\), 4 to 3600. | |
rings |
integer | 8 | \(N_r\), 1 to 128. | |
elevation_min_deg, elevation_max_deg |
number | deg | -20, 10 | The limits of the rings without a table. |
elevations_deg |
array | deg | none | The elevation of each ring. It also sets \(N_r\). |
azimuth_order |
string | ccw |
The direction of the column index: ccw or cw. |
|
azimuth_offsets_deg |
array | deg | none | \(\delta_r\), -30 to 30. |
lens_center_m |
number | m | 0 | \(l\), 0 to 0.5. |
range_m |
number | m | 60 | \(R_{max}\), 1 to 200. |
min_range_m |
number | m | 0.5 | \(R_{min}\). |
std_m |
number | m | 0.02 | \(\sigma_0\). |
position_flu_m |
array of 3 | m | [0, 0, 1.9] | The position in the mount frame. |
rpy_deg |
array of 3 | deg | [0, 0, 0] | Roll, pitch and yaw of the mount. |
wavelength_nm |
number | nm | 905 | \(\lambda\), for the fog extinction. |
beam_divergence_mrad |
number | mrad | 3.0 | \(\beta\), 0 to 20. |
beam_divergence_v_mrad |
number | mrad | 0 | \(\beta_v\). The value 0 gives \(\beta_v = \beta\). |
pulse_length_m |
number | m | 0 | \(L\). The value 0 sets the echo stretch off. |
sub_rays |
integer | 7 | 1 or 7. | |
max_hits_per_sub_ray |
integer | 4 | \(K\), 1 to 8. | |
continue_step_m |
number | m | 0.25 | The distance behind a foliage hit where the search continues. |
min_return_separation_m |
number | m | 0.75 | The range width of an echo. |
return_mode |
string | dual_strongest_last |
Refer to Return Mode. | |
pond_threshold_m |
number | m | 0.003 | The water depth that makes ground a water surface. |
motion_distortion |
boolean | true | The firing time for each column. | |
spin_direction |
string | cw |
The spin direction: cw or ccw. |
|
scan_start_azimuth_deg |
number | deg | 180 | \(\alpha_s\). |
delay_s |
number | s | delay_s (0.05) |
The delay of the scan row. |
output |
string | ply |
ply, pcd or both. |
|
frame_id |
string | lidar |
The frame name of the cloud. | |
self_returns |
string | none | The file of the self-return table. |
| Name | Type | Unit | Default | Description |
|---|---|---|---|---|
radiometry.range_at_10pct_m |
number | m | 60 | \(R_{10}\). |
radiometry.ring_range_at_10pct_m |
array | m | none | \(R_{10}\) for each ring. |
radiometry.signal_at_10pct_counts |
number | counts | 100 | \(S_{10}\). |
radiometry.ambient_counts |
number | counts | 20 | \(B\). |
radiometry.detection_probability_at_spec |
number | 0.9 | \(p_{spec}\), between 0.5 and 0.9999. | |
radiometry.overlap_range_m |
number | m | 1.0 | \(R_{ov}\). |
radiometry.saturation_counts |
number | counts | 65535 | \(S_{sat}\). |
radiometry.range_std_low_snr_m |
number | m | 0.03 | \(\sigma_{low}\). |
weather.enabled |
boolean | true | Rain and fog. | |
weather.rain_extinction_db_per_km |
number | dB/km | 1.076 | \(a\). |
weather.rain_extinction_exponent |
number | 0.67 | \(b\). | |
weather.fog_lidar_ratio_sr |
number | sr | 18.8 | \(S_{fog}\). |
weather.haze_lidar_ratio_sr |
number | sr | 60 | \(S_{haze}\). |
weather.rain_lidar_ratio_sr |
number | sr | 500 | \(S_{rain}\). An engineering estimate. |
weather.rain_drops_per_m3 |
number | 1/m³ | 1951 | The factor of \(n_d\). |
weather.rain_drops_exponent |
number | 0.21 | The exponent of \(n_d\). | |
weather.exit_diameter_m |
number | m | 0.01 | \(d_{exit}\). |
weather.clutter_max_range_m |
number | m | 10 | The range limit of the clutter. |
Materials of sensors.json#
Each entry of lidar.materials can have these keys.
| Name | Type | Unit | Default | Description |
|---|---|---|---|---|
reflectivity |
number | table below | \(\rho\), 0 to 1. | |
specular |
number | 0 | \(s\), 0 to 100. | |
specular_width_deg |
number | deg | 8 | \(\theta_s\). |
transmission |
number | 0 | \(\tau\), 0 to less than 1. | |
wet_darkening |
number | 0 | \(a_w\), 0 to 1. | |
incidence_exponent |
number | 1 | \(k\), 0 to 4. | |
micro_slope_deg |
number | deg | 0 | \(\sigma_t\), 0 to 30. |
reflectivity_sd |
number | 0 | \(s_\rho\), 0 to 2. | |
roughness_m |
number | m | 0 | \(\sigma_h\), 0 to 0.5. |
The material number is the index in this table. The PLY field material uses it.
| Number | Material | \(\rho\) | \(s\) | \(\theta_s\) (deg) | \(\tau\) | \(a_w\) |
|---|---|---|---|---|---|---|
| 0 | other |
0.30 | ||||
| 1 | soil |
0.25 | 0.5 | |||
| 2 | grass |
0.45 | 0.5 | 0.1 | ||
| 3 | gravel |
0.35 | 0.35 | |||
| 4 | asphalt |
0.12 | 0.3 | |||
| 5 | concrete |
0.35 | 0.35 | |||
| 6 | water |
0.02 | 5.0 | 3 | ||
| 7 | corn_leaf |
0.45 | 0.5 | |||
| 8 | soybean_leaf |
0.48 | 0.45 | |||
| 9 | tree_foliage |
0.45 | 0.6 | |||
| 10 | bark |
0.30 | ||||
| 11 | shrub |
0.42 | 0.5 | |||
| 12 | building |
0.45 | 0.3 | 8 | ||
| 13 | metal_bin |
0.30 | 2.0 | 6 | ||
| 14 | vehicle |
0.30 | 0.5 | 8 | ||
| 15 | mud |
0.12 | ||||
| 16 | potato_leaf |
0.47 | 0.45 |
The values are estimates for 905 nm from public spectral libraries. They are not measurements at ACRE.
The LiDAR Block of sensors_polaris.json#
The table shows the keys of the overlay that change a default. The Polaris LiDAR gives the beam table and the materials.
| Name | Type | Unit | Default | Description |
|---|---|---|---|---|
columns |
integer | 1800 | \(N_c\). | |
elevations_deg |
array | deg | 32 values | The beam table. \(N_r = 32\). |
azimuth_offsets_deg |
array | deg | 32 values | \(\delta_r\). |
azimuth_order |
string | cw |
Column \(c\) at \(-0.2\,c\) degrees. | |
lens_center_m |
number | m | 0.035 | \(l\). |
range_m, min_range_m |
number | m | 161.3, 0.2 | \(R_{max}\), \(R_{min}\). |
position_flu_m |
array of 3 | m | [2.5, 0, 2.039] | From base_footprint. |
rpy_deg |
array of 3 | deg | [-1.518, 5.317, 0] | The mount of the URDF. |
sub_rays |
integer | 1 | One ray for each beam. | |
beam_divergence_mrad, beam_divergence_v_mrad |
number | mrad | 1.6, 6.9 | \(\beta\), \(\beta_v\). |
pulse_length_m |
number | m | 1.0 | \(L\). |
return_mode |
string | strongest |
One return. | |
spin_direction, scan_start_azimuth_deg |
string, number | -, deg | cw, 0 |
The sweep. |
std_m |
number | m | 0.0022 | \(\sigma_0\). |
delay_s |
number | s | 0.012 | The delay of the scan row. |
output |
string | pcd |
The organised cloud. | |
self_returns |
string | polaris_lidar_self.f32 |
The body table. | |
radiometry.range_at_10pct_m |
number | m | 87.0 | \(R_{10}\) without a ring value. |
radiometry.ring_range_at_10pct_m |
array | m | 87.0, 38.7, 19.3, 9.7 | \(R_{10}\) for each ring, 32 values. |
radiometry.range_std_low_snr_m |
number | m | 0.0393 | \(\sigma_{low}\). |
Command-Line Options#
| Name | Type | Unit | Default | Description |
|---|---|---|---|---|
-SensorLidarHz= |
number | Hz | file value | The scan rate. |
-SensorLidarColumns= |
integer | file value | \(N_c\). | |
-SensorLidarRings= |
integer | file value | \(N_r\). | |
-SensorLidarRange= |
number | m | file value | \(R_{max}\). |
-SensorLidarReturnMode= |
string | file value | The return mode. | |
-SensorLidarSubRays= |
integer | file value | 1 or 7. | |
-SensorLidarDivergence= |
number | mrad | file value | \(\beta\). |
-SensorLidarIdeal |
flag | off | No motion distortion and no weather. The noise stays. | |
-SensorLidarNoMotion |
flag | off | No motion distortion. | |
-SensorLidarNoWeather |
flag | off | No rain and no fog. | |
-SensorLidarRain= |
number | mm/h | weather model | The rain rate, 500 maximum. |
-SensorLidarVisibility= |
number | m | weather model | The visibility, 10 minimum. |
-SensorCpuLidar |
flag | off | Uses the CPU path. | |
-SensorNoLidar |
flag | off | Does not record the LiDAR. | |
-SensorNoNoise |
flag | off | No signal noise and no range noise. |
Tests#
Tools/LidarModel/build.sh builds and runs the tests of AcresLidarPhysics.h without Unreal Engine.
The tests compare the C++ functions with the values of the Python fit Calibration/Polaris/lidar_model.py.
Expected Result
The last line is 22 passed, 0 failed. The detection threshold line shows 85.961306 counts.
The tests of the LiDAR of ACRES Core are in Core/Tests/lidar_tests.cpp.
Expected Result
The last line is 13 passed, 0 failed.
Code Map#
| Item | File | Function |
|---|---|---|
| Scan geometry, CPU trace, proxies | Acres/Source/Acres/AcresSensors.cpp |
FAcresSensorRecorder::BeginScan, BuildStaticProxies |
| Scan request and pose history | Acres/Source/Acres/AcresSensors.cpp |
FAcresSensorRecorder::PhysicsSample, SweepPoseAt |
| GPU dispatch and readback | Acres/Source/Acres/AcresLidarGpu.cpp |
FAcresLidarViewExtension::PostTLASBuild_RenderThread |
| GPU shader | Acres/Shaders/AcresLidar.usf |
MainCS |
| Merge of the GPU hits | Acres/Source/Acres/AcresSensors.cpp |
FAcresSensorRecorder::MergeGpuScan |
| Materials, ground terms, self returns, files | Acres/Source/Acres/AcresSensors.cpp |
FAcresSensorRecorder::FinishScan |
| Sub-ray pattern | Acres/Source/Acres/AcresLidarModel.cpp |
AcresLidar::SubRayPattern |
| Backscatter, signal, intensity byte | Acres/Source/Acres/AcresLidarModel.cpp |
AcresLidar::Backscatter, SignalCounts, IntensityByte |
| Echoes, detection, returns, clutter | Acres/Source/Acres/AcresLidarModel.cpp |
AcresLidar::ProcessBeam |
| Detection threshold | Acres/Source/Acres/AcresLidarModel.cpp |
FAcresLidarModelConfig::DetectionThreshold |
| Rain and fog | Acres/Source/Acres/AcresLidarModel.cpp |
AcresLidar::Atmosphere, RainExtinction, FogExtinction |
| Column timing | Acres/Source/Acres/AcresLidarModel.cpp |
AcresLidar::ColumnTimeOffset |
| Minnaert law, echo stretch, hash numbers | Acres/Source/Acres/AcresLidarPhysics.h |
MinnaertBackscatter, FootprintRangeSpreadM, PulsePeakFactor, RoughRangeSigmaM, TiltedCos, PlaceNormal, InPlaneDivergence |
| LiDAR of ACRES Core | Core/Source/AcresCoreLidar.cpp |
FAcresLidar::Scan, FAcresLidar::CastRay, FAcresLidarPattern::FromSensorsJson |
| Comparison of ACRES Core with the game | Core/Scripts/compare_lidar.py, Core/Tools/lidar_compare.cpp |
|
| Tests | Tools/LidarModel/lidar_tests.cpp, Core/Tests/lidar_tests.cpp |
|
| Calibration | Calibration/Polaris/lidar_stats.py, lidar_model.py, lidar_sim.py, lidar_report.py |
Limitations#
- The default LiDAR uses placeholder values. Only the Polaris LiDAR has a calibration, and its radiometry fit uses hard ground only.
- \(S_{10}\) is not identifiable from the recorded data. It keeps 100 counts.
- The foliage model is a transmission for each layer. A crop canopy at a low angle returns more beams than the real canopy.
- The game uses one divergence for ground and one for other solids. It does not compute the plane of incidence for each hit.
- The body of the Polaris is a table for one mount and one sensor. The motion of the vehicle does not move the body cells.
- All rings of a column fire together. The trace sees the world at one game frame.
- Rain and fog are homogeneous. The model has no water film on the window, no spray, no snow and no multiple scattering.
- The model has no false returns from sun glare, no crosstalk and no blooming.
- On the GPU path, leaf cards with an alpha mask are solid triangles.
- The noise stream repeats only when the geometry repeats.
References#
- Carbonneau, T. H., and Wisely, D. R. (1998). Opportunities and challenges for optical wireless: the competitive advantage of free space telecommunications links in today's crowded marketplace. Proceedings of SPIE, 3232, 119-128.
- Hapke, B. (1981). Bidirectional reflectance spectroscopy: 1. Theory. Journal of Geophysical Research, 86(B4), 3039-3054.
- Jutzi, B., and Stilla, U. (2003). Laser pulse analysis for reconstruction and classification of urban objects. International Archives of Photogrammetry and Remote Sensing, 34(3/W8), 151-156.
- Kim, I. I., McArthur, B., and Korevaar, E. J. (2001). Comparison of laser beam propagation at 785 nm and 1550 nm in fog and haze for optical wireless communications. Proceedings of SPIE, 4214, 26-37.
- Koschmieder, H. (1924). Theorie der horizontalen Sichtweite. Beiträge zur Physik der freien Atmosphäre, 12, 33-53.
- Lobell, D. B., and Asner, G. P. (2002). Moisture effects on soil reflectance. Soil Science Society of America Journal, 66(3), 722-727.
- Marshall, J. S., and Palmer, W. McK. (1948). The distribution of raindrops with size. Journal of Meteorology, 5(4), 165-166.
- Minnaert, M. (1941). The reciprocity principle in lunar photometry. Astrophysical Journal, 93, 403-410.
- Müller, D., Ansmann, A., Mattis, I., Tesche, M., Wandinger, U., Althausen, D., and Pisani, G. (2007). Aerosol-type-dependent lidar ratios observed with Raman lidar. Journal of Geophysical Research, 112, D16202.
- O'Connor, E. J., Illingworth, A. J., and Hogan, R. J. (2004). A technique for autocalibration of cloud lidar. Journal of Atmospheric and Oceanic Technology, 21(5), 777-786.
- O'Neill, M. E. (2014). PCG: A family of simple fast space-efficient statistically good algorithms for random number generation. Technical report HMC-CS-2014-0905, Harvey Mudd College.
- Rasshofer, R. H., Spies, M., and Spies, H. (2011). Influences of weather phenomena on automotive laser radar systems. Advances in Radio Science, 9, 49-60.
- RoboSense. RS-Helios-5515 User Manual, version 3.0.1.
- Steele, G. L., Lea, D., and Flood, C. H. (2014). Fast splittable pseudorandom number generators. Proceedings of OOPSLA 2014, 453-472.
- Wagner, W., Ullrich, A., Ducic, V., Melzer, T., and Studnicka, N. (2006). Gaussian decomposition and calibration of a novel small-footprint full-waveform digitising airborne laser scanner. ISPRS Journal of Photogrammetry and Remote Sensing, 60(2), 100-112.
- Wojtanowski, J., Zygmunt, M., Kaszczuk, M., Mierczyk, Z., and Muzal, M. (2014). Comparison of 905 nm and 1550 nm semiconductor laser rangefinders' performance deterioration due to adverse environmental conditions. Opto-Electronics Review, 22(3), 183-190.