Skip to content

Sensors#

The Polaris carries the sensors of the real vehicle: a Helios LiDAR, a front camera, GNSS, an IMU and an INS. This page shows each sensor, how to change it, and how to read it from the game, from a stream or from ACRES Core.

Before you start

The Sensors of the Polaris#

The values come from Acres/Content/Simulation/sensors_polaris.json. The game makes every sensor. ACRES Core makes the LiDAR only.

Sensor Rate Main Values Model
LiDAR (Helios) 10 Hz 32 rings x 1800 columns, 161.3 m range, 0.2 m minimum range, strongest return, motion during the sweep LiDAR
Camera 10 Hz 896 x 512 pixels, 73.5 degrees horizontal field of view, rolling shutter, exposure, noise Camera
GNSS 10 Hz RTK fix states, multipath, outages INS and GNSS
IMU 120 Hz Accelerations and turn rates with bias and noise INS and GNSS
INS 100 Hz A fused pose in NAD83 UTM zone 16 INS and GNSS
CAN Wheel speeds and drive-by-wire signals. Off for the Polaris by default. Wheel and CAN Signals
Sensor rigs of the Maxxum and the Polaris: schematic side views with the positions of the LiDAR, the GNSS antenna, the camera, the IMU and the frame base_footprint. MAXXUM, sensors.json Side view. The file gives the mounts from the chassis origin. wheelbase 2.642 m x z 1 2 3 4 B Position, FLU metres, from the chassis origin 1 LiDAR[0.0, 0.0, 1.9] 2 GNSS antenna[0.0, 0.0, 1.7] 3 Camera[0.6, 0.0, 1.65] 4 IMU, chassis origin[0.0, 0.0, 0.0] B base_footprint[-1.057, 0.0, -1.1] POLARIS, sensors_polaris.json Side view. The file gives the mounts from base_footprint. wheelbase 2.870 m x z 1 2 3 4 B Position, FLU metres, from base_footprint; rpy, degrees 1 LiDAR[2.169, -0.056, 2.049] rpy [-0.16, 5.543, 0.361] 2 GNSS antenna[1.371, -0.756, 2.162] 3 Camera[2.411, 0.554, 1.91] rpy [-2.09, 16.8, -2.0] 4 IMU[1.285, 0.4, 1.646] B INS output point[0.0, 0.0, 0.0]
The mount of each sensor, in the frame of the vehicle: x forward, y left, z up, in meters. The values come from the sensor files. Open the diagram
The sensor pipeline: the physics step is the clock, a due-time scheduler starts the samples, the physics thread, the game thread and the thread pool make the data, and the data go to the session log and to the sensor stream. CLOCK AND SAMPLE SCHEDULING Physics step clock of all sensors, 1/120 s Due-time scheduler first sample at t = 0, then one sample each 1/f, in the first step at or after the due time truth, actions, IMU 120 Hz · INS 100 Hz · CAN 20 Hz · GNSS, LiDAR, camera 10 Hz samples that are due camera due time SENSOR MODELS, BY THREAD Physics thread truth, actions and CAN rows IMU, GNSS and INS models pose history of the LiDAR sweep request for each LiDAR scan Game thread LiDAR: beam trace, GPU ray pass, echoes, detection and returns camera: warm-up renders, sample render, GPU readback Thread pool LiDAR: organised cloud, PLY and PCD encode camera: camera model, PNG encode LATENCY AND FILES Latency buffer available = sample + delay_s release in a physics step Writer thread one thread for all rows queue limit 200 000 lines Session log JSONL rows, episode.json PLY, PCD and PNG files The LiDAR row and the camera row go into the latency buffer when their files exist. The truth, actions and CAN rows go to the writer thread without a delay. rows PLY, PCD, PNG clock, INS cloud, image Sensor stream sender thread binary frames on 127.0.0.1, option -SensorStream=, no modelled delay, queue limit 96 MiB for clouds and images ROS 2 bridge sim_bridge publishes the ROS 2 topics
From the physics step to the files and the stream: each sensor samples the state, adds its errors and writes its rows. Open the diagram

Get the Sensor Data#

Packaged/Linux/Acres.sh -VehicleDemo -Vehicle=polaris -SensorRecord

The game writes an episode folder with images, scans and JSON Lines files. The key F6 starts and stops an episode. Refer to Collect Data.

Packaged/Linux/Acres.sh -VehicleDemo -Vehicle=polaris -SensorStream=5601

The game sends the samples on a TCP port. The ROS 2 bridge publishes them as ROS 2 topics. Refer to ROS 2 and the Sensor Stream reference.

ranges, material = batch.lidar(None, pattern="helios", crops=True)

No rendering: many scans at the same time. Refer to Scan with the LiDAR in ACRES Core.

The HUD with the camera image and the LiDAR view

Change a Sensor#

Option Function
-PolarisSensorConfig=<file> Uses a different sensor file for the Polaris. Copy sensors_polaris.json and change the copy.
-SensorLidarHz=, -SensorLidarRings=, -SensorLidarColumns=, -SensorLidarRange= The rate, the rings, the columns and the range of the LiDAR
-SensorLidarReturnMode= The return mode of the LiDAR
-SensorLidarIdeal A LiDAR with no noise, no divergence and no weather
-SensorCameraHz=, -SensorCameraWidth=, -SensorCameraHeight=, -SensorCameraHfov= The rate, the size and the field of view of the camera
-SensorCameraIdeal A camera with no distortion, no noise and no rolling shutter
-SensorGnssHz=, -SensorGnssOutage= The GNSS rate and an outage
-SensorNoNoise Removes the noise of all sensors
-SensorNoCamera, -SensorNoLidar, -SensorNoGnss, -SensorNoImu, -SensorNoIns, -SensorNoCan Removes one sensor
-SensorSeed= The seed of the sensor noise

In an agents file, sensor_overlay changes values of the sensor file for one vehicle. Refer to Vehicles.

Scan with the LiDAR in ACRES Core#

ACRES Core casts the rays of the Helios pattern on the same scene and crops. Each return has a material code.

lidar_materials.py
"""One Helios scan in a soybean field, with the crops, and the returns of each material."""
import math
from collections import Counter

import numpy as np
import acres_core as ac

batch = ac.Batch(".", num_envs=1)
batch.reset([0], np.array([[-585.0, 75.0, math.pi / 2, 0.0]]))       # field 14
names = {code: name for name, code in ac.LIDAR_MATERIALS.items()}      # material code -> name
for crops in (False, True):
    ranges, material = batch.lidar([0], pattern="helios", crops=crops)
    hit = np.isfinite(ranges[0])
    counts = Counter(names.get(int(m), str(int(m))) for m in material[0][hit])
    print(f"crops={crops}: {hit.sum()} returns", dict(counts.most_common(4)))

Expected result

With the crops, the soybean plants take the place of approximately half of the ground returns. self is the body of the vehicle.

crops=False: 35004 returns {'self': 18221, 'terrain': 16783}
crops=True: 33872 returns {'self': 18221, 'terrain': 7915, 'crop_soybean': 7736}
Argument of lidar Function
pattern planar (360 beams in one ring at the sensor height) or helios (1800 x 32)
crops=True Adds the corn, soybean and potato plants and the grass cover
intensity=True Adds a third array: the calibrated reflectivity byte of each return
sweep=True Casts each column from the pose at its firing time, with the motion of the vehicle
noise=True, seed Adds the range noise
objects How the scan sees the objects on the path: shape, column, none or auto

Next Steps#

  • Collect Data

    Record a camera and LiDAR data set and read it in Python.

  • LiDAR Model

    The beam pattern, the materials, the weather and the noise.

  • Camera Model

    The lens, the exposure, the rolling shutter and the noise.