Skip to content

Design Choices#

This page gives the main design decisions of ACRES. Each decision has its reason and its cost.

Summary#

Decision Reason
A surveyed farm Sensor data and vehicle results must agree with a real site.
Engine-free models One set of equations for the game, ACRES Core and the tests.
Parity of ACRES Core and the game Learning needs speed. Evaluation needs the full world.
A fixed physics step of 120 Hz The forces must not change with the frame rate.
Lockstep A slow client must see a deterministic simulation.
The timestamped command queue A command must apply at the correct physics step.
Render tiers and the pinned sensor profile A dataset must not change with the GPU.
Local sockets between the simulators and ROS 2 The game must build and run without ROS 2.
The DDS loopback fence No simulator message can reach the real vehicle.
Calibration from real to simulation to real The models must agree with the real Polaris.
Configuration as files, sessions as folders Each session must be reproducible.
One episode log format The game, ACRES Core and ROS 2 tools must read the same file.

A Surveyed Farm#

Decision. The world is a measured tile of ACRE. The terrain comes from a LiDAR survey. The soils come from the USDA SSURGO database. The fields, roads and buildings come from imagery, RTK drives and footprints.

Reason. Data for training and tests is useful only when the scene has the properties of a real site. A real site also permits a comparison of the simulator with the real vehicle on the same ground.

Cost. The map pipeline is large. Some items are estimates, for example the height of a bin. The ACRE Scene gives the sources and the error budget.

Engine-Free Models#

Decision. The mathematics of each model is in standard C++ without Unreal types. The file names end in Model. The Unreal layer owns the actors, the threads and the rendering.

Reason. A person who does not know Unreal can read and test the models. A standard compiler builds the model tests in seconds. ACRES Core compiles the same files, so the two simulators cannot use different equations.

Cost. The Unreal layer must copy data into and out of the model structs at each step.

Parity of ACRES Core and the Game#

Decision. ACRES has two simulators. The game has the full world, the renderer and all sensors. ACRES Core has the Polaris, the terrain, the soil, the crops, the obstacles and a ray-cast LiDAR. The two simulators share the engine-free models, the drive-by-wire code, the episode log and the map products.

ACRES Core and the game side by side: the two simulators compile the same engine-free model sources and read the same data. The game adds the Chaos body, the renderer, the GPU sensors, the weather and the Maxxum. ACRES Core adds its own rigid body, terrain, obstacle scene, ray-cast LiDAR and batch. The Python module and core_sim use ACRES Core. Two simulators, one set of model sources THE GAME ADDS SHARED BY THE TWO SIMULATORS ACRES CORE ADDS The game (Unreal Engine) Chaos rigid body chassis integration, collisions Collision mesh ray casts for the wheel contacts Renderer and GPU sensors camera, GPU LiDAR, GNSS, INS Weather and water grid sky, rain, soil-water thread Traffic and workers farm vehicles, persons Maxxum and implements tractor, hitch, PTO Engine-free sources Vehicle and drive-by-wire AcresUtvModel AcresVehicleModel AcresSimModel Soil, tyre and energy AcresSoilModel AcresPowerModel Farm, surface and weather AcresFarmModel AcresSurfaceMap AcresEnvironmentModel Episode log writer Configuration and ACRE map compiles compiles reads reads ACRES Core (libacres_core) Rigid body of the Polaris semi-implicit Euler, 1/120 s Terrain survey triangles, exact ray casts Obstacle scene buildings, bins, trees, collisions Ray-cast LiDAR Helios pattern, planar pattern Farm state crushed crop, ruts, soil-water scenario Batch N environments, thread pool INTERFACES AND USERS Three local sockets bridge, control channel, sensor stream ROS 2 bridge sim_bridge same topics for the two simulators core_sim same sockets acres_core Python module Learners A ROS 2 node sees the same interfaces from the game and from core_sim. A learner steps thousands of Core environments from Python.
The game and ACRES Core compile the same model sources. Each side adds its own world layer. Open the diagram

Reason. Reinforcement learning needs many steps. A batch of Core environments runs much faster than real time. The evaluation and the datasets need the camera, the full scene and the traffic. Only the game has them.

A policy that trains in Core must see the same vehicle in the game. Shared sources give this property.

Cost. Core has its own rigid-body integration and terrain contact. The trajectories of the two simulators are thus not identical. The repository measures the difference. Time Stepping and Determinism gives the numbers. Core has no camera and no Maxxum.

A Fixed Physics Step#

Decision. The vehicle physics runs on the asynchronous physics tick of Unreal at a fixed step of 1/120 s. The tyre and soil model of ACRES computes all wheel forces. Chaos integrates only the chassis body.

Reason. A tyre can change from grip to spin in a few milliseconds. With a variable step, the result changes with the frame rate. A fixed step also gives the sensors an exact clock. Each sample has a physics step number.

Cost. One frame can advance the physics by 0.1 s at most. A slower frame makes the simulation slower than real time.

Lockstep#

Decision. A client can hold the world and advance it by N physics steps. The game then uses a fixed frame time of one physics step. The reply to a step request arrives after the sensors completed their work for these steps. Each data channel carries a barrier that marks the end of the step.

The flow of one lockstep step request: the client sends commands and a step request, the simulator releases the world for N frames with one physics step in each frame, holds the world, lets the sensors finish, sends a barrier on each data channel and then sends the reply. CLIENT GAME OR CORE SERVER DATA CHANNELS 1 Send the commands on the vehicle bridge, before the step request Vehicle bridge the commands go into the command queue of the agent commands 2 Send the step request {"op":"step","steps":N} 3 Start the request frame time = one physics step, 1/120 s wait until no step is in progress 4 Release the world for one frame exactly one physics step for each agent the queued commands apply apply at the next step 5 Hold the world while sensors work the sensors complete the scans and images that were due in the step steps 4 and 5: N times 6 Send a barrier on each channel the N steps are complete, the world is held and all sensors are idle Barrier: sensor stream frame type 6, payload step S Barrier: vehicle bridge {"type":"barrier","step":S} Barrier: CAN transport frame 0xFF1E, payload step S 7 Send the reply {"ok":true,"step":S,"barrier":S} 8 Receive the reply S is the physics step that the world reached the client waits 9 Read each channel to its barrier. All data of the N steps is before the barrier. sensor data and reports of the steps, then the barrier
One lockstep step: the world advances, the sensors complete, each channel sends a barrier, then the client gets the reply. Open the diagram

Reason. A learner or a slow controller needs more wall time than one control period. Without lockstep, the simulation continues while the client computes. With lockstep, two runs with the same commands give the same trajectory.

Cost. A lockstep run with rendering is slower than a free run. The physics of Chaos and the GPU LiDAR limit the determinism of the game. Lockstep Stepping shows the procedure and the measured results.

The Timestamped Command Queue#

Decision. A receiver thread stamps each bridge command with its arrival time. The game maps the arrival time onto the solver time of the frame. The first physics step at or after that time applies the command.

Drive-by-wire command path: a ROS 2 command becomes a line on the vehicle bridge, the receiver thread stamps its arrival, the command clock gives it a solver time, the command queue holds it until the matching physics step, the watchdog examines its age, the actuators and the ULC move the vehicle model, and the reports go out at 50 Hz. The real vehicle has the same commands and reports. Drive-by-wire command path SIMULATOR: A COMMAND ARRIVES ROS 2 command /vehicle/*/cmd ds_dbw_msgs ROS 2 bridge sim_bridge one line for each Vehicle bridge -RlPort= JSON lines, TCP Receiver thread 1000 reads each second, arrival time Command clock arrival time to solver time EACH PHYSICS STEP, 1/120 S Driver keyboard, replay, drive script Watchdog command age: 0.1 s maximum Enable, override system enable, fresh flags Take the snapshots that are due at this step Command queue inbox snapshots, each with its time Drive-by-wire actuators StepDbw Steering servo ULC Gear actuator Pedal emulation Brake pressure commands that are fresh without DBW control Vehicle model engine, CVT, wheels, chassis Reports, 50 Hz dbw_report /vehicle/*/report wheel speeds for the ULC REAL VEHICLE: THE SAME COMMANDS AND REPORTS ROS 2 command /vehicle/*/cmd the same topics Dataspeed gateway commands on the CAN bus DBW firmware timeout, actuators, ULC Polaris Ranger steering column, pedal, brake Reports, 50 Hz /vehicle/*/report the same topics
A command goes through the timestamped queue to the physics step that agrees with its arrival time. Open the diagram

Reason. The game reads its sockets one time for each frame. A long frame contains up to 12 physics steps. Without the queue, all commands of the frame reach the first step and the other steps see no command. The drive-by-wire then reports a command timeout of 0.1 s, although the controller sent commands at 50 Hz. On the real vehicle, the timeout occurs only when the controller stops.

Cost. A command applies up to one frame after its arrival. In lockstep, each command applies at the next step. Drive-by-Wire and ULC gives the details.

Render Tiers and the Pinned Sensor Profile#

Decision. The game selects a render tier from the memory of the GPU one time for each session. The tier changes the quality of the main view only. While a sensor camera records, the game pins the render state to a sensor profile. The high tier pins the profile calibrated. The low tier pins the profile reduced-6gb, unless an option selects calibrated.

Render tiers: the dedicated GPU memory selects the low tier or the high tier, each tier sets the main view, and a recording camera or LiDAR pins the render state to a sensor profile, which the datasets name in the label sensor_profile. TIER SELECTION AT THE START OF A SESSION GPU memory dedicated memory from the RHI, in MiB Tier selection 11 776 MiB or more: high tier less than 11 776 MiB: low tier Option -RenderTier=low|high replaces the selection low high Low tier render state when no sensor records main view at 60 % resolution, TSR to the output size anti-aliasing, shadow, post-process, effects: level 2 global illumination and reflection: level 1 surface cache 2048, shadow pool 2048 pages Nanite 4 pixels for each edge, mesh LOD scale 2 cloud shadow map 128, leaf masks to 60 m High tier render state when no sensor records main view at 100 % resolution, TSR anti-aliasing all scalability groups: level 3 full Lumen global illumination and reflections surface cache 8192, shadow pool 4096 pages Nanite 1 pixel for each edge, mesh LOD scale 1 cloud shadow map 2048, authored leaf masks while a camera or a LiDAR records while a camera or a LiDAR records Sensor profile reduced-6gb the calibrated list with: surface cache 2048, texture pool 2048 MiB, shadow pool 2048 pages, radiance cache 16 texels for each probe also on the high tier with -SensorRenderProfile=reduced Sensor profile calibrated the render state of the Polaris camera and LiDAR validation, the same list as the high tier also on the low tier with -SensorRenderProfile=calibrated Label sensor_profile in episode.json and in the episode log polaris-calibrated · polaris-reduced-6gb · tractor-default · tractor-default-reduced-6gb · custom
The tier follows the memory of the GPU. The sensors use a pinned profile. Open the diagram

Reason. A dataset must not depend on the GPU of the machine that recorded it. The calibration of the camera and the LiDAR is valid only for the render state of the calibration.

Cost. A GPU with little memory cannot hold the calibrated profile. The dataset then contains the label of the reduced profile, so a user cannot confuse the two. Platforms and GPU Tiers gives the limits and the settings.

Local Sockets between the Simulators and ROS 2#

Decision. The game has no ROS 2 code. It opens three sockets on 127.0.0.1. These are the vehicle bridge, the simulator control channel and the sensor stream. The ROS 2 bridge acres_sim is a separate process that connects the sockets to ROS 2. The Core server acres_core_sim opens the same three sockets for ACRES Core.

Reason. The Unreal build does not depend on a ROS 2 installation. A Python client can use the sockets directly without ROS 2. One ROS 2 bridge operates with the game and with ACRES Core.

Cost. Each message crosses one more process. The sensor stream copies each cloud and image one more time.

The DDS Loopback Fence#

Decision. Each shell of the ROS 2 environment limits DDS to the loopback interface. The environment sets a private domain identifier and stops multicast discovery. A script proves the fence with a trace of each sent datagram.

Reason. The workstation can share a network with the PC of the real Polaris. The simulator publishes drive-by-wire commands on the topic names of the real vehicle. A command of the simulator must never reach the real vehicle.

Cost. The ROS 2 graph of the simulator stays on one machine. Build the ROS 2 Workspace shows the settings and the proof.

Calibration from Real to Simulation to Real#

Decision. The parameters of the Polaris come from the logs of the real vehicle. The simulator publishes the Polaris on the topics, types and frames of the real vehicle. The software of the vehicle thus drives the simulated Polaris without a change.

Calibration loop: the real Polaris gives logs that are read only, the extracts give event tables and episodes, the fit gives the parameters, the simulator loads the parameters, the replay comparison gives metrics, and the field-day kit gets new runs from the vehicle. The calibration loop REAL FIT Real Polaris Dataspeed DBW, the lab drives it Real logs ROS 2 bags, read only Extracts event tables, episodes polaris_dbw.py Fit least squares polaris_dbw.py fit Parameters polaris.json sensor overlay TEST ON THE VEHICLE COMPARE Field-day kit new runs with safety limits Metrics RMSE, tracking error fit_results.json Replay comparison recorded commands replay_logs.py Simulator test stand, Core, game polaris_tests loaded at the start recorded commands and reports the lab runs it ACRES reads the logs of the vehicle. The workstation does not send commands to the vehicle.
Real logs give the parameters. The simulator replays the logs. The lab then examines the result on the vehicle. Open the diagram

Reason. A policy or a controller must behave on the vehicle as it behaved in the simulator. The loop makes each difference measurable. The same recorded run goes through the model, and the tools compare the result with the log.

Cost. The Maxxum and its implements have no such data. Their values come from publications and datasheets. ACRES reads the vehicle logs but never sends a command to the vehicle. The lab operates the vehicle. Calibrate against Real Logs shows the procedure.

Configuration as Files, Sessions as Folders#

Decision. All tunable values are in JSON files. The simulation reads only files and command-line options. The menu is an editor for these files. When the menu starts a session, the game writes a session folder with copies of the files and loads the map again.

Reason. Each session contains the exact configuration of its run. A script can do all that the menu does. The simulation code does not depend on the UI.

Cost. A session start loads the map two times when it comes from the menu.

One Episode Log Format#

Decision. The episode log is an MCAP file of ROS 2 messages in CDR encoding. One C++ file without dependencies writes and reads it. The game and ACRES Core compile this file.

Reason. ROS 2 tools read the log without a conversion. The game can replay a log of ACRES Core. This permits the render-later method: record states at high speed, then make the images in the game.

Cost. The message layouts exist two times: in acres_interfaces and in the writer. A test compares them. Episode Log gives the layout.

Other Decisions#

Decision Reason Cost
The soil-water model has its own thread and clock. Water moves in seconds to hours. The physics step has no time for the full grid. In lockstep, the physics thread must step the water at fixed times.
Each noise source has its own seeded generator. A user can generate a dataset again. The seeds are part of the configuration.
The camera renders one image, then a sensor model changes it. A scene capture gives the image at low cost. The noise applies to the 8-bit image, not to linear light.
The LiDAR traces rays on the GPU against the scene of the renderer. The LiDAR hits the same geometry that the camera shows. A machine without hardware ray tracing uses a CPU ray caster.
The optimizer calls tyre and soil functions of the game source. A plan must use the equations of the simulation. The optimizer uses the legacy soil model, and the simulated wheels use the soil library. A plan can thus fail in the simulation.
The farm traffic is kinematic. No user records the wheel slip of the traffic. The traffic does not react to the soil.
The UI is C++ (Slate), not Blueprint assets. A reviewer can read a change as text. The UI code is longer.
The repository contains copies of the vehicle message packages. The definitions must agree byte for byte with the recorded bags. A new release of the vendor needs a manual update.