Skip to content

Architecture#

This page shows the parts of ACRES, the connections between them and the threads that run them. Read Unreal for C++ Developers first when the Unreal terms are new to you.

The Parts#

ACRES has five parts.

Part Location Function
The game Acres/ The full simulator in Unreal Engine 5.8: the farm, both vehicles, the implements, the weather and all sensors.
ACRES Core Core/ The simulator without Unreal: the Polaris on the same terrain, soil and crops, with a ray-cast LiDAR.
The ROS 2 workspace ROS/ The messages, the ROS 2 bridge, the Core server and the Polaris description.
The learning stack Learning/ The tasks, the environments, the training and the deployment program.
The calibration studies Calibration/ The comparisons of the simulator with real data.

The system overview shows the data flows between these parts.

The Layers of the Game#

The game is one Unreal module with three layers.

The layers of the game module: clients, the Unreal layer, the engine-free models, the files, and ACRES Core that compiles the same model files. Keyboard and menu a person drives Socket clients Python programs, CAN devices ROS 2 bridge acres_sim Unreal layer actors, threads, rendering, UI. Chaos integrates the chassis body. Shell and menu AcresShell AcresMenu AcresSession AcresAgents Vehicle pawn AcresVehicle AcresPolaris AcresVehicleControl Bridges AcresRlBridge AcresSimControl AcresSensorStream Farm and weather runtime AcresFarmRuntime AcresEnvironment AcresFieldSetup Sensors AcresSensors AcresLidarGpu AcresCameraModel AcresLidarModel AcresGnssModel Rigs, traffic, view AcresRig AcresNpc AcresHud AcresRenderTier AcresSoilVisuals one call for each physics step, all values in SI units Engine-free models standard C++, no Unreal types Vehicle models AcresVehicleModel AcresUtvModel AcresPowerModel Implement models AcresImplementModel AcresImplementCoupling AcresFieldWorkModel Soil and farm models AcresSoilModel AcresFarmModel AcresSurfaceMap Weather model AcresEnvironmentModel Sensor mathematics AcresLidarPhysics AcresGeoUtm Timing and episode log AcresCommandTiming AcresEpisodeLog compiles the same model files ACRES Core its own world layer for the Polaris: terrain, scene, rigid body, ray-cast LiDAR, batch Files Configuration Content/Simulation/ read at the start Map products Simulation/ACRE/ read at the start Session folder Saved/Sessions/ logs of the session ACRES Core reads the same files
The Unreal layer calls the engine-free models at each physics step. ACRES Core compiles the same model files. Open the diagram
  1. Clients. A person at the keyboard, a program on a socket, or the ROS 2 bridge.
  2. The Unreal layer. It owns the actors, the threads, the rendering and the UI.
  3. The engine-free models. They contain the mathematics. They use standard C++ and SI units.

The Unreal layer reads the configuration and the map products at the start of a session. At each physics step it fills the input structs of the models and applies their outputs. Chaos, the physics engine of Unreal, integrates only the chassis body from the total force and torque.

File Groups#

All game code is in Acres/Source/Acres. A file name without an extension stands for the .h file and the .cpp file.

Group Files Function
Vehicle pawn AcresVehicle, AcresVehicleControl.cpp, AcresPolaris.cpp The pawn of a vehicle, its physics step, its controls and the Polaris branch.
Vehicle models AcresVehicleModel, AcresUtvModel, AcresSimModel, AcresPowerModel The Maxxum model, the Polaris model with the drive-by-wire, the power and fuel model.
Soil AcresSoilModel, AcresSurfaceMap The soil library, the tyre and soil forces, the surface class at a point.
Implements AcresImplementModel, AcresImplementCoupling, AcresFieldWorkModel The implement forces, their path to the chassis, the marks of field work.
Equipment rigs AcresRig The skeletal meshes of the vehicles and implements that follow the physics state.
Agents AcresAgents, AcresPlaces, AcresGeoUtm.h The vehicles of a session, the named places, the UTM projection.
Farm AcresFarmRuntime, AcresFarmModel, AcresFieldSetup, AcresSoilVisuals, AcresCrops.h Crops, soil water, ruts, field settings and the look of the ground.
Weather AcresEnvironment, AcresEnvironmentModel Sun, clouds, temperature, wind, rain and the clock.
Sensors AcresSensors, AcresCameraModel, AcresLidarModel, AcresLidarPhysics.h, AcresLidarGpu, AcresGnssModel The sensor recorder and the model of each sensor.
Rendering AcresRenderTier, AcresVideoCapture The render tiers, the sensor profile and the video capture.
Bridges AcresRlBridge, AcresCommandTiming.h, AcresSensorStream, AcresSimControl The vehicle bridge with the timestamped command queue, the sensor stream, the simulator control channel.
Logs AcresSessionLog, AcresEpisodeLog, AcresEpisodeSchemas.h, AcresReplay The session log, the episode log and the replay input.
Application AcresShell, AcresMenu, AcresSession, AcresHud.cpp, AcresOptimizer, AcresNpc The game mode, the menu, the session configuration, the HUD, the optimizer and the farm traffic.

C++ Headers lists the public declarations of each file.

How a Session Starts#

The menu does not build the simulation in the running map. It writes a session folder with copies of the configuration files. Then it loads the map again with options that point to the folder. A command line can start the same session without the menu. It gives the folder with the option -VehicleOutput=.

How a session starts: the menu prepares a session folder with configuration copies and reloads the map with flags, or the command line gives the flags directly. The vehicle pawn reads the files and writes the logs. End Simulation loads the menu again. START GAME FILES Session folder Menu New Simulation, Replay Simulation, Pilot Prepare the session examines the settings, makes the session folder FAcresSession::Prepare Configuration copies session-config.json environment.json tractor.json sensors.json field-setup.json Command line -VehicleDemo -SessionLog -VehicleOutput= Map load with flags The game mode selects the pawn from the flags. /Game/Maps/V03ACRE Vehicle pawn reads the files, runs the physics at 120 Hz Logs tractor.csv session-summary.json sensors/episode-NNN/ farm-state.json and more farm files Start writes the copies flags and file paths, then the map loads again flags reads the copies logs Esc, then End Simulation: the logs close and the map loads with the menu
The menu or a command line gives the options. The pawn reads the files of the session folder. Open the diagram

The simulation reads only files and options. Each session is thus reproducible from its folder. Run the Packaged Game and Configuration Files give the details.

What the Vehicle Pawn Owns#

AAcresVehiclePawn is the hub of a session. One instance exists for each agent. The Maxxum and the Polaris use the same class with different models and parameters. BeginPlay of agent 0 does these steps in this sequence.

  1. It reads the agent list of the session and its own configuration.
  2. It goes to the spawn pose.
  3. It makes the weather runtime and the farm runtime. The farm runtime starts the water thread.
  4. It makes the body, the wheels, the equipment rig and the implement.
  5. It opens the session log and starts the sensor recorder and the sensor stream.
  6. It opens the vehicle bridge.
  7. It spawns the other agents. They share the farm and the weather.
  8. It makes the simulator control when the command line requests it.

Each agent has its own controls, sensors, bridge ports and logs. Run Several Vehicles shows a session with two agents.

One Physics Step#

Each 1/120 s the physics thread calls AsyncPhysicsTickActor of each vehicle pawn.

One physics step of a vehicle: the pawn exchanges the controls, advances the weather and the farm, runs the steering and the drivetrain, examines each wheel contact, solves the driveline, sums the forces, gives them to Chaos and publishes the step. One physics step of a vehicle, 1/120 s INPUTS PHYSICS THREAD, IN THIS SEQUENCE OUTPUTS 1 Exchange the controls controls, bridge command, reset request 2 Examine the body mass properties, energy of the last step, reset 3 Advance the weather and the farm rain, soil water, crops (agent 0 only) 4 Select the control source hand throttle, park brake, automatic drivers 5 Steering and drivetrain StepSteering, StepDrivetrain 6 Each wheel: contact ray cast, surface, soil state, PrepareWheel 7 Driveline solve SolveDriveline wheel speeds, clutch, fuel 8 Each wheel: forces and marks tyre forces, water drag, ruts, crushed crop 9 Air drag and implement implement forces, mass and engine loads 10 Forces to Chaos AddForce, AddTorque 11 Publish the step telemetry, logs, sensor sample The Polaris runs the same step. Its own functions replace the drivetrain parts of steps 5 to 7. Game thread keyboard controls Vehicle bridge command queue Weather and farm models shared by all agents Automatic drivers scripted test, replay, drive script Ground collision mesh, surface map, soil class, soil water, ruts Farm state ruts and tread prints crushed crop field-work marks Chaos rigid body integrates the chassis Telemetry HUD, vehicle bridge Session log, episode log one row for each step Sensor recorder sensor stream, sensor files
The sequence of operations in one physics step. Open the diagram

The pawn computes each force itself and gives Chaos only the totals. The physics is thus the same as in the model tests and does not change with the frame rate. Time Stepping and Determinism gives the sequence and the timing rules.

One Frame#

On the game thread, Tick does the visible work of a frame.

  • It moves the equipment rig to the last physics state.
  • It reads the keyboard and the sockets and puts commands into the timestamped command queue.
  • It schedules the camera captures and the LiDAR scans and collects their results.
  • It updates the sky, the sun, the fog, the rain and the weather materials.
  • It applies the changes of the crops, the ruts and the water to the ground.

The HUD draws the speed, the time, the weather, the minimap and the panels after the 3D scene.

Threads#

The threads of the game and the data that moves between them: physics, game and render threads in the centre, worker threads at the sides. Water thread AcresWater soil water on its own clock flow, infiltration, drains Physics thread fixed step, 120 Hz Vehicle step: commands, models, wheel forces, forces to Chaos, farm marks, log row, sensor samples at exact step numbers. Sensor writer AcresSensorWriter writes JSONL rows, images and point clouds moisture rain, ruts rows controls and timed commands telemetry: a struct copy under a lock Bridge receiver AcresRlReceiver reads the vehicle bridge and stamps each message Game thread each frame Actors, UI, visuals, keyboard, sockets, sensor schedule (camera captures, LiDAR scans), lockstep control. Sensor stream AcresSensorStream sends sensor frames to the ROS 2 bridge messages frames render commands images and scans Thread pool camera sensor model and PNG encoding, one task for each image Render thread each frame Draws the frame, reads the sensor camera image back from the GPU, runs the GPU LiDAR pass. Video writer AcresVideoWriter sends window frames to ffmpeg for a video capture pixels frames
The threads of the game and the data that moves between them. Open the diagram
Thread Rate Work
Game Each frame Actors, UI, visuals, sockets, the sensor schedule, lockstep control.
Physics Fixed, 120 Hz The vehicle step: models, forces, farm marks, log rows, sensor samples.
Render Each frame Drawing, the camera readback, the GPU LiDAR pass.
AcresWater Its own clock The soil-water model. In lockstep, the physics thread steps it at fixed simulation times.
AcresSensorWriter When rows arrive Writes the sensor files in the sequence of delivery.
AcresSensorStream When frames arrive Sends the sensor frames to the ROS 2 bridge.
AcresRlReceiver When messages arrive Reads the vehicle bridge and stamps each message with its arrival time.
AcresVideoWriter Each captured frame Sends the frames of the window to ffmpeg.
Thread pool Each camera image The camera sensor model and the PNG encoding.

Data moves between threads in three ways.

Method Use
A small struct that the code copies under a lock Telemetry, controls, the weather state.
A snapshot with three buffers The soil-water state. A reader never waits.
A bounded queue Sensor rows, scan requests, stream frames, timed commands. A full queue drops data and counts the drops.

The Bridges#

A session can open three sockets for each agent on 127.0.0.1.

Socket Option Direction Content
Vehicle bridge -RlPort= Both Commands to the vehicle, reports from the vehicle. The Maxxum also has a J1939 CAN bus.
Simulator control channel -SimControl= Both Pause, steps, resets, entities, conditions, recording. One channel for the session.
Sensor stream -SensorStream= Out The clock, the INS data, the LiDAR clouds and the camera images.
The three local sockets of one session: the simulator control channel for the session, and a vehicle bridge and a sensor stream for each agent, with the clients and the data that flows on each socket. CLIENTS SOCKETS ON 127.0.0.1 GAME OR CORE SERVER Session client sim_control.py ROS 2 bridge: services and the step action Simulator control channel JSON lines on TCP, one client one socket for the session -SimControl=5600 Session state, lockstep steps, resets, conditions, episode log requests replies, events Controller dbw_client.py ROS 2 bridge: command and report topics Vehicle bridge JSON lines on TCP, one client -RlPort=5556 J1939 CAN frames, UDP or SocketCAN -RlCanUdp= -RlCan= commands, resets reports, barrier ROS 2 bridge sim_bridge publishes the sensor topics and the clock Sensor stream binary frames on TCP, one client INS, LiDAR, camera, clock -SensorStream=5601 frames, barrier Agent 0 one vehicle: Polaris or Maxxum sensor rig of the vehicle More clients one controller for each agent Sockets of agent 1, 2, ... -RlPorts=5556,5557 -SensorStreams=5601,5602 Agent 1, 2, ... own vehicle bridge and sensor stream
The three sockets of a session and their clients. Open the diagram

The ROS 2 bridge acres_sim connects to the three sockets and publishes the topics of the real vehicle. The Core server acres_core_sim opens the same sockets for ACRES Core, so one ROS 2 bridge operates with the two simulators. Interfaces and Ports lists the ports and the protocols.

ACRES Core#

ACRES Core compiles the engine-free model files of the game and adds its own world layer.

ACRES Core and the game side by side: the two simulators compile the same engine-free model sources and read the same data. The game adds the Chaos body, the renderer, the GPU sensors, the weather and the Maxxum. ACRES Core adds its own rigid body, terrain, obstacle scene, ray-cast LiDAR and batch. The Python module and core_sim use ACRES Core. Two simulators, one set of model sources THE GAME ADDS SHARED BY THE TWO SIMULATORS ACRES CORE ADDS The game (Unreal Engine) Chaos rigid body chassis integration, collisions Collision mesh ray casts for the wheel contacts Renderer and GPU sensors camera, GPU LiDAR, GNSS, INS Weather and water grid sky, rain, soil-water thread Traffic and workers farm vehicles, persons Maxxum and implements tractor, hitch, PTO Engine-free sources Vehicle and drive-by-wire AcresUtvModel AcresVehicleModel AcresSimModel Soil, tyre and energy AcresSoilModel AcresPowerModel Farm, surface and weather AcresFarmModel AcresSurfaceMap AcresEnvironmentModel Episode log writer Configuration and ACRE map compiles compiles reads reads ACRES Core (libacres_core) Rigid body of the Polaris semi-implicit Euler, 1/120 s Terrain survey triangles, exact ray casts Obstacle scene buildings, bins, trees, collisions Ray-cast LiDAR Helios pattern, planar pattern Farm state crushed crop, ruts, soil-water scenario Batch N environments, thread pool INTERFACES AND USERS Three local sockets bridge, control channel, sensor stream ROS 2 bridge sim_bridge same topics for the two simulators core_sim same sockets acres_core Python module Learners A ROS 2 node sees the same interfaces from the game and from core_sim. A learner steps thousands of Core environments from Python.
The shared model sources and the parts that each simulator adds. Open the diagram
Item The Game ACRES Core
Vehicles Maxxum with implements, Polaris Polaris
Chassis integration Chaos Its own rigid body
Terrain The collision mesh of the map The terrain triangles of the same survey
Sensors Camera, GPU LiDAR, INS, GNSS, IMU, CAN Ray-cast LiDAR, INS, wheel signals
Environments in one process One world, more than one agent A batch of independent environments
Episode log Yes Yes, the same format

Python API and Headless Core Runs show how to use it.

Data on Disk#

Location Content Reader
Acres/Content/Simulation/*.json The parameters of the vehicles, soils, sensors, weather and farm. The simulation at the start, the menu for the defaults, ACRES Core.
Acres/Content/Simulation/ACRE/ The map products. The farm, the weather, the traffic, the optimizer, the HUD, ACRES Core.
Saved/Sessions/<stamp>-<mode>/ One session that the menu started: configuration copies, the session log, the sensor episodes. The user.
Saved/Vehicle/ or the folder of -VehicleOutput= One session that a command line started. The user.
Saved/Configs/ The presets that the menu saved. The menu.
Saved/Optimizer/ The results of the optimizer. The user.

Session Log, Episode Log and Map Products give the formats.