AcresRlBridge.h#

Acres/Source/Acres/AcresRlBridge.h Generated

Live reinforcement-learning bridge: lets an external agent (a Python client such as acres_learn, or ROS/acres_sim) drive the player's tractor in the running game, with full rendering and sensors running, over one of two transports:

JSON over TCP (-RlPort=<port>): newline-delimited JSON on 127.0.0.1:<port>, one client at a time.
CAN bus (-RlCan=<interface>, Linux SocketCAN, e.g. vcan0; or -RlCanUdp=<port>, the same 16-byte frames over UDP on 127.0.0.1 for machines without vcan): SAE J1939 / ISO 11783 (ISOBUS) frames, like a real auto-steer or robot controller on the tractor bus.

Both transports carry the same observation and action and share the reset / episode logic, so the Python side can switch with one flag. Several may be open at once; the controls follow whichever sent the latest action.

JSON protocol. Game -> client, every 0.1 s of simulation time while a client is connected: {"t": physics time s, "episode_t": time since the last reset s, "x", "y": body origin, world metres (X east, Y south), "z", "yaw_deg": Unreal yaw (0 = east, positive toward south), "vx", "vy": body-frame velocity m/s (x forward, y right), "yaw_rate": rad/s (positive = turning right, Unreal convention), "cross_slope": rad (positive = ground falls away to the right), "slip": [4] wheel slip ratios (FL, FR, RL, RR), "steer_rad": actual steering angle (positive = left), "rpm", "difflock": 0/1/2, "theta_here", "theta_5", "theta_10": soil water content under the tractor and 5 m / 10 m ahead, "lidar": [16] ranges in m (front half-ring, -90..+90 degrees from the heading, sector centres, 30 m max), "bunker_kg", "harvest_kg", "crushed_m2", "draft_n", "implement", "implement_depth_m", "pto_kw", "hitch_lever", "hitch_lift_deg", "hitch_mode", "raised", "pto_on", "pto_rpm": the Maxxum implement's state, "struck": true if a worker was struck since the last message, "episode": resets requested by agents, "reset_pending": true until the physics thread has applied the last reset, "reset_epoch": resets applied (the R key also bumps it), "workers": workers the bridge has placed, "stale": true while the safety brake holds} Client -> game (any subset of keys in one object): {"steer": -1..1 (fraction of HALF the steering lock, positive = left), "pedal": -1..1 (>0 throttle, <0 brake), "difflock": >0 locks the rear axle} driving action (gear fixed at F10, forward) {"reset": {"x": m, "y": m, "yaw_deg": deg, "speed": m/s}} teleport to that pose on the ground (new episode); the farm's marks stay (they belong to the simulation); "restore_field": true clears the field under the pose {"env_clock": {"date": "YYYY-MM-DD", "hour": h}} restart the weather clock at that local date and time {"workers": [[x, y], ...]} replace the bridge's workers (needs -NpcWorkers) {"cmd": "harvester", "on": true|false} switch the harvester header / digger {"hud": "text", "reward": x} status line for the HUD (see StatusText) {"curvature": 1/m} guidance: path curvature (positive = left), steering angle atan(curvature x wheelbase), as the CAN Guidance System Command; a driving action {"implement": {"raise": bool, "pto": bool, "transport": bool, "hitch_lever": 0..1, "hitch_mode": "position" | "draft" | "float", "draft_setpoint_n": N, "hand_throttle": 0..1, "park_brake": bool}} implement and operator controls (Maxxum; the keyboard's Q, E, O, Up/Down, F, PageUp/Down, +/-, P)

Every observation also carries "agent" (the agent's name, AcresAgents.h) and "vehicle" ("maxxum" or "polaris"). In a multi-agent session each agent has its own port (-RlPorts=5555,5556 or the agent's "rl_port"), so one client drives one vehicle.

Polaris drive-by-wire (the Dataspeed interface, ds_dbw_msgs 2.3.11 field names and units, so a ROS 2 bridge maps each message one to one). Client -> game, any of these keys in one object, every command at 20-100 Hz: the model drops a subsystem 0.1 s after its last message, as the vehicle does (Documentation/Modelling/polaris.md): {"dbw": {"enable": true}} / {"dbw": {"disable": true}} /vehicle/enable, /vehicle/disable (std_msgs/Empty) {"steering_cmd": {"cmd": deg | 1/m | rad/s | %, "cmd_type": 2 angle | 3 curvature | 4 yaw rate | 14 percent, "cmd_rate": deg/s, "cmd_accel": deg/s2 (0 = default), "enable": true}} {"throttle_cmd": {"cmd": %, "cmd_type": 14 percent | 13 raw pedal, "rate_inc": %/s, "rate_dec": %/s, "enable": true}} {"brake_cmd": {"cmd": % | bar, "cmd_type": 14 percent | 1 pressure, "rate_inc", "rate_dec", "enable": true}} {"gear_cmd": {"cmd": 1 P | 2 R | 3 N | 4 H (DRIVE) | 5 L (LOW)}} (also {"cmd": {"value": n}}) {"ulc_cmd": {"cmd": m/s | m/s2, "cmd_type": 1 velocity | 2 accel, "limit_accel", "limit_decel", "limit_jerk_throttle", "limit_jerk_brake" (0 = default), "enable": true, "coast_decel": false}} {"drive_mode": "awd" | "2wd" | "turf"} the Ranger's AWD switch (not a DBW message) Unsupported command types (steering torque, brake torque or acceleration) are rejected with a warning. "clear", "ignore", "enable_shift" and "enable_shift_park" are accepted and ignored. Driver input on the keyboard while the system is enabled is an override: it disables the system until the next enable. Game -> client, at 50 Hz of simulation time while a client is connected (one line per report instant): {"type": "dbw_report", "agent", "t": physics time s, "steering_report": {"steering_wheel_angle", "cmd", "cmd_type", "enabled", "override_active", "timeout"}, "throttle_report": {"percent_input", "percent_cmd", "percent_output", "cmd_type", "enabled", ...}, "brake_report": {"pressure_input", "pressure_cmd", "pressure_output", "percent_cmd", "percent_output", "cmd_type", "enabled", ...}, "gear_report": {"gear", "cmd", "driver"}, "ulc_report": {"cmd_type", "vel_ref", "vel_meas", "accel_ref", "accel_meas", "enabled", "timeout"}, "vehicle_velocity": {"vehicle_velocity_brake", "vehicle_velocity_propulsion", "dir_src": 1}, "system_report": {"enabled", "override"}, "drive_mode", "road_wheel_rad", "stop_hold"} A Polaris client that sends only DBW messages is never treated as a stale RL agent (no safety brake): the DBW timeouts are the safety net, as on the vehicle. The RL "steer" / "pedal" actions still work on the Polaris as the driver's steering wheel (a fraction of half the road-wheel lock) and pedals.

CAN protocol (29-bit J1939 identifiers: priority << 26 | PGN << 8 | source address; for PDU1 PGNs, PF < 0xF0, the PS byte carries the destination address). Addresses: RL controller 0x2A, engine 0x00, tractor ECU / guidance 0x80, brakes 0x0B. Multi-byte fields are little-endian. Controller -> tractor, 10 Hz: Guidance System Command, PGN 44032 (0xAC00) to 0x80, id 0x0CAC802A: bytes 0-1 commanded curvature, 0.25 km^-1/bit, offset -8032 km^-1, positive = left (ISO 11783-7); byte 2 bits 0-1 steering command status, 01 = intended to steer (otherwise the wheels are centred). Steering angle = atan(curvature x wheelbase). TSC1, PGN 0 to the engine 0x00, id 0x0C00002A: byte 0 bits 0-1 override control mode; 10 = torque control: byte 3 requested torque, 1 %/bit, offset -125 -> throttle = torque % / 100 (the pedal); 01 = speed control: bytes 1-2 requested speed, 0.125 rpm/bit, followed by a proportional governor (throttle = (requested - engine rpm) / 400); 00 = no override (throttle 0). XBR, PGN 1024 (0x0400) to the brakes 0x0B, id 0x0C040B2A: bytes 0-1 external acceleration demand, 0.00048828125 m/s2/bit, offset -15.687 m/s2; byte 2 bits 6-7 XBR control mode (00 = override disabled). Brake = -demand / 4 m/s2 (full brake at -4 m/s2) while the mode is not 00. Proprietary A, PGN 61184 (0xEF00) to 0x80, id 0x18EF802A, byte 0 selects the message: 0x01 control: byte 1 diff lock (0 off, 1 rear, 2 all, 0xFF keep), byte 2 harvester (0 off, 1 on, 0xFF keep), byte 3 = 1 resets to the pose set by 0x10-0x12 (a new episode; byte 6 = 1 also clears the field under the pose, otherwise the farm's marks stay), byte 4 = 1 places the workers staged by 0x21 (count in byte 5; 0 removes them). 0x10 reset X: bytes 1-4 int32, 0.01 m. 0x11 reset Y: bytes 1-4 int32, 0.01 m. 0x12 reset heading and speed: bytes 1-2 yaw, 0.01 deg (Unreal yaw 0..360), bytes 3-4 start speed, 0.001 m/s. 0x21 stage a worker: byte 1 index (0-7), bytes 2-4 X int24 0.01 m, bytes 5-7 Y int24 0.01 m. 0x30 HUD: bytes 1-4 float32 reward / return to display, byte 5 success rate % (0xFF = none). Tractor -> controller, every 0.1 s of simulation time: EEC1, PGN 61444 (0xF004) from 0x00, id 0x0CF00400: bytes 3-4 engine speed 0.125 rpm/bit (as the sensor recorder's CAN log; other bytes 0xFF). WBSD, PGN 65096 (0xFE48) from 0x80, id 0x0CFE4880 (ISO 11783-7): bytes 0-1 wheel-based speed 0.001 m/s/bit, bytes 2-5 distance 0.001 m, byte 7 bits 0-1 direction (00 reverse, 01 forward). Guidance Machine Status, PGN 44288 (0xAD00) to all (0xFF) from 0x80, id 0x0CADFF80: bytes 0-1 estimated curvature (same scaling as the command, from the actual steering angle), byte 2 bits 2-3 = 01 ready. Proprietary B from 0x80, priority 6, ids 0x18FFxx80 with PGN 0xFF10 + n: 0xFF10 pose: bytes 0-3 X, 4-7 Y, int32 0.01 m (world metres, X east, Y south). 0xFF11 motion: bytes 0-1 yaw uint16 0.01 deg (0..360), 2-3 vx, 4-5 vy int16 0.001 m/s (body frame, x forward, y right), 6-7 yaw rate int16 0.0001 rad/s (positive = turning right). 0xFF12 wheel slip: 4 x int16 0.001 (FL, FR, RL, RR). 0xFF13 soil and steering: 3 x uint16 0.0001 theta (here, 5 m, 10 m ahead), bytes 6-7 steering int16 0.0001 rad (positive = left). 0xFF14 LiDAR sectors 0-7, 0xFF15 sectors 8-15: uint8, range = value x 30 / 250 m. 0xFF16 state: bytes 0-1 cross slope int16 0.0001 rad, byte 2 diff lock, byte 3 flags (bit 0 worker struck since the last observation, bit 1 reset pending, bit 2 safety brake held, bit 3 agent in control), bytes 4-5 episode uint16, byte 6 workers placed, byte 7 resets applied (low byte). 0xFF17 farm: bytes 0-1 bunker kg, 2-3 harvested kg (uint16, 1 kg), 4-5 crushed crop uint16 0.1 m2, 6-7 live draft uint16 10 N. 0xFF1F observation complete (sent last): bytes 0-3 physics time uint32 ms, 4-5 time since the reset uint16 0.01 s, byte 6 sequence counter. 0xFF1E lockstep barrier (priority 7, only in lockstep, AcresSimControl.h): bytes 0-7 the world step a step request reached, uint64; every frame of those steps was sent before it.

Safety: while an agent is in control (a JSON client connected, or CAN commands from 0x2A within the last 3 s) its actions replace the keyboard's throttle, brake, steering, gear (F10, forward) and diff lock; if no action arrives for 1 s (wall clock) the tractor holds the brake. R (reset) and Esc still work. Otherwise the keyboard drives. In lockstep (AcresSimControl.h) wall-clock time does not count: a client that sent an action keeps control however long it thinks between steps.

LiDAR here is not the sensor recorder's: the nearest return anywhere within each of 16 sectors of 11.25 degrees (sector k spans -90 + 11.25 k .. -90 + 11.25 (k + 1) degrees from the heading), measured from the cab 1 m ahead of the body origin, 30 m max, as the fast training environment. World geometry: 6 horizontal line traces per sector on ECC_Visibility, 1.2 m above the ground, ignoring the tractor, its trailer and the ground (actors tagged ACREGround); that channel hits buildings, trees and other ACREObstacle geometry and the NPC vehicles' BlockAll boxes. Workers have no collision while alive (AcresNpc.cpp detects strikes with a box test), so they are added analytically as 0.4 m radius cylinders (nearest point D - 0.4 m when the person's angular extent overlaps the sector).

Command timing: a receiver thread reads the transports 1000 times a second and stamps every JSON line and CAN frame with its arrival time (wall clock). The game thread handles them once per frame, and the commands that drive the vehicle (the DBW commands, the RL steering, pedal, curvature and diff lock actions, the CAN guidance, TSC1, XBR and diff lock) apply at the physics step matching their arrival (AcresCommandTiming.h): a slow frame's many physics steps see them in the order and at the spacing they arrived, so a controller sending at 20-100 Hz keeps the 0.1 s DBW command timeouts fresh however slowly the game renders, and a controller that goes silent still trips them 0.1 s after its last command. In lockstep (AcresSimControl.h) every command applies at the next step, as the step request expects. Resets, workers, the environment clock, the harvester, implement commands and the drive mode apply when the frame handles them.

Threading: the receiver thread only reads the sockets (under ReceiveMutex) and queues what arrived; everything else runs on the game thread from AAcresVehiclePawn::Tick; controls and reset requests go to the physics thread through the pawn's StateMutex, like the keyboard's.

FAcresRlBridge#

Opens the requested transports.

Argument	Description
`InPawn`	The player's tractor (not the trailer). Must outlive the bridge.
`JsonPort`	TCP port for the JSON transport, or 0 for none.
`CanInterface`	SocketCAN interface name (e.g. "vcan0"), or empty for none (Linux only).
`CanUdpPort`	UDP port for CAN frames over 127.0.0.1, or 0 for none (Linux only).

FAcresRlBridge(AAcresVehiclePawn* InPawn, int JsonPort, const FString& CanInterface, int CanUdpPort);

~FAcresRlBridge#

Closes every socket.

~FAcresRlBridge();

Tick#

Game-thread update, called by the pawn's Tick after the keyboard block: accepts a JSON client, reads and applies messages and CAN frames, overrides the controls and sends an observation every 0.1 s of simulation time.

Argument	Description
`State`	The pawn's latest telemetry.

void Tick(const FAcresVehicleTelemetry& State);

IsConnected#

True while an agent is in control (a JSON client connected or CAN commands arriving).

bool IsConnected() const;

Episode#

Episodes started by the agent (reset commands) so far.

int Episode() const ;

Reward#

Last reward / return the agent reported for display (0 until one arrives).

double Reward() const ;

StatusText#

One-line status for the HUD, e.g. "RL: connected (JSON), episode 12, reward 48.3 | <agent text>".

FString StatusText() const;

SendDbwReports#

Sends Polaris drive-by-wire reports (FAcresDbwReport, 50 Hz) to the JSON client as "dbw_report" lines. Game thread; does nothing without a client.

Argument	Description
`Reports`	Reports in time order since the last call.

void SendDbwReports(const TArray<struct FAcresDbwReport>& Reports);

SendBarrier#

Lockstep (AcresSimControl.h): sends {"type": "barrier", "step": Step} to the JSON client and the frame 0xFF1E (bytes 0-7 the step, uint64) on the CAN transports, after everything the steps produced (reports, observations, CAN frames). Game thread; does nothing without a client or CAN peer.

Argument	Description
`Step`	The world step the step request reached.

void SendBarrier(uint64 Step);