Skip to content

Replay#

A replay drives a vehicle from a file or shows a recorded episode again. This tutorial gives the procedures for a command replay, a path replay, a drive script, an episode replay and a ROS 2 bag.

Before You Start#

Select the replay type for your task.

Replay Input Physics Models Use
Command replay A CSV file with speed and curvature. They run. To apply recorded commands to the vehicle without feedback.
Path replay A CSV file with positions. They run. To follow a recorded or planned path with a speed profile.
Drive script A JSON file with timed controls. They run. To make a demonstration or a test run that you can repeat.
Several replays An agents file. They run. To give a different replay or script to each vehicle of a session.
Episode replay An episode log. They do not run. To show an episode from a different camera, or to make its images later.
Bag replay A ROS 2 bag. No simulator. To test ROS 2 nodes with recorded data.

The Replay File#

A command replay and a path replay read a CSV file with a header row and two or more data rows. The file can be the file tractor.csv of a session log, the session folder that contains it, or a file from a different program. The game does not use the case of the column names. For each quantity, the first name of the table that the file contains applies.

Quantity Column Names Unit Necessary Description
Time time_s, t s Yes The time. It must increase. The first value can be different from 0.
Speed speed_mps, v_forward_mps, v m/s Yes The forward speed. A negative value is reverse.
Curvature curvature_1pm, curvature_command_1pm, curvature_measured_1pm, kappa 1/m Yes The curvature of the path. Positive is left.
Position ue_x_m and ue_y_m m No The position with x east and y south.
Position east_m and north_m m No The position with x east and y north. The game uses these columns when the file has no ue_x_m and ue_y_m.
Heading heading_deg deg No The heading, clockwise from north. It sets the start pose.
Implement implement, harvest No 1 lowers the implement or starts the harvest function. 0 lifts or stops it.
Gear gear No The forward gear of the Maxxum, from 1 to 16. 0 lets the replay select the gear.

The game applies these rules when it reads the file.

  • It ignores a row that is less than 0.05 s after the last row that it kept. A session log of 120 Hz thus becomes a track of 20 Hz.
  • It ignores a row with fewer fields than the header.
  • It refuses the file when the time does not increase.
  • It refuses the file when a speed is more than 20 m/s or a curvature is more than 1 per metre.

When the game refuses the file, the log contains ACRES_REPLAY_REJECTED with the cause.

Command Replay#

A command replay applies the speed and the curvature of the file as a function of time. The steering angle is \( \delta = \arctan(\kappa L) \) with the curvature \( \kappa \) and the wheelbase \( L \). The replay has no position feedback. The position error thus increases with time.

  1. Write the command file /data/commands.csv.

    time_s,speed_mps,curvature_1pm
0,0,0
2,3,0
10,3,0.02
20,3,0
25,0,0

  2. Start the game with the file and a session log. The spawn options put the Maxxum on field F21, where it has space.

    Packaged/Linux/Acres.sh -VehicleDemo -ReplayFile=/data/commands.csv -ReplayMode=commands \
    -VehicleSpawnU=480 -VehicleSpawnV=405 -VehicleSpawnYaw=-90 \
    -SessionLog -VehicleOutput=/data/replay-commands

    Expected Result

    The Maxxum waits for 3 s, then it accelerates to 3 m/s, makes a slow left turn and stops.

    ACRES_REPLAY_READY rows=5 duration=25.0 s path=0 mode=commands source=/data/commands.csv

  3. Close the game with Esc after the vehicle stopped. Then read the summary and the tracking file.

    cat /data/replay-commands/replay-summary.json
sed -n '1p;120p' /data/replay-commands/replay-tracking.csv

    Expected Result

    The replay is complete. At 11.8 s the speed is 3.0 m/s and the steering angle follows the curvature of the file. A command file has no positions, thus the cross-track error is 0.

    {"source":"/data/commands.csv","mode":"commands","reference_duration_s":25.000,"path_available":false,
 "rms_cross_track_m":0.0000,"max_cross_track_m":0.0000,"completed":true,"vehicle":"maxxum_150",
 "shift_east_north_m":[0.000,0.000]}
time_s,ref_x_m,ref_y_m,x_m,y_m,cross_track_m,ref_speed_mps,speed_mps,ref_curvature_1pm,steer_rad
11.800,-33.791,-175.794,-33.791,-175.794,0.000,3.000,2.999,0.01640,0.04330

The replay holds the brake for the first 3 s and after the end of the file. A speed controller sets the throttle and the brake. On the Maxxum, the replay selects the highest gear that keeps the engine at 1500 rev/min or more. The Polaris gets the speed as a command to its ULC and the steering angle as a steering command.

Path Replay#

A path replay follows the positions of the file with a pure-pursuit controller. It reads the speed, the implement state and the gear at the position of the vehicle along the path, not at the time. The vehicle starts at the first position of the file.

  1. Start the game with the path of the repository and the mode path.

    Packaged/Linux/Acres.sh -VehicleDemo -ReplayFile=Demo/harvest_path.csv -ReplayMode=path \
    -SessionLog -VehicleOutput=/data/replay-path

    Expected Result

    The Maxxum starts at the first point of the path and follows the straight line to the east at 1.5 m/s.

    ACRES_REPLAY_READY rows=43 duration=43.0 s path=1 mode=path source=Demo/harvest_path.csv

  2. Close the game with Esc when the vehicle stopped. Then read the summary and the tracking file.

    cat /data/replay-path/replay-summary.json
tail -n 1 /data/replay-path/replay-tracking.csv

    Expected Result

    The largest distance from the path is 8 mm.

    {"source":"Demo/harvest_path.csv","mode":"path","reference_duration_s":43.000,"path_available":true,
 "rms_cross_track_m":0.0037,"max_cross_track_m":0.0077,"completed":false,"vehicle":"maxxum_150",
 "shift_east_north_m":[0.000,0.000]}
52.000,256.570,-86.870,253.081,-86.864,0.006,0.000,0.001,0.00000,0.00261

Note

The speed of the last row of this file is 0. The Maxxum thus stops 3.5 m before the last point, and completed stays false.

To replay a session, give its session folder: -ReplayFile=<session folder> -ReplayMode=path.

Name Type Unit Default Description
-ReplayFile= path no replay The CSV file or the session folder.
-ReplayMode= string commands commands or path. The mode path needs the position columns.
-ReplayShift= two numbers m 0,0 A shift of the path to the east and to the north, as <east>,<north>.

The controller of the path replay has these properties.

Property Value
Look-ahead distance 2.5 m + 0.6 s × speed, 8 m maximum.
Speed The lowest speed of the file in the braking distance and the look-ahead distance.
More than 1.5 m from the path The speed is 2 m/s maximum until the vehicle is on the path again.
End The replay is complete when the vehicle is less than 1.5 m from the last point.
Stall After 10 s below 0.15 m/s with a target above 0.5 m/s, stop and report stalled.
Time limit After three times the reference duration plus 30 s, stop and report time_limit.

A failed path replay applies the brake, requests an implement lift and writes ACRES_REPLAY_FAILED to the log. Its summary has completed=false and a failure reason. An empty failure string means that no failure was detected; check completed as well, because the operator can end a session early.

The game writes two files into the session folder when the session ends.

File Content
replay-tracking.csv One row each 0.1 s: time_s, ref_x_m, ref_y_m, x_m, y_m, cross_track_m, ref_speed_mps, speed_mps, ref_curvature_1pm, steer_rad.
replay-summary.json source, mode, reference_duration_s, path_available, rms_cross_track_m, max_cross_track_m, completed, failure, vehicle, shift_east_north_m.

In an agents file, the keys replay_file, replay_mode and replay_shift set the replay of one agent. The menu has the same function: select Replay Simulation and then the tab Replay.

Drive Script#

A drive script is a JSON file with a list of keys. Each key has a time t and one or more fields. The time is the physics time since the start of the session in seconds. A field keeps its value until a later key sets it again.

  1. Write the script /data/drive.json.

    {"frame": "spawn", "path": [[0, 0], [60, 0]],
 "keys": [
  {"t": 0, "gear": 6, "speed_kmh": 0, "camera": [-30, -14, 11]},
  {"t": 1, "speed_kmh": 6},
  {"t": 8, "camera": [-95, -7, 8], "camera_blend_s": 2},
  {"t": 12, "speed_kmh": 0},
  {"t": 15, "exit": true}]}

  2. Start the game with the script.

    Packaged/Linux/Acres.sh -VehicleDemo -DriveScript=/data/drive.json -SessionLog -VehicleOutput=/data/drive-script

    Expected Result

    The Maxxum follows the path forward from its spawn pose and stops at 12 s. The camera moves to the side at 8 s. The game stops at 15 s. The session summary gives a distance of 16.3 m and a maximum speed of 1.56 m/s.

    ACRES_DRIVE_SCRIPT keys=5 path_points=2 file=/data/drive.json
ACRES_DRIVE_SCRIPT_DONE t=15.00

The top level of the file has these keys.

Name Type Unit Default Description
keys object list The timed keys. This list is necessary.
frame string spawn The frame of the points. spawn: metres forward and left of the spawn pose. world: x east and y south.
path list of [x, y] m none The path that the steering follows.
bales list of [x, y] m none Round bales for the loader.
windrow list of [x, y] m none A hay windrow for the baler.

A key can have these fields.

Name Type Unit Description
t number s The time of the key.
speed_kmh number km/h A speed that a controller holds with the throttle and the brake.
pedal number The throttle, from 0 to 1, when the key has no speed_kmh.
hand_throttle number The hand throttle, from 0 to 1.
brake number The brake, from 0 to 1.
park boolean true applies the brake fully.
gear integer The forward gear, from 1.
direction integer 1 forward, -1 reverse.
steer number or string deg A fixed steering angle, positive left. The string path follows the path.
diff_lock integer 0 open, 1 rear, 2 front and rear.
raise, pto, transport boolean The implement lift, the PTO and the transport position.
hitch_mode string position, draft or float.
depth_m number m The work depth of the implement.
lever number The hitch lever, from 0 to 1.
draft_setpoint_kn number kN The draft set-point.
spray, meter, pickup, stabilizers boolean The switches of the sprayer, the seed meter, the baler pickup and the backhoe stabilizers.
boom_fold number The fold of the sprayer boom, from 0 to 1.
windrow_kg_m number kg/m The mass of the windrow for each metre.
loader_lift, loader_curl number The loader controls, from -1 to 1.
swing, boom, dipper, bucket number The backhoe controls, from -1 to 1.
grab boolean true takes the nearest round bale. false releases it.
camera number list deg, deg, m The camera behind the vehicle: yaw, pitch and distance.
camera_blend_s number s The time in which the camera moves to the new camera value.
cab boolean true selects the view from the cab.
cab_look number list deg The yaw and the pitch of the view from the cab.
view string The name of the agent that the view follows. Agent 0 only.
exit boolean true stops the game.

With a path and without a steer key, the steering follows the path. A script with only camera, view and exit fields does not change the steering. You can use it together with a replay or a vehicle bridge. The scripts of the repository are in Demo/SoilWeather and Demo/Multi.

Several Replays in One Session#

In an agents file, each agent has its own keys replay_file, replay_mode, replay_shift and drive_script. The exit field has an effect only in the script of agent 0.

  1. Write the agents file /data/agents.json. Agent 0 uses the drive script. Change the time of its exit key to 55 s.

    {"agents": [
  {"name": "script", "vehicle": "maxxum", "drive_script": "/data/drive.json"},
  {"name": "commands", "vehicle": "maxxum", "spawn": [480, 405, -90],
   "replay_file": "/data/commands.csv", "replay_mode": "commands"},
  {"name": "path", "vehicle": "maxxum", "replay_file": "Demo/harvest_path.csv",
   "replay_mode": "path"}
]}

  2. Start the session.

    Packaged/Linux/Acres.sh -VehicleDemo -Agents=/data/agents.json -SessionLog -VehicleOutput=/data/replay-session

    Expected Result

    Three Maxxum tractors drive at the same time. The session folder has one folder for each agent.

    ACRES_DRIVE_SCRIPT keys=5 path_points=2 file=/data/drive.json
ACRES_REPLAY_READY rows=5 duration=25.0 s path=0 mode=commands source=/data/commands.csv
ACRES_AGENT_SPAWNED index=1 name=commands vehicle=maxxum
ACRES_REPLAY_READY rows=43 duration=43.0 s path=1 mode=path source=Demo/harvest_path.csv
ACRES_AGENT_SPAWNED index=2 name=path vehicle=maxxum
ACRES_DRIVE_SCRIPT_DONE t=55.00

    /data/replay-session/commands:  replay-summary.json  replay-tracking.csv  session-summary.json  tractor.csv
/data/replay-session/path:      replay-summary.json  replay-tracking.csv  session-summary.json  tractor.csv
/data/replay-session/script:    session-summary.json  tractor.csv

The agents file is on the page Run Several Vehicles.

Episode Replay#

An episode replay shows an Episode Log in the game. Each vehicle follows its logged states, and the farm gets the logged contacts and events again. The physics models do not run, thus the replay shows the episode without a change.

Item Behaviour
Agents Without -Vehicle=, -Vehicles= or -Agents=, the game makes the agents of the log at their first logged poses.
Sky The replay starts with the date, the hour and the weather preset of the first conditions message. Then the weather model runs.
Farm The game applies the logged farm stamps and farm events at their steps.
End After the last logged step, the vehicles keep their last state. The log contains ACRES_REPLAY_DONE.
Cameras All views are available: the chase view, the cab view (-CameraCab) and the camera fields of a drive script.
Sensors The sensors operate on the replayed poses: -SensorRecord and -SensorStream= are available.

The two procedures below show the two uses of an episode replay.

Render-Later#

Render-later records an episode log without images and makes the images later with an episode replay. ACRES Core runs the episode much faster than real time, and the game then shows it.

  1. Record an episode with ACRES Core in Python. Refer to Headless Core Runs for the Python module.

    import math
import acres_core
import numpy as np

batch = acres_core.Batch(".", num_envs=1)
batch.reset([0], np.array([[-38.71, -140.513, math.pi / 2, 0.0]]))
batch.start_log(0, "/data/core-episode.mcap")
for k in range(300):
    curvature = 0.08 * math.sin(2 * math.pi * k / 200)
    batch.step_curvature_speed(np.array([curvature]), np.array([2.0]))
batch.stop_log(0)

    Expected Result

    ACRES Core simulates 30 s in 0.06 s and writes 8405 messages into the file.

  2. Show the episode in the game from the cab view and record a video. The option -EpisodeReplayExit stops the game at the end of the log.

    Packaged/Linux/Acres.sh -VehicleDemo -EpisodeReplay=/data/core-episode.mcap -EpisodeReplayExit \
    -CameraCab -RenderOffscreen -ResX=1280 -ResY=720 -CaptureVideo=/data/core-episode.mp4

    Expected Result

    The game makes the Polaris of the log, shows the drive from the cab in real time and stops after 30 s.

    ACRES_REPLAY_LOADED /data/core-episode.mcap states=3600 agents=1 farm_events=0 farm_stamps=3600 duration_s=29.99
ACRES_SIM_CONTROL_READY port=0 lockstep=0 replay=1 log=
ACRES_CAPTURE_START /data/core-episode.mp4 1280x720 at 30 fps
ACRES_REPLAY_DONE steps=3600
ACRES_CAPTURE_DONE /data/core-episode.mp4: 899 frames (30.0 s at 30 fps), 0 dropped

  3. Examine the video.

    ffprobe -v error -show_entries format=duration -of csv=p=0 /data/core-episode.mp4

    Expected Result

    29.966667

Images at Exact Steps#

With -Lockstep and -SimControl=, a replay advances only on step requests. Each image then belongs to an exact physics step. This procedure uses an episode log of the game: the file run0.mcap of the procedure Lockstep Stepping.

  1. Start the replay in lockstep.

    Packaged/Linux/Acres.sh -VehicleDemo -EpisodeReplay=/data/determinism/run0.mcap -SimControl=5600 -Lockstep

    Expected Result

    ACRES_REPLAY_LOADED /data/determinism/run0.mcap states=3720 agents=1 farm_events=1102 farm_stamps=3705 duration_s=30.99
ACRES_SIM_CONTROL_READY port=5600 lockstep=1 replay=1 log=

  2. Step to physics step 1200, read the pose and write the main view to a file.

    import sys
sys.path.insert(0, "Tools/SimControl")
from sim_control import SimControl

ctl = SimControl(5600)
print(ctl.call("hello")["replay"])
reply = ctl.call("step", steps=1200)
print(reply["step"], reply["time_s"])
print(ctl.call("get_entity_state", entity="polaris")["pose"]["position"])
ctl.call("screenshot", path="/data/step-1200.png")

    Expected Result

    The pose is the pose of step 1200 in the log. The image shows the Polaris at this pose under the logged sky.

    True
1200 10.000000521540642
[-43.48095200704194, -129.01587965046375, 0.27569996426740656]

  3. Stop the game.

    ctl.call("set_state", state=3)

Replay a ROS 2 Bag#

A bag of the ROS 2 bridge or of the real Polaris contains the sensor topics, the odometry and the transforms. The launch file bag_replay.launch.py plays a bag with its clock and starts the description of the Polaris. ROS 2 nodes and RViz then get the data as from the vehicle. This procedure does not use a simulator.

  1. Record a bag from a session with the ROS 2 bridge: refer to Collect Data.

    source ROS/Env/setup_env.sh
ros2 bag record --use-sim-time -s mcap -o /data/bag /tf /tf_static /vehicle/odom /oxts/imu /lidar/points

    Note

    Use --use-sim-time for a bag of the simulator and do not record /clock. The times in the bag are then simulation times, and the replay makes the clock from them.

  2. Examine the bag.

    ros2 bag info /data/bag

    Expected Result

    The duration is the simulation time of the recording.

    Files:             bag_0.mcap
Bag size:          178.5 MiB
Storage id:        mcap
Duration:          19.991667709s
Start:             Dec 31 1969 19:00:11.508333934 (11.508333934)
End:               Dec 31 1969 19:00:31.500001643 (31.500001643)
Messages:          7199

  3. Play the bag with the description of the Polaris.

    ros2 launch acres_description bag_replay.launch.py bag:=/data/bag rviz:=false

    Expected Result

    The launch file starts the robot state publisher and the path node, then it plays the bag after 4 s. The topic /clock has simulation time, and the transform from odom_origin to lidar is available.

    [robot_state_publisher-1] [INFO] [robot_state_publisher]: got segment base_footprint
[robot_state_publisher-1] [INFO] [robot_state_publisher]: got segment lidar
[ros2-3] [INFO] [rosbag2_player]: Set rate to 1
[odom_path-2] [INFO] [odom_path]: path origin odom_origin at (500434.63, 4479976.18, 181.52) in utm

Without rviz:=false, the launch file also starts RViz with the model of the Polaris, the LiDAR points and the path of the odometry.

Name Type Unit Default Description
bag path The folder of the bag.
rate number 1.0 The speed of the replay.
loop boolean false true plays the bag again and again.
rviz boolean true false starts no RViz.
delay number s 4.0 The time before the replay starts.

An episode log is not a bag of sensor topics. To publish its messages, use ros2 bag play -s mcap: refer to Episode Log.

Next Steps#