Scene Fidelity and Streaming Plan#
Keep accurate physics, sensor geometry and visible detail around every active vehicle. Load distant detail only when it affects a camera, sensor, shadow, reflection or interaction. Nanite, Lumen and Niagara remain part of the design. Their presence does not establish realism or real-time performance.
This page describes the regional ground pilot and the remaining scene work. The optional ground pilot exists. New house and tree placement remain incomplete. See Polaris Transfer Plan for calibration, control implementation and real-vehicle acceptance.
Current Evidence#
The selected pink area covers approximately 8.93 km². Surveyed ground covers 20.90 km². Keep the original origin, corrected central ground and 59 task fields. The complete 2 October recording is local: 56.84 GB, with source hashes verified. SQLite integrity, sensor timing and image quality checks remain separate gates.
With -DriveRegion alone, regional ground creates 128 procedural mesh components with 16 million triangles.
Each chunk spans approximately 381 m. This mode retains all exact geometry and complex collision.
Its 2.5 km distance limit stops drawing distant chunks. It does not unload their resources.
Add -DriveRegionStreaming to use the protected ground pilot.
The original central terrain is additional to this count.
The RTX 5060 Ti has 16,311 MiB of device memory. These native 1920 × 1080 tests use all six sensors and the balanced cloud setting:
| Scenario | Average FPS | Frame Time, p95 | Physics Steps per Wall Second | Peak Total GPU Memory |
|---|---|---|---|---|
| Moving parking area, dry | 35.15 | 59.32 ms | 120.001 | 12,464 MiB |
| Moving field, steady rain | 19.99 | 76.22 ms | 119.855 | 12,413 MiB |
The scenarios differ. Their ratio does not isolate the cost of rain. Both tests fail the 60 FPS and sensor deadline gates.
In steady rain, the shadow pass averages 16.03 ms. LiDAR scan completion averages 13.36 ms per rendered frame.
CPU and GPU work overlap. Do not add these timings to predict total frame time.
The measurements are in Calibration/Map/cloud_budget_performance_2026-10-02.json.
Expected Resource Reduction#
Streaming reduces the active resources. It does not reduce the source map area or guarantee the same reduction in frame time.
The estimate uses circle-to-chunk intersection at each of the 24,518 finite recorded positions. It preserves whole chunks, including their boundary triangles. It counts only additional regional ground. It excludes central ground, distant representations, objects, textures and streaming overhead.
| Example Protected Radius | Maximum Regional Chunks on This Route | Maximum Detailed Regional Triangles | Reduction from 16 Million |
|---|---|---|---|
| 250 m | 7 of 128 | 875,000 | 94.53 % |
| 300 m | 9 of 128 | 1,125,000 | 92.97 % |
These are capacity estimates for one vehicle. They are not live performance results. Multiple vehicles need the union of their protected areas. Prefetch and unload delays increase the active resource count.
Visible distant ground remains available through a cheaper representation.
Measure memory after implementation; these triangle ratios do not predict total VRAM savings.
The calculation and input identities are in Calibration/Map/scene_streaming_capacity_2026-10-02.json.
Implemented Ground Pilot#
Start the game with both -DriveRegion and -DriveRegionStreaming.
The central authored ground remains unchanged. Regional height, surface and texture arrays remain in memory.
Only the additional exact mesh and collision resources load and unload.
Packaged/Linux/Acres.sh -VehicleDemo -Vehicle=polaris \
-DriveRegion -DriveRegionStreaming -SimControl=5600 \
-RegionMaxChunks=32 -RegionPrefetchSeconds=3 -RegionPadding=100 \
-RegionUnloadGrace=5 -RegionLoadsPerTick=1
| Option | Unit | Default | Function |
|---|---|---|---|
-RegionMaxChunks= |
chunks | 32 | Maximum number of exact resident regional chunks |
-RegionPrefetchSeconds= |
s | 3 | Predicts a swept loading area along current velocity |
-RegionPadding= |
m | 100 | Adds optional loading margin; the policy limits this margin to 300 m |
-RegionUnloadGrace= |
simulation s | 5 | Retains unused chunks before release; current protection takes priority under capacity pressure |
-RegionLoadsPerTick= |
chunks | 1 | Limits optional loads per game-thread update |
Each exact chunk contains 250 × 250 survey cells. It retains the source triangle order and full collision. Distant regional ground uses every twenty-fifth survey node, approximately 38.1 m apart. Its 3 m downward skirts conceal boundary gaps. This distant representation has no collision. It is not a validated camera substitute for every scene.
Required ground loads synchronously before spawn, reset, teleport or a sensor request. Optional loads follow the update limit. Synchronous mesh creation can cause a frame delay. Publish completed collision availability before physics uses it. Retain resources through each physics callback and unfinished historical scan or camera readback.
If required collision is missing, hold the vehicle at its last safe pose with zero velocity.
The game thread then requests the missing ground. Continue only when the required collision is ready.
The log reports ACRES_REGION_CONTACT_HOLD and ACRES_REGION_CONTACT_READY.
Reject teleports outside the required survey coverage.
This pilot does not convert the world to World Partition or Nanite. It does not extend regional soil-water, crop, rut or drying state. These models and their leave-and-return checks remain separate work.
Recorded Route Policy Check#
The engine-free check uses the production selection policy at 24,518 finite recorded positions. It checks multiple source areas, historical scan retention, unload delay and the 32-chunk limit.
| Result | Value |
|---|---|
| Maximum resident regional chunks | 9 of 32 |
| Chunk creates | 71 |
| Chunk releases | 71 |
| Required missing chunks at startup, before loading | 3 |
| Required missing chunks during the remaining route, before loading | 0 |
| Required missing chunks after loading | 0 |
This check completes each load immediately. It does not measure cooking time, frame delay, sensor output or vehicle motion.
Live full-route acceptance remains pending.
The report is Calibration/Map/streaming_route_policy_2026-10-02.json.
The recorded predicted sensor envelope fits the surveyed ground with at least 15.60 m of spare margin. The advisory 300 m circle extends outside the survey at 3,756 positions. The route does not need more ground for the checked envelope. This result does not certify every position or speed in the pink area.
The check treats the navigation position as base_footprint. Confirm the navigation lever arm before final calibration.
The report is Calibration/Map/protected_drive_coverage_2026-10-02.json.
Use region_probe to inspect actual loaded ground collision without loading extra chunks.
Packaged Pilot Checks#
The headless and rendered checks pass eight locations, a moving chunk crossing and return visits. Each run checks 192 vertical ground probes and settled contact on all four wheels. Maximum collision height error against the source DEM is 0.00000971 m. This is numerical agreement, not survey accuracy.
Both runs create and release 23 chunks. Peak residency is 12 chunks. No contact hold or sensor error occurs.
The rendered check runs all six sensors with no missed camera frame, busy LiDAR scan or dropped row. Its peak total GPU memory is 11,015 MiB. The guard limit is 14,775 MiB. The largest measured synchronous chunk build is 47.43 ms.
The moving rain comparison uses native 1920 × 1080, all six sensors and 20 mm/h rainfall.
| Mode | Average FPS | Frame Time, p95 | Physics Steps per Wall Second | Peak Total GPU Memory |
|---|---|---|---|---|
| Fully resident regional ground | 20.01 | 76.05 ms | 120.133 | 12,414 MiB |
| Protected ground pilot | 21.35 | 84.07 ms | 120.011 | 10,940 MiB |
This short pair uses matching settings. Its free-running paths differ slightly. It does not establish a repeatable frame-rate improvement.
The pilot reduces peak device memory by 1,474 MiB in this pair. Both runs fail rendering and sensor deadline gates. Synchronous loading still exceeds one 16.67 ms frame. Cooked or asynchronous loading needs a separate implementation and comparison.
Full live route, multiple vehicles, reverse travel, hold and resume, camera agreement and complete LiDAR comparisons remain open.
The compact evidence is Calibration/Map/protected_ground_pilot_2026-10-02.json.
python Tools/SimControl/check_region_streaming.py --output Acres/Saved/Streaming/headless-check
python Tools/SimControl/check_region_streaming.py --output Acres/Saved/Streaming/rendered-check --rendered
Protected Areas#
Each active Polaris and tractor supplies a streaming source. A spectator camera supplies an additional visual source. Do not use only the player's position or forward camera view. The Polaris LiDAR needs geometry in all directions to 161.3 m. The default tractor LiDAR range is 60 m. Read the active sensor configuration, including overrides, when calculating protection.
The pilot uses a 10 m contact radius for each vehicle. Other dynamic rigid bodies use their bounding radius plus 5 m. LiDAR protection includes its full configured range, mount and lens offset, plus a 5 m margin. Predicted loading follows the swept path. Sensor protection never clips at the survey boundary.
Current and pending camera sources use a 160 m visual radius. A spectator camera also supplies visual protection. Only these visual radii clip at the survey boundary. This rule does not reduce LiDAR range.
| Area | Proposed Representation | Required Preservation |
|---|---|---|
| Physical and sensor area | Detailed geometry, exact collision, full vehicle physics and sensor processing | Ground contacts, obstacles, LiDAR returns, surface classes and current wetness |
| Nearby visual area | Shared detailed assets, streamed texture mips, Nanite and local effects | Camera silhouettes, materials, markings, shadows, reflections and visible motion |
| Middle distance | HLOD groups, shared tree instances, cheaper ground and effects | Camera appearance and relevant lighting; replace detail only outside sensor protection |
| Far distance | Streamed forest and ground silhouettes; compact state for distant actors | Horizon, sky and visible scene continuity |
HLOD means a distant representation of a group of objects. It cannot replace sensor geometry inside the protected area unless conformance tests establish equivalent returns.
The 300 m comparison radius is an advisory example, not a fixed safety bound. Calculate the minimum from sensor range, vehicle extent, physical interactions, measured load delay and vehicle speed. At 52 mph, three seconds of travel adds approximately 70 m to the loading requirement.
The nearest recorded position is only 212.56 m from the surveyed ground boundary. A 300 m bubble therefore extends beyond this ground at some positions. Retain mapped coverage for each required contact and sensor ray. Detect requests beyond measured coverage before replay acceptance. Do not replace missing ground with an unreported flat plane.
Camera protection also depends on projected size, shadows, reflections and indirect light. A distant building can occupy many pixels. A nearby offscreen tree can cast a visible shadow. Set visual distances from image comparisons rather than the physical radius alone.
Preload cells ahead of motion and route turns. Use separate load and unload thresholds to prevent repeated cell changes. Keep geometry referenced by an unfinished physics step, sensor scan or GPU readback until that work completes. If required cells are unavailable, reject the sensor sample and hold vehicle motion until the scene is ready. Keep deterministic object identities and PCG seeds. Streaming must not move trees or reset crop and soil state.
The current procedural components do not acquire World Partition streaming automatically. The pilot uses bounded regional chunks with the central map unchanged. Then select cooked cell assets or World Partition from measured load cost and project compatibility. World Partition supports multiple streaming sources; actor references can keep additional cells loaded. See World Partition.
Performance Work#
Terrain and Shadows#
Bake regional ground into streamable assets with stable boundaries and exact source heights. Keep the original collision triangulation for contacts and sensor intersections. Test Nanite on the new opaque regional ground before wider conversion. The original masked terrain has a GPU-hang history. Its conversion needs a separate guarded test.
Inspect non-Nanite shadow marking overflow, foliage overlap, wind updates and virtual shadow map invalidation. Keep important shadows around both vehicle cameras. Limit distant shadow work after camera comparison passes. Use shared tree meshes and material instances. Measure masked leaf overdraw and ray-tracing instance cost. Nanite does not remove texture, shadow, transparency or instance costs.
Lumen's Epic preset targets 30 FPS in Epic's performance guide. Its High preset targets 60 FPS. Feature support does not require maximum settings for every distant object. Retain Lumen and measure each proposed setting against sensor images. See Lumen Performance Guide.
Sensors and State#
Move LiDAR material, radiometry and noise processing away from the game thread with bounded jobs. Each job needs an owned snapshot of wetness and other mutable state. Preserve beam order, random-number order, timestamps, scan order and shutdown behavior. Workers must not read mutable Unreal objects.
Reduce CPU fallback trace cost only after nearest-hit comparisons pass. A terrain height query alone cannot represent vehicles, fences, crops or other obstacles. Cache component classes with revision-based invalidation for changes to crops, ruts and streamed cells. Do not reduce LiDAR rays, camera resolution, noise detail or physics rate to pass the benchmark.
The traffic director already updates custom traffic actors together. These are not full Unreal Character instances. Use compact state and lower update rates only for distant actors outside relevant interaction and sensor areas. Keep soil water, crop growth and mission state independent of visual actor loading.
Keep compact GNSS sky-view and multipath obstacle records independent of rendered actor loading. Refresh these records when scene geometry changes. A distant representation must not change satellite obstruction.
Asset and Map Work#
Audit one house and one tree before importing the full library. Measure final mesh counts after modifiers, scale, material cost, texture residency, pivots and collision. Convert source materials to consistent Unreal materials. Test leaf geometry, wind and LiDAR visibility. Keep shared meshes and textures across instances.
Complete one representative scene near Beck and ICSC before filling the pink area. Include real road width, verges, parking, the large tree, building surroundings and wet ground. Compare it with recorded camera frames and iPhone photographs. Use image views and aerial footprints for building proportions. DEM supplies foundation elevation only. Preserve existing main buildings unless image evidence shows a clear mismatch.
Then extend the same asset rules through the selected 8.93 km² and recorded roadside surroundings. Outside this scope, use deterministic forest and distant scenery. South of US 52, preserve the actual road and measured surroundings before adding forest. Do not place random trees where the recording shows a road, yard or open sightline.
Implementation Order#
| Order | Work | Acceptance before Expansion |
|---|---|---|
| 1 | Verify Blender and Unreal MCP; audit source assets; finish sensor QA and time alignment | Read-only tool calls succeed; complete bags pass integrity and quality checks |
| 2 | Validate the implemented ground pilot with protected sources and prefetch | Central map remains identical; contact, sensor and frame-delay tests pass |
| 3 | Correct shadow costs and move scan processing into bounded jobs | Matched dry and steady-rain tests improve without sensor or physics changes |
| 4 | Build the Beck–ICSC scene with one approved house and tree family | Camera, LiDAR and collision comparisons pass; native 1080p performance passes |
| 5 | Extend roads, surfaces, buildings and vegetation across the selected area | Full recorded-route replay passes, including boundaries, turns, rain and two vehicles |
| 6 | Fit and validate physics and sensor parameters; export matching Core geometry and cache identities | Held-out motion and sensor residuals improve across surfaces and speeds |
| 7 | Finish integration defects and the production residual framework | Classical mission gates and Lean conformance tests pass against runtime code |
| 8 | Train local residual PPO and evaluate unseen fields and conditions | Improvement over pure pursuit without increased failures or unsafe commands |
| 9 | Deploy through ROS 2 replay, shadow operation and supervised real trials | Real sensor freshness, latency, stop behavior and mission results meet the stated limits |
Data review can proceed during the scene work. Training waits for the preceding acceptance gates. The exact planner handles mission order. The phase controller handles mission progress. PPO handles only the local correction.
Release Gates#
Use matched, warmed tests at native 1920 × 1080. Also measure cold loading separately. Run central and regional dry, rain and night cases, both vehicles, reversing, fast turns and cell boundary crossings. Report p95, p99 and worst frame time. Test route lookahead at the recorded high-speed envelope.
- Average rendering reaches 60 FPS; frame time p95 is at most 16.67 ms.
- Wall-clock physics reaches at least 119 steps per second with a configured 120 Hz step.
- Total GPU memory stays below 14,775 MiB on this device, including desktop use and loading peaks.
- All requested sensor rates and deadlines pass. No steady-state camera misses, busy LiDAR scans, errors or dropped rows occur.
- Streaming and fully resident runs agree on ground contacts, collision and sensor intersections under identical inputs.
- Camera differences stay within thresholds set from repeat-run variation and real sensor QA.
- Core and Unreal use matching geometry, materials, calibration and map identities where their models overlap.
- The game compiles, packages and completes the route without a GPU crash or missing required scene resources.
Run one guarded GPU-heavy game or editor at a time. The benchmark keeps a 1536 MiB device reserve. The reserve reduces risk; it cannot guarantee that a driver or shader will never crash.
Zero-shot transfer means successful deployment without fitting or training on the held-out target conditions. Scene realism supports that result. It does not prove it. Keep unseen drives, fields and conditions outside fitting and training, then report closed-loop real performance.
Official MCP Setup#
Both official integrations passed local protocol checks on 2 October 2026. Blender also executed a read-only scene query through MCP. The check used Blender 5.2.2 without GPU rendering. Unreal compiled the current editor target and returned its available toolsets from an empty editor map with NullRHI. The test processes stopped after the checks.
The official Blender checkout uses v1.0.3, commit 2cea8d566dde07fbac28a61d698909d69724e853.
The MCP package uses torchenv. The background bridge registers the official add-on without changing Blender preferences.
The source checkout is local at ~/.local/share/acres-tools/blender_mcp.
See Blender MCP and its official source.
The bridge listens on loopback port 9876. The separate MCP process uses stdio.
Set BLENDER_MCP_SOURCE if the official source checkout is elsewhere.
Save scene changes into working copies before exporting approved runtime assets.
Unreal's ModelContextProtocol and AllToolsets plugins are already enabled for the editor.
Start the editor server with -ModelContextProtocolStartServer -ModelContextProtocolPort=8000.
Its local endpoint is http://127.0.0.1:8000/mcp.
The Unreal integration is experimental. See Unreal MCP.
The local Codex configuration now contains both servers. A new client session loads these registrations.
This running session verified them through local MCP clients; its original tool catalogue does not automatically refresh.
Compact tool-check evidence is in Calibration/Map/scene_tools_2026-10-02.json.