fix(shadow): repair field routing and harden evidence collection

docs/runbooks/ft546-shadow-20260907-repair-notes.md

@@ -0,0 +1,108 @@
+# FT546 Shadow Repair: 2026-09-07
+
+## Delivery Boundary
+
+Source changes in the existing `optimize/wcs` worktree, based on `9829b9c0fa71d8a4623a9f968178183217724db1`.
+Prior uncommitted field fixes are preserved. During the repair/validation stage, no commit,
+push, deployment, live SQL write, schema migration, production application startup,
+PLC command or WMS call was performed. The user subsequently authorized committing and
+pushing the repaired source to `origin/optimize/wcs`; this does not authorize deployment.
+No deployment scripts or JAR deliverables are included.
+
+This is a shadow repair, not approval to replace the production scheduler.
+
+## Repairs
+
+- Keep virtual TD/WY transit nodes in route planning and resource protection. Generate
+  PLC preview points from numeric start/end/turn nodes, rather than every transit node.
+  Missing geometry, nonnumeric command points and more than six turns remain rejected.
+- Deduplicate protected resource keys when approach and execution routes overlap.
+  Preserve ordered route nodes, including repeated visits and turns.
+- Tag new hypothetical reservations with `predictionReservationIds`. On the next poll
+  or restart, remove only tagged predictions from the working ledger. Preserve untagged
+  legacy reservations and all OCCUPIED, command-unknown or recovery-unresolved protection.
+  Keep snapshot CAS, fencing tokens, HOLD counters and transactional evidence rollback.
+- Use a UUID startup prefix for new reservation IDs, preventing reuse after restart.
+- Bound evidence growth and persist current run state, including empty polls and shutdown.
+
+## Capacity Settings
+
+Existing local external configuration remains usable; omitted new settings use these defaults:
+
+```yaml
+shadow:
+  evidence:
+    maximum-database-bytes: 17179869184
+    maximum-batch-payload-bytes: 33554432
+```
+
+The database limit is 16 GiB of allocated data plus log files, read from the evidence
+database's `sys.database_files`. It is not a table row-count limit, used-page measurement,
+SQL Server MAXSIZE setting or exact physical disk quota. A batch/file-growth increment can
+overshoot the threshold before the next poll. The existing local `minimum-free-bytes`
+check still applies to `output-directory`; it does not measure every remote SQL volume.
+
+The 32 MiB batch limit counts newly inserted large text payloads as UTF-16 bytes.
+It excludes database page, index and transaction-log overhead. Existing identical replays
+do not consume the insert budget. Over-budget batches roll back with their traffic state.
+
+Collection stops with a specific reason: `EVIDENCE_DATABASE_SIZE_LIMIT`,
+`EVIDENCE_DATABASE_SIZE_CHECK_FAILED`, `EVIDENCE_BATCH_PAYLOAD_LIMIT`, or the existing
+storage availability/free-space reason. It does not automatically resume or delete rows.
+Raw `captured_json`, its content hash, evidence hashes and existing replay identity remain
+unchanged. Lossless deduplication, archival and retention require a separate reviewed
+migration/operational decision. Existing evidence is not compacted by this repair.
+
+## Durable Runtime State
+
+Each new process uses its own run UUID. In the already-authorized evidence tables:
+
+- `shadow_run.status` records STARTING, RUNNING (status endpoint READY), DEGRADED or STOPPED.
+- `shadow_traffic_state.state_key = FT546:RUN:<runId>` contains a `RUN_HEARTBEAT` JSON record.
+- `heartbeatAt`, `lastAttemptAt`, `lastCompletedCycle` and `checkedAt` use UTC ISO timestamps.
+- `collectionStopped`, `reasonCode` and `cycleDiagnostics` distinguish process liveness,
+  attempted collection, completed collection and permanent collection failure.
+- Pre/post-poll writes and scheduled backoff/stopped ticks refresh the same bounded row.
+  Runtime JSON is limited to 32,768 characters, independently of business evidence size.
+- Runtime persistence failure stops further source reads. Failed writes cannot guarantee
+  that the database contains the latest stop reason; check logs and heartbeat freshness.
+- Normal close records STOPPED. Forced termination/power loss can only be detected by a
+  stale heartbeat. A RUNNING row alone is not proof of a live process or successful work.
+
+Traffic-ledger readers should select `FT546:TOPOLOGY:<version>` rows; runtime rows have
+their own key namespace and `kind`. Historical runs without a heartbeat are not rewritten
+or retrospectively marked stopped. With default polling, investigate a heartbeat older
+than 60 seconds; adjust this operational threshold when poll/deadline settings change.
+
+## Local Verification
+
+Final offline Maven command used Maven 3.6.0 and the supplied complete-v2 capture ZIP:
+`mvn -o -pl ruoyi-wcs-shadow -am test -Dshadow.replay.zip=<complete-v2-zip>`.
+
+- 298 tests passed: 128 WCS, 10 protocol, 160 shadow; zero failures, errors or skipped tests.
+- 300 independent captured inputs: 600 available legacy projections, 300 normal
+  NO_ELIGIBLE_VEHICLE holds, zero new grants. This sample does not prove live dispatch readiness.
+- 10 additional captured field batches: 9 encodable grants, 9 normal holds,
+  zero input or grant-projection errors. Independent empty initial traffic state per batch.
+- 6 historical queue paths: zero legacy, routing or command-point projection errors.
+  These checks are protocol-only, not a reconstruction of historical vehicle eligibility.
+- Focused regressions cover overlap, virtual transit nodes, invalid command points,
+  prediction expiry, protected state, ID restart, storage limits, atomic rollback,
+  heartbeat, retries, timeout permanence, normal stop and runtime write failure.
+- Independent read-only review found no blocking issues. `git diff --check` passed.
+  Legacy Admin and existing PLC/WMS implementation files have no changes.
+
+Detailed local logs are under
+`D:/workSpace/PROJECT_SPACE/FT546/outputs/wcs-shadow-review-20260907/`:
+`repair-full-with-captured-replay.log` and `live-database-repair-final-replay.log`.
+
+## Remaining Field Validation
+
+SQL persistence tests use isolated H2 databases. The new metadata query and runtime writes
+have not been exercised against SQL Server 2016 under the deployed shadow login in this
+repair turn. No physical PLC/HIL test, forced network-stall shutdown test or sustained
+production-load test was run. Query/pool timeouts are not an OS-level hard shutdown bound.
+
+After an explicitly authorized shadow update, verify the new run's heartbeat, collection
+progress, capacity reasons, source latency and evidence completeness under real load.
+Review new shadow evidence before deciding on any production scheduler cutover.

docs/superpowers/plans/2026-09-07-shadow-field-repair.md

@@ -0,0 +1,66 @@
+# Shadow Field Repair Implementation Plan
+
+> **For agentic workers:** Execute inline with superpowers:executing-plans. Scouts are read-only; the main agent owns edits and verification.
+
+**Goal:** Repair the field-reproduced shadow failures without changing legacy dispatch or granting production write authority.
+
+**Architecture:** Keep topology routes separate from FT546 command points. Shadow predictions must not become permanent physical reservations. Preserve existing evidence JSON and database schema, add bounded collection and durable runtime metadata using existing authorized tables.
+
+**Tech Stack:** Java 8-compatible source, Maven 3.6.0, JUnit 5, Spring JDBC, SQL Server 2016; H2 for isolated persistence tests.
+
+**Spec:** Existing `docs/superpowers/specs/2026-08-12-ft546-shadow-first-safe-cutover-design.md`, plus the read-only field assessment at `D:/workSpace/PROJECT_SPACE/FT546/outputs/wcs-shadow-review-20260907/live-database-assessment.md` and the user's approval to repair the shadow version.
+
+## Global Constraints
+
+- Work only in the existing optimize/wcs worktree. Preserve prior uncommitted fixes and unrelated files.
+- No production SQL writes, migrations, PLC/WMS connections, application startup or deployment. The repair/validation stage excluded commit/push; subsequent user approval authorizes publishing only the repaired source to origin/optimize/wcs.
+- Keep legacy Admin/PLC behavior unchanged. Generic route changes must be backward compatible and tested across ruoyi-wcs.
+- Do not delete, truncate, silently sample, or change the raw evidence JSON/hash contract.
+- Keep unsupported phase/route representations explicitly unavailable; do not manufacture successful comparisons.
+
+## Task 1: Field Path and Protocol Regression
+
+Files: `ruoyi-wcs-shadow/.../facts/LegacyQueueProtocolProjector.java`, `.../kernel/ShadowCommandIntentFactory.java`, `ruoyi-wcs/.../domain/traffic/RoutePlan.java`, `AStarRoutePlanner.java`, `.../domain/dispatch/DefaultVehicleAssignmentPolicy.java`, focused tests in both modules.
+
+- [x] Add literal expected 4004 payload tests for task 31312, straight numeric paths, long virtual paths, invalid endpoint/turn and overflow cases.
+- [x] Add the overlapping approach/execution reservation regression; verify ordered nodes survive while unique resources are reserved.
+- [x] Run the tests red against current code.
+- [x] Carry geometry-derived turn-node metadata on planned routes; project start/end/turns in the shadow factory. Accept only known virtual transit tokens in historical reconstruction, with numeric validated endpoints/turns.
+- [x] Deduplicate reservation resources at route combination, not ordered route nodes or the reservation safety boundary.
+- [x] Run focused tests green and repeat the captured database probe.
+
+## Task 2: Shadow Prediction Lifecycle
+
+Files: `ruoyi-wcs-shadow/.../kernel/JdbcShadowTrafficStateRepository.java`, snapshot assembly and associated H2/end-to-end tests.
+
+- [x] Write a repeated-poll/restart test showing a prior hypothetical grant cannot consume capacity forever.
+- [x] Keep physical/unknown protection distinct from hypothetical reservations; never interpret a new prediction as PLC execution.
+- [x] Preserve snapshot CAS, evidence transaction rollback and hold-count behavior; tag new state JSON with PER_SNAPSHOT_PREDICTION and explicit predictionReservationIds. Preserve all untagged legacy state and OCCUPIED/unknown/recovery protection; remove only tagged predictions from the next working ledger.
+- [x] Run repeat, restart, conflict and rollback regressions. Add per-startup reservation ID prefix to avoid collision with retained state.
+
+## Task 3: Evidence Growth Guard
+
+Files: shadow evidence storage health/configuration, repository metadata access, property defaults and tests.
+
+- [x] Add tests for database allocation threshold, unavailable capacity metadata, local free-space threshold and preservation of existing evidence rows.
+- [x] Use bounded database-file metadata reads, not whole captured_json scans or broad server-state permission grants.
+- [x] Stop new collection with a specific durable reason when limits are reached. Keep raw evidence/export compatibility unchanged.
+- [x] Document that lossless deduplication/archive remains a separate migration decision, not an automatic deletion policy.
+
+## Task 4: Durable Run State
+
+Files: shadow evidence repository/runtime monitor, scheduler, status report/configuration and tests.
+
+- [x] Test heartbeat on empty polls, last success vs attempt, retryable failure, permanent timeout, orderly stop and persistence failure.
+- [x] Persist bounded run metadata in FT546:RUN:<runId> in shadow_traffic_state and update shadow_run status. No DDL or new privileges.
+- [x] Ensure timeout/stop reasons cannot be overwritten by a late worker completion; monitor failure must not claim healthy collection.
+- [x] Keep timestamps explicitly UTC in serialized metadata and expose freshness for external stale detection.
+
+## Task 5: Acceptance
+
+- [x] Run the full offline Maven reactor tests for shadow and its dependencies, plus existing charging and new field-input replay. 298 tests passed, none skipped; 300 captured inputs and 10 additional field batches replayed without processing/projection errors.
+- [x] Request independent read-only code review of safety boundaries, protocol payloads and new operational paths; fix important findings. No blocking findings; SQL Server integration and blocked-shutdown coverage limits recorded.
+- [x] Verify no legacy source edits, no secrets in outputs, no new database/PLC authority and no background test jobs left running.
+- [x] Record results and remaining field/HIL boundaries; deliver source only, without claiming production-scheduler acceptance. See docs/runbooks/ft546-shadow-20260907-repair-notes.md.
+
+Baseline: offline Maven reactor test command succeeded on 2026-09-07 before this repair. The field probe from the preceding turn reproducibly failed on virtual nodes and duplicate route resources.