# Part 2 — Cross-engine policy transfer: experiment plan Draft plan for `policy-transfer/index.html`. Everything runs in simulation and reuses part one's robot (Panda, Menagerie asset, matched pads), scenes, per-scene controllers (explicit 1 kHz joint PD; task-space impedance for insertion) and engine selection. The proposal below is a publication plan; completed exploratory results are summarized in the implementation checkpoint and linked article. ## Implementation pilot checkpoint — 2026-10-05 ### Review follow-up — 2026-10-07 Work resumed on instance **54550992**. The active main scope is insertion and 90 mm push, with reaching as a control. Grasp/place and narrow-channel task redesign are deferred; historical pilot results remain visible. The original proposal and hypotheses below are retained, not silently presented as completed. `setup/rl_review_suite.py` freezes five CPU training seeds, 262,144 transitions each, 24 validation starts, and the existing eight plus 50 newly randomized test starts (balanced 0.10/0.25 mm clearance). A matched five-seed force-penalty cohort uses coefficient 10, fixed before its results. These are still pilot budgets. The old eight tests are regression cases; the 50 random starts are new holdouts. No test result selects a checkpoint. Evaluation standardizes reward weight to zero; success and overload criteria are unchanged and training weights retained. The insertion contract stays `insertion_v4`; push/reach stay `bounded_v3`. These are frozen per-task contracts with different action spaces, not pooled versions of the same task. A new version requires fresh training and a separate cohort. Do not relabel or reinterpret old checkpoints. Immediate gates: scripted/learned matched-start PhysX controls, native floor and contact-offset checks, raw plus causal 5/10 ms force diagnostics, Newton wrench versus generalized-force consistency, native GPU batching at existing solver settings, then PhysX-trained insertion ± force penalty. Audit historical Genesis grasp/push collisions before using them to support H3. Full end-to-end throughput must include controller, observations, reward, episode resets and PPO; physics profiling alone cannot establish a feasible training budget. Run-count reconciliation: the **original proposal** is 80 baseline + 120 core robustness = 200 trainings, or 240 including the optional 40 LOO+DR runs. The October 4 harness status counted that optional condition as mandatory. The narrowed scope will require a new publication budget after throughput gates; neither 200 nor 240 is an already launched queue. The proposal below is still the publication plan. Exploratory implementation now lives in the sibling `state-of-robot-aug-2026/experiments/simcompare` harness; see `rl/STRUCTURED-PILOT-2026-10-04.md` there for frozen inputs and complete pilot outcomes. The article now reports these pilots explicitly, with local native-pose videos, the policy-feedback diagnostic, the initial nine-run learning curves and `assets/pilot-2026-10-05/summary.json` (including per-episode outcomes and hashes). All source training reported so far is MuJoCo CPU; no full training-engine matrix, force-penalty comparison or multi-engine training has been completed. The structured cohort completed 11 trainings and 1,703,936 transitions. Wider insertion scores 8/8 on fresh source starts versus 4/8 for its own nominal action tape. Its three-case transfer probe scores 3/3 in MuJoCo/Newton/Genesis and 1/3 in PhysX, with two overloads. The longer place run lifts in 16/16 tests but completes 0/16; initial grasp pilots remain unsolved. Earlier v3 reaching and 90 mm push results appear separately and are excluded from this cohort total. These are small exploratory samples, not a ranking or a publication-complete study. The pilot compares SB3 PPO with 64×64 and 256×256 tanh MLPs; the proposed 256–256–128 ELU architecture below has not been adopted or frozen. It adds a Cartesian grasp/place interface and a structured insertion interface while preserving the historical controller contracts and their recorded results. Task structure is now an explicit design check. A pick-and-place extension requires lifting, transport, lowering, release and a final hold at varying goals. It remains a learning diagnostic until a source policy solves it. These are custom Panda tasks inspired by Meta-World's task structure, not official Meta-World environment ports or benchmark scores. The original insertion conditions can be solved by replaying one learned nominal action sequence. A wider, separately frozen insertion distribution requires initial lateral alignment beyond the entry chamfer. A diagnostic compares each policy with its own fixed nominal action tape on the same layouts, to establish whether feedback matters. Large-tilt variants must first pass action-workspace feasibility checks: rotating the mounted rod requires lateral wrist travel. The rod remains attached to the hand; this is not a complete pick-up-and-insert sequence. Keep the original nine conditions as contact/regression controls. All pilot and adaptive follow-up findings are exploratory. Preserve the original hypotheses below; do not rewrite them around observed successes or failures. ## Task selection (which part-one scenes are RL problems) | Part-one scene | Role | Reason | | --- | --- | --- | | Sliding, drop, hinge, collision chain | pipeline controls | no agent | | Free-space tracking | reaching = interface check | easy; engines agree within 0.2 mm | | Grasp, lift & release | **RL task** | standard dense-reward task, sustained contact | | Stacking at the edge | excluded | long-horizon, sparse; outcome set by geometry, agreed everywhere | | Channel push | **RL task** | dense progress reward; largest layout sensitivity (MuJoCo family) | | Crowded sweep | excluded | sparse count reward, 12.5 s, GPU noise ≈ engine differences | | Precision insertion | **RL task** | canonical contact-rich RL (Factory, IndustReal); PhysX overloads vs. soft-contact success | | Soft ball | excluded | unvalidated in PhysX/Genesis; 50 µs steps too slow | ## Research questions 1. If a policy is trained in engine A, how well does it perform in engine B? 2. Do gaps follow part one's structure (MuJoCo family vs. PhysX; contact richness)? 3. What do policies exploit in their training engine, and does that predict transfer? 4. Does multi-engine training beat single-engine domain randomization on a held-out engine? ## Shared recipe (freeze before training) | Item | Proposal | | --- | --- | | Algorithm | PPO, one implementation for all engines (e.g. rsl_rl or a small in-repo PPO) | | Network | MLP [256-256-128], ELU; same init seeds per training seed | | Observations | joint pos/vel, hand pose, object poses, goal; canonical world frame; **no contact forces** | | Actions | delta joint targets at [50 Hz] (decimation [20] at 1 ms) + finger target; delta hand pose for insertion | | Controller | part one's per-scene controller: explicit PD + 4 N finger cap; impedance with 20 N / 2 Nm caps for insertion | | Reward | one implementation over the canonical state; overload (40 N / 4 Nm) and numerical failure terminate as failure | | Budget | equal env steps per run, e.g. [200M] (set after pilot) | | Seeds | 5 per (engine, task) | | Evaluation | frozen layout set per task, [200] episodes per (policy, eval path) | Training paths: PhysX GPU (Isaac Lab), Newton/MJWarp (batched), MuJoCo CPU (multi-process, matched step budget, wall-clock reported separately), Genesis (batched). Evaluation paths: the four above + PhysX CPU (part one's path). ## Phases and run counts | Phase | Content | Runs | | --- | --- | --- | | P0 pilot | reach task, 1 seed per engine; measure throughput at audit-grade physics settings | 4 trainings | | P1 pipeline controls | scripted push/grasp through new wrappers (1 vs. N worlds); PhysX CPU vs. GPU; seed + repeat noise floor | sim only, cheap | | P2 reach (interface check) | 4 engines × 5 seeds | 20 trainings, 20 × 5 evals | | P3 grasp & lift | 4 × 5 | 20 trainings | | P4 channel push | train on widths U[80, 95] mm, offsets ±3 mm; eval at 80/85/90 mm | 20 trainings | | P5 peg insertion | train on clearance {0.10, 0.25} mm, offset ≤ [0.5 mm], tilt ≤ [1°]; eval on part one's 9 conditions + perturbed starts | 20 trainings | | P6 diagnosis | open-loop action replay per engine pair; exploit audit (penetration, peak force, overloads, wall tunnelling) | analysis + replays | | P7 robustness | push + insertion: single+DR (40), single+force penalty (40), leave-one-engine-out (40), LOO+DR (optional, 40) | 120–160 trainings | | P8 cost | learning curves vs. steps and wall-clock, from P3–P5 logs | none extra | Core total ≈ 80 single-engine trainings + 120 robustness trainings. PhysX GPU parity (P1) must include insertion with the 64-facet geometry (128-facet failed PhysX's import check). ## Metrics - Success with 95 % CI over seeds; transfer ratio = target / source success. - Behaviour gap vs. training engine: hand-path deviation, peak pad/wall force, control effort (part one's normalized squared effort, same window), failure severity. - Open-loop divergence time: first time any object deviates > [5 mm]. - Noise floor: seed spread and unchanged-repeat spread per engine. ## Hypotheses (pre-registered in the draft) H1 reach transfers; H2 MuJoCo CPU ↔ MJWarp near-perfect, PhysX furthest on contact; H3 gaps follow part one's sensitivities (push: into MuJoCo family; insertion: soft-contact policies overload in PhysX); H4 success overstates behavioural transfer; H5 multi-engine > single-engine DR on a held-out engine; H6 a contact-force penalty narrows push/insertion gaps. ## Open decisions 1. **Audit-grade vs. training-grade physics.** If 100 iterations / tight tolerances are too slow for RL, freeze a cheaper per-engine configuration and rerun part one's sliding/grasp/push controls with it. Do not tune for task success. 2. **One env codebase.** Recommended: part one's adapters behind one vectorized-env API. Isaac Lab (PhysX + Newton backends) as an optional cross-check only. 3. **MuJoCo CPU in multi-engine training** or evaluation-only (throughput bottleneck). 4. **Observation noise / latency** in the base recipe: none (cleanest sim-to-sim signal) vs. small shared noise (closer to part three). Recommended: none in the base recipe; latency only as a DR factor. 5. **Imitation-learning variant** (BC/diffusion on scripted demos from one engine): out of scope unless time allows; would connect to Manda's policy-evaluation work. ## Directory conventions - `assets//` — plans, metrics JSON, SVG figures, audit notes (as in part one). - `replay/` — gzip replay bundles, referenced from the shared viewer as `../replay/index.html?trace=../policy-transfer/replay/.json`. Uncompressed `*.json` originals are git-ignored. - Policy checkpoints and raw training logs stay outside this repo; link hashes only.