Comparing physics engines for robotics simulation
We compare four physics engines using matched scenes and the same robot controller. They usually agree on success or failure in our rigid-body tests, but differ in contact forces, trajectories and soft-object behavior.
- Matched carefully, the engines agree on coarse outcomes such as single-object motion, robot motion in free space, grasping and pick-and-place (Sections 2–4).
- Similar outcomes hide different forces and motion: Contact-force peaks differ up to tenfold, and identical success scores hide centimetre-scale tracking differences (2, 6).
- Tight-clearance contact separates the engines: in peg insertion, hard-contact PhysX hits force overloads where the three soft-contact engines succeed (4.4).
- Soft objects are where engines diverge most, i.e., same nominal material and grasp give different outcomes in every engine, and mesh or timestep refinement does not reconcile them (5).
- We have good choices for simulation on both CPU and GPU: MuJoCo on the CPU is fastest for a single environment; GPU engines win at RL-scale batch sizes, with throughput levelling off at the largest batches (7).
- Many apparent physics differences may be setup differences, for example imported joint armature, missing collision masks and mislabelled geometry. (2, 8).
Simulation is a core layer of the embodied-AI stack [1][2]. When we use it to benchmark and evaluate robotic behavior, we have two main goals: We want simulation to enable scalability and reduce cost compared with real-world testing, and we want it to model real-world physics faithfully (a.k.a. close the Sim2Real gap [3]), so that it predicts actual deployment performance.
Strong agentic models are poised to revolutionize asset generation and Real2Sim workflows, making simulation even more scalable and accessible. While visual fidelity of generated assets is already impressive, modeling physical behavior is just as crucial for robotic simulations. Many details of the real world are hard to replicate accurately. A key decision to make is the choice of simulation framework or engine. In this blog post, we therefore zoom in on a key ingredient of simulation, and specifically want to answer the research question: how do different physics engines compare when simulating the same robot and the same scene? What are the intricacies and pitfalls to be aware of?
To answer this, we compare four physics-engine implementations in depth: NVIDIA PhysX in IsaacSim, native Mujoco, Newton (Mjwarp), and Genesis. We explore where and why they disagree, and discuss what that means for policy evaluation.
We also release a companion article: We discuss the landscape of simulation and physics engines, the core components, and relevant details about contact calculation, asset imports, and configuration settings.
We run the four engines through a series of controlled experiments, matching scenes and assets exactly and driving the robot with a shared controller. To maintain tractability, we add complexity step by step, from simple contact tests to multi-object interactions and robot manipulation, and measure contact forces, trajectories and other relevant metrics along the way.
This article is the first in our series on simulation engines for robotics. In follow-up posts, we will look at cross-engine RL policy training and transfer, and finally at sim-to-real transfer of engines.
- Background: An overview of simulation frameworks for robotics
- Where physics engines agree and differ
- (Coming soon) Cross-engine policy transfer
- (Coming soon) Sim-to-real transfer
1. Method
A shared benchmark across engines
Simulation is a simplification of reality. No simulator captures every detail of the real world, and even small differences can lead to divergent outcomes. Engines also make different assumptions, particularly around contacts and solver implementation, so the same scene can play out differently in each. We test where these differences become relevant for robot evaluation.
To do so, we define a common benchmark that runs on all four engines. We match scene specifications and assets exactly, and verify compatibility by gradually increasing scenario complexity.
New to physics engines? Our companion article explains simulators, physics engines and the core terms used below, maps the current landscape and describes how the four setups we compare differ.
01 Single objects Validating friction & trajectories
02 Multiple objectsTracking forces in multi-object collision
03 Moving the Panda robotic armVerifying the controller for free-form movement
04 Stacking on the edgeStability of object placement
05 Contact-rich pushingModeling simultaneous contacts between multiple objects
06 Grasping a soft object Rigid–deformable coupling
Engine selection
We chose four engines/configurations that are popular and openly accessible:
| Engine / stack | Execution path | Version & configuration |
|---|---|---|
| PhysX | Isaac Sim · CPU TGS | Isaac Sim 6.1.0.0 · float32 |
| Newton (MuJoCo-Warp) | Standalone Newton · native MuJoCo-Warp contacts · GPU | Newton 1.5.2 · MuJoCo-Warp 3.11.0 · Warp 1.16.0 · float32 |
| MuJoCo CPU | Native CPU runtime | MuJoCo 3.11.0 · float64 |
| Genesis | Native rigid-body solver · GPU | Genesis 1.4.1 · float64 |
All our experiments were performed on an RTX PRO 5000 instance, NVIDIA GPU driver 595.71.05.
Two contact families
MuJoCo CPU, MuJoCo-Warp (run through Newton) and Genesis all use soft, MuJoCo-style contact that allows slight penetration; PhysX treats contacts as hard, rigid constraints. Close agreement among the first three is therefore expected, and PhysX is the most independent comparison. All four simulate the Franka Emika Panda in joint coordinates, and we drive all of them with one explicit torque controller instead of each engine's built-in drives. For soft objects the setups diverge further: MuJoCo keeps the ball inside one solver, while Genesis and Newton couple separate rigid and soft-body solvers. How the four setups differ, with contact settings and porting pitfalls →
Robot and asset audit. We match geometry, mass, centre of mass, inertia, joint axes and limits, initial poses and collision geometry. We check these after import rather than assuming that the source file is interpreted identically. For the Franka Emika Panda, we verify the imported physical properties and forward kinematics before running the manipulation experiments.
There is no physical ground truth in this study
We compare engines with one another, not with hardware. Agreement does not tell us which engine is closer to reality. MuJoCo CPU and analytic solutions are convenient baselines, not ground truth.
Unless stated otherwise, each configuration is one matched trial. Repeats and half-timestep runs are sensitivity checks, not confidence intervals, success-rate estimates or convergence studies. A per-step momentum-balance check tests whether the recorded forces are consistent with the recorded motion. This validates the instrumentation, not the physics. Failed and superseded runs are kept in the linked audit notes rather than discarded.
The viewer replays each engine's recorded body poses in a common renderer. Force arrows use the same colors and scale throughout (1 N = 1 cm), so large impact arrows can extend beyond the view; the plots show full magnitudes.
2. Basic controls
Single-object sanity checks to verify consistency
We first check whether basic motion and contact are consistent enough to interpret. We match primitive geometry, initial state, mass, gravity and nominal surface settings, then measure stopping distance, impact response, force histories and timestep sensitivity. The objects move similarly overall, while their contact histories differ.
The sliding endpoints differ by only about 0.05 mm, and the hinged link follows the independent pendulum reference within 0.084° at 1 ms. That is a useful baseline for the harder scenes. Contact is less interchangeable: the dropping cube settles with different penetration and transient responses, and similar sliding paths hide substantially different force peaks. Agreement in overall motion is a control check—not proof of identical contact physics.
Sliding block
A 100 g, 4 cm cube slides to a stop. Nominal mass, geometry, initial velocity, friction and timestep are matched; the contact formulations differ.
Almost the same stopping distance
| Engine | Travel | Peak |ΣFcontact| |
|---|---|---|
| PhysX | 126.422 mm | 1.06 N |
| Newton / MuJoCo-Warp | 126.472 mm | 10.46 N |
| MuJoCo CPU | 126.471 mm | 10.46 N |
| Genesis | 126.447 mm | 1.50 N |
Final travel spans only about 0.05 mm across engines. For reference, an ideal, non-rotating Coulomb slider would travel 127.42 mm.
Very different contact histories
MuJoCo CPU and Newton briefly lift the cube's centre of mass about 1.12 mm above its starting height, with a net contact-force peak roughly ten times the 0.981 N resting support force. Genesis stays close to the surface, and PhysX holds the height nearly constant.
All four traces pass the momentum-balance check, which confirms consistency between recorded force and momentum change—not physical plausibility. The brief lift and large peak remain an unresolved contact transient; the smaller-step control reduces the lift but does not identify its cause.
Halving the timestep moves the cube more than switching engines
Halving the timestep to 1 ms adds about 0.50–0.51 mm of travel in every engine, roughly ten times the between-engine spread at 2 ms. The MuJoCo CPU and Newton peak force falls to about 9.17 N, and their maximum lift from about 1.12 mm to 0.23 mm. Genesis's peak falls to 1.25 N; PhysX stays near 1.06 N.
The qualitative force contrast survives, but the bounce is strongly timestep-sensitive and should not be read as an inherent property of any engine. Inspect the 1 ms comparison · Control-test measurements.
Drop, bounce & settle
The same cube is released from rest with its centre of mass 35 cm high (33 cm clearance) and simulated for one second. The main figure uses a 1 ms timestep; 2 ms runs serve as a sensitivity check.
A sharp stop versus a compliant stop
| Engine | Peak |ΣFcontact| | Max overlap | Settled by |
|---|---|---|---|
| PhysX | 253.86 N | <0.001 mm | 0.264 s |
| Newton / MuJoCo-Warp | 52.04 N | 7.844 mm | 0.362 s |
| MuJoCo CPU | 52.04 N | 7.844 mm | 0.362 s |
| Genesis | 53.44 N | 7.567 mm | 0.364 s |
Overlap is the depth of the lowest rotated cube vertex below the ideal plane, not a contact margin. “Settled” means speed stays below 1 mm/s for the rest of this one-second recording.
Recovery is not necessarily a bounce
All four engines reach contact at about 0.26 s. PhysX arrests the fall almost immediately. With the current 10 ms soft-contact time constant, MuJoCo, Newton and Genesis let the cube sink into the plane, then push it back up toward rest, without a meaningful airborne rebound.
The overlap has a predictable scale: approximating MuJoCo's solref = (0.01, 1) as a continuous critically damped contact with near-unit impedance gives ω ≈ 100 rad/s. A 0.33 m fall reaches about 2.55 m/s, giving a peak penetration estimate v/(eω) ≈ 9.4 mm—close in scale to the measured 8.7–9.0 mm at 2 ms and 7.6–7.8 mm at 1 ms. This one-dimensional estimate neglects discrete integration and multi-contact coupling; it explains why the chosen 10 ms response permits millimetres of overlap. PhysX's zero-restitution constraint is a different contact configuration.
Halve the timestep, double the peak?
PhysX's peak rises from 127.53 N at 2 ms to 253.86 N at 1 ms, yet the contact impulse over the first 40 ms after contact is 0.29234 N·s in both runs. The same impulse is delivered over a shorter interval, so the reported step-average force roughly doubles. Peak force alone would make a misleading comparison.
At 1 ms, the corresponding 40 ms impulses are 0.30430 N·s for MuJoCo CPU and Newton and 0.30341 N·s for Genesis; their different velocity histories explain the small gap.
Validation caveats
Baseline and repeat runs were collected at 2 ms, followed by a 1 ms run in each engine. All four displayed 1 ms traces satisfy the existing 5 × 10⁻⁶ N·s per-step momentum tolerance. The Newton 2 ms trace reaches 1.4 × 10⁻⁵ N·s and is not used as a validated main figure. A follow-up readback found agreement between native contact forces and generalized contact forces; the small discrepancy is between native forces and acceleration, rather than a force-frame conversion error. It remains a numerical consistency caveat.
The 2 ms soft-contact runs overlap by about 8.74–9.01 mm, reduced to 7.57–7.84 mm at 1 ms. This is not a convergence study yet. Raw trials and the audit record are retained; no force peaks were smoothed or rescaled.
Testing a single joint
A uniform 1 kg link, 30 cm long and 4 cm wide, hangs from a frictionless hinge. It starts at 0.5 rad from vertical, with zero velocity. Every engine receives the same 0.2 Nm torque pulse from 0.1 to 0.2 s. There is gravity, but no collision, policy, position drive or damping.
An import default almost looked like a physics difference
In an initial Genesis run, we observed that the link swung much more slowly, deviating from the reference by up to 51.4° within one second. The imported joint carried 0.1 kg·m² of armature (extra rotational inertia), although the MJCF specified zero. After setting it explicitly to zero and checking the readback, Genesis and MuJoCo CPU agree to floating-point precision.
The same import checks caught a fixed-base setup error and a scaled PhysX joint frame. The accepted models pass the forward-kinematics check throughout the run, with worst-case COM disagreement below 0.1 µm.
Small discretization error, no run-to-run variation
All four engines stay within about 0.084° of an independent ODE solution for the ideal rigid link at 1 ms, and within about 0.042° at 0.5 ms.
All four, including Newton/MuJoCo-Warp, reproduce their recorded frames exactly across seeds 0, 1 and 2 in this contact-free test, whereas Newton varied in the collision chain. That points to scene- or computation-dependent effects, possibly collision handling, and certainly does not support a blanket “GPU physics is nondeterministic” claim.
3. Multiple objects
When contact configurations change collision paths
We start with simple object collision to test whether differences in force peaks propagate into different paths and object orientations. We find that the hard constraint contacts of PhysX start to make a difference.
Visibly different outcomes on the same chain
All four engines produce the intended sphere–cube–prism sequence. At the final sample (after 0.9995 s), the cube centre sits at approximately (119.3, 4.9) mm in PhysX, (136.5, −2.7) mm in Newton/MuJoCo-Warp, (139.0, −3.1) mm in MuJoCo CPU and (135.1, −5.0) mm in Genesis. The prism heading is about 23.0°, 55.5°, 57.4° and 55.3°, respectively. The differences show up in where the objects end up (not only in the force plots).
Could this just be randomness?
We ran each engine three times with identical initial conditions and different seeds. PhysX, MuJoCo CPU and Genesis were identical across runs. Newton varied by up to 4.40 mm for the sphere, 0.88 mm for the cube and 1.70 mm for the prism.
The cube’s mean endpoint differs by 18.7–21.3 mm between PhysX and the other engines, well beyond this run-to-run variation. Differences among the other three are smaller, about 2.6–4.3 mm, and three runs are too few to read much into them.
Genesis and Newton both run on the GPU here; PhysX and MuJoCo run on the CPU. Genesis uses float64, Newton float32. Genesis repeats in this collision-chain test, but varies in the later crowded sweep. GPU execution alone is therefore insufficient to predict repeatability; the result depends on the scene and implementation. Parallel operation ordering, precision and the solver or contact implementation may all contribute.
The sphere has a 3 cm radius (150 g), the cube 6 cm sides (100 g), and the prism a 9 cm equilateral-triangle cross-section and 6 cm height (100 g). All start on the floor; only the sphere moves initially, at 1.4 m/s with matching rolling angular velocity. Nominal friction is 0.3, without explicit rolling or torsional friction. Mass and inertia readbacks match the authored values to numerical precision, but that does not guarantee identical cooked contact geometry or contact laws.
Halving the timestep from 1 ms to 0.5 ms moves the cube and prism endpoints by roughly 0.1–4.8 mm, depending on the engine; the still-moving ball is considerably more sensitive. PhysX uses hard contact with zero restitution; the others use soft contact.
4. Robot
Integration tests to match robot assets across simulators
Once a robot is involved, a disagreement could come from contact, or from the robot already behaving differently before it touches anything. We therefore first match imported inertias and frames, joint conventions, effort limits and reference motion, and extend the earlier hinge control to full-arm forward kinematics, identical-effort motion and feedback tracking, all without contact.
The earlier hinge check caught an import default. With that correction carried forward, the Panda moves almost identically in all four engines under matched effort. Free-space agreement does not imply equivalent contacts, which the grasping, stacking and pushing experiments examine next.
Moving the Panda
Matching a target position is not enough. Before interpreting a failed grasp, we need to separate differences in the imported model, the articulated dynamics and the feedback controller:
- Asset and kinematic parity — checked. One pinned Panda model and standard hand, not four independently bundled assets. Check joint frames, link COMs and inertias, limits and finger conventions; compare end-effector poses at identical joint configurations.
- Identical effort replay, without contacts — measured below. Send the same time-indexed joint torques with native drives, passive damping and gravity compensation explicitly controlled. Compare joint and end-effector trajectories before adding a policy.
- Shared explicit controller — longer replay below. Use the same PD equation, gains, limits and cadence. Targets match; resulting torques need not, because feedback sees different states. Report the effort traces alongside tracking error rather than calling this identical action replay.
We use all seven arm joints, with the standard hand's fingers fixed open at 4 cm each, from one pinned MuJoCo Menagerie Panda asset. Contacts, passive damping, armature and native drives are disabled in every adapter; joint limits are monitored but not enforced during this short, within-range motion. This is a deliberately simplified dynamics baseline, not a production Panda configuration.
Tracking a longer motion with a shared controller
The joint-space reference follows a smooth quintic curve to an offset pose and back over 2.5 s. One canonical model generates the shared inverse-dynamics feedforward sequence offline. Each engine then runs the same explicit PD equation with the same gains and torque clipping at 1 kHz. Targets are identical; feedback torques respond to each engine’s measured joint state.
Simply extending the original open-loop torque pulses would drive the arm past its joint limits after about one second, and an initial 100 Hz tracking trial produced large wrist errors with these gains. The displayed run uses the same 1 kHz controller configuration everywhere, without engine-specific tuning.
The hand travels about 19.3 cm from its starting position. At the 1 ms physics step, maximum joint tracking error is at most 0.026° in every engine, and the hand never strays more than 0.194 mm from its MuJoCo CPU counterpart. Three seed runs reproduce the recorded frames exactly within each engine. Halving the physics step increases PhysX’s maximum separation to 0.740 mm, so agreement does not simply improve with smaller steps. This is an unresolved cross-engine discrepancy, not evidence that either trajectory becomes less accurate: both trajectories change with timestep, and we lack an independent dynamic reference for the full arm.
Open-loop baseline: 0.5 s of identical torques
The short replay applies the same open-loop torques in every engine (the figures and numbers in this section refer only to this test).
One physical specification, checked after import
All eleven link masses, COM offsets and full inertia tensors are compared in common link frames. The largest discrepancies are below 1.7 × 10⁻⁷ kg, 2.6 × 10⁻⁹ m and 5 × 10⁻⁸ kg·m², respectively. Five prescribed joint configurations check forward kinematics before integration; the native poses are also checked against their reported joint angles throughout each rollout.
Matching effort yields closely matching motion
Every engine receives the same constant gravity-holding torque computed at the initial pose, plus +2 Nm on joint 2 from 0.05–0.15 s and −1 Nm on joint 4 from 0.15–0.25 s. The feedforward is held constant as the robot moves; there is no per-engine gravity compensation or feedback.
At 1 ms, the maximum hand-origin separation from MuJoCo CPU is 0.035 mm for PhysX and 0.0031 mm for Newton/MuJoCo-Warp, with maximum joint-angle differences of 0.024° and 0.00047°. Genesis and MuJoCo CPU overlap to floating-point precision. All four engines reproduce their recorded frames exactly across the three seeds.
At 0.5 ms, differences stay small
At 0.5 ms, PhysX’s maximum hand separation from the corresponding MuJoCo CPU run grows to 0.213 mm; Newton’s is 0.0040 mm, with maximum joint differences of 0.058° and 0.00073°. These remain small, but one refinement is not enough to claim convergence or identify the cause.
Grasp, lift & release
Can matched robots agree on a sustained contact task? The Panda grasps a 4 cm, 100 g cube with two independently actuated fingers and follows a scripted close–lift–release sequence, using the shared arm controller from the free-space test. Finger effort is capped at 4 N per finger, and all contacting surfaces have nominal friction 0.5. We test a centred cube, then shift it 8 mm along the pad width.
The contacting pads are explicitly matched boxes rather than each engine’s cooked Panda fingertip mesh, and the rest of the robot has no collision shapes. This isolates a simple articulated grasp before tackling the real hand's more complicated geometry.
Centred grasp: similar lifts and holding forces
Off-centre grasp: agreement on success, differences after release
What we observe
All eight trials lift and release the cube. In the off-centre case, final horizontal displacement is about 9.0 mm for PhysX, 6.9 mm for Genesis, 1.1 mm for Newton/MuJoCo-Warp and 0.3 mm for MuJoCo CPU: similar task success, different contact trajectories.
Limitations
These are single trials at a 1 ms physics step, and contact regularization differs between engines.
4.1 Robot · Pick-and-place
Stacking at the edge for pick-and-place
We look at a simple pick-and-place task, a very common scenario across robotics manipulation benchmarks. Because it is representative of many test cases we find in benchmarks, we ask: Do small contact differences change whether a manipulation task succeeds? We let our robot pick up a 40 mm, 100 g block, carry it 100 mm and place it on another block. Both blocks are dynamic, so the lower one is not anchored. The same controller runs in all four engines, with no object-aware corrections or engine-specific tuning.
The gripper setup matches the grasp control introduced above: two fingers capped at 4 N, matched box-shaped pads, nominal friction 0.5 and no other robot collision shapes.
We vary the intended resting position for the upper cube at different thresholds to determine whether engines agree in such a scenario.
| Nominal placement | Outcome after settling |
|---|---|
| 0 mm | Remains stacked in all four engines |
| 16 mm | Remains stacked in all four engines |
| 20 mm | Topples in all four engines |
| 22 mm | Topples in all four engines |
Newton’s centred stack remains standing, but its upper block moves through about 0.081 mm vertically in the final half-second; brief contact interruptions and speed spikes miss our stricter settling check. We report the Newton case separately rather than counting the stack as toppled.
We also found that shifting the upper block's start by −1 mm in X and +1 mm in Y from the 20 mm case preserves the stack in all four engines; the opposite diagonal shift topples it in all four. Because both coordinates changed, these runs bracket a local placement transition, not a pure X-offset tipping threshold. These auxiliary cases are reported in the audit rather than the replay dropdown. Unchanged repeats and seed-only controls reproduce the toppled outcome. PhysX and MuJoCo CPU repeat their recorded frames exactly, and Genesis’s seed-only differences stay below 0.00001 mm. Newton’s upper-block path deviates from its baseline by 4.8–6.9 mm across these reruns, including a same-seed repeat.
4.2 Robot · Pushing
Testing contact forces in a narrow passage
What happens in more complex scenes with multiple contact interactions? Here the robotic arm pushes three loose 40 mm, 100 g blocks toward a channel between two fixed walls. The two front blocks initially span 84 mm, and we compare channel widths of 90 and 80 mm without changing the controller or tuning any engine.
The hand follows one frozen straight-line target: hold for 0.5 s, push 240 mm over three seconds, then hold until 5 s. Finger targets stay fixed, and the same compliant joint controller runs at 1 kHz in every engine. Nominal friction is 0.5 throughout.
The two front blocks span 84 mm before rearrangement: the 90 mm channel provides clearance, whereas 80 mm requires reorientation or separation. This makes the two widths strongly geometry-biased controls, rather than a resolved test of the contact-dependent passage boundary. The intermediate-width threshold was not swept.
All four engines push all three blocks through the 90 mm channel; none does at 80 mm. The coarse outcome is the same, but the object paths differ: in the 80 mm trial, the final position of the initially rear block differs by about 13 cm between MuJoCo CPU and Genesis.
At 90 mm, shifting the rear block 1 mm sideways changes MuJoCo CPU’s object paths by up to 31.4 mm, versus 2.4 mm in PhysX, while both engines’ unchanged repeats match exactly. Halving the timestep changes the paths by up to 27.8 mm and 0.27 mm, respectively, without changing the passage outcome. We cannot infer physical accuracy from this sensitivity.
| Engine | Unchanged repeat | ±1 mm layout change | Half timestep |
|---|---|---|---|
| PhysX | 0 | 2.40 | 0.27 |
| Newton / MuJoCo-Warp | 2.76 | 27.99 | 5.27 |
| MuJoCo CPU | 0 | 31.36 | 27.81 |
| Genesis | 0.003 | 3.55 | 0.45 |
Values are maxima over all three objects at matching timestamps, not only endpoint distances; the layout column takes the larger of the two perturbations. All 90 mm controls preserve the passage outcome. Newton’s unchanged repeat varies more than Genesis’s, although both run on the GPU.
4.3 Robot · Crowded sweep
Repeatability in 15-object sweep
In both scenarios above, the engines largely agreed. What happens when contacts cascade? A Panda carries a 220 mm paddle diagonally along a 45-degree guide, gathers fifteen balls, cubes and cylinders, then pushes them through a 300 mm opening and down a shallow ramp into a tray. Some objects reach the tray, some remain outside it, and some fall from the table.
| Engine / route | Repeat 1 | Repeat 2 | Repeat 3 |
|---|---|---|---|
| PhysX CPU | 9 / 15 | 9 / 15 | 9 / 15 |
| Newton / MuJoCo-Warp | 11 / 15 | 8 / 15 | 7 / 15 |
| MuJoCo CPU | 10 / 15 | 10 / 15 | 10 / 15 |
| Genesis | 10 / 15 | 8 / 15 | 8 / 15 |
Counts use the same final object-centre-in-tray criterion at 12.5 s. Each repeat starts a fresh process with identical inputs and seed 0. All twelve trials passed the declared mass, centre-of-mass, inertia, kinematic and finite-state checks.
Genesis’s paddle-contact force spikes remain unresolved. Its collection counts are diagnostic outcomes; they do not validate the recorded contact forces.
Randomness can affect reported metrics
Newton collects the most objects in the first repeat, but fewer than both CPU routes in the next two. Genesis also changes its collection outcome. MuJoCo CPU and PhysX reproduce their recorded object-position trajectories exactly across these three runs.
Shared setup and recording details
All routes use the same frozen 12.5 s reference, explicit PD gains and effort caps, a 1 ms physics step, nominal friction 0.5, explicit masses and inertias, and the same collision geometry. Cylinders use shared 32-sided convex meshes. The paddle collides with objects and obstacles; the arm meshes are visual only. Contact solvers, collision cooking and force readback semantics still differ. The scene and controller were developed with MuJoCo CPU before freezing them, which makes task selection a source of bias.
All original 5 ms recordings are retained locally. The three compressed web replays total approximately 23 MB and load one repeat at a time. Impact counts exclude inactive zero-force contact candidates.
All trial outcomes and checks · Protocol and reproduction · Frozen run plan · Recording checksums
Genesis also records peak paddle-contact forces of 6.37–7.52 kN, against about 57 N in MuJoCo CPU, 50–169 N in Newton / MuJoCo-Warp and 605 N in PhysX. In the replay, these peaks occur only late in the run, when the paddle pushes hard against objects that have already come to rest, so they barely affect the collection counts. Their physical cause remains unresolved.
4.4 Robot · Precision insertion
Small clearances, different outcomes
A torque-driven Panda inserts a rigidly mounted 20 mm peg into a chamfered bore. We vary radial clearance (0.10, 0.25, 0.50 mm) and initial alignment (aligned, 0.25 mm lateral offset, or 1° tilt). Every condition runs three times in fresh processes with identical inputs and seed: 108 attempts across four engines. Geometry stays at 64 facets, physics and the shared impedance controller at 1 kHz.
Real steel-on-steel insertion can exceed 60 N of friction and 20 Nm of torque at the end effector. Our controller caps commands at 20 N / 2 Nm and stops runs above 40 N / 4 Nm, so this tests contact handling in a gentler regime, not a calibrated industrial fit.
| Engine | Target depth reached / 27 | Other outcomes |
|---|---|---|
| PhysX CPU | 21 | 6 overload stops |
| Newton / MuJoCo-Warp | 22 | 3 tilted runs stall short; 2 numerical failures |
| MuJoCo CPU | 24 | 3 tilted runs stall short |
| Genesis | 24 | 3 tilted runs stall short |
At 0.50 mm clearance, every engine reaches the target in all nine runs without the peg touching the bore, which makes it a contact-free control. At 0.10 mm, tilted insertion stalls at 15.3–15.7 mm in MuJoCo CPU, Newton and Genesis, and PhysX overload-stops in the aligned and tilted cases. Repeats are identical except in Newton, whose peak forces vary between attempts and which fails numerically twice.
Can contact feedback resolve the stall?
We rerun the tight 0.10 mm offset and tilted cases with a small contact-guided controller: the filtered contact force permits bounded lateral displacement and upward relief, and the moment about the peg tip permits bounded orientation correction. It does not use the hole position to choose a correction direction. Everything else stays unchanged, and the feedback gains are identical across all four engines.
| Engine | Final depth (mm) | Peak contact force (N) | Peak native penetration (µm) |
|---|---|---|---|
| PhysX CPU | 12.24 → 12.24 (both overload) | 125.83 → 46.32 | 8.67 → 8.34 |
| Newton / MuJoCo-Warp | 15.31 → 19.932–19.936 | 17.04 → 19.23–20.65 | 4.49 → 3.90–10.08 |
| MuJoCo CPU | 15.32 → 19.95 | 16.85 → 15.30 | 3.23 → 2.02 |
| Genesis | 15.68 → 19.96 | 11.12 → 3.59 | 2.75 → 1.24 |
With contact feedback, MuJoCo CPU, Newton and Genesis reach the target in all three tilted runs, while PhysX overload-stops in all three. Newton reaches the same depth with varying force peaks (19.2–20.6 N) and penetration (3.9–10.1 µm): successful depth tracking does not mean identical contact histories.
Validity, force limits, repeats and excluded geometry
Target depth means at least 19 mm throughout the last 0.25 s of a completed 4 s run; penetration is a separate diagnostic, not folded into this outcome. The controller caps its task command at 20 N / 2 Nm before joint-effort limits. The experiment stops after a contact resultant exceeds 40 N or a hand-origin moment exceeds 4 Nm. These are not certified Panda safeguards and cannot prevent the triggering impulse.
PhysX's tight aligned runs reach about 485 N / 331 Nm and 1.49 mm native penetration: unresolved simulated anomalies, not plausible hardware operation. All 64-facet inputs pass our sampled native geometry checks. PhysX's separate 128-facet trials fail the peg import tolerance (about 24 µm discrepancy); those are invalid imports and are not included here. The failed 16-piece geometry remedy is also separate. No engine was tuned to win this sweep.
MuJoCo CPU and Genesis tight-case repeats match at the displayed precision. Newton's tight-offset peak forces vary from roughly 7.5 to 13.4 N across attempts, one of which fails numerically; another numerical failure occurs at 0.25 mm clearance. Three repeats diagnose variability, but do not establish a reliable rare-failure probability. The replays and plots show the first repeat of each condition.
Each contact-guided pilot run has a fresh matched baseline using the same code. A final batch of eight unchanged-input runs repeats only the guided tilted condition, with no further changes to settings; MuJoCo CPU, Genesis and PhysX reproduce the tabulated metrics across all three runs. In the guided offset case, all four engines reach the depth criterion, but PhysX's peak penetration rises from 17.8 to 54.6 µm, over half the 100 µm radial clearance. Newton's offset baseline fails numerically at 3.226 s; its guided counterpart completes, but earlier repeats show that the baseline failure is intermittent, so this pair does not establish that feedback fixes it.
All 108 full-rate outcome summaries and raw-trace hashes · All 16 pilot outcomes, audit checks, controller parameters and source hashes · All 12 guided-tilt recordings. Full raw traces and frozen inputs are backed up in the research workspace.
5. Deformable objects
Soft objects: Matching settings does not yet match the task
We have established that engines largely agree on rigid body scenarios, even though modeled forces may substantially differ. We are now moving on to a scenario that is crucial to get right for many meaningful simulations of realistic scenarios, but notoriously challenging to implement: Manipulating soft objects. We are no longer comparing rigid contact alone: the material law, the internal mesh and the coupling between fingers and deformable object all matter. Can the same Panda motion squeeze, lift and put down a 60 mm ball? We tested two separate mesh variants to control for different resolutions.
In the tested 2 N configurations, MuJoCo CPU retains the ball across the numerical controls; native MuJoCo-Warp reaches the hold but fails during lowering. Newton + VBD can complete the grasp after correcting imported damping, but retention depends on mesh and timestep. Genesis fails to sustain lift on the original mesh and develops invalid elements on the refined one. PhysX's early mesh failures leave its setup unvalidated, so we cannot assess its grasp capability here.
First check: should the ball stay in the gripper?
The authored tetrahedral ball weighs approximately 105 g (1.031 N); a perfect 60 mm sphere at the same density would weigh 113 g. For a static, symmetric pinch with horizontal contact normals, Coulomb friction gives a maximum vertical support of 2 µ N, where N is the normal force at each finger. With µ = 0.5, the threshold is approximately 1.03 N normal force per finger. A 1 N pinch is slightly below that threshold; 2 N offers approximately 1.94 times the required static support.
Our 1 N and 2 N settings cap actuator effort, not measured contact-normal force. Curved contacts, deformation, joint dynamics and lift acceleration can shift the actual threshold. At 1 N, slipping is consistent with the near/below-threshold loading. Explosive motion, invalid geometry and inconsistent force feedback remain separate failures.
The 4.5 s motion closes the fingers, lifts by a commanded 80 mm, holds, lowers the ball and then opens; lowering before opening keeps grasp stability separate from a drop impact. As before, the analytical fingertip pads are the only colliders.
Detailed grasp outcomes, effort sweep and coupling diagnostics
| Route | 1 N per finger | 2 N per finger |
|---|---|---|
| MuJoCo CPU | Stopped at 1.345 s | Complete; hold lift ≈66 mm |
| Native MuJoCo-Warp | Stopped at 1.335 s | Hold lift ≈60 mm; stopped at 2.525 s |
| PhysX GPU — setup not working | Stopped at 0.066 s | Stopped at 1.536 s |
| Genesis FEM / SAP | Complete; no sustained lift | Complete; no sustained lift |
| Newton MJWarp + VBD | Completes numerically; lifts briefly, loses ball | Complete grasp/lower/release; hold COM lift ≈69 mm |
The 1 N PhysX run stops before the fingers close, requiring an initialization and contact audit. Newton completes the 2 N grasp but loses the ball at 1 N. Both native MuJoCo routes lift at 2 N, then differ during lowering; Genesis completes both runs without sustained lift.
Near the friction threshold: small effort changes, different outcomes
We tested 1.05, 1.10, 1.15 and 1.50 N actuator limits on the three routes that had already lifted the ball. Each setting was run once, without randomized inputs. The controller, trajectory, object and each route's numerical settings were held fixed; Newton retains shape damping 0.2 and a 0.5 ms step, while native MuJoCo routes retain 50 µs steps. We then added PhysX and Genesis at the two displayed settings, 1.10 and 1.50 N, retaining their original 1 ms settings and pinned environments.
| Limit per finger | MuJoCo CPU | Native MuJoCo-Warp | Newton MJWarp + VBD | PhysX GPU — setup not working | Genesis FEM / SAP |
|---|---|---|---|---|---|
| 1.05 N | Invalid at 1.410 s; before hold | Invalid at 1.365 s; before hold | Full cycle; 62.2 mm lift | Not run | Not run |
| 1.10 N | 57.8 mm hold lift; invalid at 2.340 s | Invalid at 1.390 s; before hold | Full cycle; 64.2 mm lift | Invalid at 0.216 s; before hold | Numerically complete; no sustained lift (−3.9 mm) |
| 1.15 N | Full cycle; 63.7 mm lift | Invalid at 1.445 s; before hold | Full cycle; 64.9 mm lift | Not run | Not run |
| 1.50 N | Full cycle; 65.3 mm lift | 58.7 mm hold lift; invalid at 2.410 s | Full cycle; 66.4 mm lift | Invalid at 0.656 s; before hold | Numerically complete; no sustained lift (−0.7 mm) |
At 1.10 N, CPU holds the ball before failing during lowering, native MuJoCo-Warp fails before the hold, and VBD completes the cycle. At 1.50 N, the expected-success control still does not complete on native MuJoCo-Warp. Geometry failures interrupt the trials, preventing a clean comparison of Coulomb slip thresholds. PhysX fails before the hold at both displayed settings; Genesis finishes numerically but does not sustain a lift. Negative COM displacement means the object settles below its initial position.
VBD's force feedback also needs scrutiny: its peak harvested feedback is 9.02, 165.56, 2.28 and 2.14 N at the four increasing effort limits. The 1.10 N spike occurs during lowering (about 3.00 s); its physical validity remains unresolved.
The replay selector above includes two additional five-route comparisons: 1.10 N — contrasting failure phases and 1.50 N — expected-success control. They were selected after inspecting the sweep; the full table prevents hiding the other outcomes. All measurements, run settings and source hashes.
Compression control: squeezing the ball without a robot
Two prescribed plates compress the ball by a nominal 20%, hold, release and recover over 1.5 s with gravity disabled. All five standalone routes complete the geometry capability cycle after correcting import, floor and plate-motion issues.
With the same native flex model and 50 µs timestep, MuJoCo CPU reports a hold-average opposing plate load of 2.151 N; native MuJoCo-Warp reports 1.632 N. Their observed ball extents also differ: approximately 49.3 and 53.7 mm against a 48.2 mm plate gap. Different penetration depths confound the apparent stiffness difference.
An unchanged-input repeat changes recorded vertices by at most 0 mm on CPU and 0.181 mm on native MuJoCo-Warp. Halving the timestep changes them by 0.018 and 0.176 mm. Mesh convergence and calibrated material equivalence remain outstanding.
Same parameters, different models
The nominal inputs are shared: radius 30 mm, density 1,000 kg/m³, Young's modulus 20 kPa, Poisson ratio 0.3 and friction 0.5. The starting volume mesh has 125 vertices and 384 tetrahedra. The implementations differ as follows:
- MuJoCo CPU and native MuJoCo-Warp: full-DOF volumetric flex, the same compiled mesh and volume-lumped masses; 50 µs steps.
- PhysX GPU: Isaac Sim 6.1 volumetric TetMesh, GPU dynamics, explicit contact offsets and checked rest-node mapping; 1 ms steps. Earlier rigid experiments still use their original PhysX configuration.
- Genesis: GPU float64 implicit FEM with a linear-corotated material and SAP coupling; 1 ms steps. It remeshes the shared surface, so its interior elements differ. The rigid importer does not honor the MJCF elliptic-cone request in these runs; we record this as a configuration mismatch.
- Newton MJWarp + VBD: rigid robot and soft particles use separate solvers with lagged proxy feedback; 0.5 ms steps and explicit shape contact damping 0.2. The robot controller, effort limits, friction, ball mesh and elastic parameters are unchanged. This is a native Newton implementation of the experimental coupling pattern documented by Isaac Lab, not a run through the Isaac Lab frontend. The compression-only control uses VBD directly, without MJWarp.
Force measurements
Native MuJoCo plate/finger contact readbacks are shown as provisional native solver measurements. Genesis SAP impulses and Newton VBD proxy reactions were exported too, but earlier configurations failed a zero-gravity momentum-balance check. Compression arrows remain withheld; the instrumented grasp replay shows diagnostic signals, not validated loads. The PhysX rigid-contact probe returns zero despite visible deformation, so we exclude that probe from the reaction measurements. Ball weight is computed from mesh mass and shown separately.
Instrumentation follow-up. Separate reruns record Newton's actual harvested and applied coupling buffers at every step, with the expected one-step delay. In the 2 N grasp the peak harvested feedback is 34.4 N; halving the timestep reduces it to 2.56 N, while mean hold COM lift changes from 68.6 to 66.5 mm. Lift is comparatively stable, but force peaks are not timestep-converged. The replay draws applied, not harvested, feedback. For Genesis compression, increasing the outer solver cap from 5 to 50 iterations reduces the maximum force-balance residual from 3.66 N to 0.018 N. PhysX now has raw joint-reaction recordings and an inferred leaf-finger contact load from momentum balance, checked in a no-ball run (maximum residual after the initial step: 0.000036 N). Each instrumented replay uses matching geometry and force recordings from these separate reruns. Read the force-instrumentation protocol and limitations.
Implementation lesson: global settings did not replace imported shape settings
Newton initially kept the imported shape damping of 200 despite global damping being set to zero. The resulting run failed before lifting, with a 1.92 kN force spike and invalid force measurements. Setting shape damping explicitly to 0.2, based on the Franka soft-body example, allowed the 2 N grasp to complete without changing the asset or controller. Setting shape damping to zero also completed the grasp. The correction demonstrates a working coupled grasp and isolates the imported contact configuration as the cause of this early failure. Physical calibration and transfer to other scenes remain to be tested.
Contact-damping diagnostic protocol, results and source hashes
Does the grasp survive mesh and timestep refinement?
Question: are the grasp failures caused by a mesh that is too coarse or a timestep that is too large? We test sensitivity to numerical resolution by using more mesh elements and smaller physics steps while keeping the physical inputs and commanded motion fixed.
We tested the 2 N grasp in 25 single trials: five numerical configurations for each of the five routes. The original mesh runs at the baseline, half and quarter timestep; the refined mesh runs at half and quarter timestep. The motion tape, 1 kHz controller, effort cap, nominal material and contact settings stay fixed. Newton retains explicit shape damping 0.2.
Result: within the tested range, finer meshes and smaller timesteps do not consistently eliminate failures or preserve grasp outcomes. CPU MuJoCo is comparatively stable over this tested range; native MuJoCo-Warp's lowering failure persists. Newton + VBD shows why a valid mesh throughout a run is insufficient: grasp retention itself can depend on the mesh/timestep combination. Genesis likewise becomes invalid on the refined mesh rather than simply improving. These controls expose numerical sensitivity; convergence and failure mechanisms remain open.
Inspect refinement replays, exact mesh/timestep settings and all 25 outcomes
Refinement subdivides the existing tetrahedra from 125 vertices / 384 elements to 729 vertices / 3,072 elements, preserving the same polyhedral boundary and approximately 105 g mass rather than making the ball rounder. Genesis remeshes the subdivided surface natively; its actual mesh and volume are checked separately. Timesteps remain route-specific: native MuJoCo CPU/Warp use 50 → 25 → 12.5 µs, Newton + VBD 500 → 250 → 125 µs, and PhysX/Genesis 1,000 → 500 → 250 µs. Two meshes do not establish convergence.
| Route | Numerical validity | What changes under refinement? |
|---|---|---|
| MuJoCo CPU | 5/5 complete | Holds the ball in every condition. Mean hold lift is 65.6–65.7 mm on the original mesh and 62.9–63.0 mm on the refined mesh. |
| Native MuJoCo-Warp | 5/5 invert during lowering | All reach the hold, at 59.1–60.5 mm mean lift. Inversion occurs at 2.515–2.540 s on the original mesh and 2.335–2.365 s on the refined mesh. Smaller steps do not remove the failure. |
| PhysX GPU — setup not working | 5/5 invert before lifting | The baseline stops at 0.461 s; smaller-step and refined-mesh cases stop at 0.040–0.061 s, before the commanded closure starts at 0.3 s. These results still implicate initialization/contact setup rather than establish a grasping limitation. |
| Genesis FEM / SAP | 3 original-mesh runs complete; 2 refined-mesh runs invert | The original mesh never sustains lift (mean hold displacement about −3 mm). Both refined cases invert around 1.355 s during the lift, before the hold window. |
| Newton MJWarp + VBD | 5/5 complete numerically | Original-mesh mean hold lift changes from 68.4 to 66.5 to 64.2 mm as the timestep shrinks. The refined mesh loses the ball at 250 µs, but holds it at 125 µs (64.0 mm mean lift). |
What remains open: force-readback validation and the PhysX initial-contact/import checks are separate from these numerical controls. Single trials show the VBD retention change but cannot quantify its frequency. The near-threshold 1.10 N coupling spike is outside this 2 N study. Matching nominal soft-body settings still leaves substantial differences in grasp robustness.
All 25 measurements and source hashes · Mesh/timestep protocol, caveats and reproduction. Three sampled replay bundles are deployed; full-rate raw traces stay in the research archive.
Download the trial summary · Reproduction notes and remaining checks
6. Controller tuning
Controller tuning: success hides a tracking trade-off
So far, one hand-set controller ran unchanged in every engine. Did that hide engine-specific tuning requirements? We return to the three-block pushing task and let each engine select its own controller from the same candidate grid, with the same selection rule and held-out layouts. Physics parameters are not tuned per engine.
Each engine evaluated 27 candidates, combining arm proportional gain (0.5×, 1×, 2×), damping gain (0.7×, 1×, 1.4×) and push duration (2, 3, 4 s), on three 90 mm selection layouts: 324 tuning trials, all valid. We then ran the selected controller and the original baseline on six held-out layouts, four with the 90 mm channel and two with the narrower 80 mm channel.
Every engine selects the same controller
Every candidate passed all three selection layouts, so the selection rule fell through to its tie-break: lower mean arm effort. All four engines therefore chose the same candidate, with 0.5× proportional gain, 0.7× damping gain and a 4 s push, against the baseline's 1× gains and 3 s push. The agreement reflects the effort objective, not better task success. The winner also sits at the edge of the grid on all three axes, and its advantage over the runner-up is below 0.6% in every engine.
| Controller | PhysX CPU | Newton / MJWarp | MuJoCo CPU | Genesis |
|---|---|---|---|---|
| Original baseline | 4/6 | 4/6 | 4/6 | 4/6 |
| Effort-selected controller (same in all four) | 4/6 | 4/6 | 4/6 | 4/6 |
Both controllers pass the four 90 mm layouts and fail the two 80 mm layouts in every engine. As in the pushing experiment, the channel geometry decides the outcome.
Identical success, different trajectories
The traces tell a different story than the success table. Across the six held-out layouts, the selected controller uses roughly 33–36% less arm effort in each engine, but it allows larger vertical hand excursions, although the reference keeps the hand at constant height. Tracking error and contact peaks were not part of the selection score.
| Engine | Four successful 90 mm layouts | Two failed 80 mm layouts |
|---|---|---|
| PhysX CPU | 0.8–6.2 → 1.4–16.3 | 77–84 → 245–315 |
| Newton / MJWarp | 3.3–4.7 → 22.2–28.3 | 109.5–110.1 → 296.6–297.7 |
| MuJoCo CPU | 2.6–3.8 → 17.6–19.1 | 109.2–109.3 → 298.7–299.6 |
| Genesis | 0.2–0.2 → 0.6–0.7 | 106–107 → 300–304 |
In the successful layouts, Genesis stays close to the reference height, while the selected controller reaches centimetre-scale deviations in the other three engines. In the narrow layouts, both controllers fail everywhere, but the selected one deviates by roughly 25–31 cm instead of 8–11 cm.
Is it the gains or the speed?
The selected controller changed gains and push duration together. As a quick control, we kept the baseline gains and shortened the 24 cm push from 3 s to as little as 0.75 s: five durations, three 90 mm layouts and four engines, 60 trials. Every trial succeeded and stayed within 10 mm of the reference height. Even at the fastest push, peak vertical deviation ranged only from 0.24 mm (PhysX) to 3.3 mm (Newton / MuJoCo-Warp). Speed alone therefore does not produce the 17–28 mm excursions of the selected controller; its lower gains are the more likely cause.
Selection protocol and caveats
The frozen selection rule ranked candidates by successful layouts, passed objects and capped minimum-object progress, then by lower mean normalized squared arm effort. This effort metric is not mechanical work or energy. It is averaged over the whole recorded episode, 5 s for the baseline and 6 s for the selected controller, so a longer push changes both the motion and the averaging window and can favour slower execution.
The selected p0.5-d0.7-t4 uses the lowest proportional gain, lowest damping and longest
push tested. Its effort advantage over the runner-up, p0.5-d1-t4, is 0.04% in Genesis,
0.11% in MuJoCo CPU, 0.42% in PhysX and 0.59% in Newton / MJWarp; repeat trials are needed to judge
whether these margins exceed run-to-run noise. Gains and push duration changed together, which
confounds their individual effects; the speed control above addresses this in part.
Each configuration and layout was run once, with seed 0. Held-out layouts were excluded from controller selection but may have been used during scene development. Hand deviations are measured from the engines' native hand poses.
7. Runtime
One fast CPU world, or many GPU worlds
For a single environment, MuJoCo CPU is by far the fastest: about 167,000 steps per second for the contact-free Panda, against roughly 5,700 for MuJoCo-Warp and 1,900 for Genesis on the GPU. A single small world cannot fill a GPU, so per-step overhead dominates. With many parallel worlds the picture flips: at 8,192 Panda worlds, MuJoCo-Warp reaches 21.6 million and Genesis 11.8 million aggregate steps per second. MuJoCo-Warp leads at every batch size, but the gap to Genesis narrows from about 3× at 128 worlds to 1.3× at 131,072.
| Parallel worlds | MuJoCo CPU | MuJoCo-Warp (GPU) | Genesis (GPU) |
|---|---|---|---|
| 1 | 0.17 | 0.006 | 0.002 |
| 128 | — | 0.61 | 0.21 |
| 8,192 | — | 21.6 | 11.8 |
| 32,768 | — | 33.1 | 22.4 |
| 131,072 | — | 35.2 | 26.4 |
Measured on an RTX PRO 5000 Blackwell (48 GB) with a Ryzen 9 9900X host. Three limits apply: MuJoCo CPU was timed with one world only, not across the host's 16 logical cores; PhysX was not batch-tested; and Genesis ran in float64 against MuJoCo-Warp's float32. At 131,072 worlds the GPU used 6.2 GiB (MuJoCo-Warp) and 4.9 GiB (Genesis), and batched trajectories matched their single-world runs exactly. All timings · Protocol · Batch-equivalence checks
MuJoCo CPU is fastest in our single-environment Panda benchmark. At large batch sizes, MuJoCo-Warp and Genesis deliver higher aggregate throughput. MuJoCo-Warp leads at every tested batch size, with the gap to Genesis narrowing at larger batches.
8. Discussion & outlook
What to take into your next policy evaluation
Lessons learned
From single objects to crowded scenes, soft objects and controller tuning, four lessons recur.
Verify imports before blaming the physics. A large engine difference in the hinge test turned out to be an imported joint armature, and the Panda agreed across engines only once masses, inertias and joint frames were checked after import (2, 4).
Agreeing outcomes can hide different physics. Stopping distances agree to 0.05 mm while force peaks differ tenfold, and controllers with identical success scores follow their path very differently (2, 6). Report forces, trajectories and failure severity alongside task success.
Measure variation within an engine before ranking engines. In the crowded sweep, identical-input repeats on the GPU vary as much as the engines differ from each other (4.3), and in the pushing scene a 1 mm layout change moves objects far more in some engines than in others (4.2).
Validate soft objects in each engine. Matching nominal material settings does not match grasp outcomes, and refining the mesh or timestep does not bring them together (5).
Together, these experiments form a validation workflow for cross-engine comparisons: start with single objects, add interactions, check the robot in free space and then in contact, and at every step audit the imported model, halve the timestep and repeat runs. For the choice of engine itself, MuJoCo on the CPU suits single-scene development and debugging, while the GPU engines pay off for batched training (7).
Evaluate across more than one engine
The engines disagree most where policies are most exposed: contact-rich and crowded scenes, tight insertions (4.4) and soft objects. Running an evaluation in a second engine is therefore a cheap robustness check. For learned policies, freeze the observation pipeline, action interpretation and controller together with the policy weights. Use the controls above to investigate a score gap before attributing it to policy quality; replaying recorded commands can help separate dynamics from closed-loop perception effects. Our next post applies this to policies trained in one engine and evaluated in the others.
Correct the scene, don't tune for success
Scene setup still needs per-engine care: an imported armature, a missing collision mask and a mislabelled geometry each changed our early results. Keep such corrections to the physical specification separate from tuning for task success. Document engine-specific numerical choices, keep the original controls and rerun them after every change. Our controller study shows why the objective must be explicit: lower mean effort did not improve success and allowed larger tracking deviations.
Limitations & next steps
This is a comparison of specific implementations, scenes and parameter ranges. Most settings have one trial; the repeat studies show why broader claims need repeated measurements. Some scenes were developed in MuJoCo CPU, and held-out controller layouts may have appeared during scene development. Task selection therefore favors familiar configurations. The robot uses simplified collision geometry—often only fingertip pads—and explicit scripted control. Native-drive portability, learned-policy transfer and real-world accuracy require separate tests with matched hardware measurements. We take up learned-policy transfer and real-world accuracy in the next two posts of this series.
The most urgent validation work concerns PhysX's deformable initialization and the remaining coupling-force diagnostics, followed by Genesis's late force spikes in the crowded sweep. Those warnings stay beside the affected results. Beyond these checks, repeated trials, calibrated contact and material models, and systematic convergence studies would strengthen the comparison.
Code, data & reproducibility
Each experiment links its measurements and audit notes. For the final controller study, start with the frozen plan and source hashes, transfer report and trace-analysis reproduction notes. Runtime measurements have a separate protocol. Large raw trace backups are retained locally; the replay files are sampled visualization exports.
References
- J. Nuñez Cano, M. Patel, A. Arranz et al. (NVIDIA). “The State of Simulation for Physical AI: An Overview.” Hugging Face blog, July 21, 2026. huggingface.co/blog/nvidia/state-of-simulation-for-physical-ai
- Genesis AI. genesis.ai - the role of simulation in scalable robotics
- Elie Aljalbout, Jiaxu Xing, Angel Romero, Iretiayo Akinola, Caelan Reed Garrett, et al. 2026. The Reality Gap in Robotics: Challenges, Solutions, and Best Practices. Annual Review of Control, Robotics, and Autonomous Systems 9:403-432. https://doi.org/10.1146/annurev-control-031924-100130
Acknowledgements
Thank you to Benjamin Alt and Haoyang Zhou for reading drafts of this post and giving feedback.
If you would like to work with us on custom evaluations of your robotic policies, use the collaboration form.