A validation layer for your training pipeline.
The Validation Audit
A validation layer you'll integrate with your existing training pipeline. Your scene files, configuration, and policy checkpoints will stay where they are, in your environment. The audit will run against them there and produce a Validation Report with a pass/fail stamp and the evidence behind it.
Every audit will run the same protocol: same layers, same measurement methods, same report format. The only thing that changes between audits is the pipeline being tested. That's what makes the pass/fail stamp mean something. It's the same bar for everyone.
Is your physics behaving consistently?
A simulator should produce the same physics twice. The audit will run identical episodes, repeated within your engine and across engines, and measure how much the trajectories diverge. That divergence number will be a core finding of the report: it tells you how much of your sim is physical behavior and how much is numerical artifact. You can't see this from inside one engine, because everything looks fine in there.
Is your policy robust or just lucky?
The audit will retrain your policy across seeds and configuration variants and track what happens to the success rate. If it swings from 18% to 96% because the seed changed, the policy is overfitting to noise, and it will not survive contact with real hardware. The report will show you how brittle it actually is while that's still cheap information.
Will it work when you deploy it?
This layer will produce the final verdict. The pass/fail criteria will be locked in writing before the evaluation runs, with a hash so you can prove they didn't move. Then the evaluation will run and you'll get your answer, with confidence intervals attached, and you can make your deployment decision on an actual number.
Does your sim match your actual robot?
On our own arm, this is the layer that found the delta. It will compare your sim's model against measured hardware data, and it needs measurements from your physical robot to run, which is why it ships after the first three layers. If you want it, we'll tell you exactly what measurements to collect so you're ready when it's available.
How It Works
Connect your pipeline
You give the audit tool access to your simulation environment, where it already runs: scene files, configuration, policy checkpoints. Your data stays in your environment the whole time. Intake checks that the pipeline is complete and runnable, and once it clears, your audit is in the queue.
The audit runs
The layers run in sequence against your pipeline. You see the pass/fail thresholds before anything starts, and the hash proves they didn't move afterward. There's nothing for you to do during the run. You get notified when it's done.
The report arrives
A Validation Report with pass/fail stamps per layer. The divergence numbers between runs and engines. Robustness results across seeds. A deployment readiness verdict with confidence intervals. Every finding traces back to a specific episode, scene, and measurement, so you can check the work.
What You Get Back
What the audit will need
Access to your simulation pipeline, in your environment: scene files, configuration, policy checkpoints, and the deployment scenario you're validating for. Nothing will be copied out. The audit will run where your data lives.
What you get back
- —A Validation Report (PDF) with pass/fail stamps per layer
- —The divergence numbers: how much identical episodes differ between runs and between engines
- —Policy robustness analysis across seeds and variants, with the full sweep
- —A deployment readiness verdict with confidence intervals
- —Reproducibility documentation for every finding: deterministic on CPU, statistically reproducible on GPU within a stated tolerance band, and traceable back to the specific episode and scene it came from
One thing the audit won't give you: a promise that your policy will work on hardware. The audit validates whether your simulation can be trusted. The deployment decision stays yours.
Pricing
There will be two versions of the audit.
The full protocol will run against your pipeline in your simulation engine. Layer 1 uses repeated-run consistency checks within your engine.
Everything in the Single-Engine Audit, plus your pipeline will run in a second engine. Where the two engines disagree about the same physics, that disagreement becomes a finding.
What if the audit finds nothing wrong?
Then you'll have actual evidence that your simulation is trustworthy, which is more than most teams can put in front of a safety review. A full pass will be a signed bill of health you can hand to your safety team or your investors, instead of asking them to take your word for it.