The independent check nobody runs.
Vision
Every robotic manipulation system deployed in the world should pass an independent audit before it touches a human workspace.
Why it matters
When the model is wrong, nobody finds out until the robot is standing in front of a real shelf, failing. There's no step in most pipelines where someone stops and asks whether the simulation actually matches the robot. You train in the sim, you deploy to hardware, and you hope. And the person who set up the sim is usually the same person who's about to discover it's wrong.
We know because it happened to us. On our own arm, the gripper position was off by 9.8 millimeters between sim and reality. The camera field of view was off by 23.5 degrees (the sim said 70.7, the real camera sees 47.2). Errors like that will quietly ruin a trained policy, and they just sit there until something puts the sim and the physical robot side by side. Most pipelines never do.
A failed deployment costs you a week of robot time, your engineer's attention, maybe an awkward customer visit, maybe a contract. Catching problems like that before the robot ships is a lot cheaper than finding them after.
Can't you just check this yourselves?
Independent audits are the norm across every other industry. Buildings, aircraft, financial reporting. Nobody trusts the team that built a thing to also be the one who vouches for it, and it's not because anyone is careless. It's because the people closest to a system are the least objective about it. That includes us, which is why the audit is designed so nobody has to take anyone's word for anything.
The pass/fail criteria are pre-registered and hash-committed before anyone sees results. Once they're locked, they can't quietly move to fit whatever anyone was hoping to find. You can verify this yourself, because the hash proves the thresholds didn't change after the fact. So the independence doesn't depend on you trusting us. It's a property of how the audit works.
It's also worth saying who won't build this: the simulation engine companies. They want their tools to look good, and an audit that documents where those tools fall short is not in their interest. It has to come from somewhere independent.
Who we are
Arithmancy started with a study that killed our own thesis. We pre-registered the evaluation, hash-committed the criteria, ran it, and the result told us we were wrong. It also told us something more useful: the instrument we'd built to run that study could find problems nobody else was looking for. That instrument became the Validation Audit.
We're also hiring a US or Canada-based validation engineer who has personally shipped sim-to-real policy transfer on a real arm and been burned by it.