Testing the Robot Training Loop I Drew

A month ago I wrote that a robot training center isn’t ten projects, it’s one five-stage loop: Capture, Simulate and Synthesize, Train, Validate, Deploy. That piece was a map drawn from my experience: bits and pieces of a training workflow I’d built over the last few years, each specific to a particular use case rather than a general platform view. Now that the field is moving toward generalization, I wanted to build it and run it firsthand — to see whether this could actually grow into a horizontal platform. I ended that writing with a promise to “take a task all the way around the loop myself.” This is the build that takes a first pass at it: the loop implemented as a real, orchestrated pipeline that runs end to end on a single GPU, with two tasks carried around it. This part documents the environment, the stack, the architecture, and the measured results. Part 2 covers what the runs mean and what they leave unproven.

The goal was to test those convictions rather than restate them: that it’s one loop and not ten disconnected projects; that the middle three stages are reusable while the ends are per-task; and that the “return belt” from Deploy back to Capture can be a real mechanical thing rather than a dotted line on a diagram.

What I built

The system hangs on one design decision: the filesystem is the contract. Each of the five stages is an independent script that reads its inputs and writes exactly one JSON report into a shared run folder. No stage imports another.

Two layers are worth separating. The contract — stages coordinate only through declared artifacts, never shared code or memory — is the durable design choice, and I’d keep it in a real system: it’s what let me build the dashboard mock-first with no GPU, swap a stage’s learning method without touching the others, and treat scaling to OSMO (NVIDIA’s cluster orchestrator) as an executor swap rather than a rewrite. The filesystem is just the medium, and the deliberate simplification: on one box, JSON files in a run folder are enough, where a production system would put the same artifacts in object storage or a dataset service. The pattern stays; only the medium changes. That pattern is the point: components plug in, and the loop extends progressively, which is what a horizontal capability needs.

A lightweight runner — effectively a single-node orchestrator — reads a loop.yaml file describing the stages as a dependency graph, runs them in order, and stamps a manifest after each. It plays the orchestrator role on one box; in a production design it’s the piece you’d swap for a full orchestrator like OSMO. The browser dashboard holds no logic of its own; it reads those JSON files and renders the run, mock or real.

That same decision is what makes the return belt concrete. The Deploy stage writes a next_run hint into its report, and the runner reads it to seed the next iteration. The flywheel the article kept insisting on is, in practice, a file one stage writes and the next run reads.

The dashboard

Before the stack and the numbers, here’s what it looks like. The front end is a single-page dashboard served from the GPU host and opened in a browser. It’s the control panel and the live view in one: you pick a task and settings at the top, hit Run, and the five-stage ring fills in as each stage finishes, with a metrics row (total time, iteration, environments, train iterations, eval, deploy latency), a time-per-stage bar, and one expandable card per stage showing its tech, its metrics, and links to download its artifacts. It makes the loop concrete: you watch each stage finish instead of trusting a log.

Because the whole point is to iterate, most of what I built into the dashboard is about making the second, third, and tenth run cheap:

• A mock mode that runs the entire loop with no GPU and no simulator, emitting realistic stage reports. I built and debugged the dashboard against it before spending a GPU minute, and it’s still how I test front-end changes.
• A real/mock toggle and per-field config — task, demos source, environments, train iterations, trials, loops — each with a one-click ↺ reset, so changing one knob and re-running takes seconds.
• Live per-stage timers and status, so a run shows progress instead of going dark for 95 minutes.
• Auto-capture on completion: the dashboard downloads the run’s report PDF, data JSON, rollout video, and every artifact (dataset, checkpoint, policy), each named by run id, so every run documents itself.
• A recent-runs table, newest first, to compare a new run’s reward, success rate, and latency against past runs at a glance.
• An optional live view that streams the running simulator into a desktop WebRTC client, for when you want to watch the rollout instead of reading its numbers.

None of these change what the loop computes. They change how fast I can turn the crank, which is the entire job of a testbed. I didn’t want to sink time into the front end — just enough to show the loop generalizes across tasks and to have something I can run a live demo from.

Environment

Everything ran on a single cloud GPU instance, driven from a browser on a Mac. No cluster, no second machine.

Instance	AWS EC2 g6e.2xlarge (8 vCPU, 64 GB RAM)
GPU	1× NVIDIA L40S, 48 GB
OS / driver	Ubuntu 22.04, NVIDIA driver 580.126.16
Python	3.12 (conda env env_isaaclab)
Client	Browser on macOS, over an SSH tunnel to :8080

Two environment constraints turned out to be load-bearing, and I’ll return to both in the stuck section: Isaac Sim 6 requires Python 3.12 specifically, and it pairs with a particular Isaac Lab release branch rather than main.

The stack

The whole thing runs on the open/free stack; the only cost is GPU hours. Components by role:

Role	Component	Notes
Simulator	Isaac Sim 6.0.1.0	native WebRTC streaming
Task framework	Isaac Lab release/3.0.0-beta2	main pairs with Isaac Sim 5.1, not 6
Deep learning	PyTorch 2.10.0 + cu128	installed by the Isaac Lab beta
RL training	rsl_rl (PPO)	native ONNX export — the reason I chose it
Imitation data-gen	Isaac Lab Mimic	annotate + generate demonstrations
Imitation training	robomimic	behavior cloning (bc_rnn_low_dim)
Deploy benchmark	onnxruntime	measures p50 / p99 inference latency
Orchestrator	custom single-node runner + PyYAML	interprets the loop.yaml DAG
Server / dashboard	Python stdlib HTTP + vanilla HTML/JS	no framework, no build step

I built the single-node runner deliberately instead of standing up NVIDIA’s OSMO, which assumes a Kubernetes cluster, object storage, and a registry. But I built it to mirror OSMO’s model — a declarative DAG, content-addressable artifacts stored by hash, and stages that declare the compute they need — so that moving to OSMO at scale would be an executor swap rather than a rewrite. That portability is a design intention I haven’t yet tested against a real cluster.

One caveat on the whole stack: these are the versions that worked together when I tested this, in July 2026. Isaac Sim 6 is a recent release, and I deliberately ran on the newest line to put it through its paces — so expect the exact pairing (Isaac Sim, the Isaac Lab branch, the PyTorch build) to move over time.

What I used at each stage — and what I skipped

The table above is what I ran. This one is the fuller decision: at each stage, the realistic options and my call. The interesting column is the last one. Almost everything I skipped, I skipped for one of two reasons — to keep the demo simple, or because it’s the next layer to add — not because it wouldn’t fit. Legend: ✅ used · ➖ skipped.

Stage	Option	What it does	Used?	Why / why not
Capture	Teleoperation (SpaceMouse / keyboard / VR)	A human drives the robot to record demos	➖	Simplicity — I used NVIDIA’s published seed demos; live teleop is a manual pre-step, not needed to show the loop
	robomimic HDF5	Demonstration dataset format	✅	The native format for robomimic and Mimic
Simulate & Synthesize	Isaac Sim	GPU physics and rendering	✅	The simulator everything runs in
	Isaac Lab	Task / environment framework	✅	Provides the Cartpole and Franka tasks
	Isaac Lab Mimic	Trajectory-adaptation data generation	✅ (imitation)	Expands 10 → 1,000 demos from the seed set
	Domain randomization (Isaac Lab events)	Varies textures, lighting, dynamics	✅ (config)	Built into the environment config
	Cosmos (world foundation models)	Generative synthetic data	➖	Scope — Mimic augmentation is enough here; Cosmos is the intended next layer
	Omniverse Replicator	Synthetic perception data	➖	No vision task — both policies use low-dimensional observations
Train	rsl_rl (PPO)	On-policy RL trainer	✅ (RL)	Native ONNX export — enables the real deploy-latency measurement
	rl_games	Alternative RL trainer	➖	No ONNX export in this setup
	robomimic (BC)	Behavior cloning	✅ (imitation)	Standard BC on the generated demos
	VLA post-train (OpenVLA / Octo + LoRA)	Fine-tune a vision-language-action model	➖	Scope — the next phase; BC is enough to close the loop
Validate	Isaac Lab play (rsl_rl / robomimic)	Rolls the policy out in sim and scores it	✅	Reuses the same env; gives success rate and reward
	External benchmark suites	Standardized evaluation batteries	➖	Sim-rollout success is enough for the demo
Deploy	ONNX + onnxruntime	Portable inference plus a latency benchmark	✅ (RL)	Cheap, portable proof of control-rate latency
	TensorRT	GPU-optimized inference	➖	Production optimization — onnxruntime is enough to show latency
	Triton Inference Server	Production model serving	➖	This is deploy-as-inference locally; Triton is for production serving
	ROS 2 / Isaac ROS	Run the policy on a real robot	➖	No hardware — sim-to-real is out of scope
Orchestration	OSMO	NVIDIA’s cluster workflow orchestrator	➖	Demo simplicity — I built a single-node runner in OSMO’s shape; OSMO (or Kubernetes / Airflow / Ray) is overkill for one box and right only at production scale

Cosmos is the clearest deliberate deferral. I left it out to start because running it in parallel with everything else on a single GPU wasn’t where I wanted to begin. I already have a Cosmos synthetic-data pipeline working on its own, and the plan is to drop it into the Simulate stage as a pluggable source of generated data — exactly the kind of extension the contract is meant to allow. It’s on the backlog for now.

Architecture and data flow

The control path is one direction and the data path is the other. The dashboard sends a single POST /api/run (task, mode, stage flags) and then polls /api/runs/<id> for state; it renders whatever JSON it finds and nothing more. On the host, server.py spawns runner.py, which topologically orders the stages from loop.yaml, runs each as a subprocess that shells out to isaaclab.sh, and writes per-stage timings and reports into runs/<id>/. That folder is the loop’s entire memory: the manifest (loop.json), the five stage reports, and the artifacts (datasets, checkpoints, exported policies) referenced by hash. Running the server inside the env_isaaclab conda environment, so its spawned subprocess inherits the simulator’s heavy Python env, was the specific thing that let a browser button launch a real GPU run.

The same structure in text, for any view that can’t load the diagram:

CLIENT · your Mac
  Dashboard SPA (pure reader)
     │  POST /api/run  ·  poll /api/runs → JSON
     ▼
HOST · AWS L40S · Isaac Sim 6
  server.py :8080  ─spawns─▶  runner.py (loop.yaml DAG)
     │ runs the 5 stages; each shells to isaaclab.sh
     ▼
  capture ▶ simulate ▶ train ▶ validate ▶ deploy
     │ every stage writes one JSON into runs/<id>/
     ▼
  runs/<id>/  ·  loop.json + 01–05_*.json + artifacts
     └ deploy writes next_run ▶ seeds the next run

I kept this simple enough to build solo, but principled enough to carry the design ideas a horizontal service would need — the same contract, pluggable stages, and compute-abstracted steps, just at one-box scale.

How a full run flows

Clicking Run kicks off the same five stages in the same order for either task. What differs is what each stage does inside — but first, the two tasks themselves:

For Cartpole (reinforcement learning), Capture is skipped (no demonstrations, only a reward) and reported as skipped-rl. Simulate writes a small config report; the environment itself is instantiated by Train, which runs PPO across 4,096 parallel environments for 150 iterations, until the reward curve plateaus. Validate reloads the trained policy, rolls it out over a set of episodes, and records a rollout video. Deploy exports the policy to ONNX, benchmarks p50/p99 inference latency, and writes next_run. The whole sequence takes about three minutes. Cartpole is the smoke test here — the fastest task that runs all five stages for real — so its low absolute reward doesn’t matter; a working loop does.

For Franka cube-stack (imitation learning; a 7-DOF arm stacking cubes), Capture is real: it loads ten seed demonstrations, NVIDIA’s published human-teleop set. Simulate is now the heavy stage — Isaac Lab Mimic annotates those demos and generates 1,000 variations across randomized cube positions. Train fits a behavior-cloning policy (robomimic bc_rnn_low_dim, ~2,000 epochs, no tuning) to that dataset offline, with no simulator running. Validate rolls the policy out over 50 episodes and reports a success rate; Deploy records it as inference-only (ONNX export isn’t wired for behavior cloning yet) and writes next_run. End to end, about 95 minutes, most of it in Mimic and training.

Same pipeline, two shapes:

RL   (Cartpole):  Capture(skip) ─▶ Simulate(config) ─▶ Train(PPO in sim) ─▶ Validate(rollout) ─▶ Deploy(ONNX + latency) ─▶ next_run ↺
Imit (Franka):    Capture(10 demos) ─▶ Simulate(Mimic→1000) ─▶ Train(BC, offline) ─▶ Validate(rollout · 72%) ─▶ Deploy(inference) ─▶ next_run ↺

The why behind those differences — why manipulation copies instead of tries, and why one path runs the simulator during training while the other doesn’t — is Part 2. Here the point is only that one runner and one contract carry both without changing shape.

The stage contract

Every stage emits one JSON report with a shared envelope: stage, status, mode, metrics, artifacts, and — on Deploy — a next block that carries the return belt. The manifest tracks the task, the current iteration, and a stage-status map. That contract is the only coupling between the stages and the dashboard, which is why either can be developed in isolation. Here’s the real Deploy report from a Cartpole run, trimmed:

{
  "stage": "deploy", "status": "done", "mode": "real",
  "summary": "0.009 ms/step p50",
  "metrics": {
    "export_format": "onnx",
    "latency_p50_ms": 0.009,
    "latency_p99_ms": 0.027,
    "on_robot": false
  },
  "artifacts": [{ "name": "policy", "kind": "policy", "path": ".../exported/policy.onnx" }],
  "next": { "suggest": "widen domain randomization and re-run", "params": { "dr_scale": 0.1 } }
}

The next block is the return belt in its entirety: the runner reads it and applies params to the following iteration.

The two runs

I ran two tasks on purpose, because they exercise the two different ways a robot learns a skill, and the loop is supposed to carry both. Here are the measured per-stage results from the two recorded runs whose media is in the repo:

Stage	Cartpole (RL)	Franka cube-stack (imitation)
Capture	skipped-rl (no demos)	10 seed demos (download)
Simulate & Synthesize	config only, ~0:00	Mimic 10 → 1,000 demos, 46:10
Train	PPO, reward 4.94, 2:27	behavior cloning, 39:57
Validate	32 episodes, 0:40	50 episodes, 72% success, 9:00
Deploy	ONNX, p50 0.009 ms / p99 0.027 ms	inference-only (ONNX not wired for BC)
Total	~3 min	95 min

Everything here ran on the single 48 GB card without batching or memory tuning — the 4,096-environment RL job and the 1,000-demo Mimic generation each fit as-is.

Both policies are recorded if you want to watch them run: Cartpole and Franka (8 of 10 in that recording; 72% over the full 50-episode eval), alongside the auto-generated run report (PDF).

One measurement is worth isolating. At 50 generated demos the Franka policy succeeded about 2% of the time; at 1,000 it reached 72%, from the same ten seed demonstrations. Almost the entire gap is manufactured variety. Why the two tasks take such different shapes inside the same loop — Capture present or skipped, the sim running during training or only to generate data — is the subject of Part 2.

The scorecard

Building the loop turns each of the article’s claims into a checkable result:

Article claim	How the build treated it	Where it landed
It’s one loop, not ten	One run flows through all five stages; the return belt is a file Deploy writes	✅ Held — closed in code
Own the middle, rent the edges	Same middle pipeline and contract carried both tasks (the insides flip by method); Capture and Deploy swap per method	✅ Held — proven in the wiring
Capture is the hard part	RL skips it; imitation makes it real, and everything downstream is capped by those ten seed demos	✅ Shown by contrast
Synthetic data unlocks behavior	Mimic turned 10 demos into 1,000; success moved from ~2% to 72%	◑ Partly — a generative layer (Cosmos) is the next rung
Adaptation over invention	Behavior cloning got 72% out of the box; foundation-model post-training is untested	→ Deferred to the next build
Validate is the unsolved link	Sim-eval works; safety_certified is false on every run	⚠ Left open on purpose

Where I actually got stuck

Almost no time went into the loop logic. It went into the stack and into one class of bug.

The stack needed a specific, non-obvious pairing. Isaac Sim 6 wants Python 3.12; my packages first installed against 3.13 and silently did nothing useful. Isaac Lab’s main branch pairs with Isaac Sim 5.1 and an older PyTorch, so getting Isaac Sim 6 meant pinning Isaac Lab to a specific beta release branch. I couldn’t find it documented as a single instruction anywhere. I also started on rl_games and moved the entire training and validation path to rsl_rl for one reason: rsl_rl exports an ONNX policy natively, which is what let Deploy benchmark real latency instead of faking it.

The bug worth recording: the Isaac Lab launcher script returns exit code 0 even when the Python process inside it crashes. A stage would shell out, the inner training would die, the launcher would report success, and the stage would reuse artifacts from the previous run — reporting a completed 1,000-demo generation in about twenty seconds with the last run’s numbers. It worked the first time and silently reused stale data every time after. The fix was to stop trusting exit codes: every real stage records a timestamp before starting and rejects any output older than that, marking itself failed if outputs are stale or missing. Robot pipelines are full of tools built to be run once by hand, so re-runnability also meant every stage deletes its own prior outputs first — half of them refuse to overwrite, and the other half block on an interactive “are you sure?” prompt that hangs with no terminal attached.

One smaller gotcha shaped the media: on the imitation path, robomimic can’t write a rollout video; its player only streams to a live client. The clean footage of the arm stacking cubes comes from a separate offscreen recorder I wrote — it loads the checkpoint, runs the rollouts headless with cameras enabled, and captures frames straight to an MP4.

Limitations

Two limits are structural, and I’ve flagged them throughout — but they’re worth stating plainly in one place. Deploy reports on_robot: false on every run: this is deploy-as-inference, where the policy is exported, loaded, and benchmarked, but never touches physical hardware. And a 72% success rate in simulation measures capability, not safety — there is no trusted way to turn one into the other, so safety_certified is false on every run. That second one isn’t a gap in the build; it’s an open problem for the field, and the loop’s job here is to mark the seam precisely instead of covering it.

What’s next in the build

The loop is complete but shallow in two places, and the next phase adds depth without changing the shape. The behavior-cloning policy tops out near 70% because copying demonstrations has a ceiling; the “adaptation over invention” claim only gets tested by post-training a vision-language-action foundation model on the same captured data and checking whether it beats the from-scratch baseline. The return belt is closed mechanically but hasn’t been run for the two or three iterations that would show a success rate moving as data flows back. A generative synthesis layer (Cosmos) sits on top of Mimic and isn’t wired yet, and the scale-out to real OSMO would test the “executor swap” claim against actual heterogeneous compute. None of that requires changing the loop’s architecture — which, if “it’s one loop” is right, is exactly the expected result.

Code, the full build log with every version pin and fix, the stage reports, and reproduce-from-scratch steps are in the repo: github.com/pr9868/robot-training-loop-demo. Part 2 covers what the runs mean, and where I’m still not sure.

Disclaimer: The views and opinions expressed in this account are those of my own and do not represent those of my employer, NVIDIA. This is a work in progress and I’ve almost certainly gotten something wrong or missed a detail, so I’m always happy to be corrected.