I Built a Live USD Viewer That Never Launches Kit

I wanted to know if the library-first pitch (build real things on Omniverse without ever touching Kit) actually holds up for something with genuine interactivity, not just a static render. So I built one from scratch: a browser-based USD viewer that renders server-side at full RTX quality and streams the frames live over WebRTC, with no Omniverse install and no GPU required on the client. Took a few hours, start to finish.

The rendering ran on a single AWS EC2 g6e.2xlarge, one NVIDIA L40S (48 GB), and I drove the whole thing from a Mac in a browser. The GPU stayed in the cloud; the client needed none.

The short cut below moves through one browser session: inspect and edit a prim, measure between two prims, save a camera position, configure telemetry, and then play the stage live.

The stack

Everything runs as plain Python and a normal npm frontend, no .kit file, no extension, no app to launch:

• ovrtx: the RTX renderer. In the version I built against, it loaded a USD stage internally and rendered a frame every time I called step().
• ovstream: the WebRTC server. Handles signaling, the data channel for camera/input events, and the actual video pipeline.
• warp-lang: a small GPU kernel to convert the renderer’s RGBA output to BGRA before it hits the stream encoder, entirely on-GPU, no CPU round-trip.
• pxr (usd-core): the open-source USD library, for every read/write the HTTP server itself needs: walking the prim hierarchy, editing attributes, building session layers, generating baked animation.
• React + TypeScript + Vite on the frontend, talking to the server over a plain REST API.

In the version I tested, pxr and ovrtx did not share the same stage. That turned out to matter a lot (more below).

What it actually does

By the time I stopped adding to it, the viewer had grown well past “stream a frame to a browser.” It had a scene browser that lists USD files and loads them on demand, a lazy-loaded prim hierarchy tree, GPU picking (click anywhere in the viewport, get back the prim under the cursor), a prim inspector for type/transform/visibility/variants, and transform editing that writes straight back to the stage with a live reload. On top of that: session-layer authoring (create Sphere/Cube/Cylinder/Xform/DomeLight prims, deactivate prims, undo/redo), camera bookmarks, timeline playback controls, PNG snapshot download, and a telemetry mode that binds prims to motion channels (oscillate, rotate, alert pulse, conveyor) and plays back live animation on the stage. Every one of those is also exposed over a REST API, so none of it is locked to the bundled frontend. Any script or client can drive the same server.

That’s basically USD Composer’s core editing feature set, minus Kit, running as a Python process that starts in seconds.

The application took shape in layers. The first useful version was already more than a streamed viewport: I could choose a stage, navigate the scene, keep camera bookmarks and inspect scene state without opening a desktop application.

The later pass made the browser an authoring surface. The hierarchy, render-mode controls, prim creation, undo and redo stayed beside the streamed scene, while the server remained the only owner of rendering and USD writes.

Architecture

The Python server is the only thing that touches the USD stage and the only thing that renders. The browser is pure UI: it sends commands over REST (load scene, pick a pixel, edit a prim) and receives rendered frames plus server events over WebRTC. Nothing runs client-side except the React app.

Where I actually got stuck

The single biggest architectural cost in the version I built was that pxr and ovrtx held separate stages. pxr opened the USD file on disk; ovrtx kept its own in-memory copy for rendering. Every edit had to go through a file round-trip:

# pxr writes to its own copy of the stage, not ovrtx's
stage = pxr.Usd.Stage.Open("/path/to/scene.usda")
prim = stage.GetPrimAtPath("/World/Table")
prim.GetAttribute("xformOp:translate").Set(Gf.Vec3f(1.0, 0.0, 0.0))
stage.GetRootLayer().Save()

# ovrtx has no idea anything changed until you force it to reload
renderer.open_inline_root(usda_string)   # ~2 second blink in the stream

That reload is a visible black flash on every single edit: drag a transform slider, see the stream blink. Session-layer authoring (the create/deactivate/undo-redo feature) makes this worse: since there’s no proper session-layer API available outside Kit, I ended up hand-generating USDA strings and injecting them as a sublayer. That’s fragile in a specific, annoying way: a single formatting error in the generated string doesn’t throw, it just silently drops the edit. The telemetry feature made this sharper still: baking 720 frames of animation into a timeSamples block is mostly string templating, and a missing comma anywhere in those 720 entries quietly kills the animation with no error at all. I lost real time to exactly this: the fix was writing a small validator that re-parses the generated USDA before injecting it, specifically so a bad template fails loudly instead of just not animating.

A Kit app avoided the same costs at the time:

	pxr (what I used)	Shared-stage model (Kit in the version I tested)
Stage model	Separate copy from the renderer	One shared object: Kit, the renderer, and your extension all hold the same pxr.Usd.Stage
Edit → visible	Save to disk, then force a full reload (~2s blink)	Next frame, no save, no reload
Session layer	Hand-built USDA string, injected as a sublayer	stage.GetSessionLayer(), a real API
timeSamples write	String templating, silently breaks on a formatting error	Direct attr.Set(value, time_code) call
Failure mode	Silent: a malformed string is just skipped	Typed exception at the call site
Available in the build I tested	✅ Yes, standalone, no Kit needed	✅ Yes, but only inside a full Kit app

Kit got this for free because omni.usd.get_context().get_stage() handed my code the same Python object the renderer was already holding. There was nothing to synchronize. The standalone libraries did not yet give me an equivalent stage owner, so I built the file round-trip above.

What changed since I built it

That missing layer has now landed as pre-release software. Starting with ovrtx 0.4, the renderer integrates with ovstage, a standalone shared scene substrate for runtime data and changes. The application owns the ovstage instance and decides when each library reads or writes it. The renderer-owned stage APIs I used have been deprecated in favor of that shared-stage path.

So the two-stage architecture above is still an honest record of the build, but it is no longer the architecture I would choose now. If I rebuilt it, the application would own one ovstage instance, ovrtx would render from it, and ovstream would deliver the result to the browser. That is much closer to the clean separation I wanted in the first place.

I have not rerun this whole application on the new path yet. I therefore cannot claim that every black flash, session-layer workaround, or reconnect problem disappears. What I can say is narrower and useful: the main architectural limitation I found is no longer a missing public library.

The other mistakes were smaller but cost real debugging time:

• Two ovrtx renderers on one GPU deadlock. The CUDA driver serializes access and neither process can proceed. I hit this by accident restarting the server without killing the old process first. Now step one of every restart is confirming the old process is actually dead.
• Cold start burns about 90 seconds compiling shaders the first time ovrtx renders a frame. Warm restarts are ~15 seconds. Not a bug, just something you have to know to wait for before assuming the server is broken.
• WebRTC reconnect after a disconnect can come back as a black frame: the underlying stream handle corrupts and doesn’t recover on its own. Fix is a server restart plus a hard client refresh, and nothing fixable from the browser side.
• On Python 3.13, UsdTokens isn’t exported from pxr anymore. Cost me an import error before I found UsdGeom.Tokens as the replacement.

What held up

None of the above is a knock on the pattern. It is what the stack looked like when I tested it, and every item came from a real debugging session. The thing I set out to test did hold up: I built a fully interactive editing tool in a few hours, with no Kit install anywhere in the stack. The dual-stage design was the largest cost in that build; ovstage means it should not be treated as the permanent cost of going library-first.

Code, full setup instructions, and the complete REST API reference: github.com/pr9868/omniverse-realtime-viewer. Status: working, and still my own testbed for this stack.

Disclaimer: The views and opinions expressed in this account are those of my own and do not represent those of my employer, NVIDIA.