Checking Motion in Agent-Generated 3D

A two-second animation passed in 0.083 seconds because the contract checked time codes

The brief gave the panel two seconds to move one metre. My first motion contract accepted a version that finished in about 0.083 seconds.

The constructed test changed only the USD clock rate. Every sampled position still matched because the requirements used time codes, the timeline’s numeric positions, without enforcing elapsed duration.

I added a separate group of checks, motion.timing, for duration and positions in seconds. It rejects that clock-only change while accepting a properly rescaled animation. The original checks and their contracts keep their meaning; the fix is available in a pinned public development revision, with commands below.

The constructed panel tests also expose a wrong midpoint and an excursion between passing samples. They are separate from the Astra assembly demonstration, where the producer caught an export problem during its work.

Diagram of the retained fixtures and results. Dots mark the saved keys; each scene moves the panel from X = 0 to X = 1 metre. Changing only the clock compresses the elapsed time. The timing contract rejects that duration before checking positions, so its position result remains unresolved. These selected checks do not establish the whole path.

Define the position requirement

The fixture contains a triangular panel at /World/Panel. Its origin should move along X from zero to one metre over two seconds. The original contract checks three positions: zero at time code 0, half a metre at time code 1 and one metre at time code 2.

The stage declares one time code per second, so time codes 0, 1 and 2 correspond to seconds in this fixture. That is not a universal USD convention. OpenUSD’s stage API exposes the timing metadata and interpolation setting. The pack records the rate but does not require a particular value. In the clock-rate test below, changing only that rate still passes the original contract.

A passing time code is not a duration check

During an adversarial review, the coding assistant copied the passing fixture and changed only timeCodesPerSecond from 1 to 24. The positions, sample times and contract stayed unchanged. Both evaluations returned ACCEPT_FOR_USE.

Scroll sideways for more columns.

Saved scene	Time-code rate	Duration from time code 0 to 2	Recorded decision
Original passing fixture	1 per second	2 seconds	Accept.
Constructed clock-rate change	24 per second	About 0.083 seconds	Accept.

The samples still satisfy the declared contract. The second scene fails the stronger brief of taking two seconds to move the panel. This is a gap in the acceptance definition, not evidence that USD evaluated time incorrectly or that an agent made this mistake.

To reproduce the change, follow the core article’s installation steps for v0.3.0. From that extracted folder with its environment active, create a new temporary copy:

python - <<'PY'
from pathlib import Path
import shutil

source = Path("evaluation/packs-v1/fixtures/motion_correct")
target = Path("/tmp/motion-clock-01")
shutil.copytree(source, target)
scene = target / "scene.usda"
text = scene.read_text()
assert text.count("timeCodesPerSecond = 1\n") == 1
scene.write_text(text.replace("timeCodesPerSecond = 1\n",
                              "timeCodesPerSecond = 24\n"))
PY

check-3d --bundle-root /tmp/motion-clock-01 \
  --contract contract.json --candidate scene.usda \
  --out /tmp/motion-clock-result-01

For a brief stated in seconds, I need a checked time-code rate and a duration requirement. The original motion 1.0.0 pack exposes neither as a parameter, so copying its sample contract alone cannot establish them. Its report contains the changed rate, but reporting it does not enforce the two-second brief.

The fix: check duration and elapsed seconds

I added motion.timing as a separately selected pack in the 0.4 development revision. The original time-code examples above remain unchanged. The new pack has a clock check for the authored stage duration and an optional exact rate, plus a positions check whose samples use elapsed seconds from the authored stage start.

For this brief, I require two seconds and the origins at zero, one and two elapsed seconds. I leave the choice of clock rate to the producer. A rate of 24 is acceptable when its time range and keyframes are rescaled together; changing only the rate still fails the brief.

Scroll sideways for more columns.

Submitted motion	New timing result	Reason
Original two-second animation	Accept.	Duration and sampled origins match.
Clock changed to 24; keyframes unchanged	Reject.	The stage spans about 0.083 seconds.
Clock and keyframes rescaled to 24	Accept.	The same requested motion spans two seconds.
Stage extended to two seconds; early keyframes retained	Reject.	Duration passes, but the panel is already at the end at one second.

The last case tests the fix itself. A declared stage duration does not tell me how long an object actively moves. I need the position requirement too. Missing or unusable clock metadata stays unresolved, and these samples still do not prove the whole path.

Regression tests for release 0.5 corrected three edge cases: an authored framesPerSecond can supply USD’s effective rate; floating-point rounding at the final time must not put a valid sample outside playback; and valid keys before or after playback must not fail merely for being there. A wholly implicit clock remains unresolved under this delivery policy. These compatibility changes are separate from the pinned replay below.

To try the fix in a separate checkout and environment, use the exact tested development commit:

git clone https://github.com/pr9868/scene-acceptance.git scene-acceptance-timing
cd scene-acceptance-timing
git checkout 33e57211a1ddd2f624be07b8eeae9cfbb1cd5b6d
python3.12 -m venv .venv
source .venv/bin/activate
python -m pip install -r requirements-test.lock
python -m pip install --no-deps --no-build-isolation .

check-3d --bundle-root evaluation/motion-timing-v1/fixtures/clock_rate_changed \
  --contract contract.json --candidate scene.usda --out /tmp/timing-rate-01

check-3d --bundle-root evaluation/motion-timing-v1/fixtures/rescaled_24 \
  --contract contract.json --candidate scene.usda --out /tmp/timing-rescaled-01

The first evaluation rejects; the second accepts. The complete contract and pack settings show the duration_s, tolerance_s and elapsed_s fields. The application can add an exact time_codes_per_second requirement when its consuming tool needs one.

All 17 timing controls matched their expected decisions. The 29 timing software tests also cover shifted start times, fractional clock rates, units and absent targets; all 224 software tests passed. The old 22 pack cases and 32 mesh cases retained their decisions. These counts overlap and describe evaluator development tests, not agent performance.

Test the gap between samples

Checking duration and selected positions still leaves the motion between those positions open. A separate probe of the original contract makes that gap visible. Its brief requires uniform translation. The candidate has the correct X values at time codes 0, 1 and 2, but adds a keyframe at 0.5 with X = 0.90 m. Uniform translation would put it at 0.25 m.

Two evaluations use identical scene bytes:

Scroll sideways for more columns.

Probe	Required times	Observed decision
`between_samples`	0, 1, 2	`ACCEPT_FOR_USE`; the extra excursion is unexamined.
`inspect_added_time`	0, 0.5, 1, 2	`REJECT`; the half-second position differs by 650 mm.

A second USD trace reaches x 0.9 metres at half a second, then returns to x 0.5 at one second before ending at x 1. The checks at zero, one and two seconds all pass. A newly required half-second check finds a 0.65 metre discrepancy. — The second probe passes at every time in the three-sample contract. The extra check finds a 650 mm discrepancy. Increasing the number of samples improved detection for this known case; it did not establish continuous coverage.

To reproduce this probe after trying the timing fix, return to the original extracted v0.3.0 folder and reactivate its .venv. Install that project’s pinned experiment dependencies and run:

python -m pip install -r requirements-articles.txt
python article-evidence/run-content-probes.py \
  /tmp/content-probes-01

This shared runner executes the texture probes from the materials article as well. Inspect summary.json and the individual motion reports under the new output directory. The plot uses positions read back from the saved USD, rather than an illustration of how the animation was expected to behave.

For this piecewise-linear translation, a specialized check could inspect every authored sample and compare each linear segment with the requested line. The current pack does not implement that proof. Rotating parts and deforming objects need a different argument; a passing origin does not establish that the occupied geometry stays clear.

The assembly’s motion required more than a clock check

The panel controls isolate translation and timing. In the recorded assembly task, rotation produced a different failure between the planned inspection times.

Playback of the final delivered GLB. The crank rotates and the slider follows its guide. The corrected delivery is shown here; the intermediate export failure is retained in the producer's logs.

The assembly brief required one crank cycle in four seconds while the linkage stayed connected. Its harness report passed a four-second duration check through motion.timing. The coordinator separately checked the pin relationships and phase at nine elapsed times. Neither result establishes every pose between those times.

The producer’s initial failure log records nearly 70 mm of rod-to-pin separation caused by angle interpolation between frames. The demonstration note records its correction and denser readback. That check was the producer’s, outside the harness. A reusable assembly profile needs both elapsed-time requirements and linkage measurements at relevant subframes; a duration pass alone cannot establish that the rod stays connected.

The original position check is still useful

The v0.3 sample contract checks the panel’s world origin at explicit time codes. OpenUSD evaluates each composed transform, including the parents, and the pack converts its translation to metres. A coordinate difference beyond the supplied tolerance fails the sample. The one-micrometre tolerance is for an exact synthetic test, not a recommendation for measured geometry.

Play or pause the replay. The solid panel follows the saved USD motion; the dashed outline follows the reference path. At one second, the bad panel is 300 mm ahead. Both panels still finish correctly. The required midpoint check catches the error. Playback is slowed two times, with holds at the start and end; this is a projection of saved transforms, not a simulation.

The supplied motion_wrong_midpoint fixture puts the origin at X = 0.8 m instead of 0.5 m at time code 1. Its report identifies /World/Panel, the time, and the expected and observed positions; the 300 mm error follows from those positions. The companion endpoints_miss_midpoint contract accepts that same motion because it selects only the start and end. Both are useful controls when adapting the public harness example.

A fuller brief changes which motion can be judged

The later 24-scene study makes the specification boundary visible. All six medium-brief deliveries met their supplied endpoints. Five missed the fuller path used only during assessment. The producer had not received that path, so those discrepancies cannot be counted as ignored instructions.

All six detailed deliveries received 21 path checkpoints and passed them. They also passed a later 201-position scan. This cohort produced no sparse-pass/dense-fail example; the constructed excursion above remains the evidence for that particular sampling weakness.

What the pack evaluates

The implementation uses UsdGeom.XformCache at each requested Usd.TimeCode. USD can evaluate an interpolated value even when that exact time is not an authored keyframe.

Scroll sideways for more columns.

Evaluation	What it captures	Result when the requirement cannot be met
Named object	The prim exists and supports a transform.	Missing or unsuitable object: `FAIL`.
Units	An authored, finite, positive metres-per-unit value.	Missing or unusable units: `UNKNOWN`.
Time coverage	An authored time range containing every requested time.	Missing range or request outside it: `UNKNOWN`.
Sample definition	Times are strictly increasing and parameters follow the schema.	Invalid contract: `ERROR`.
World position	X, Y and Z compared at every selected time, including parent transforms.	A coordinate outside tolerance: `FAIL`.
Usable measurement	Evaluated positions contain finite values.	Non-finite observation: `UNKNOWN`.

A required failed comparison prevents acceptance. An unresolved required check also prevents acceptance, but it tells the caller to obtain usable evidence instead of treating an unmeasured position as a known violation. The core article explains how the overall decisions are assembled.

Position tolerance applies separately to X, Y and Z. A 1 mm allowance on each axis can admit a position about 1.73 mm from the required point. The connection check instead compares straight-line distance and records which moving attachment points and times it inspected. A brief that says only “within 1 mm” needs that metric clarified before either test can enforce it.

The report records the checked times, timing metadata and interpolation setting. It does not evaluate orientation, deformation, the space swept by the object or physical feasibility. A rotating door can keep its origin fixed while sweeping through nearby equipment; every position sample could pass that case.

Leave the clock choice open when the brief allows it

For a new animation, I would keep four timing controls: the intended motion, a clock-only change, an equivalent rescaling, and an animation that finishes early but leaves a longer stage range. Together they check that acceptance enforces the requested duration and motion while allowing a different representation.

The coding assistant constructed the fixtures, implemented the evaluator and assessed the results under my direction. These controls expose specific contract gaps; they are not a detection rate for arbitrary animation.

Requiring 24 time codes per second would have been an easy response to the first failure. It would also have rejected a valid one-code-per-second delivery. Checking elapsed seconds lets the producer choose either clock while keeping the two-second requirement fixed.

Disclaimer: The views and opinions expressed in this account are those of my own and do not represent those of my employer, NVIDIA.