A 3D Scene Can Pass Its Checks and Still Miss the Brief

In a constructed 3D packaging-cell example, the request was to move a pallet one metre forward while keeping the front aisle clear. With the pallet’s existing dimensions, both conditions could not be met: the requested position put it 300 mm into the aisle. A clear alternative was possible, but it changed the requested destination.

The unchanged scene and a different, unrequested placement nevertheless passed all eight geometry checks. The checker verified dimensions, the reserved aisle and unchanged equipment. It never asked whether the pallet had reached the destination in the brief.

I asked for an adversarial review of the example. The coding agent inspected the checker, recorded expected outcomes and ran four controlled cases. That exposed two separate acceptance questions: did the requested edit happen, and could it happen without violating a constraint that still applied?

In the delegation article, I argued for giving an agent room to choose its production method. This example shows where that freedom needs a boundary: the agent can propose a different destination when requirements conflict, but a passing layout report cannot approve that change to the brief.

The layout and its conflicting request

The example is an independently designed carton-packaging cell, occupying a 9 by 5 metre rectangle. Conveyors connect a sealer and labeler, with a pallet beside the line. A 1.2 metre strip along the front is reserved as an aisle. These are illustrative dimensions, not a survey of a plant or a recommended clearance standard.

My direction was the lifecycle question and the need for a separate example. The coding agent chose the layout and dimensions, wrote the generator and checker, and later carried out the adversarial review. The result is available to inspect and reproduce; it has not had independent engineering validation.

The pallet is 1.00 metre deep. Its baseline centre is at Y = 2.40 metres, so its front edge is at Y = 1.90 metres. The constructed request moves its centre one metre forward, to Y = 1.40 metres. Its front edge would then be at Y = 0.90 metres, inside the reserved strip by 300 mm.

There is no placement with that exact centre and depth whose front edge also stays outside the aisle. I would have to resolve the conflict before accepting the work. The proposed centre at Y = 1.75 metres restores the clearance, but changes the specified destination.

What the eight checks answered

The original checker used OpenUSD to reopen the stages and calculate world-space bounds. It checked metre units, Z-up, equipment identities, containment within the cell, aisle clearance, pallet dimensions and conveyor surface height. The eighth check compared non-pallet cube bounds and display colors with the baseline.

These checks went beyond whether the file could open, but covered only the listed scene properties. They did not certify the scene generally or establish that arbitrary topology, materials or behavior remained unchanged. There is no physics or operational connection in these files.

The earlier SplatStage export problem explains why I care about reopening the artifact the next tool will consume. This review asks a further question: once I have the correct saved file, am I checking what the user requested?

Four cases against the unchanged checker

The review copied the baseline, requested move and proposed alternative. It added one deliberately wrong but clear placement at Y = 1.90 metres. Expected outcomes were recorded after inspecting the code and before running the cases, making this a targeted test of an omission rather than a blind benchmark.

The unchanged scene and the different clear placement both passed. Neither delivered either destination. The exact original request reached its destination but violated the standing aisle constraint. No row satisfied both parts of that original brief.

For a second test contract, the review explicitly made Y = 1.75 metres the target. Only the proposed alternative then met both the original geometry requirements and the destination check. That was an explicit test input, not a claim that a plant owner had approved the alternative.

The runner reused the original inspection function and repeated its comparison of non-pallet cube bounds and colors. The three copied cases matched the original saved hashes and all eight reported outcomes. It also calculated each deck centre from transformed cube corners; those values agreed with bounds readback. Both calculations use OpenUSD transforms, so their agreement is a numerical cross-check rather than independent validation of the requirements.

Make the change itself part of acceptance

For these axis-aligned proxies, the missing comparison is small:

# Read from the reopened candidate, not the generator's inputs.
centre_y = (deck_min_y + deck_max_y) / 2
reached_target = abs(centre_y - agreed_target_y) <= 1e-6
accepted = all(protected_geometry_checks.values()) and reached_target

The agreed target needs to come from the change record. The agent can propose another destination when it finds a conflict; the original request remains unresolved until that alternative becomes the requirement.

I would keep the starting scene, resulting artifact and checks in that record too, because a passing report from an earlier revision can otherwise be attached to a later change. A filename alone is weak evidence when the file can be overwritten.

The exact-coordinate test fits this brief. A different task might allow a range of positions or ask for the best layout under an objective. Its completion check should express that requirement instead. The numerical tolerance here is also part of the exercise; a real layout needs one appropriate to its measurements and intended use.

The later harness can propose checks from a brief. Its caller still has to review the interpretation. For this request, no interpreter can make the exact destination and the standing aisle constraint compatible.

Evidence and the scope for a harness

The companion evidence bundle contains the brief, checker code, saved stages, four-case protocol and complete results. Its README provides setup instructions for Python 3.12 and usd-core==25.11; reproduce.py verifies file hashes and reruns the checks in temporary copies. The bundle was tested after extraction into a fresh directory. It does not call a model or overwrite the recorded evidence. The code is available in this download; there is no separate public repository for the exercise.

The later Scene Acceptance project on GitHub makes these explicit completion requirements part of a reusable harness. This packaging exercise’s checker and four saved cases remain in the companion download above.

These are constructed review cases. They demonstrate the missing completion criterion, without establishing how often an agent would make a mistake or whether Astra builds scenes faster. The scope is a translation in a simple layout. The corner calculation and checksum checks do not turn it into an accepted plant design.

For the original request, I would keep the result unresolved until the destination or standing constraint is explicitly changed. The reusable harness project can carry that agreed requirement and saved-revision evidence into a common report; it cannot approve the alternative on the owner’s behalf.

The third article examines claims made about a scene after these checks. For the change itself, I would keep two negative controls: the untouched starting scene and a wrong but feasible placement. The first tests whether acceptance requires any edit; the second tests whether it requires the requested one.

Disclaimer: The views and opinions expressed in this account are those of my own and do not represent those of my employer, NVIDIA.