Checking Material Delivery in Agent-Generated 3D

Three false acceptances exposed a missing texture reference in the checker itself

The first pack replay gave three false acceptances: two cases with a missing texture file and one with an undeclared texture dependency. The harness’s reader missed a path authored on a shader attribute because its layer-level dependency call did not enumerate that path. The later checks never got the reference they needed.

The fix explicitly inspected asset-valued attributes before stage composition. The fixtures and expected decisions stayed fixed; all 22 pack cases then matched their expectations. The repository keeps the first reports alongside the corrected ones. The coding assistant implemented and tested that fix under my direction.

That failure changed how I test delivery. I need controls that exercise the path from a shader reference to a local file, as well as checks that the right material reaches the intended surface. This article uses the public v0.3 fixtures, then shows a separate decoding probe that exposed the next gap.

A material can exist without reaching the surface

The controlled scene contains a single triangular panel and two materials, Coating and Other. The requirement is for the panel to use Coating. Binding it to Other fails the check even though Coating is still present.

The same triangular panel with two resolved material bindings. Coating connects the supplied white texture and passes. Other has no texture connection and fails the required binding. The diagnostic preview displays Other in neutral gray. — The panel must use `Coating`. The bad delivery binds `Other`, even though `Coating` still exists in the scene. This diagnostic preview uses the delivered white texture on the left and a neutral display fallback on the right. The harness catches the saved binding mismatch; it does not compare these image pixels.

The check uses OpenUSD’s resolved material-binding API, then resolves the selected surface source. It isn’t a text search for a material name. I would preserve that distinction when an asset uses inherited bindings or several materials, because a material’s presence and the surface that uses it are different facts.

A delivered file can still fail to decode

The next probe changed only the bytes of pixel.png. One bundle contains a valid tiny PNG. The other contains a plain-text sentence with the same filename. Their USD scene hashes and bindings are identical.

Both pass the recorded v0.3 delivery contract. A separate Pillow decode reads the valid PNG and raises UnidentifiedImageError for the text file.

To replay that probe, follow the core installation steps for v0.3.0. From the extracted folder with its environment active, add the pinned article dependencies and run the project’s content-probe runner:

python -m pip install -r requirements-articles.txt
python article-evidence/run-content-probes.py /tmp/content-probes-01

The output summary.json contains valid_texture and unreadable_texture, including the harness verdict and png_decodes. The runner also replays the two temporal probes from the motion article. All saved inputs are checked against their frozen hashes before and after evaluation.

For the unreadable file, the delivery verdict is ACCEPT_FOR_USE while png_decodes is false. Pillow’s decoder already supplies the missing measurement. The v0.3 gap was integrating that result into required acceptance, with supported formats, resource limits and findings tied to the same delivered files. The textures.decode pack in release 0.5 makes that comparison using equivalent passing and failing controls: the saved USD matches, but the newer image files have different bytes.

Decoding still leaves the image’s content and its rendered appearance untested. A readable texture can contain the wrong picture, or appear differently under another renderer, lighting setup or color-management policy.

The saved USD bundle feeds an installed material check using OpenUSD and a separate Pillow decoding probe. In the recorded unreadable-texture example, bindings and file presence satisfy delivery, while decoding fails. A dashed boundary identifies the separate executed probe, whose result is not yet part of the delivery verdict. — Historical v0.3 flow: gray identifies inputs and reused libraries; green identifies project evaluation. The dashed box is the then-separate executed experiment. This workflow diagram uses the recorded unreadable-texture outcomes; it does not depict an integrated decoder pack.

The material check on the Astra assembly

I also applied the binding check to a recorded producer delivery. These results concern the submitted assembly; the faulty panel and texture files above were constructed controls.

The delivered crank-slider rendered from its Blender source, with blue structure, an orange slider, metallic linkage and dark feet. — The assembly rendered from its Blender source. The material pack inspected the saved USD bindings separately; it did not decide from these pixels.

The submitted scene contains seven materials. The assembly’s acceptance contract names four specific surfaces and their expected bindings:

Scroll sideways for more columns.

Surface checked	Expected material in the submitted USD
Blue base	`Structure___cobalt_enamel`
Orange slider carriage	`Slider___signal_orange`
Connecting rod	`Linkage___satin_steel`
One rubber foot	`Feet___dark_rubber`

Those four bindings and their UsdPreviewSurface shader IDs passed in the assembly report. The referenced environment file was present. The profile was mapped to the delivered names after submission, within the same coordinated demonstration; it is not an independent assessment of all seven materials or every surface.

A material name containing “orange” does not prove an orange appearance. That required looking at the actual render and browser delivery. Nor does the metallic linkage shader establish mass or friction. Full USD shader compliance remained unverified because the checker installation could not resolve the standard shader definitions. I would retain those distinctions even when the delivery pack passes.

Give the pack a specific surface requirement

For the panel comparison, the contract requires /World/Looks/Coating to be bound to /World/Panel, with a UsdPreviewSurface surface shader and its referenced texture delivered.

The complete example is material_correct/contract.json in the GitHub project. This is its material check entry inside the checks array:

{
  "id": "appearance.delivery",
  "pack": "materials",
  "check": "delivery",
  "parameters": {
    "bindings": {
      "/World/Panel": "/World/Looks/Coating"
    },
    "purpose": "",
    "render_context": "",
    "surface_shader_id": "UsdPreviewSurface"
  },
  "required": true,
  "after": []
}

The outer contract pins the materials pack to 1.0.0 and declares pixel.png as an allowed dependency. It also selects native USD format checks. The check ID appearance.delivery is a label chosen for this fixture; it does not mean the implementation renders or compares appearance.

The empty purpose and render-context values select the default binding purpose and universal surface context in this example. For another application, I would set those to the context it consumes and keep the expected shader ID explicit. Copy the complete contract before adapting it; the fragment above is one check, not a standalone contract.

Run a correct delivery and a wrong binding

Install the project using the core project’s setup instructions, activate its environment and stay in the extracted folder. The material pack uses the base OpenUSD installation; it needs no renderer or NVIDIA dependency.

check-3d \
  --bundle-root evaluation/packs-v1/fixtures/material_correct \
  --contract contract.json --candidate scene.usda \
  --out /tmp/material-ok-01

check-3d \
  --bundle-root evaluation/packs-v1/fixtures/material_wrong_binding \
  --contract contract.json --candidate scene.usda \
  --out /tmp/material-wrong-01

The first accepts. The second rejects with exit code 2. Open the respective report.html files or read result.json. Each output directory must be new.

The rejection includes this finding; these are selected fields from the recorded result:

{
  "object": "/World/Panel",
  "property": "material:binding",
  "expected": "/World/Looks/Coating",
  "observed": "/World/Looks/Other",
  "status": "FAIL"
}

The finding sits under the check’s evidence.observations.findings. The caller can return the object path and its expected and observed bindings to the producer. The harness doesn’t rebind the surface itself.

What the evaluation captures

The pack compares resolved bindings, surface shader IDs and referenced-file presence. Binding and shader checks use default time. Referenced-file inspection covers authored asset values, including time samples, and can include assets outside the named binding. It does not decode images, inspect their contents, validate UVs or render the scene.

The core also checks dependency admission before these comparisons. An undeclared dependency makes coverage unresolved; the material pack doesn’t get to silently add it to the contract.

The review for release 0.5 found two compatibility gaps. A texture path connected through a material input was being read as empty; the decoder now follows supported static input connections within the admitted bundle. MDL shaders identify their source differently from UsdPreviewSurface; the material check now reads their source asset and subidentifier. A renderer library such as OmniPBR.mdl that cannot be resolved locally stays unknown by default, while an ordinary missing texture still fails. A caller can require the library to be delivered locally. The 0.5 workflow also accepts an explicit caller attestation of availability, tied to the saved scene, exact MDL reference and selected renderer environment. That policy needs approval in a prepared scope, and the original resolver error stays in the report. Neither route proves MDL compilation or rendered appearance; these changes are not in the public v0.3 replay.

These recorded cases show how those conditions become decisions. All five are folders under evaluation/packs-v1/fixtures:

Scroll sideways for more columns.

Case	What changed	Recorded decision
`material_correct`	Required binding, surface shader and referenced file are present.	Accept.
`material_wrong_binding`	Panel resolves to `Other` instead of `Coating`.	Reject.
`material_missing_texture`	The shader still references `pixel.png`, but the file is absent.	Reject, with incomplete dependency evidence also retained.
`undeclared_texture`	A referenced file wasn’t declared in the contract.	Insufficient evidence.
`omitted_material_requirement`	Wrong binding, but no binding requirement selected.	Accept under that weaker contract.

Replace the fixture name in the CLI command to inspect another case. For an accepting application, I would make the binding and delivery check required. An advisory failure remains visible but does not block that contract’s acceptance.

Keep a missing-file control beside the good bundle

For a new viewer, copy the complete contract, name the surfaces and bindings it needs, and select the shader context it actually consumes. Keep a correct bundle, a wrong binding and a referenced file removed from disk. That last case would have caught the reader mistake even though the material code itself looked reasonable.

For textured content I would also require decoding. Release 0.5 integrates it, while the v0.3 replay above leaves it as a separate probe. The texture follow-up shows why a readable image can still miss its reference, and why the comparison region belongs in the result. The current harness article places those source-image checks beside optional rendered review.

The file check should report which dependency is missing, and the binding check should name the surface and expected material. Those are repairs a producer can act on. A general instruction to “fix the appearance” would hide the very distinction these controls uncovered.

Disclaimer: The views and opinions expressed in this account are those of my own and do not represent those of my employer, NVIDIA.