Last month I published the harness I built after my agent kept skipping steps. The fix was to move the plan out of the model’s context, declare it as a graph, and let code decide which step was ready. The agent still did the work inside each step, but it could no longer choose the order or declare a blocked step ready.
That worked. Anchor’s tests still prove that a failed verification keeps its successor blocked, that a gate holds its output, and that an incomplete run cannot end normally. I have not changed my mind about the central design.
I did change my mind about what that design proved. I had treated deterministic ordering, mechanical checks and a run record as most of the safety boundary. They solve one failure mode—the agent forgetting the plan—while leaving crashes between writes, repeated external effects and competing runs untouched.
Ratchet is the second draft I needed when I stopped asking only whether the agent followed the plan and started asking what a restart would do.
The missing part was restart
The practical dividing line for me was supervision. Most of my early Anchor runs were scratch tests I was watching. If something stopped, I could inspect the files, decide whether a lock looked abandoned, check whether an output had landed, and choose whether to retry; that was acceptable while I was proving the graph pattern, but it was also evidence that I was still part of the runtime.
The recurring dashboard and report work that motivated Anchor has a less tidy failure surface: its sources finish at different times, so one slow read can overlap my manual retry or the next scheduled attempt, a publish call can succeed remotely and die before local state records the result, and one source can advance while another fails. A scheduler can also fail before the runner creates any audit record.
The graph answers which node should run next. Anchor made the work repeatable while I watched it. I wanted interrupted state to be legible enough that a restart did not depend on my memory.
The lock was the first warning
Anchor used a deliberately plain lock file. At the beginning of a run it checked whether the file existed, then wrote the run ID. At the end it removed the file. That was enough to stop me from casually starting the same workflow twice in a scratch project.
Reading the same code as unattended infrastructure produced a different result. The existence check and write were separate operations, so two processes could both pass the check. A crashed run left the lock behind forever because there was no heartbeat or stale-lock recovery. The resume option bypassed the existing lock and replaced it. Ending an older run removed the lock by workflow name without proving that the older run still owned it, so it could release a newer run’s lock.
The graph had not failed. The deterministic code around it had.
I had also been too generous with the word verified. Anchor could evaluate row_count > 0, but the value might still be metadata supplied by the same agent whose work was being checked. I acknowledged that limitation in the original article, then treated independently deriving the metric as a future enhancement. The sharper conclusion is that a runner which cannot observe the post-condition for itself has recorded a self-assessment rather than established completion.
My first Ratchet contract was wrong too
I did not start the rewrite with code. I wrote down what I thought an unattended runner should guarantee, separated from any particular workflow. The first Ratchet contract had four guarantees.
An adversarial review found that three were stronger than their mechanisms. The completion guarantee trusted the agent’s own ok result. The lock could expire, but nothing required a run that lost it to stop writing. The recovery story repeated a failed window without proving that its external actions were safe to repeat.
Those were not wording problems. Fixing them changed the state model. Ordering was only the first guarantee.
That is the change in how I think about deterministic execution now. It is not one property. It is several different questions that must each have a mechanical answer.
The implementation kept correcting the design
Writing the contract first did not make the implementation obvious. I initially chose a monotonic clock for lock heartbeats because it is the right tool for measuring elapsed time inside one process. A heartbeat persisted for another process to read needs a clock both processes can interpret. Monotonic values do not share a reference point across processes or hosts, so Ratchet had to persist wall-clock time and get its real protection elsewhere.
The first atomic-lock correction was incomplete too. Exclusive file creation prevented two creators, but the creator could expose an empty file before writing the lock record. A second process could observe that half-state and diagnose corruption during a healthy acquisition.
The most uncomfortable correction came after I required stale-lock recovery to terminate the old process. A saved process ID does not remain attached to one process forever; the operating system can reuse it. In one run, that logic terminated the shell that had invoked the runner. Ratchet stopped signaling old process IDs. A new owner now takes a higher ownership number, and a displaced run must discover that its number is no longer current before it writes again.
Each of those failures changed the mechanism, but the tests still found a way to flatter me. Four tests were placed after the suite’s entry point and never registered. After I moved them, 39 of 39 checks passed while the code still contradicted the contract in three places: post-conditions were optional, a run that had lost ownership could still write its final diagnosis, and two simultaneous stale-lock recoveries could receive the same ownership number. A heartbeat also ran only between steps, which made one long, healthy call look abandoned.
The lesson was not to accumulate more green checks. It was to make every guarantee survive a test that attacks the mechanism without asking an agent to cooperate. If a guarantee needs the model to behave correctly to pass, it is not a runner guarantee.
What I mean by deterministic now
Anchor taught me that the agent should not hold the plan in its head. That remains the right first move. Ratchet taught me that code owning the plan begins the safety argument; it does not finish it.
A repeatable run also needs an explicit answer for overlap, interruption and ambiguity. Who may write? What happens if ownership changes mid-step? Which progress can commit? Did the external action land? Is absence different from “I cannot tell”? What evidence earns the completion marker? Those answers need to live in state transitions and tests rather than in the operator’s memory.
The technical design and executable evidence live in the separate Ratchet Runtime Build Note.
That narrower realization is the whole second draft: taking order away from the agent stopped one class of failure. Taking recovery decisions away from me was the next step.
Disclaimer: The views and opinions expressed in this account are those of my own and do not represent those of my employer, NVIDIA.