Checkpointing a long agent task: progress files, idempotent steps and resumption

Goal

Make any interruption of a long task cost at most one step of work and zero duplicated side effects, whether the interruption is a process crash, a context window that had to be reset, or a hand-over to another agent.

Prerequisites

A task that can be written as a sequence of steps with observable completion criteria, a place to write files that survives the interruption, and external systems whose write operations either accept an idempotency key or can be queried for what already happened.

Steps

1. Before the first side effect, write a plan file: numbered steps, the completion check for each ("file exists and parses", "record 4711 has status sent", "test suite green"), and the inputs the steps depend on.
2. Keep a progress file next to it and update it after each completed step, not before: step number, timestamp, the evidence of completion (a hash, an identifier, a command's exit status) and anything the next step needs that was learned in this one.
3. Give every external write an idempotency key derived from the task and the step, so that a repeated request after a crash is recognised by the receiver instead of executed twice; where the receiver has no such mechanism, query for the effect first and skip the write if it is present.
4. Make each step restartable in isolation: it reads its inputs from files or identifiers, never from the conversation, and it tolerates its own partial output from an earlier attempt (overwrite or clean up, never append blindly).
5. On resumption, read the plan and the progress file, verify the last recorded step's evidence against the real state (the record really is sent, the file really parses), and continue from the first unverified step. Never trust the progress file alone; verify, then proceed.
6. Record decisions taken along the way in the progress file with their reason, so a resumed run does not reopen them, and record open unknowns explicitly so they are not mistaken for settled matters.
7. At the end, write the final summary from the progress file, which by then contains every step with evidence; then archive both files with the run's identifier.

Expected result

A run that is interrupted at any point can be resumed by the same or another agent from the files alone, repeats at most the step that was in flight, and produces no duplicate external effect.

Limits and test basis

A proposed protocol. It assumes steps with checkable completion; work whose only artefact is understanding (reading, exploring) needs a written note as its evidence, which is a weaker check. Idempotency depends on the receiving system honouring the key.

Unqueryable effects: record the intent first

For a step whose effect cannot be queried afterwards (a sent message, a webhook delivered to a third party, a payment captured without a lookup endpoint), reverse the order of step 2: write an intent record with the reserved idempotency key before the effect, perform the effect, then write the completion record. A resumed run that finds an intent without a completion knows the effect may or may not have happened and must ask the receiver with the same key, or hand the step to a person, rather than repeat it. Write both records atomically (temporary file, then rename) so that a crash during the write cannot leave a half-written progress file.