ADR 0057 — Atomic cursor advance closes the field-migration replay window

Context

The MigrateField workflow (ADR 0024) converts every document in a collection when an editor changes a field's type. Its convertBatch step walks the collection in ULID order, reading the resume cursor from migration_runs.last_processed_id so a retried step picks up where the last one left off. For each document it did two separate writes: first documentsRepo.update committed the converted value and minted an entity_versions snapshot, then migrationRunsRepo.recordBatchProgress advanced the cursor and the converted count.

Cloudflare Workflows steps are at-least-once. A step body can run, have its side effects commit, and still be replayed if the platform does not record the acknowledgement — a Worker eviction or an infrastructure blip in the window between the two writes is enough. When that happened, the conversion committed but the cursor did not. The replay re-fetched the same document, because the cursor never moved past it, and re-ran the conversion rule on a value that was already in the target shape.

That second pass is unsafe. The v1 conversion rules are not idempotent: text to number, called again on the number it already produced, throws MigrationFailValueError and fails the whole run with the field left half-migrated and its declared type never flipped; text to richtext re-wraps its own Portable Text array as a string, silently corrupting the value and minting a duplicate version. The workflow's comments claimed this window was closed. It was not.

Options considered

• Make every conversion rule idempotent — return an already-target-typed value unchanged instead of throwing. Rejected: it demands per-rule, second-pass reasoning across the whole registry, and several pairs are ambiguous on a second pass (boolean to number turns false into 0, and 0 is a valid number-to-boolean source). The risk of silently corrupting data outweighs the saving.
• Detect, before converting, a value already in the target shape and skip it. Rejected for the same reason: a robust check needs the target type's value validator and still cannot disambiguate the overlapping cases above, so it trades a loud failure for a quiet one.
• Make the conversion and the cursor advance atomic. Chosen. It needs no per-rule reasoning and is provably correct on replay: the cursor moves with the data or not at all.

Decision

The per-document conversion and the cursor advance commit in a single D1 batch. The versioned-write helper already runs the live-row UPDATE and the entity_versions snapshot insert in one db.batch (ADR 0007), and a D1 batch is one transaction — every statement commits or the whole batch rolls back. The cursor advance is now a third statement in that same batch.

Two small seams carry it. migrationRunsRepo exposes recordBatchProgressStatement, which builds the cursor-advance UPDATE as a prepared statement without running it. documentsRepo.update takes an optional extraStatements array that it appends to the batch it hands the versioned-write helper. The workflow builds the cursor statement for the document it is about to convert and passes it through update, so the conversion, its snapshot, and the cursor advance to that document's id land as one transaction. migration_runs stays a direct, non-versioned write (ADR 0024) — the change is only that the write now rides inside the conversion's batch rather than following it.

After this change there is no observable state in which a document is converted while the cursor still points before it. A replay re-reads the cursor, the id-greater-than-cursor predicate skips every document that committed, and the conversion rule is never re-invoked on its own output.

Consequences

• Positive — the at-least-once replay window is closed by construction, with no per-rule idempotency to reason about and no path to silent corruption. ADR 0024's promise that a mid-migration restart double-converts nothing now holds in fact, not just in intent.
• Positive — the cost is nil. The cursor advance was already one UPDATE per document; it now shares the batch the conversion already issued, removing a round-trip rather than adding one.
• Neutral — documentsRepo.update gains an optional extraStatements seam. It is generic (run these statements atomically with the document write) and defaults to empty, so every ordinary edit path is untouched. The migration is its only caller in v1.
• Neutral — a migration left half-converted by a pre-fix crash, carried across the exact deploy that ships this change, is not healed retroactively; it fails loudly on resume rather than corrupting data, and is re-run from the admin. The fix is forward-looking, which is the correct scope for a window that only the old two-write path could open.