Async work with Workflows

Anything that takes more than a request to finish runs as a Cloudflare Workflow. Publishing a document, scheduling it for next Tuesday, importing a backup, running the nightly export — none block the editor's request, and none are fire-and-forget. They are durable, multi-step jobs that the platform persists, retries step by step, and lets the admin watch to completion.

This is a system-wide choice. Cusp does not use Cloudflare Queues, and it does not poll a cron trigger to find work that is due. Both were rejected in favour of one primitive: the Workflow.

Why a Workflow for every durable job

A Workflow gives Cusp three things a CMS needs: durable execution that survives a Worker eviction, per-step retries with their own budgets, and a status the admin can read back over the Workflows REST API. A queue gives you delivery but not durable multi-step state; a cron trigger gives you a wakeup but no per-instance observability and double the failure surface. The Workflow gives all of it in one model.

Cusp binds twelve Workflow classes in wrangler.jsonc, all registered on the one Worker:

• PublishDocument — verifies the document is still published, renders it to R2, re-renders the listing it appears in, indexes it for search, notifies subscribed plugins, purges the affected surrogate keys, then regenerates the sitemap, feeds, and home through a debounced aggregates step that issues a second conditional purge.
• UnpublishDocument — the mirror of publish, re-rendering the listing and home, regenerating the sitemap and feeds, and purging the broader key set.
• ScheduledPublish — sleeps until the scheduled time, then hands off to the publish pipeline.
• ImportContent — restores a backup tarball from R2, verifying the manifest and per-table checksums, inserting parent rows before children with cursor-resume on retry.
• MigrateField — applies a field-definition change across existing documents in batches.
• ExportBackup — a long-lived instance that self-schedules and wakes nightly to dump D1, tar the media and rendered output, and write a manifest to R2.
• InstallPackage — downloads a theme or plugin package, verifies its signature, writes it to R2, and registers its contributions in D1.
• RestoreVersion — restores the whole site to an earlier point in time, walking every versioned entity to the target timestamp and snapshotting current state first so the restore is itself reversible.
• SeedExampleSite — seeds the worked example so a fresh deployment has content to explore.
• PrunePolicies — a long-lived instance that wakes nightly (04:00 UTC) to enforce per-entity-type version-retention policies, hard-deleting the D1 version rows beyond the configured cap and best-effort deleting each pruned version's R2 spillover blob. There is no archive-before-delete step; the ExportBackup that runs an hour earlier (03:00 UTC) is the protection story.
• SweepMedia — a long-lived instance that wakes nightly (02:00 UTC) to hard-delete media that has been soft-deleted past its retention window, after a reference scan confirms no document still points at it.
• RebuildSearchIndex — replays every published document to rebuild the FTS5 search_index from scratch, run on demand rather than on a schedule. The first-boot enqueue no-ops if the index is already populated; the manual Health-page rebuild passes force and repopulates the index unconditionally.

Steps are independently retryable

Inside a Workflow, each unit of work is a step.do call with its own retry budget and backoff. If a step fails — a renderer bug, an R2 hiccup, a transient D1 error — only that step retries; the steps that already succeeded keep their result and are not re-run. Cusp tunes the budget per step: a render step typically gets three retries with exponential backoff, while a step that would only rebuild the same rejection (an invalid package signature, say) gets a retry limit of zero so it fails fast.

await step.do(
  "render-document",
  { retries: { limit: 3, delay: "2 seconds", backoff: "exponential" } },
  async () => {
    const html = await renderDocument(/* ... */);
    await this.env.BUCKET.put(rendererR2KeyFor(permalink), html);
  },
);

Steps are written to be idempotent: each one writes to a deterministic R2 key derived from the path and reads the live state from D1, so a retry recomputes from current state and overwrites the same key with the correct content. No step holds non-idempotent state in Workflow memory between attempts. That is what makes a mid-flight retry safe.

The publish pipeline as the canonical shape

PublishDocument is the pattern the other Workflows mirror. When an editor clicks publish, the admin handler does the synchronous part — the versioned status flip to published, published_at, and the revision snapshot — and then creates the Workflow instance with env.PUBLISH.create. Instance creation is sub-millisecond, so the editor sees confirmation immediately while the rendering happens behind the response.

A key consequence of that split: if the Workflow fails entirely, the document is still correctly marked published in D1, and the public render handler falls back to live SSR on the next request. The editor never sees a broken state, only a slower first paint until the render step succeeds on retry.

Scheduling with step.sleepUntil — no cron polling

Scheduled publishing is a Workflow, not a polled queue or a cron trigger. When an editor schedules a document, the admin creates a ScheduledPublish instance that calls step.sleepUntil with the target time, then runs the publish hand-off when it wakes. Sleeping instances do not count toward the active-instance concurrency ceiling, so a million documents waiting to publish cost effectively nothing.

await step.sleepUntil("wait-for-scheduled-time", new Date(scheduledAt * 1000));

const stillScheduled = await step.do("verify-still-scheduled", async () => {
  const document = await documentsRepo.findById(this.env.DB, this.env.BUCKET, documentId);
  if (!document) return false;
  if (document.status !== "scheduled") return false;
  return document.scheduledAt === scheduledAt;
});
if (!stillScheduled) return;

Cancellation falls out for free. Re-scheduling creates a new instance with the new time; the old one wakes, sees the document is no longer at its scheduledAt, and exits as a no-op. There is no instance to find and terminate, and no scheduling Durable Object to maintain.

The same self-scheduling pattern drives periodic work. Cusp has no triggers.crons entry and no scheduled() handler on the Worker. Instead, the nightly ExportBackup, PrunePolicies, and SweepMedia run as long-lived Workflow instances bootstrapped at install time. Each does its work, then loops with step.sleepUntil to the next run. A cron trigger would double the failure surface and give no per-instance observability; the long-lived Workflow gives both.

Each Workflow step has its own thirty-minute cap, and step.waitForEvent is available for anything that needs to pause for a human decision. Long waits never busy-loop.

Status surfaced in the admin

Because a Workflow's step state is observable through the Workflows REST API, the admin can show the editor exactly where a job is. The document detail view shows a rendering indicator during publish that flips to published when the final purge step completes. The admin Health page surfaces recent runs of the backup, import, and restore Workflows, the persisted long-lived instance IDs, and the failed-step detail for anything that errored. A failure is visible and retryable, not silent.

Surrogate-key purges are the last step of the publish and unpublish flows, never a global zone purge. Rendering writes the R2 origin in the earlier steps; the purge step then invalidates the edge cache for exactly the affected keys. Doing it in that order means the next request cannot re-cache stale content before the re-render lands.

What is deliberately not a Workflow

Bounded work that the editor expects to be instant stays synchronous. Unpublish flips status, deletes the document's own rendered HTML from R2, and purges its doc-scoped cache key in the request, so the page 404s the moment the response returns; the listing, home, sitemap, and feed regeneration move to the UnpublishDocument Workflow. A settings change purges its caches inline rather than spinning up a Workflow, because it regenerates no derived files. The boundary is the same every time: bounded, must-be-immediate work is synchronous; unbounded, multi-step, retry-worthy work is a Workflow.

Related reading

• ADR 0017 records why unpublish splits an immediate, in-request 404 from the deferred listing and feed regeneration that the UnpublishDocument Workflow handles.
• Search and media covers the two features that RebuildSearchIndex and SweepMedia keep correct: the FTS5 search index and the soft-delete lifecycle for uploaded media.