Search

Search is part of Cusp core, not an add-on. It works on first boot: no plugin to install, no external service to sign up for, no extra binding to provision. The index lives in the same D1 database as the rest of your content, so search stays inside the one-Worker, one-database model that defines Cusp.

How the index works

Cusp maintains a single full-text index using SQLite's FTS5 extension, which is built into the SQLite that D1 already runs. The index is a virtual table named search_index, populated by migration 0015.

The index is a derived projection of your content, not a versioned entity. It carries no version history and never spills to R2, because rebuilding it from the document corpus is cheap and the real history lives in documents and their revisions. If the table is ever dropped or corrupted, the rebuild workflow regenerates it from scratch.

CREATE VIRTUAL TABLE IF NOT EXISTS search_index USING fts5 (
  document_id      UNINDEXED,
  collection_slug  UNINDEXED,
  title,
  body,
  status           UNINDEXED,
  updated_at       UNINDEXED,
  tokenize = "unicode61 remove_diacritics 2",
  prefix = "2 3"
);

• Tokenizer — unicode61 remove_diacritics 2 folds accented Latin scripts so a search for "creme" matches "crème". Typo tolerance and semantic ranking are deliberate non-goals.
• Prefix index — prefix = "2 3" builds prefix-token indexes for 2- and 3-character prefixes. The query helper appends a trailing * to each token, which powers the admin command palette's type-as-you-go behaviour without a second engine.
• Ranking — results are ordered by BM25 with the title weighted 4x the body (bm25(search_index, 0, 0, 4, 1)), applied at query time rather than stored.
• Body — the body column is a flattened string of the document's block content only, built by extractSearchableText. Per-field data is deliberately excluded so the anonymous public search endpoint never exposes the text of custom fields a theme does not render (ADR 0058). Only the title and body columns are tokenised; everything else is UNINDEXED metadata.

The schema carries a status column ('published' or 'draft') so the same table can serve both surfaces. In practice the index holds published documents only: the publish workflow writes rows with status 'published', and the rebuild workflow indexes the published corpus. Draft indexing is not wired up today, so the admin palette returns published documents — the status-aware query path is in place for when draft indexing lands.

The admin command palette

Press Cmd+K (or Ctrl+K) from any admin page to open the command palette. As you type, the palette's documents scope queries GET /api/admin/search and shows matching documents so you can jump straight to the editor. The client debounces keystrokes and the prefix index keeps responses fast enough to feel live as you type. The endpoint caps results at 20 rows and sets Cache-Control: private, no-store so a shared CDN never serves another admin's session.

The palette is gated to administrators, editors, and authors; subscribers cannot reach it. Per-document permissions — for example, limiting an author to their own documents — are a known gap; the role gate matches the rest of the admin document routes.

Public search

External clients call GET /api/v1/search, which returns only published documents and requires no authentication — it is mounted outside the bearer-token gate. Each result carries a BM25 rank and an FTS5 snippet with the matching terms wrapped in <mark> tags.

curl 'https://your-site.example/api/v1/search?q=spiced+buns&collection=recipes&perPage=20'

• Query — the q parameter is required and must be 2 to 100 characters; anything outside that range returns a 400.
• Collections — filter by collection slug; repeat the collection parameter to query several collections at once (up to 20).
• Pagination — page defaults to 1; perPage defaults to 20 and is capped at 50, lower than the standard list cap because search is heavier per request. meta.total carries the full result count.
• Caching — responses set Cache-Control: public, max-age=60, stale-while-revalidate=600 and a Surrogate-Key listing the search family plus a listing:{collection} key for each collection queried, so the next publish in a collection invalidates stale result pages.
• Rate limit — 60 requests per minute per IP, enforced through the RateLimit Durable Object; over the limit returns 429 with Retry-After.

Raw query input is never passed straight into the FTS5 MATCH clause. The query helper keeps only word tokens (letters, numbers, underscore, hyphen), caps them at 16, joins them with an implicit AND, and appends a trailing prefix marker to each so partial words still match. That closes the FTS5 syntax-injection surface that parentheses, quotes, and column operators would otherwise open.

Keeping the index current

Because search is core, a failed index write is a defect that should be retried and surfaced — not silently swallowed the way a plugin hook would be. So the index write lives inside the publish and unpublish workflows as an ordinary, independently retryable step.

• Publish — after the document goes live, the PublishDocument workflow upserts the row into search_index (a DELETE-then-INSERT, since FTS5 has no ON CONFLICT). The step runs after the page is published, so a transient index failure never blocks the page from going live; the workflow's per-step retry budget covers the rest.
• Unpublish — the UnpublishDocument workflow deletes the document's row, removing it from public results.
• Delete — deleting a published document leaves its row behind until the next rebuild reaps it; the row is harmless but the document is still findable until then. Unpublish first if you need it gone from search immediately.

The recovery action for a stale or damaged index is a single button on the admin Health page: Rebuild search index. It runs the RebuildSearchIndex workflow, which clears the table and re-indexes every published document in chunks of 50. Search results are unavailable for the duration of the rebuild — a brief outage is accepted over a partially built index that lies about result counts. The same workflow runs once after a fresh deploy adds the table, and no-ops if the index is already populated.

Search is part of core, served by a single D1 FTS5 table, rather than a plugin. Requiring a plugin install to get keyword search is friction against Cusp's editor-first promise, and the index write needs to be a retryable, observable workflow step rather than a fire-and-forget hook. Semantic and vector search remain plugin territory; keyword search is too core to leave one indirection away.

ADR 0045 (search in core via D1 FTS5), ADR 0046 (index writes in workflows)