The keyword index behind the site worked well when people typed a product name and returned nothing when they didn't. Real queries aren't product names — they're phrases (what do I need for a car loan) and typos, in two languages. Rather than replace the index, this adds an LLM query-understanding layer in front of it and a grounded answer card above the results.

query ───▶ rewrite (LLM) ───▶ keyword index ───▶ ranked results     blocking · fast path
            │                                       │
            ├─ single token → spelling correction   │
            └─ phrase       → keyword extraction    │
                                                    ▼
                            top-2 result pages ──▶ LLM ──▶ answer card     cached, non-blocking

Decisions worth defending

Two prompts, not one clever one. Correcting a misspelled term and reading intent out of a sentence are different tasks that fail in different directions — a spelling prompt handed a question invents a product, an extraction prompt handed one typo'd word “corrects” it into a different product. A deterministic check on the shape of the query — not a model call — routes it to one or the other, so each prompt has a single job and a checkable output:

// one dispatcher, two single-purpose prompts, one fallback contract
const rewrite = async (query, lang) => {
  const task = query.trim().includes(' ') ? EXTRACTION : SPELLING;
  try {
    const out = await callModel(promptFor(task, query, lang), MODEL_OPTS);
    return out?.trim() || query;   // empty or unusable → raw query
  } catch (err) {
    log(err);
    return query;                  // model failure → pre-AI search, never an error
  }
};

Two endpoints, not one. The rewrite is cheap and on the critical path; summarization needs to fetch and read the result pages, so it can't be. Results are a blocking GET; the summary is a second POST fired after results paint, into a skeleton placeholder so nothing reflows. The expensive call cannot regress the latency of the list.

Failure degrades to the old behavior, never to an error. Every model call is wrapped so its fallback value is the user's raw query — a provider outage silently returns the pre-AI keyword search. The summary returns null and the card simply doesn't render. The whole layer is gated on a config key plus a request parameter, so it can be switched off without a deploy.

The cheapest model tier, on purpose. The rewrite is a classification-shaped task whose answer is a single term, and the summary is extractive — neither rewards a frontier model, and both are sensitive to the latency and cost one adds. Picking the smallest tier was a decision the design then had to earn: because the output contract is a bounded token count and a small fixed object, the tier is a config line rather than an architectural commitment, revisitable per call site if quality ever demands it.

Cache on the rewritten keyword, not the raw query. This is the highest-leverage line in the feature: hundreds of phrasings and misspellings collapse onto one (language, keyword) cache entry, so the long tail hits cache instead of paying for page fetches and generation. The TTL is set against the content-release cadence rather than picked for cost, which makes it a staleness budget worth naming on a bank surface. Two things bound it. Only the generated summary is cached — the ranked results are not, so the list stays current even when the card is behind it — and the card renders directly above those results and cites the pages it read, so the live page is always one click beneath the summary of it. Responses also carry whether the rewrite ran and whether the summary came from cache, so AI-path share and hit rate are readable from ordinary request logs instead of needing their own instrumentation.

Constrain the model by construction, not by asking. Deterministic decoding, and an output cap sized for a term rather than a sentence: sixteen tokens for a rewrite, a few hundred for a summary. The cap bounds cost and truncates runaway output before it reaches the index; it doesn't make a sentence impossible, so the caller still normalizes what comes back and falls through to the raw query when the result is empty. Separate connect and request timeouts on every call: a third party on the critical path never gets an unbounded await. JSON responses are fence-stripped before parsing, because models wrap output in markdown code fences regardless of instructions, and validated before reaching the client.

Reading the site the way a crawler does. The pages being summarized belong to a single-page app, so fetching one server-side returns an empty shell — the text worth summarizing doesn't exist until a client renders it. The fetch therefore reuses the same pre-render path the site already exposes to search-engine bots, in parallel across the selected pages and under a hard timeout. Only the document body is extracted before the HTML reaches the prompt, which drops the scripts and metadata that would otherwise dominate the token count.

Grounding over free-form answering. On a bank surface, an invented answer is the failure mode that matters. The summary is generated only from the HTML of the top two ranked pages the index already returned, fetched server-side at generation time, and it renders only if the model produced both fields of the expected contract. It sits directly above the real results, so the source is one click away. Provider safety thresholds are set explicitly rather than left at defaults.

Partial degradation across independent sources. A result page draws on more than one upstream index plus an editorial exclusion list, and they don't share a failure domain. They're issued concurrently rather than awaited in sequence, and the non-essential ones resolve to an empty set on failure instead of rejecting: one source being down narrows the results, it doesn't produce a blank page. Exactly one upstream is a hard dependency, which is a deliberate choice about which failure the user should be allowed to see.

Dedupe on normalized identity, not on rendered text. The same page legitimately arrives from more than one source, and the index returns titles with the matched span wrapped in highlight markup — so the naive comparison never matches, because the markup itself varies with the query. Identity is the URL with trailing slash and case normalized, plus the title with highlight markup stripped; the visible, highlighted title is left untouched for rendering. Small function, and the difference between a clean result list and the same product listed twice.

Bilingual output is enforced, not hoped for. Models drift to English on non-English input. Target language is pinned in the prompt and is part of the cache key, so the two locales can never serve each other's text.

Client-side race correctness. A search-as-you-type box with a slow second request is a race generator. Input is debounced at half a second behind a three-character minimum, each request carries a monotonic id, and in-flight requests are aborted on supersede — for the results and the summary independently. A stale response can't overwrite a newer one, which is the bug users report as “it showed me the wrong results for a second.”

this._requestId = (this._requestId || 0) + 1;
const requestId = this._requestId;
this._pending?.cancel();                      // abort the in-flight request

this._pending = this.apiCall(searchUrl(term));
this._pending.then((res) => {
  if (requestId !== this._requestId) return;  // superseded → drop the response
  render(res);
});

The same guard doubles as the cheapest cost control in the feature: the minimum query length and the debounce window mean the overwhelming majority of keystrokes never become a request at all, so the model only ever sees queries a user paused on.

One component, two consumers. The search logic — fetch, debounce, race handling, state — lives in a base Lit element shared through a Module Federation remote; the header modal and the full results page extend it. Summarization hangs off a single _onProductsLoaded hook the subclass may implement, so the lightweight modal never pays for it and there is no if (isModal) branching in shared code.

Impact

• •Questions, phrasings and misspellings resolve to indexed terms instead of an empty-state page.
• •The pre-AI keyword path is the floor, not the fallback of last resort: no query can return worse results than it did before, and the layer can be turned off from config.
• •Cost per search is bounded by construction — a capped-length rewrite and a cache keyed on the normalized keyword, so repeat traffic never reaches the provider.
• •Popular queries serve their answer card from cache, so the expensive path is paid once per keyword per TTL window rather than once per search.
• •The answer card is generated only from pages the index already ranked and cites, so every claim on it has a source one click away.