mdmem/SECURITY_ANALYSIS.md

Security, Correctness, and Plan-Drift Analysis

Date: 2026-09-12
Reviewed revision: 4cb9639 (master)
Reference contract: PLAN.md
Scope: src/mdmem, tests, eval, deploy, Containerfile, README.md

Executive summary

The current test suite is green (227 passed) and ruff check src tests passes, but important concurrency, data-integrity, authentication, and recovery behavior remains untested. This review found:

• 2 critical findings
• 12 high-severity findings
• 18 medium-severity findings
• several lower-severity plan and coverage gaps

The two most urgent defects are:

1. lifecycle/version decisions are made before the service write lock is acquired, allowing concurrent requests to resurrect rejected notes, overwrite accepted edits, or recreate a deleted note; and
2. an unterminated frontmatter block is still classified as a new hand-written note and destructively rewritten.

The next tier is dominated by stale embedding races, filename/frontmatter identity split-brain, idempotency races, secret leakage through secondary write channels, and startup behavior that can prevent the documented lexical-only degradation mode.

Severity here measures likely impact to confidentiality, integrity, availability, or the core human-promotion trust boundary. It is not a CVSS score. No fixes were made as part of this review.

Method

The review combined independent passes over storage/retrieval, service/MCP, and UI/deployment/eval, then rechecked findings against the current working tree and PLAN.md. High-impact claims were confirmed with controlled temporary-vault reproductions. No production vault or configured live index was used.

Representative observed results included:

• reject/update race: reject -> archive, then stale update -> inbox; final body was the agent's text and rejected was gone;
• two concurrent identical memory_write calls with one request_id created two distinct notes;
• delayed old embedding and new embedding both used chunk ID 1; the old vector was stored against the new chunk and vec_pending became 0;
• deleting a closing --- renamed and rewrote an active fact, changed its type to note, emptied its tags, and moved the former frontmatter into the body without reporting an error;
• a valid allowed Origin paired with Host: evil.example was accepted;
• signed ID tokens with exp: NaN, exp: Infinity, multiple audiences plus a mismatched azp, and a missing sub were accepted;
• a note written during the scan-to-watcher startup gap remained on disk but absent from the index;
• MCP_TOKEN=..., AWS_SECRET_ACCESS_KEY=..., github_pat_..., glpat-..., npm_..., and hf_... examples all evaded the secret scanner.

Critical findings

C-01 — Lifecycle and optimistic-version checks are outside the serialized operation

Affected code: src/mdmem/service.py:465-545, 572-605, 613-619, 655-719
Plan: §2.4, §6 single-writer rule, §9 optimistic versioning, §14.10

update_note, approve, reject, edit, and hard_delete read the note and make lifecycle or version decisions before acquiring _write_lock. Only the eventual _persist or deletion block is serialized. A queued operation therefore acts on a stale Note object after an earlier queued operation has changed the canonical file.

Confirmed consequences:

• Two concurrent inbox updates with the same expected_version both succeed; one silently overwrites the other and one returned version is stale immediately.
• A rejected note can be rewritten back into inbox, losing rejected: true.
• An update queued behind hard_delete can recreate the deleted ULID and file after deletion reported success.
• An agent update queued behind approval can move an owner-approved note back to inbox with the agent's text.

Required regression coverage: deterministic barrier tests for update/update, reject/update, approve/update, archive/update, edit/update, and delete/update. The read, validation, mutation, file write, and synchronous index update must be one serialized operation.

C-02 — Missing closing frontmatter delimiter causes destructive “repair”

Affected code: src/mdmem/notes.py:67-95, src/mdmem/index.py:264-305
Plan: §4 identity/frontmatter preservation, §6.3 safe parsing, §14.25/§14.26

Strict parsing only raises for malformed YAML after the frontmatter regex has matched both delimiters. If the closing --- is missing, both lenient and strict parsing return an empty map. read_and_fix then treats an existing canonical note as a clean hand-written note.

In the reproduction, an indexed active fact was renamed and rewritten, its real title was replaced by the old filename stem, tags were erased, and the former YAML became body text. The degraded metadata also replaced the good index row. Clearing superseded_by this way can make an obsolete fact searchable again. The history snapshot made during repair contains the already broken text and is not a dependable last-good recovery point.

Required regression coverage: remove only the closing delimiter, then exercise both full scan and watcher sync. The file, path, identity, and last-good index row must remain unchanged while a clear error is reported.

High-severity findings

H-01 — Delayed embeddings can be attached to newer text

Affected code: src/mdmem/index.py:24-29,61-127,360-367, src/mdmem/service.py:227-242,714-719
Plan: §6 single-writer consistency, §7 recovery, §14.7

PendingEmbed records key_hash, but _embed_items discards it and store_vectors stores solely by numeric chunk ID. Embedding happens after _write_lock is released. Reindexing deletes and reinserts chunks, and SQLite can immediately reuse the old integer ID.

A delayed response for hash A can therefore overwrite the vector for current hash B and mark B as complete. This affects ordinary writes, background retries, and reembed_all. If the old ID is not reused, the code may instead attempt to create an orphan vec row.

Required regression: hold an old embed request open, complete an edit/reindex, then release it. Store a vector only when both chunk ID and embed_text_hash still match in the same transaction.

H-02 — Filename/frontmatter disagreement creates identity and lifecycle split-brain

Affected code: src/mdmem/vaultio.py:55-69, src/mdmem/index.py:210-305, src/mdmem/db.py:25-32
Plan: §4 identity and folder lifecycle source of truth

read_and_fix trusts any syntactically valid frontmatter ULID without checking the ULID encoded in the filename. Confirmed failure modes:

• Editing a canonical file's frontmatter ID to another ULID makes indexing fail on the unique path constraint while preserving a stale row under the filename ID. The file's new ID cannot be found by Vault.find.
• An active original plus an archive copy carrying the original frontmatter ID is accepted with no conflict; last-file-wins indexing can hide the active original as archived while Vault.find still reads the active file.
• A file such as active/manual.md with a valid frontmatter ID is indexed, but Vault.find(id) and memory_get cannot locate it because lookup uses the filename ID.

Required regression: reject mismatched IDs and duplicate lifecycle copies without changing the last-good row. Also test valid-frontmatter-ID files with noncanonical names.

H-03 — Normal vault deletion leaves inbound links invisibly dangling

Affected code: src/mdmem/index.py:153-158, src/mdmem/service.py:180-190
Plan: §5 unresolved links, §6 vanished-file handling, §10 health report

The UI hard-delete path now reindexes inbound sources, but Indexer.remove—used by ordinary filesystem deletion through scan/watcher—deletes only outgoing links. Inbound rows retain the deleted ULID rather than reverting to unresolved:<target>, so the unresolved health count remains zero and the broken reference is invisible until its source is reindexed for another reason.

Required regression: delete a target through both scan and watcher paths; surviving source links must become unresolved and flags for the vanished note must be handled explicitly.

H-04 — Service updates can write NFC to disk while indexing NFD

Affected code: src/mdmem/notes.py:98-130, src/mdmem/vaultio.py:88-119, src/mdmem/service.py:531-545,655-684,714-719
Plan: §4 and §14.8 NFC normalization

dumps normalizes a deep copy, not the live Note. _persist writes the normalized serialization and then indexes the original object. An inbox update with NFD text produced NFC on disk but NFD in chunks.body and in the embedding input. Because the indexed file hash is the canonical disk hash, the watcher considers the service event already handled and does not repair the mismatch.

Required regression: update and UI-edit with NFD in body, frontmatter, relations, and link targets; assert the live note, disk, chunks, FTS/trigram text, links, and embedding input all use NFC.

H-05 — Scan-to-watcher startup gap can permanently miss changes

Affected code: src/mdmem/service.py:117-157
Plan: §6 startup scan/watcher, P1 live-edit latency

The watcher starts only after scan, secret scan, self-test, and pending embeddings. A file changed after its scan pass but before watcher registration emits no event. A controlled reproduction wrote a second valid note while embed_pending was blocked; after startup finished, the file existed but the note was not indexed.

Required regression: mutate a file after initial scan while startup embedding is paused. Start watching before reconciliation or run a final scan after watcher registration.

H-06 — A Bifrost outage can put Kubernetes into a cold-start restart loop

Affected code: src/mdmem/server.py:67-72, src/mdmem/service.py:117-157, src/mdmem/embed.py:160-199, deploy/kubernetes/apps/apps/mdmem/mdmem/app/deployment.yaml:58-69
Plan: §7, §13a.3, §14.17 lexical-only degradation

The process awaits svc.start() before either listener binds. Startup synchronously awaits an embedding self-test and pending embeddings. Each can spend three 60-second attempts. The deployment has liveness starting after 30 seconds, no startupProbe, and the default failure threshold. Kubernetes can kill the pod before /healthz ever binds, repeatedly preventing the promised lexical-only mode.

Required regression: start the real server with a blocking/blackholed embedder and prove liveness, readiness, and lexical search bind before the probe deadline. Deployment tests must cover the startup probe budget.

H-07 — Common explicit credential forms evade the secret scanner

Affected code: src/mdmem/secrets.py:11-29
Plan: §9 and §14.12

The generic assignment regex requires a word boundary before names such as token and secret. Because underscore is a word character, common environment variable names evade it. Provider patterns are also incomplete. Confirmed misses include MCP_TOKEN=..., AWS_SECRET_ACCESS_KEY=..., OPENAI_API_KEY=..., GITHUB_TOKEN=..., github_pat_..., glpat-..., npm_..., and hf_....

Required regression: current provider token formats and standard environment assignments, including export, quoting, mixed case where applicable, and every serialized note field.

H-08 — Secondary persistent channels bypass secret rejection and can leak into audit

Affected code: src/mdmem/service.py:556-569, src/mdmem/mcp_server.py:90-128,350-354
Plan: §4 untrusted source, §9 audit, §14.12

flag.reason is agent-controlled and persisted in SQLite without a scan. request_id is stored and logged raw without a size or secret check. Audit source is copied from self-reported note metadata before note validation. A write rejected for source.harness=AKIAIOSFODNN7EXAMPLE still copied that full key into audit.jsonl; the same value in a flag reason was stored and returned by memory_get.

This is also provenance confusion: the audit UI presents an untrusted, caller-asserted source as attribution.

Required regression: inspect the audit file after rejected source/request ID values, and test flag reason storage. Store bounded/redacted identifiers, not unchecked caller strings.

H-09 — Idempotency is race-prone and has inconsistent key semantics

Affected code: src/mdmem/mcp_server.py:83-129, src/mdmem/db.py:58-59, src/mdmem/service.py:777-813
Plan: §9 replay guarantee

The guard performs get -> side effect -> put without synchronization. Two concurrent identical writes with one request ID both missed and created different notes. Concurrent different arguments also both succeeded instead of returning IDEMPOTENCY_CONFLICT.

The table primary key is request_id, while lookup treats (request_id, tool) as the key. INSERT OR REPLACE for a second tool deletes the first tool's replay record. Replaying the original operation can then repeat its side effect.

Required regression: concurrent same/different arguments, same ID across tools, in-progress requests, and failure after mutation but before idempotency persistence. Claim and commit an idempotency key atomically before performing a mutating operation.

H-10 — Revision approval has no target-version or target-lifecycle check

Affected code: src/mdmem/service.py:494-528,572-605
Plan: §2.4 and §9 revision review

A proposal records only revises, not the version it was based on. _merge_revision blindly loads the current target and overwrites it. Confirmed outcomes:

• active v1 -> agent proposal v2 -> owner edit v3 -> approve proposal results in v2, silently losing the newer owner edit;
• active target -> proposal -> reject/archive target -> approve proposal rewrites the archived target while retaining rejection state, so a later restore promotes the agent text.

Required regression: target edited, archived, rejected, deleted, or superseded after proposal creation. Approval should refuse stale proposals or require an explicit owner conflict resolution.

H-11 — Supersede proposals bypass lifecycle and exclusivity invariants

Affected code: src/mdmem/service.py:548-554,572-593
Plan: §2.4, §2.6, §9

supersede accepts archived, rejected, inbox, and already-superseded targets. A rejected item can be re-proposed through this path despite the explicit archived-note guard on update. Two successor proposals can also both be approved: both remain active and point back to the old fact, while the old note's singular superseded_by points only to the second. Search can expose contradictory “current” facts.

Required regression: archived/rejected/inbox/already-superseded/nonexistent targets, concurrent and sequential competing successors, and bidirectional chain consistency.

H-12 — Zero and non-finite embeddings are accepted as healthy

Affected code: src/mdmem/embed.py:39-42,178-188, src/mdmem/search.py:72-74
Plan: §2.1 known all-zero failure, §7 startup self-test, §14.1

Normalization leaves zero vectors at zero and propagates NaN/Infinity. Shape checks accept them, including during startup self-test. Two zero vectors have distance zero, which search converts to cosine 1.0, creating a falsely perfect semantic match.

Required regression: zero, NaN, and infinite vectors for self-test, documents, and queries. Invalid vectors must not be cached or stored, and search must degrade to lexical.

Medium-severity findings

M-01 — Flags violate the Markdown source-of-truth and recovery contract

Affected code: src/mdmem/db.py:53-56, src/mdmem/service.py:556-569
Plan: §0 Markdown canonical; SQLite rebuildable

Flags exist only in SQLite. Deleting/rebuilding index.db from an unchanged vault changes flags_for(id) from populated to empty. Either make flags canonical in Markdown or explicitly change the backup/recovery contract and back up this mutable database state.

Missing test: flags survive full index recreation.

M-02 — Temporal values are unvalidated strings compared lexicographically

Affected code: src/mdmem/service.py:417-450, src/mdmem/search.py:194-199,327-332
Plan: §2.6, §4 UTC timestamps, §8 filters

Malformed values, offsets, and valid_from >= valid_until are accepted. Lexical comparison is not instant comparison: for example, equivalent offset and Z timestamps can filter differently.

Missing tests: RFC3339 validity, UTC normalization, ordering, equivalent instants, and boundary semantics for stored values and search arguments.

M-03 — Unicode and trigram lexical hits are not merged by better rank

Affected code: src/mdmem/search.py:120-150
Plan: §8.1

The implementation mixes raw BM25 magnitudes from different FTS tables and ignores trigram whenever the same chunk has a unicode61 hit. This does not implement “keep the better rank per chunk.”

Missing test: controlled rank lists where a chunk is rank 50 in unicode61 and rank 1 in trigram.

M-04 — Late link resolution misses equivalent slug spellings

Affected code: src/mdmem/index.py:182-207
Plan: §5 exact-title then slug resolution

Initial resolution slugifies a target, but late resolution updates only two literal unresolved keys. [[Later Note]] remains unresolved when a later note titled Later Note arrives, although both slugify identically.

Missing tests: whitespace, punctuation, capitalization, and umlaut variants on forward links.

M-05 — Search accepts invalid bounds and can exceed max_chars

Affected code: src/mdmem/search.py:303-318
Plan: §8.7, §14.11

The first result is always emitted even when it exceeds the requested budget; zero or negative budgets still return one result. A negative internal k uses Python slicing and can return nearly all candidates. The current test explicitly permits an 800-character overrun, contradicting the plan.

Missing tests: k <= 0, max_chars <= 0, oversized first title/snippet, exact-budget boundary, and structured MCP validation.

M-06 — One long token bypasses the chunk target

Affected code: src/mdmem/chunker.py:50-89
Plan: §6.4

_split_words does not split a single token longer than the limit. Agent content can reach 8,000 characters and hand-authored notes are unbounded, producing oversized chunks.

Missing test: a no-whitespace token larger than the chunk limit, with bounded lossless output.

M-07 — is_ulid accepts overflow encodings rejected by the ULID library

Affected code: src/mdmem/ids.py:13,38-39
Plan: §4/§6 valid identity

Validation checks only length and alphabet. Values such as ZZZZZZZZZZZZZZZZZZZZZZZZZZ pass but ULID.from_str rejects them as overflow.

Missing property: is_ulid(value) implies successful parsing by the canonical ULID implementation.

M-08 — OIDC claim validation accepts non-finite expiry, missing subject, and bad azp

Affected code: src/mdmem/ui/auth.py:210-245
Plan: §10 fail closed

Signed tokens with exp: NaN and exp: Infinity pass because any int/float is accepted and non-finite comparisons do not expire them. sub is not required, allowing an empty-subject session. When aud contains multiple values, azp is not required or checked; a token with mdmem among its audiences but azp naming another client was accepted.

Missing tests: finite expiry, required nonempty subject, and multi-audience authorized-party rules.

M-09 — Scalar group claim is accepted contrary to the fail-closed plan

Affected code: src/mdmem/ui/auth.py:60-90, tests/test_ui_auth.py:59-60
Plan: §10, P4, §14.21

The code converts a scalar string group to a list, and a test deliberately blesses it. The plan requires a present-but-wrong-type group claim to return 403.

Required change to coverage: only list[str] should pass; scalar, null, mapping, and mixed lists must fail.

M-10 — UI POST protection still does not validate Host

Affected code: src/mdmem/ui/app.py:85-105,464-480
Plan: §10 Origin/Host allowlist

The recent Origin/Referer fix correctly rejects headerless writes, but when configured origins are present, Host is ignored. A request with allowed Origin and Host: evil.example was accepted.

Missing tests: invalid Host with valid Origin, proxy-forwarded host behavior, and the exact deployed host allowlist.

M-11 — JWKS is cached forever, breaking key rotation until restart

Affected code: src/mdmem/ui/auth.py:149-165
Plan: §10 operationally reliable OIDC

An unknown kid does not trigger one forced JWKS refresh. New logins fail after PocketID signing-key rotation until the pod restarts.

Missing test: cache key A, publish key B, present a B-signed token, and verify one refresh succeeds.

M-12 — Audit hashes omit actual arguments and unexpected errors vanish

Affected code: src/mdmem/mcp_server.py:83-129,167,335-339
Plan: §9 every tool call logged with argument hash, source, latency, IDs

Search hashes only query and k, omitting filters. Flag hashes only ID, omitting reason. Different calls therefore receive identical audit fingerprints. _guard catches only MdmemError; an injected database/programming exception produced no audit entry at all.

Missing tests: vary every argument and require a hash change; inject unexpected exceptions and require ok:false, latency, request ID, and a bounded error class.

M-13 — memory_flag is non-idempotent

Affected code: src/mdmem/mcp_server.py:328-339
Plan: §9 idempotency

The mutating flag tool has no request_id. A transport retry creates duplicate review items. Search and get also omit the plan's broadly stated optional request ID, though they have no side effect.

Missing test: replaying one flag request creates exactly one flag.

M-14 — Evaluation can recommend a variant with a catastrophic category regression

Affected code: src/mdmem/evalharness.py:340-372
Plan: §11 “without a regression in any category beyond noise”

pick_default checks only overall nDCG and confidence intervals. It never examines per_category.

Missing test: a candidate with a clear overall win but severe temporal or update regression must remain unselected.

M-15 — Eval output is not standards-compliant JSON

Affected code: src/mdmem/evalharness.py:90-117,160-174,395-425, eval/results/2026-09-12-synthetic-smoke.json:538-541
Plan: §11 committed JSON result artifact

Unavailable metrics remain float("nan"); json.dumps emits raw NaN, which RFC 8259 does not permit. The test named as a CLI writer test manually writes a different object and never calls run_eval_cli, leaving serialization false-green.

Missing test: invoke the actual CLI and parse with a strict JSON parser; unavailable values should serialize as null.

M-16 — Eval latency is order-biased by a shared embedding cache

Affected code: src/mdmem/evalharness.py:293-312, src/mdmem/service.py:99-115
Plan: §11 fair ablation comparison

Each variant gets an isolated index but retains the configured cache_db. The first real-model variant pays query embedding latency while later variants reuse cached query embeddings, making p95 latency incomparable.

Missing test: instrument backend query calls and prove equivalent cache state or perform an explicit warm-up excluded from every variant's timing.

M-17 — Required Prometheus signals are absent

Affected code: src/mdmem/ui/app.py:75-145
Plan: §13a.7

Embedding error count, search-latency histogram, and MCP calls by tool/outcome are not exported. Search/write totals are Gauges rather than Counters. Existing tests only check that generic metrics exist and omit note content.

Missing test: scrape and assert all required metric families and bounded label cardinality.

M-18 — UI state filtering and state transitions do not enforce the displayed workflow

Affected code: src/mdmem/ui/app.py:261-300, src/mdmem/service.py:638-653
Plan: §2.4, §10 Search filters and review workflow

Search state=inbox includes both active and inbox notes, while state=archive includes active, inbox, and archive. In addition, the generic state endpoint can promote an inbox revision or supersede proposal directly to active, bypassing approve merge/chain behavior if called manually.

Missing tests: exact results for every state filter and forbidden lifecycle transitions that must go through approval.

Lower-severity plan drift and test-suite gaps

• Health has no “run eval” action required by §10 (src/mdmem/ui/templates/health.html, src/mdmem/ui/app.py:443-460).
• History compares one snapshot only to current, not any two revisions as required by §10 (src/mdmem/ui/app.py:313-324).
• Browse does not offer sorting by tags as required by §10.
• Health audit rows do not display the recorded source field.
• “Edit & approve” converts an intentionally empty body to None, approving the old body instead (src/mdmem/ui/app.py:235-239). Empty must be saved or explicitly rejected, not ignored.
• AuditLog.recent reads persisted history only while its in-memory list is completely empty. After restart and one new call, Health shows only that call rather than the last 100 across restart (src/mdmem/logging.py:77-92).
• memory_supersede omits relations, validity bounds, and source metadata available to memory_write; either complete the §9 schema or narrow the contract.
• Bearer configuration accepts whitespace/non-ASCII values that clients cannot reliably present; startup should validate a trimmed ASCII token with an explicit minimum length.
• validate_id is dead code. ID arguments have no schema pattern or maximum length and are echoed in NOT_FOUND responses.
• The session test named “round trip and expiry” never advances time or tests expiration.
• No route-level OIDC test covers discovery, state, PKCE, nonce carriage, callback errors, cookie flags, expiry, logout, or signing-key rotation.
• The OIDC HTTP client created in build_ui_app has no application shutdown hook calling OidcClient.aclose.
• CLI tests cover help/version but not serve, doctor, reindex, eval, contradictory mode flags, startup failures, or graceful shutdown.
• The note round-trip property test generates a body but does not assert its round trip and does not property-test arbitrary unknown-key order/comments.
• The clean manifest template is not directly buildable: app/kustomization.yaml references the intentionally absent encrypted secret.yaml; image digest and PocketID client ID remain release placeholders. These are documented setup blockers, but there is no CI smoke test for the rendered deployment overlay.
• The only committed evaluation is a synthetic fake-embedding smoke run. It proves harness wiring, not the P5 acceptance criterion of 50–100 real labelled queries and a production-model tuning report.

Test integrity assessment

The suite is useful for sequential happy paths but systematically underrepresents interleavings, recovery from canonical Markdown, deployment startup, and strict external-format validation.

Highest-value missing test groups, in order:

1. deterministic service concurrency tests around the whole read/check/write operation;
2. chunk-hash-guarded embedding races, including background retry and re-embed;
3. malformed frontmatter delimiter and filename/frontmatter identity conflict cases;
4. startup with a blocked embedding backend plus a mutation during watcher registration;
5. concurrent idempotency and crash-between-side-effect-and-record cases;
6. secret scanning and audit redaction across every persistent channel;
7. full OIDC route tests with strict claims and signing-key rotation;
8. actual CLI eval serialization through a strict JSON parser;
9. disaster rebuild proving all canonical state survives; and
10. deployment render/probe smoke tests.

Some current tests actively encode plan drift rather than merely omitting it:

• tests/test_ui_auth.py accepts a scalar group claim despite §14.21;
• tests/test_search.py permits the first result to exceed max_chars by up to one snippet;
• tests/test_evalharness.py::test_cli_reports_and_writes_a_json_file does not invoke the CLI it names.

Recommended remediation order

1. Serialize lifecycle/version operations end-to-end and add deterministic race tests.
2. Reject unterminated/mismatched frontmatter without touching the file or last-good index.
3. Guard vector writes by chunk ID plus embedding-text hash.
4. Make startup bind in lexical mode independently of Bifrost and close the scan/watch gap.
5. Make idempotency atomic and reconcile its database key.
6. Close all secret-bearing secondary channels and redact audit fields.
7. Enforce revision/supersede target version, lifecycle, and exclusivity.
8. Validate embeddings, timestamps, ULIDs, OIDC claims, Host, and response bounds.
9. Restore the Markdown-only recovery guarantee for flags or explicitly redesign it.
10. Repair eval selection/serialization/fairness, observability, and the lower-priority UI drift.

Baseline notes

The following earlier findings are fixed in the reviewed revision and are intentionally not listed as open defects above: full-field note secret scanning, mandatory expected_version, archived-note update rejection in the sequential case, approval validation, audit-file wiring, ordinary hard-delete link repair, background retry of pending vectors, safe re-embed on a simple outage, trigram backfill after toggling it off/on, missing-Origin CSRF rejection, nonce/exp/iat presence checks, eval index isolation, reciprocal synthetic supersede data, connection/cache shutdown, and the direct fusion formula tests.