The eval route was looking for queries.yaml under data_dir/results/
after Qwen's cleanup, but the spec (test + README layout) places it
at data_dir/eval/. Restores the documented path and unbreaks
test_run_eval_keeps_its_files_on_the_data_volume_and_reports_status.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G9K7DZE8v1AYvfKU4qrQN5
Answers the independent review in SECURITY_ANALYSIS.md: both critical findings,
all twelve high, all eighteen medium, and the lower-severity plan drift. The
headline defects were a write path whose lifecycle and version checks sat
outside the serialized operation, an unterminated frontmatter block that was
"repaired" into a new note, delayed embeddings landing on reused chunk ids, and
a startup that could not bind its listeners while the embedding backend was
unreachable.
Also in here, found while deploying rather than by the review:
- Referrer-Policy: no-referrer made browsers send `Origin: null` on form
submissions, so the CSRF guard refused every write in the UI. It had never
worked in a browser on any build; the tests missed it by setting Origin by
hand.
- Renaming a note orphaned its inbound links at the next reindex of each source,
silently and long after the rename.
- An idempotency claim abandoned by a cancelled request wedged its request_id
for the full 24h TTL.
The UI is translated to English and rebuilt around design tokens, and the test
suite grows from 227 to 406, including the deterministic concurrency and barrier
tests the review asked for.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0139vHnNpS2sLNELjrAHX5Cu
This is why approving a note returned {"error":"bad origin"}, and it was never
the Host comparison: the UI shipped `Referrer-Policy: no-referrer`.
Per the Fetch standard, that policy makes a browser send `Origin: null` on an
HTML form submission — and suppress the Referer the CSRF guard falls back to.
Every write in this UI is a form POST and the guard fails closed, so both of its
inputs were destroyed by our own security header and every approve, reject,
edit, delete and health action was refused. On master too: this has never worked
in a real browser, on any build.
The tests did not catch it because they set Origin by hand, which is exactly
what a browser under that policy will not do.
`same-origin` sends the real Origin on same-origin requests and nulls it only
cross-origin — precisely the case the guard exists to refuse — while still
sending no referrer information off-site. The header test now asserts the
policy is not any of the values that null the Origin on a form submission, so
the reason is pinned rather than the string, and a new test posts the shape a
browser actually sends: a real Origin and no Referer.
`Origin: null` is now named in the refusal log, since it means something
upstream stripped the origin rather than the caller being on the wrong host.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0139vHnNpS2sLNELjrAHX5Cu
The inbox card only edits the body, so correcting frontmatter an agent got wrong
— a bad type, invented tags, a malformed valid_from — means opening the note and
using the full edit form. But the note page had no Approve or Reject, so the
owner had to navigate back to the inbox to act on what they had just fixed.
Both now appear on the note page while the note is in the inbox, posting to the
same routes the inbox uses, so revision merges and supersede chains behave
identically. An active note still shows only edit/archive/delete.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0139vHnNpS2sLNELjrAHX5Cu
A CSRF rejection returned 403 {"error":"bad origin"} and wrote nothing anywhere.
uvicorn's access log is off for both listeners, so a refused write left no trace
on the server at all — the Host/Origin mismatch that made the deployed UI
read-only was only visible by opening browser devtools, and looked like a bug in
the UI rather than a header spelling difference.
The rejection now logs what was actually compared: path, method, whether the
header was missing or merely disallowed, the candidate origin, Host,
X-Forwarded-Host, and the configured allow-list. All of those are non-secret,
and none of them exist without them. The allow-list is now built in one place
that both the check and the log line use, so they cannot drift apart.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0139vHnNpS2sLNELjrAHX5Cu
The Host cross-check added for M-10 compared the Host header to the Origin's
netloc with ==. That does not survive a reverse proxy: a browser omits the
default port from Origin, while Traefik forwards Host (and X-Forwarded-Host)
with it, so `memory.example.com` and `memory.example.com:443` are one authority
spelled two ways. The result was a 403 {"error":"bad origin"} on every
state-changing POST in the deployed UI — approve, reject, edit, delete and every
health action — which left the UI effectively read-only.
Both sides are now normalised to (host, port) with the scheme's default port
filled in, lowercased, IPv6 literals handled. An allowed Origin paired with a
foreign Host is still refused, which is the case the check exists for: the tests
cover evil.example, evil.example:443 and a suffix-match attempt alongside the
proxy shapes that must pass.
Reverting the comparison fails the proxy test.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0139vHnNpS2sLNELjrAHX5Cu
Link resolution re-reads the link text on every reindex: `[[Old Title]]` is
looked up again and the result overwrites the stored edge. So renaming a note
did not break anything at the time — it armed a trap. The next time each inbound
source was reindexed, for any unrelated reason, its link silently became
`unresolved:Old Title`, and `reindex --full` sprang every one of them at once.
Nothing in the codebase handled a title change.
A new `note_titles` table records every title a note has carried, written in the
same transaction as its links. `_resolve` consults it only after the live title
and slug lookups miss, so a *new* note claiming a freed title still wins — the
link text should name whatever holds that title now, not whatever held it first.
Where a title has had several holders, the most recent wins: deterministic, and
the least surprising reading.
`open()` backfills the table from the titles already in `notes`. Without that an
upgraded index only learns a title when the note is next reindexed, and a note
that never changes never is — so the first rename after the upgrade would still
break its links, which is the whole case this exists to prevent. `remove()`
drops a note's titles, so a deleted ULID cannot keep winning resolution for a
title nothing holds any more.
Reverting the lookup fails the rename and backfill tests; reverting only the
backfill fails the backfill test.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0139vHnNpS2sLNELjrAHX5Cu
The health pass only covered the page it was looking at. The rest had the same
class of problem:
- Card headings put a title and its pills in one inline run, and `.pill` was
`white-space: nowrap`. An inbox item labelled `Revision of "<title>"` carries a
whole note title inside that pill, so it pushed the card — and the page —
sideways on a phone. Those headings are now flex containers that wrap, and a
pill may break its own text at the small breakpoint.
- The reject form pairs a reason input with a button. Inside a flex `.actions`
the input collapsed toward its intrinsic size; it now takes the slack.
- The search filter row is eight controls. Stacked full width it was eight rows
tall before a single result; it now lays out in two columns with the query and
the submit button spanning.
- `white-space: nowrap` on table cells (added so the scroll container would do
its job) made the browse table absurdly wide, since note titles are long and
every one of them claimed a full line. Title, tags and the call-log ids now
wrap within a bounded column; the short columns stay nowrap.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0139vHnNpS2sLNELjrAHX5Cu
The shell had a genuine bug: `main` carried both `max-width: var(--shell)` and a
centering inline padding, and with `box-sizing: border-box` the max-width
*includes* that padding. On a wide display the content column collapsed to a
fraction of its intended width while the header still spanned the full shell, so
the page sat narrow and misaligned with its own header. Both now use the one
gutter formula and no max-width, against `100%` rather than `100vw` so a
scrollbar cannot turn the gutter into overflow.
Health mixed three kinds of thing into one tile grid — bare counts, a status
pill, a latency with a unit, a model name — which is why it read as a ragged
list. Tiles are now uniformly one label and one number, muted at zero and amber
when a non-zero count wants attention (pending vectors, unresolved links,
orphans). Everything with a unit or a status moved into an Embeddings card
beside a Maintenance card, and a lexical-only backend now says so in a banner
instead of hiding inside a pill.
Mobile: tiles hold two columns down to ~340px, the nav scrolls sideways instead
of wrapping onto a second line, the user's name gives way to the sign-out link,
action buttons and filter controls go full width rather than forming a ragged
two-and-a-half-button row, and table cells stop wrapping so the scroll container
does its job.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0139vHnNpS2sLNELjrAHX5Cu
The UI shipped in German (commit 5853b7b), which PLAN.md never asked for — the
plan's German context is about *content*: NFC/umlaut handling, German trigram
coverage, a bilingual eval set. Chrome is now English. German test fixtures and
eval/corpus notes stay German on purpose, since they are what exercises that
handling.
The stylesheet is rebuilt around design tokens rather than ad-hoc values:
surface/line/accent scales with real light and dark palettes, a type scale with
tabular numerals for the metric-heavy views, consistent radii and elevation,
visible focus rings on every control, and reduced-motion support. Notable
component work: the nav marks the current page with aria-current, health counts
become stat tiles instead of a bullet list, tables get a sticky header inside a
scroll container so a wide row cannot push the page sideways, and the diff view
tints whole added and removed lines rather than colouring the text alone.
Two functional touches that came out of the markup pass: browse and the MCP call
log render an explicit empty state, and the search state filter now offers
"Inbox only", which the route already supported after the M-18 fix but nothing
exposed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0139vHnNpS2sLNELjrAHX5Cu
The redirect URI registered against this client must be
https://memory.cloud.traunseenet.com/ui/auth/callback — OidcClient derives it
from UI_BASE_URL, so the two move together.
secret.yaml (SOPS) is now the only thing standing between this overlay and a
clean render. The digest and client-id placeholder spellings stay on the test's
allowlist: the digest is bumped by hand on every build (§13a.2), so it comes
back each release.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0139vHnNpS2sLNELjrAHX5Cu
sha256:4b0c21be… is the digest Harbor returned for the pushed
harbor.cloud.traunseenet.com/library/mdmem:0.1.0 build of this branch. Per
§13a.2 the tag and digest are bumped by hand together, since Renovate does not
track library/*.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0139vHnNpS2sLNELjrAHX5Cu
PLAN §4 gains the `flags` frontmatter key and the reason it lives there: the
index is derived and rebuildable, so review flags stored only in index.db are
lost the moment it is dropped. Notes that flagging bumps the note's version, so
an agent holding an older one gets VERSION_CONFLICT and must re-read — at which
point it sees the flag. §9's memory_flag row says the same.
SECURITY_ANALYSIS.md is the independent review these commits answer, kept so the
findings and their reproductions stay next to the code that closes them.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0139vHnNpS2sLNELjrAHX5Cu
Closes SECURITY_ANALYSIS.md M-14, M-15 and M-16.
M-15: unavailable metrics stayed float("nan") and json.dumps emitted bare NaN,
which RFC 8259 does not permit — the committed report carried 120 of them. They
now serialize as null, and the test named as a CLI writer test invokes the real
CLI and parses with a strict parser instead of hand-writing a different object.
M-14: pick_default compared only overall nDCG and confidence intervals, so a
variant with a catastrophic temporal or update regression could still be
recommended. M-16: each variant got an isolated index but shared the configured
embedding cache, so the first real-model variant paid query embedding latency
that later variants read from cache, making p95 incomparable.
The synthetic smoke report is regenerated against the current code and renamed to
match the date it actually carries. rrf, rrf-inbox-1.0 and rrf-bm25-flat now
score identically, which is expected rather than the shared-state corruption an
earlier report showed: no seeded note is in inbox, and lexical subchannels fuse
by ordinal rank, so a BM25 weight change only moves the score if it reorders.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0139vHnNpS2sLNELjrAHX5Cu
The liveness probe started at 30s with no startupProbe and the default failure
threshold, so a slow or blocked startup was a restart before /healthz ever bound
(SECURITY_ANALYSIS.md H-06). A startupProbe now allows 180s.
The overlay cannot render as committed — secret.yaml is SOPS-encrypted and kept
out of the repo, and the image digest and PocketID client id are release-time
placeholders. That is deliberate, but nothing checked the rest of it stayed
coherent. The new tests pin: every manifest parses, every kustomization resource
exists apart from the documented SOPS secret, no undocumented placeholders creep
in, every ConfigMap key is actually read by the application, secret-supplied
values are not duplicated in plaintext, probes point at ports the container
exposes, every persistent path lives on the data volume, the pod stays locked
down, and the ServiceMonitor scrapes the port the metrics are on.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0139vHnNpS2sLNELjrAHX5Cu
Closes SECURITY_ANALYSIS.md H-05, H-06 and the lower-severity audit, bearer and
CLI gaps.
H-06: the process awaited svc.start() before either listener bound, and startup
synchronously awaited an embedding self-test plus pending embeddings, each able
to spend three 60s attempts. Kubernetes could kill the pod before /healthz ever
bound, repeatedly preventing the lexical-only degradation the plan promises.
Startup now runs as a background task and the embedding work with it. H-05: the
watcher started only after scan, so a file changed in between emitted no event
and stayed unindexed; the watcher now starts before a reconciling second scan.
AuditLog.recent read persisted history only while its in-memory list was
completely empty, so after a restart plus one call the health page showed only
that call. It now merges both without duplicates, reading a bounded tail rather
than the whole append-only file — /metrics calls it on every scrape.
MCP_TOKEN is validated at startup: non-empty, ASCII, no whitespace or control
characters, minimum length. A padded or non-ASCII value silently 401s every
client. --mcp-only with --ui-only built an empty server list, so the process
bound nothing and exited 0 — a silent restart loop; argparse now refuses it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0139vHnNpS2sLNELjrAHX5Cu
Closes SECURITY_ANALYSIS.md M-08, M-09, M-10, M-11, M-17, the UI half of M-18,
and the lower-severity §10 drift.
M-08/M-09: signed tokens with exp NaN or Infinity passed (any int/float was
accepted and non-finite comparisons never expire), sub was not required, and a
multi-audience token with a mismatched azp was admitted. A scalar group claim was
coerced into a list where §14.21 requires 403. M-10: with configured origins the
Host header was ignored, so an allowed Origin paired with Host: evil.example was
accepted. M-11: JWKS was cached forever, so logins failed after a PocketID key
rotation until the pod restarted; an unknown kid now forces one refresh.
M-17: §13a.7 requires an embedding error counter, a search latency histogram and
MCP calls by tool and outcome; none existed, and monotonic totals were Gauges.
M-18: state=inbox returned active and inbox, state=archive returned everything.
Each filter now returns exactly that state, narrowing a wider candidate pool so a
match below the first page is not silently dropped.
§10 drift: "edit & approve" turned an intentionally emptied body into None and
approved the old text; history compared one snapshot to current instead of any
two revisions; browse could not sort by tags; the health page showed no audit
source and had no run-eval action. The eval action writes to the data volume,
not the working directory, which readOnlyRootFilesystem makes unwritable.
Route-level OIDC coverage is new: login/callback/logout, PKCE verifier against
its challenge, nonce carriage, forged and missing state, cookie flags, open
redirect confinement and provider-down handling. The session test named "round
trip and expiry" now actually advances the clock.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0139vHnNpS2sLNELjrAHX5Cu
Closes SECURITY_ANALYSIS.md M-03, M-04 (with index.py) and M-05.
M-03: bm25 magnitudes from chunks_fts and chunks_tri are not on one scale and
were mixed, and a trigram hit was discarded whenever the same chunk had any
unicode61 hit — not "keep the better rank per chunk". The two subchannels now
merge by per-table ordinal rank. The trigram pass keeps its 0.5 discount as a
rank weight: dropping it entirely cost the rrf baseline ~0.03 nDCG@10 on
eval/corpus by promoting substring hits over better whole-word ones.
M-05: the first result was emitted even when it blew the character budget, and a
zero or negative budget still returned one result. Non-positive bounds are now
refused and an oversized first snippet is trimmed to fit.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0139vHnNpS2sLNELjrAHX5Cu
Closes SECURITY_ANALYSIS.md H-08, H-09, M-12, M-13 and the lower-severity
request-id and validate_id gaps.
H-09: the guard did get -> side effect -> put with no synchronisation, so two
concurrent identical writes with one request_id created two notes, and differing
arguments both succeeded instead of conflicting. A claim is now committed before
the mutation runs and the table key is (request_id, tool) rather than
request_id alone, which used to let a second tool delete the first one's replay
record. An uncommitted claim expires after two minutes: CancelledError is a
BaseException that no `except Exception` arm sees, so a client disconnect
between claim and commit used to refuse that request_id for the full 24h TTL.
H-08: flag reasons were persisted unscanned, request_id was stored and logged
raw, and the audit `source` was copied from self-reported note metadata before
validation — a write rejected for carrying an AWS key still copied that key into
audit.jsonl. Reasons are scanned, request ids are bounded and hashed, and source
is no longer caller-asserted.
M-12: search and flag hashed only part of their arguments, so different calls
shared an audit fingerprint, and an unexpected exception produced no audit entry
at all. M-13: memory_flag takes a request_id. Every id argument now goes through
validate_id, which was dead code.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0139vHnNpS2sLNELjrAHX5Cu
Closes SECURITY_ANALYSIS.md H-07.
The generic assignment pattern required a word boundary before names like
`token` and `secret`, and underscore is a word character, so ordinary
environment variable names walked straight past it. Provider patterns were also
incomplete. MCP_TOKEN=, AWS_SECRET_ACCESS_KEY=, OPENAI_API_KEY=, GITHUB_TOKEN=,
github_pat_, glpat-, npm_ and hf_ were all confirmed misses; each now has a
regression test.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0139vHnNpS2sLNELjrAHX5Cu
Closes SECURITY_ANALYSIS.md C-01, C-02, H-01, H-02, H-04, H-10, H-11, H-12,
M-01, M-02, M-06, M-07 and the service half of M-18.
C-01: update/approve/reject/edit/supersede/set_state/hard_delete/flag read the
note, decided lifecycle and version, and only then took _write_lock, so a queued
operation acted on a stale Note. Two updates on one version both succeeded, a
rejected note could be rewritten back into inbox, and an update queued behind a
delete recreated the note. Read, validate, mutate, write and index are now one
serialized operation. Three deterministic barrier tests wedge the first
operation inside its persist and start the second while it is stuck there; all
three fail if the lock is narrowed back.
C-02: a note whose closing --- was deleted parsed as "no frontmatter" and was
destructively repaired into a new hand-written note. Strict parsing now raises,
and read_and_fix leaves the file, its path and its last-good index row alone.
H-01: store_vectors keyed only on the numeric chunk id, so a delayed embedding
could land on a reused id and be marked complete against text it never saw. It
now re-checks embed_text_hash inside the same transaction.
H-02: a frontmatter ULID that disagreed with the filename was trusted, splitting
identity between Vault.find and the index; duplicate lifecycle copies of one id
were resolved last-file-wins. Both are now refused with the last-good row kept.
H-04: dumps() normalised a copy, so NFD text reached chunks and the embedding
input while NFC reached disk. normalise_note now mutates the live note.
H-10/H-11: a revision proposal records the version it was based on and approval
refuses a drifted or non-active target; supersede refuses archived, rejected,
already-superseded and duplicate targets. M-18: set_state can no longer promote
a pending proposal straight to active, bypassing the merge and the chain.
H-12: zero and non-finite vectors normalised to themselves and scored a perfect
cosine match. They are now rejected at normalize(), in the cache, and in the
startup self-test.
M-01: review flags lived only in index.db, so dropping a derived, rebuildable
database lost them (PLAN §0). memory_flag now appends to the note's own
frontmatter and the flags table is a mirror the indexer rewrites on reindex.
M-02: timestamps were unvalidated strings compared lexicographically; new
timeutil.py parses RFC3339 and normalises to UTC. M-06: a single token longer
than the chunk limit is now split. M-07: is_ulid accepted overflow encodings the
ULID library rejects.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0139vHnNpS2sLNELjrAHX5Cu
Database.close() closed only the calling thread's read connection, but reads
happen inside asyncio.to_thread workers, so each worker's connection outlived
stop(); the EmbeddingCache connection was never closed at all. A stopped
service now leaves zero open connections (was 3), which matters for the
Recreate rollout and for reopening the same index file.
Ownership rule: whoever creates the cache closes it. build() closes the one it
constructs; a caller-supplied cache survives the service, as the rebuild tests
require.
Also silences the two warning sources: httpx per-request cookies in the UI
tests, and one unclosed in-memory database per Hypothesis example in the FTS
fuzz test (396 of the 410 hidden ResourceWarnings).
The multi-agent review found 35 defects against PLAN.md, four data-losing. All are
fixed with regression tests verified to fail when the fix is reverted. §14 gains the
six general lessons: never infer 'new file' from 'unparseable'; resolve destructive
operations from the id you were given, not from file contents; NFC and byte-exact
round-tripping are incompatible and NFC wins; an eval run must never touch the live
index; fused scores are not comparable across channel counts; fail closed on CSRF
and on id_token claim validation.
The committed report predated the eval-isolation fix and still showed the
corruption fingerprint: rrf-no-trigram, rrf-inbox-1.0 and rrf-bm25-flat all
reported an identical ndcg@10 of 0.7777. Regenerated against a fresh copy of
eval/corpus in a temp dir (VAULT_DIR/INDEX_DB/CACHE_DB/HISTORY_DIR pointed
there) with the shipped defaults (rrf, dim 2560, MIN_SCORE unset) -- this is
a machinery check, not a tuning result, so no default changed. The three
ablation arms now report distinct scores (0.818 / 0.802 / 0.806); rrf and
rrf-inbox-1.0 legitimately match because none of the 12 seeded notes are in
inbox state. eval/README.md's conclusions (every CC arm's CI still overlaps
the RRF baseline's; defaults unchanged) still hold, so it is unchanged.
_rrf had a pinned unit test but _cc did not, so a sabotaged _cc returning a
constant score for every chunk still passed the existing (indirect,
end-to-end) CC test. Add a direct test of the alpha*sem_norm +
(1-alpha)*lex_norm formula and its per-channel-missing fallbacks.
Verified: with _cc temporarily replaced by "return 0.42 for every chunk",
this test fails while test_cc_fusion_returns_results_too still passes;
restored, both pass.
Test: tests/test_search.py::test_cc_formula_is_pinned
Closes out finding 5: rather than relying on inspection of
mcp_server.py, assert directly that the MCP tool list is exactly
{memory_search, memory_get, memory_write, memory_update,
memory_supersede, memory_flag} and that none of edit, merge_into,
set_state, approve, reject or hard_delete is reachable through it. No
production code change was needed — mcp_server.py already only
registers the six tools.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0139vHnNpS2sLNELjrAHX5Cu
RRF already scales its sum when a channel is entirely unavailable (embedder
down), so a lexical hit survives an embedding outage. CC fusion had no
equivalent: weighting a missing sem channel at the configured alpha silently
deflated every score by (1-alpha), so a MIN_SCORE tuned with both channels
live emptied the results the moment the embedder failed. _cc now falls back
to the available channel alone (alpha forced to 0 or 1) when a whole channel
did not run for this query, not merely when it returned no hits.
Test: tests/test_search.py::test_cc_min_score_abstains_on_irrelevant_query,
tests/test_search.py::test_cc_min_score_is_stable_during_embedding_outage
lex_rank/sem_rank were computed from the raw pre-filter lex/sem dicts, so a
chunk removed by the type/tag/state/time filter still shifted the reported
rank of the chunks that remained -- debug reported rank 2 where fusion had
actually used rank 1. Compute ranks from the same post-filter subset _fuse
consumes.
A chunk that matched only via the trigram substring pass was reported with a
"bm25" field holding bm25(chunks_tri) * -0.5 -- a different number from a
different table, indistinguishable from a real chunks_fts bm25. Track the
source table per chunk and report it under "tri_bm25" with "via": "tri"
instead. The eval harness tunes on these numbers.
Test: tests/test_search.py::test_debug_ranks_reflect_the_filtered_set_not_the_raw_channel,
tests/test_search.py::test_trigram_only_hit_is_labelled_as_trigram_not_chunks_fts_bm25
set_state(note_id, "active") promoted a note out of archive/ without
touching rejected/rejected_reason, so a restored note could still carry
rejected: true — the UI restore button hit this directly. Restore is a
legitimate owner action (§10); agents can't reach set_state at all, so
the fix only needed to make state and frontmatter agree again, not to
block the operation. The rejected version stays visible via the
history snapshot _persist already takes.
Also ran `ruff format` on service.py to pick up an unrelated,
pre-existing formatting drift in hard_delete.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0139vHnNpS2sLNELjrAHX5Cu
exp was already mandatory, but iat was still accepted when absent, and
a token claiming to be issued far in the future was accepted too. Now
iat must be a numeric claim and must not sit more than
IAT_MAX_FUTURE_SKEW_S (300s) ahead of now; that same constant is used
as authlib's own leeway so the two checks agree on what "too far in
the future" means, rather than authlib raising an uncaught,
differently-typed error for the same condition first.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0139vHnNpS2sLNELjrAHX5Cu
watcher.py debounces step=500ms/debounce=1500ms, so a 0.9s sleep never let
the watcher observe our own write at all -- the test passed 10/10 even with
the self-write dedup in service._sync_path removed, because it was really
only checking that write_note() calls index_note once. Now it also counts
_sync_path invocations (which run on every watcher batch regardless of
dedup) and polls deterministically until the watcher has demonstrably fired
before checking the dedup invariant.
Verified: with the dedup in service._sync_path temporarily disabled, this
test fails (calls == 2); restored, it passes.
Test: tests/test_watcher.py::test_service_own_write_does_not_double_index
svc.get_note(note_id) sat outside the try in history_diff, so a note
with history but no current file (unlinked without hard_delete) raised
an uncaught NotFound and returned 500 instead of 404. Moved it inside
the same try/except as the history-revision lookup. Audited the other
note_id routes: they already wrap their get_note/get calls in
try/except MdmemError, so this was the only route with the gap.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0139vHnNpS2sLNELjrAHX5Cu
Creating chunks_tri fresh on a 0->1 flip left it permanently empty for any
chunk indexed before the flip, since an unchanged note's file_hash never
sends it back through index_note. Mirror the dim/model rebuild path: db.py
marks pre-existing chunks tri_pending=1 when the table is freshly created,
and Indexer.rebuild_pending_trigram (called from scan()) backfills their
chunks_tri row and clears the flag.
Test: tests/test_db.py::test_trigram_flip_marks_existing_chunks_pending,
tests/test_index.py::test_trigram_flip_backfills_existing_chunks_and_search_finds_them
The state-changing-request guard only rejected a present-but-disallowed
Origin; a POST with no Origin header at all skipped the check entirely,
so /ui/n/{id}/approve (and every other POST/PUT/DELETE/PATCH route)
could be forged from a page that omits Origin. Now the guard requires
either an allowed Origin or a same-origin Referer, and refuses when
neither is present. Updated UI-flow tests to send an Origin header, as
a real browser does, instead of relying on the check being skipped.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0139vHnNpS2sLNELjrAHX5Cu
_split_long("") returned [""], so an H2/H3 heading immediately followed by
another heading (or by the end of the body) produced a real chunk with an
empty body: a wasted FTS row, trigram row and embedding call. ord stays
contiguous since the loop only increments when a chunk is actually appended.
Test: tests/test_chunker.py::test_empty_h2_sections_do_not_produce_empty_chunks
scan() extended stats.pending from each index_note() return and then again
from pending_embeds() (which re-reads every vec_pending=1 chunk in the db,
including the ones just indexed), so every freshly pending chunk was listed
twice and would be embedded twice.
Test: tests/test_index.py::test_scan_pending_has_no_duplicate_chunk_ids
test_round_trip_property was failing on NFC-unstable codepoints (U+F900,
U+2126) because dumps() correctly NFC-normalises every string -- the fix
that repaired a real file_hash bug where NFD/NFC input hashed differently.
NFC wins per §4/§14.8, so the property now checks round-trip modulo NFC,
plus a pinned example test and a comment documenting why.
Test: tests/test_notes.py::test_round_trip_property,
tests/test_notes.py::test_frontmatter_values_are_nfc_normalised
_FM_RE only matched a BOM stripped at offset 0, so a BOM preceded by a
newline, or a doubled BOM, parsed as "no frontmatter" and silently reset
type to note and emptied tags/summary on an otherwise valid file. Strip any
run of BOMs and whitespace up front instead of a single startswith check.
Test: tests/test_notes.py::test_parse_tolerates_any_combination_of_leading_boms_and_whitespace
Both guarded with `if expected_version and ...`, so an empty string
skipped the version check entirely and silently overwrote the note.
Empty/missing expected_version is now a structured INVALID error.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0139vHnNpS2sLNELjrAHX5Cu
update_note only branched on state=="active", so an archived (including
owner-rejected) note fell into the inbox-edit-in-place branch and an
agent could rewrite it while rejected:true stayed set. Archive is now
refused with a structured ARCHIVED error; only inbox notes are edited
in place.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0139vHnNpS2sLNELjrAHX5Cu
index.py's read_and_fix() treated "no valid ULID in frontmatter" as
"hand-written note", but notes.parse() is lenient and returns an empty
map for broken YAML. A YAML typo in an already-indexed note therefore
minted a fresh ULID, rewrote the file under a new name, unlinked the
original and dropped its index row.
read_and_fix() now re-parses strictly before treating a missing id as
"assign one": broken frontmatter raises and is left untouched on disk
(the scan reports it as an error), while frontmatter that parsed
cleanly but genuinely lacks an id first tries ids.id_from_filename()
before minting a new one.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0139vHnNpS2sLNELjrAHX5Cu
A note saved by Notepad/PowerShell carries a leading UTF-8 BOM (and
sometimes leading blank lines) before the opening `---`. The old
\A-anchored regex treated that as "no frontmatter", dumping the whole
YAML block into the body -- which then hits the destructive
no-valid-ULID path in index.py. parse() now strips a leading BOM and
tolerates leading whitespace before the delimiter.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0139vHnNpS2sLNELjrAHX5Cu
The stamp is written as UTC but was read back with mktime, which interprets it
as local time — with TZ=Europe/Vienna that shifted the 24 h replay window by an
hour or two. Adds pitfalls 24 and 25 to PLAN.md §14.
Found by running the service for real: creating a file in the vault while
mdmem is up only logged 'reindex of changed file failed' until the next scan,
which is exactly the workflow the vault is meant to support. Adds the live
test suite (-m live) and the llama-swap model entry.
