Compare commits
2
Commits
c0529d4072
...
8fdec33037
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
8fdec33037 | ||
|
|
8fd3e37b67 |
No files matched your search
@@ -4,16 +4,14 @@ Markdown-canonical agent memory service. One self-hosted process that gives ever
|
||||
harness (Claude Code, claude.ai, Open WebUI, Hermes, opencode…) a single shared
|
||||
long-term memory over MCP, with markdown files as the only source of truth.
|
||||
|
||||
Full design and its evidence: [`PLAN.md`](PLAN.md). Section numbers below refer to it.
|
||||
|
||||
- **Markdown is canonical.** `/data/vault/{inbox,active,archive}/*.md`. The SQLite
|
||||
index is derived and rebuildable; delete it and the service rebuilds from the files.
|
||||
- **Hybrid retrieval.** FTS5 (BM25, `unicode61 remove_diacritics 2`, optional trigram)
|
||||
+ sqlite-vec KNN, fused with RRF (default) or convex combination. No graph traversal
|
||||
in the retrieval path (§2.7).
|
||||
in the retrieval path.
|
||||
- **Agents propose, the owner disposes.** Agent writes land in `inbox/`; an agent
|
||||
editing an approved note creates a *proposed revision*, never an in-place change.
|
||||
There is no delete tool (§2.4).
|
||||
There is no delete tool.
|
||||
- **Memory content is untrusted data.** Tool descriptions and the MCP `instructions`
|
||||
say so; the UI renders markdown with raw HTML disabled.
|
||||
|
||||
@@ -21,20 +19,19 @@ Full design and its evidence: [`PLAN.md`](PLAN.md). Section numbers below refer
|
||||
|
||||
```
|
||||
src/mdmem/
|
||||
config.py settings from the environment (§12)
|
||||
config.py settings from the environment
|
||||
notes.py frontmatter round-trip (ruamel): unknown keys, order and comments survive
|
||||
vaultio.py atomic write → fsync → replace, pre-write history snapshot
|
||||
chunker.py atomic notes are one chunk; long ones split on H2/H3 (§6)
|
||||
db.py schema, sqlite-vec, single-writer rule (§5)
|
||||
chunker.py atomic notes are one chunk; long ones split on H2/H3
|
||||
db.py schema, sqlite-vec, single-writer rule
|
||||
index.py vault → rows, links, duplicate hints
|
||||
embed.py Bifrost client, content-hash cache, FakeEmbedder for tests (§7)
|
||||
search.py the retrieval algorithm (§8)
|
||||
embed.py Bifrost client, content-hash cache, FakeEmbedder for tests
|
||||
search.py the retrieval algorithm
|
||||
service.py the one code path every write goes through — MCP and UI alike
|
||||
mcp_server.py six memory_* tools, bearer auth, idempotency, audit log (§9)
|
||||
ui/ FastAPI + Jinja + HTMX, PocketID OIDC with a group check (§10)
|
||||
evalharness.py metrics, bootstrap CIs, ablation matrix (§11)
|
||||
deploy/ Kubernetes manifests, laid out for the fluxcd repo (§13a)
|
||||
eval/ query-set template, synthetic corpus, results
|
||||
mcp_server.py six memory_* tools, bearer auth, idempotency, audit log
|
||||
ui/ FastAPI + Jinja + HTMX, PocketID OIDC with a group check
|
||||
evalharness.py metrics, bootstrap CIs, ablation matrix
|
||||
tests/ pytest suite
|
||||
```
|
||||
|
||||
## Local development
|
||||
@@ -55,14 +52,14 @@ UI_AUTH_DISABLED=1 LOG_STYLE=text uv run mdmem serve
|
||||
```
|
||||
|
||||
`UI_AUTH_DISABLED=1` is for a port-forwarded local UI only. It logs an error on
|
||||
startup, and **an HTTPRoute must never point at a listener running with it** (§14.19).
|
||||
startup, and **an HTTPRoute must never point at a listener running with it**.
|
||||
|
||||
Other commands: `mdmem reindex [--full] [--no-embed]`, `mdmem doctor`,
|
||||
`mdmem eval --queries eval/queries.yaml [--ablate]`.
|
||||
`mdmem eval --queries queries.yaml [--ablate]`.
|
||||
|
||||
## Build and push
|
||||
|
||||
There is no CI runner in the cluster, so builds are manual from the workstation:
|
||||
Builds are manual from the workstation:
|
||||
|
||||
```bash
|
||||
podman build -t harbor.cloud.traunseenet.com/library/mdmem:0.1.0 -f Containerfile .
|
||||
@@ -72,13 +69,14 @@ podman inspect --format '{{index .RepoDigests 0}}' harbor.cloud.traunseenet.com/
|
||||
|
||||
Put tag **and** digest in `deployment.yaml`. Never `:latest` + `imagePullPolicy: Always`
|
||||
for this service: without a runner, an unpinned tag means the running code is not in
|
||||
git, and a pod restart becomes an unplanned upgrade of your memory store (§13a.2, §14.20).
|
||||
git, and a pod restart becomes an unplanned upgrade of your memory store.
|
||||
|
||||
## Deploy (first install)
|
||||
## Deployment (FluxCD on RKE2)
|
||||
|
||||
`deploy/kubernetes/apps/apps/mdmem/**` mirrors the fluxcd repo's nesting; copy it in.
|
||||
The manifests were written from §13a and validated with `kubectl kustomize`, but **not**
|
||||
against a live cluster — verify each step.
|
||||
Deployment manifests are maintained in a separate [fluxcd repo](https://github.com/traunseenet/fluxcd).
|
||||
The mdmem manifests live under `kubernetes/apps/apps/mdmem/` and are managed by FluxCD.
|
||||
|
||||
### First install
|
||||
|
||||
1. Build and push the image; record the digest and put it in `deployment.yaml`.
|
||||
2. Seed Vault:
|
||||
@@ -96,8 +94,8 @@ against a live cluster — verify each step.
|
||||
assume. Client id → `mdmem-env`, client secret → Vault.
|
||||
4. Create the restic repo for `restic-mdmem`; copy `secret.example.yaml` to
|
||||
`secret.yaml`, fill it in, SOPS-encrypt it.
|
||||
5. Commit `kubernetes/apps/apps/mdmem/**`. Check: ExternalSecret `Ready`, PVC `Bound`,
|
||||
Deployment healthy, `/readyz` 200.
|
||||
5. Commit `kubernetes/apps/apps/mdmem/**` to the fluxcd repo. Check: ExternalSecret
|
||||
`Ready`, PVC `Bound`, Deployment healthy, `/readyz` 200.
|
||||
6. Verify embeddings end to end from inside the cluster:
|
||||
```bash
|
||||
kubectl -n apps run curl --rm -it --restart=Never \
|
||||
@@ -111,19 +109,19 @@ against a live cluster — verify each step.
|
||||
While you are there, confirm Bifrost's `semantic_cache` plugin does **not** intercept
|
||||
`/v1/embeddings` — send two slightly different inputs and check the vectors differ. A
|
||||
cache hit there would return a stale vector for changed text and silently corrupt the
|
||||
index (§14.13). `uv run pytest -m live` checks this and the pooling/prefix behaviour.
|
||||
7. Only then add `deploy/bifrost-mcp-client.json` to Bifrost's `config.json` and commit,
|
||||
index. `uv run pytest -m live` checks this and the pooling/prefix behaviour.
|
||||
7. Only then add the Bifrost MCP client configuration to Bifrost's `config.json` and commit,
|
||||
so a failure is attributable to one change at a time. Create a VK scoped to the six
|
||||
`memory_*` tools.
|
||||
8. Apply the NetworkPolicy and re-check the two things it can silently break: the pod
|
||||
stays Ready (kubelet probes) and the Prometheus target stays `UP` (§14.23).
|
||||
stays Ready (kubelet probes) and the Prometheus target stays `UP`.
|
||||
9. Run a `ReplicationSource` and **test a restore** into a throwaway PVC before the
|
||||
vault holds anything you would miss.
|
||||
|
||||
### Rotating MCP_TOKEN
|
||||
|
||||
The token lives in two Vault paths. Rotating one and not the other gives every agent a
|
||||
401 that looks like an mdmem bug (§14.15):
|
||||
401 that looks like an mdmem bug:
|
||||
|
||||
```bash
|
||||
NEW=$(openssl rand -hex 32)
|
||||
@@ -145,7 +143,7 @@ All harnesses go through Bifrost — mdmem's own port is never exposed.
|
||||
a browser-side connector flow may need `https://claude.ai` added.
|
||||
- **Hermes / opencode** — remote MCP config, same URL + VK header.
|
||||
|
||||
## Where the implementation deviates from PLAN.md
|
||||
## Where the implementation deviates from the plan
|
||||
|
||||
Each of these is a decision made while building, with the reason:
|
||||
|
||||
@@ -155,9 +153,9 @@ Each of these is a decision made while building, with the reason:
|
||||
| 5 | `chunks_fts`/`chunks_tri` are `content=''` | ordinary FTS5 tables | Contentless FTS5 cannot delete a row before SQLite 3.43 (bookworm ships 3.40) and cannot `snippet()`. The index is rewritten on every edit, so deletes are the hot path. Costs a few MB of duplicated text. |
|
||||
| 6 | "one asyncio task owns all SQLite writes (queue)" | one write connection behind a lock; async paths use `asyncio.to_thread` | Same invariant (exactly one writer), and it also holds for the CLI and the tests, which have no event loop of the service's. |
|
||||
| 8 | vec0 cosine | vec0 L2 over L2-normalised vectors, `cos = 1 − d²/2` | Monotone-equivalent and does not depend on a `distance_metric=` option in a pre-v1 extension. |
|
||||
| 9 | — | `MCP_ALLOWED_HOSTS` (new setting) | The SDK's DNS-rebinding protection checks the `Host` header. Without the in-cluster service name on the allow-list, every Bifrost call returns 421. Added to §14 as pitfall 24. |
|
||||
| 10 | — | `UI_AUTH_DISABLED` (new setting) | Makes §10's "P4 may ship without OIDC, but then without an HTTPRoute" explicit and loud instead of implicit. |
|
||||
| 11 | `eval/queries.yaml` with 50–100 real queries | `eval/queries.example.yaml` + a synthetic corpus and `--fake-embeddings` | Real queries are Markus's to write, and the real model is not reachable from the build environment. The harness is complete and tested; the committed run is a machinery check, and the defaults stay as §2.5 specifies. |
|
||||
| 9 | — | `MCP_ALLOWED_HOSTS` (new setting) | The SDK's DNS-rebinding protection checks the `Host` header. Without the in-cluster service name on the allow-list, every Bifrost call returns 421. Added to pitfalls as pitfall 24. |
|
||||
| 10 | — | `UI_AUTH_DISABLED` (new setting) | Makes the "may ship without OIDC, but then without an HTTPRoute" rule explicit and loud instead of implicit. |
|
||||
| 11 | `queries.yaml` with 50–100 real queries | `queries.example.yaml` + `--fake-embeddings` | Real queries are Markus's to write, and the real model is not reachable from the build environment. The harness is complete and tested; the defaults stay as specified. |
|
||||
|
||||
## Status
|
||||
|
||||
@@ -166,13 +164,13 @@ Bifrost endpoint). P6 is written but unverified: no cluster, registry, Vault or
|
||||
was reachable from the build environment, so the image was never built, the manifests
|
||||
were never applied, and the live checks in the deploy list above are still to do.
|
||||
|
||||
The implementation has been through a multi-agent review against `PLAN.md`, which found
|
||||
The implementation has been through a multi-agent review which found
|
||||
35 defects — four of them data-losing (a YAML typo or a UTF-8 BOM destroying a note's
|
||||
identity; `Vault.delete()` resolving the target from file contents and `rmtree`-ing
|
||||
outside the vault; `Vault.write()` escaping the vault via a caller-supplied id; and
|
||||
`mdmem eval --ablate` dropping the trigram table from the live index). All are fixed,
|
||||
each with a regression test that was checked to fail when the fix is reverted. The
|
||||
lessons are recorded as pitfalls 26–31 in `PLAN.md` §14.
|
||||
lessons are recorded as pitfalls 26–31.
|
||||
|
||||
Three tests were also found to be green for the wrong reason — a dense-retrieval test
|
||||
whose query was the note body verbatim, a double-index test that slept less than the
|
||||
|
||||
@@ -1,572 +0,0 @@
|
||||
# Security, Correctness, and Plan-Drift Analysis
|
||||
|
||||
Date: 2026-09-12
|
||||
Reviewed revision: `4cb9639` (`master`)
|
||||
Reference contract: `PLAN.md`
|
||||
Scope: `src/mdmem`, `tests`, `eval`, `deploy`, `Containerfile`, `README.md`
|
||||
|
||||
## Executive summary
|
||||
|
||||
The current test suite is green (`227 passed`) and `ruff check src tests` passes, but important
|
||||
concurrency, data-integrity, authentication, and recovery behavior remains untested. This review
|
||||
found:
|
||||
|
||||
- 2 critical findings
|
||||
- 12 high-severity findings
|
||||
- 18 medium-severity findings
|
||||
- several lower-severity plan and coverage gaps
|
||||
|
||||
The two most urgent defects are:
|
||||
|
||||
1. lifecycle/version decisions are made before the service write lock is acquired, allowing
|
||||
concurrent requests to resurrect rejected notes, overwrite accepted edits, or recreate a
|
||||
deleted note; and
|
||||
2. an unterminated frontmatter block is still classified as a new hand-written note and
|
||||
destructively rewritten.
|
||||
|
||||
The next tier is dominated by stale embedding races, filename/frontmatter identity split-brain,
|
||||
idempotency races, secret leakage through secondary write channels, and startup behavior that can
|
||||
prevent the documented lexical-only degradation mode.
|
||||
|
||||
Severity here measures likely impact to confidentiality, integrity, availability, or the core
|
||||
human-promotion trust boundary. It is not a CVSS score. No fixes were made as part of this review.
|
||||
|
||||
## Method
|
||||
|
||||
The review combined independent passes over storage/retrieval, service/MCP, and UI/deployment/eval,
|
||||
then rechecked findings against the current working tree and `PLAN.md`. High-impact claims were
|
||||
confirmed with controlled temporary-vault reproductions. No production vault or configured live
|
||||
index was used.
|
||||
|
||||
Representative observed results included:
|
||||
|
||||
- reject/update race: `reject -> archive`, then stale update -> `inbox`; final body was the agent's
|
||||
text and `rejected` was gone;
|
||||
- two concurrent identical `memory_write` calls with one `request_id` created two distinct notes;
|
||||
- delayed old embedding and new embedding both used chunk ID `1`; the old vector was stored against
|
||||
the new chunk and `vec_pending` became `0`;
|
||||
- deleting a closing `---` renamed and rewrote an active fact, changed its type to `note`, emptied
|
||||
its tags, and moved the former frontmatter into the body without reporting an error;
|
||||
- a valid allowed `Origin` paired with `Host: evil.example` was accepted;
|
||||
- signed ID tokens with `exp: NaN`, `exp: Infinity`, multiple audiences plus a mismatched `azp`, and
|
||||
a missing `sub` were accepted;
|
||||
- a note written during the scan-to-watcher startup gap remained on disk but absent from the index;
|
||||
- `MCP_TOKEN=...`, `AWS_SECRET_ACCESS_KEY=...`, `github_pat_...`, `glpat-...`, `npm_...`, and `hf_...`
|
||||
examples all evaded the secret scanner.
|
||||
|
||||
## Critical findings
|
||||
|
||||
### C-01 — Lifecycle and optimistic-version checks are outside the serialized operation
|
||||
|
||||
Affected code: `src/mdmem/service.py:465-545`, `572-605`, `613-619`, `655-719`
|
||||
Plan: §2.4, §6 single-writer rule, §9 optimistic versioning, §14.10
|
||||
|
||||
`update_note`, `approve`, `reject`, `edit`, and `hard_delete` read the note and make lifecycle or
|
||||
version decisions before acquiring `_write_lock`. Only the eventual `_persist` or deletion block
|
||||
is serialized. A queued operation therefore acts on a stale `Note` object after an earlier queued
|
||||
operation has changed the canonical file.
|
||||
|
||||
Confirmed consequences:
|
||||
|
||||
- Two concurrent inbox updates with the same `expected_version` both succeed; one silently
|
||||
overwrites the other and one returned version is stale immediately.
|
||||
- A rejected note can be rewritten back into `inbox`, losing `rejected: true`.
|
||||
- An update queued behind `hard_delete` can recreate the deleted ULID and file after deletion
|
||||
reported success.
|
||||
- An agent update queued behind approval can move an owner-approved note back to inbox with the
|
||||
agent's text.
|
||||
|
||||
Required regression coverage: deterministic barrier tests for update/update, reject/update,
|
||||
approve/update, archive/update, edit/update, and delete/update. The read, validation, mutation,
|
||||
file write, and synchronous index update must be one serialized operation.
|
||||
|
||||
### C-02 — Missing closing frontmatter delimiter causes destructive “repair”
|
||||
|
||||
Affected code: `src/mdmem/notes.py:67-95`, `src/mdmem/index.py:264-305`
|
||||
Plan: §4 identity/frontmatter preservation, §6.3 safe parsing, §14.25/§14.26
|
||||
|
||||
Strict parsing only raises for malformed YAML after the frontmatter regex has matched both
|
||||
delimiters. If the closing `---` is missing, both lenient and strict parsing return an empty map.
|
||||
`read_and_fix` then treats an existing canonical note as a clean hand-written note.
|
||||
|
||||
In the reproduction, an indexed active `fact` was renamed and rewritten, its real title was
|
||||
replaced by the old filename stem, tags were erased, and the former YAML became body text. The
|
||||
degraded metadata also replaced the good index row. Clearing `superseded_by` this way can make an
|
||||
obsolete fact searchable again. The history snapshot made during repair contains the already
|
||||
broken text and is not a dependable last-good recovery point.
|
||||
|
||||
Required regression coverage: remove only the closing delimiter, then exercise both full scan and
|
||||
watcher sync. The file, path, identity, and last-good index row must remain unchanged while a clear
|
||||
error is reported.
|
||||
|
||||
## High-severity findings
|
||||
|
||||
### H-01 — Delayed embeddings can be attached to newer text
|
||||
|
||||
Affected code: `src/mdmem/index.py:24-29,61-127,360-367`,
|
||||
`src/mdmem/service.py:227-242,714-719`
|
||||
Plan: §6 single-writer consistency, §7 recovery, §14.7
|
||||
|
||||
`PendingEmbed` records `key_hash`, but `_embed_items` discards it and `store_vectors` stores solely
|
||||
by numeric chunk ID. Embedding happens after `_write_lock` is released. Reindexing deletes and
|
||||
reinserts chunks, and SQLite can immediately reuse the old integer ID.
|
||||
|
||||
A delayed response for hash A can therefore overwrite the vector for current hash B and mark B as
|
||||
complete. This affects ordinary writes, background retries, and `reembed_all`. If the old ID is not
|
||||
reused, the code may instead attempt to create an orphan vec row.
|
||||
|
||||
Required regression: hold an old embed request open, complete an edit/reindex, then release it.
|
||||
Store a vector only when both chunk ID and `embed_text_hash` still match in the same transaction.
|
||||
|
||||
### H-02 — Filename/frontmatter disagreement creates identity and lifecycle split-brain
|
||||
|
||||
Affected code: `src/mdmem/vaultio.py:55-69`, `src/mdmem/index.py:210-305`,
|
||||
`src/mdmem/db.py:25-32`
|
||||
Plan: §4 identity and folder lifecycle source of truth
|
||||
|
||||
`read_and_fix` trusts any syntactically valid frontmatter ULID without checking the ULID encoded in
|
||||
the filename. Confirmed failure modes:
|
||||
|
||||
- Editing a canonical file's frontmatter ID to another ULID makes indexing fail on the unique path
|
||||
constraint while preserving a stale row under the filename ID. The file's new ID cannot be found
|
||||
by `Vault.find`.
|
||||
- An active original plus an archive copy carrying the original frontmatter ID is accepted with no
|
||||
conflict; last-file-wins indexing can hide the active original as archived while `Vault.find`
|
||||
still reads the active file.
|
||||
- A file such as `active/manual.md` with a valid frontmatter ID is indexed, but `Vault.find(id)` and
|
||||
`memory_get` cannot locate it because lookup uses the filename ID.
|
||||
|
||||
Required regression: reject mismatched IDs and duplicate lifecycle copies without changing the
|
||||
last-good row. Also test valid-frontmatter-ID files with noncanonical names.
|
||||
|
||||
### H-03 — Normal vault deletion leaves inbound links invisibly dangling
|
||||
|
||||
Affected code: `src/mdmem/index.py:153-158`, `src/mdmem/service.py:180-190`
|
||||
Plan: §5 unresolved links, §6 vanished-file handling, §10 health report
|
||||
|
||||
The UI hard-delete path now reindexes inbound sources, but `Indexer.remove`—used by ordinary
|
||||
filesystem deletion through scan/watcher—deletes only outgoing links. Inbound rows retain the
|
||||
deleted ULID rather than reverting to `unresolved:<target>`, so the unresolved health count remains
|
||||
zero and the broken reference is invisible until its source is reindexed for another reason.
|
||||
|
||||
Required regression: delete a target through both scan and watcher paths; surviving source links
|
||||
must become unresolved and flags for the vanished note must be handled explicitly.
|
||||
|
||||
### H-04 — Service updates can write NFC to disk while indexing NFD
|
||||
|
||||
Affected code: `src/mdmem/notes.py:98-130`, `src/mdmem/vaultio.py:88-119`,
|
||||
`src/mdmem/service.py:531-545,655-684,714-719`
|
||||
Plan: §4 and §14.8 NFC normalization
|
||||
|
||||
`dumps` normalizes a deep copy, not the live `Note`. `_persist` writes the normalized serialization
|
||||
and then indexes the original object. An inbox update with NFD text produced NFC on disk but NFD in
|
||||
`chunks.body` and in the embedding input. Because the indexed file hash is the canonical disk hash,
|
||||
the watcher considers the service event already handled and does not repair the mismatch.
|
||||
|
||||
Required regression: update and UI-edit with NFD in body, frontmatter, relations, and link targets;
|
||||
assert the live note, disk, chunks, FTS/trigram text, links, and embedding input all use NFC.
|
||||
|
||||
### H-05 — Scan-to-watcher startup gap can permanently miss changes
|
||||
|
||||
Affected code: `src/mdmem/service.py:117-157`
|
||||
Plan: §6 startup scan/watcher, P1 live-edit latency
|
||||
|
||||
The watcher starts only after scan, secret scan, self-test, and pending embeddings. A file changed
|
||||
after its scan pass but before watcher registration emits no event. A controlled reproduction wrote
|
||||
a second valid note while `embed_pending` was blocked; after startup finished, the file existed but
|
||||
the note was not indexed.
|
||||
|
||||
Required regression: mutate a file after initial scan while startup embedding is paused. Start
|
||||
watching before reconciliation or run a final scan after watcher registration.
|
||||
|
||||
### H-06 — A Bifrost outage can put Kubernetes into a cold-start restart loop
|
||||
|
||||
Affected code: `src/mdmem/server.py:67-72`, `src/mdmem/service.py:117-157`,
|
||||
`src/mdmem/embed.py:160-199`, `deploy/kubernetes/apps/apps/mdmem/mdmem/app/deployment.yaml:58-69`
|
||||
Plan: §7, §13a.3, §14.17 lexical-only degradation
|
||||
|
||||
The process awaits `svc.start()` before either listener binds. Startup synchronously awaits an
|
||||
embedding self-test and pending embeddings. Each can spend three 60-second attempts. The deployment
|
||||
has liveness starting after 30 seconds, no `startupProbe`, and the default failure threshold.
|
||||
Kubernetes can kill the pod before `/healthz` ever binds, repeatedly preventing the promised
|
||||
lexical-only mode.
|
||||
|
||||
Required regression: start the real server with a blocking/blackholed embedder and prove liveness,
|
||||
readiness, and lexical search bind before the probe deadline. Deployment tests must cover the
|
||||
startup probe budget.
|
||||
|
||||
### H-07 — Common explicit credential forms evade the secret scanner
|
||||
|
||||
Affected code: `src/mdmem/secrets.py:11-29`
|
||||
Plan: §9 and §14.12
|
||||
|
||||
The generic assignment regex requires a word boundary before names such as `token` and `secret`.
|
||||
Because underscore is a word character, common environment variable names evade it. Provider
|
||||
patterns are also incomplete. Confirmed misses include `MCP_TOKEN=...`,
|
||||
`AWS_SECRET_ACCESS_KEY=...`, `OPENAI_API_KEY=...`, `GITHUB_TOKEN=...`, `github_pat_...`, `glpat-...`,
|
||||
`npm_...`, and `hf_...`.
|
||||
|
||||
Required regression: current provider token formats and standard environment assignments,
|
||||
including `export`, quoting, mixed case where applicable, and every serialized note field.
|
||||
|
||||
### H-08 — Secondary persistent channels bypass secret rejection and can leak into audit
|
||||
|
||||
Affected code: `src/mdmem/service.py:556-569`, `src/mdmem/mcp_server.py:90-128,350-354`
|
||||
Plan: §4 untrusted `source`, §9 audit, §14.12
|
||||
|
||||
`flag.reason` is agent-controlled and persisted in SQLite without a scan. `request_id` is stored and
|
||||
logged raw without a size or secret check. Audit `source` is copied from self-reported note metadata
|
||||
before note validation. A write rejected for `source.harness=AKIAIOSFODNN7EXAMPLE` still copied that
|
||||
full key into `audit.jsonl`; the same value in a flag reason was stored and returned by `memory_get`.
|
||||
|
||||
This is also provenance confusion: the audit UI presents an untrusted, caller-asserted source as
|
||||
attribution.
|
||||
|
||||
Required regression: inspect the audit file after rejected source/request ID values, and test flag
|
||||
reason storage. Store bounded/redacted identifiers, not unchecked caller strings.
|
||||
|
||||
### H-09 — Idempotency is race-prone and has inconsistent key semantics
|
||||
|
||||
Affected code: `src/mdmem/mcp_server.py:83-129`, `src/mdmem/db.py:58-59`,
|
||||
`src/mdmem/service.py:777-813`
|
||||
Plan: §9 replay guarantee
|
||||
|
||||
The guard performs `get -> side effect -> put` without synchronization. Two concurrent identical
|
||||
writes with one request ID both missed and created different notes. Concurrent different arguments
|
||||
also both succeeded instead of returning `IDEMPOTENCY_CONFLICT`.
|
||||
|
||||
The table primary key is `request_id`, while lookup treats `(request_id, tool)` as the key.
|
||||
`INSERT OR REPLACE` for a second tool deletes the first tool's replay record. Replaying the original
|
||||
operation can then repeat its side effect.
|
||||
|
||||
Required regression: concurrent same/different arguments, same ID across tools, in-progress
|
||||
requests, and failure after mutation but before idempotency persistence. Claim and commit an
|
||||
idempotency key atomically before performing a mutating operation.
|
||||
|
||||
### H-10 — Revision approval has no target-version or target-lifecycle check
|
||||
|
||||
Affected code: `src/mdmem/service.py:494-528,572-605`
|
||||
Plan: §2.4 and §9 revision review
|
||||
|
||||
A proposal records only `revises`, not the version it was based on. `_merge_revision` blindly loads
|
||||
the current target and overwrites it. Confirmed outcomes:
|
||||
|
||||
- active v1 -> agent proposal v2 -> owner edit v3 -> approve proposal results in v2, silently losing
|
||||
the newer owner edit;
|
||||
- active target -> proposal -> reject/archive target -> approve proposal rewrites the archived
|
||||
target while retaining rejection state, so a later restore promotes the agent text.
|
||||
|
||||
Required regression: target edited, archived, rejected, deleted, or superseded after proposal
|
||||
creation. Approval should refuse stale proposals or require an explicit owner conflict resolution.
|
||||
|
||||
### H-11 — Supersede proposals bypass lifecycle and exclusivity invariants
|
||||
|
||||
Affected code: `src/mdmem/service.py:548-554,572-593`
|
||||
Plan: §2.4, §2.6, §9
|
||||
|
||||
`supersede` accepts archived, rejected, inbox, and already-superseded targets. A rejected item can be
|
||||
re-proposed through this path despite the explicit archived-note guard on update. Two successor
|
||||
proposals can also both be approved: both remain active and point back to the old fact, while the
|
||||
old note's singular `superseded_by` points only to the second. Search can expose contradictory
|
||||
“current” facts.
|
||||
|
||||
Required regression: archived/rejected/inbox/already-superseded/nonexistent targets, concurrent and
|
||||
sequential competing successors, and bidirectional chain consistency.
|
||||
|
||||
### H-12 — Zero and non-finite embeddings are accepted as healthy
|
||||
|
||||
Affected code: `src/mdmem/embed.py:39-42,178-188`, `src/mdmem/search.py:72-74`
|
||||
Plan: §2.1 known all-zero failure, §7 startup self-test, §14.1
|
||||
|
||||
Normalization leaves zero vectors at zero and propagates NaN/Infinity. Shape checks accept them,
|
||||
including during startup self-test. Two zero vectors have distance zero, which search converts to
|
||||
cosine `1.0`, creating a falsely perfect semantic match.
|
||||
|
||||
Required regression: zero, NaN, and infinite vectors for self-test, documents, and queries. Invalid
|
||||
vectors must not be cached or stored, and search must degrade to lexical.
|
||||
|
||||
## Medium-severity findings
|
||||
|
||||
### M-01 — Flags violate the Markdown source-of-truth and recovery contract
|
||||
|
||||
Affected code: `src/mdmem/db.py:53-56`, `src/mdmem/service.py:556-569`
|
||||
Plan: §0 Markdown canonical; SQLite rebuildable
|
||||
|
||||
Flags exist only in SQLite. Deleting/rebuilding `index.db` from an unchanged vault changes
|
||||
`flags_for(id)` from populated to empty. Either make flags canonical in Markdown or explicitly
|
||||
change the backup/recovery contract and back up this mutable database state.
|
||||
|
||||
Missing test: flags survive full index recreation.
|
||||
|
||||
### M-02 — Temporal values are unvalidated strings compared lexicographically
|
||||
|
||||
Affected code: `src/mdmem/service.py:417-450`, `src/mdmem/search.py:194-199,327-332`
|
||||
Plan: §2.6, §4 UTC timestamps, §8 filters
|
||||
|
||||
Malformed values, offsets, and `valid_from >= valid_until` are accepted. Lexical comparison is not
|
||||
instant comparison: for example, equivalent offset and `Z` timestamps can filter differently.
|
||||
|
||||
Missing tests: RFC3339 validity, UTC normalization, ordering, equivalent instants, and boundary
|
||||
semantics for stored values and search arguments.
|
||||
|
||||
### M-03 — Unicode and trigram lexical hits are not merged by better rank
|
||||
|
||||
Affected code: `src/mdmem/search.py:120-150`
|
||||
Plan: §8.1
|
||||
|
||||
The implementation mixes raw BM25 magnitudes from different FTS tables and ignores trigram whenever
|
||||
the same chunk has a unicode61 hit. This does not implement “keep the better rank per chunk.”
|
||||
|
||||
Missing test: controlled rank lists where a chunk is rank 50 in unicode61 and rank 1 in trigram.
|
||||
|
||||
### M-04 — Late link resolution misses equivalent slug spellings
|
||||
|
||||
Affected code: `src/mdmem/index.py:182-207`
|
||||
Plan: §5 exact-title then slug resolution
|
||||
|
||||
Initial resolution slugifies a target, but late resolution updates only two literal unresolved
|
||||
keys. `[[Later Note]]` remains unresolved when a later note titled `Later Note` arrives, although
|
||||
both slugify identically.
|
||||
|
||||
Missing tests: whitespace, punctuation, capitalization, and umlaut variants on forward links.
|
||||
|
||||
### M-05 — Search accepts invalid bounds and can exceed `max_chars`
|
||||
|
||||
Affected code: `src/mdmem/search.py:303-318`
|
||||
Plan: §8.7, §14.11
|
||||
|
||||
The first result is always emitted even when it exceeds the requested budget; zero or negative
|
||||
budgets still return one result. A negative internal `k` uses Python slicing and can return nearly
|
||||
all candidates. The current test explicitly permits an 800-character overrun, contradicting the
|
||||
plan.
|
||||
|
||||
Missing tests: `k <= 0`, `max_chars <= 0`, oversized first title/snippet, exact-budget boundary, and
|
||||
structured MCP validation.
|
||||
|
||||
### M-06 — One long token bypasses the chunk target
|
||||
|
||||
Affected code: `src/mdmem/chunker.py:50-89`
|
||||
Plan: §6.4
|
||||
|
||||
`_split_words` does not split a single token longer than the limit. Agent content can reach 8,000
|
||||
characters and hand-authored notes are unbounded, producing oversized chunks.
|
||||
|
||||
Missing test: a no-whitespace token larger than the chunk limit, with bounded lossless output.
|
||||
|
||||
### M-07 — `is_ulid` accepts overflow encodings rejected by the ULID library
|
||||
|
||||
Affected code: `src/mdmem/ids.py:13,38-39`
|
||||
Plan: §4/§6 valid identity
|
||||
|
||||
Validation checks only length and alphabet. Values such as
|
||||
`ZZZZZZZZZZZZZZZZZZZZZZZZZZ` pass but `ULID.from_str` rejects them as overflow.
|
||||
|
||||
Missing property: `is_ulid(value)` implies successful parsing by the canonical ULID implementation.
|
||||
|
||||
### M-08 — OIDC claim validation accepts non-finite expiry, missing subject, and bad `azp`
|
||||
|
||||
Affected code: `src/mdmem/ui/auth.py:210-245`
|
||||
Plan: §10 fail closed
|
||||
|
||||
Signed tokens with `exp: NaN` and `exp: Infinity` pass because any int/float is accepted and
|
||||
non-finite comparisons do not expire them. `sub` is not required, allowing an empty-subject session.
|
||||
When `aud` contains multiple values, `azp` is not required or checked; a token with mdmem among its
|
||||
audiences but `azp` naming another client was accepted.
|
||||
|
||||
Missing tests: finite expiry, required nonempty subject, and multi-audience authorized-party rules.
|
||||
|
||||
### M-09 — Scalar group claim is accepted contrary to the fail-closed plan
|
||||
|
||||
Affected code: `src/mdmem/ui/auth.py:60-90`, `tests/test_ui_auth.py:59-60`
|
||||
Plan: §10, P4, §14.21
|
||||
|
||||
The code converts a scalar string group to a list, and a test deliberately blesses it. The plan
|
||||
requires a present-but-wrong-type group claim to return 403.
|
||||
|
||||
Required change to coverage: only `list[str]` should pass; scalar, null, mapping, and mixed lists
|
||||
must fail.
|
||||
|
||||
### M-10 — UI POST protection still does not validate Host
|
||||
|
||||
Affected code: `src/mdmem/ui/app.py:85-105,464-480`
|
||||
Plan: §10 Origin/Host allowlist
|
||||
|
||||
The recent Origin/Referer fix correctly rejects headerless writes, but when configured origins are
|
||||
present, Host is ignored. A request with allowed `Origin` and `Host: evil.example` was accepted.
|
||||
|
||||
Missing tests: invalid Host with valid Origin, proxy-forwarded host behavior, and the exact deployed
|
||||
host allowlist.
|
||||
|
||||
### M-11 — JWKS is cached forever, breaking key rotation until restart
|
||||
|
||||
Affected code: `src/mdmem/ui/auth.py:149-165`
|
||||
Plan: §10 operationally reliable OIDC
|
||||
|
||||
An unknown `kid` does not trigger one forced JWKS refresh. New logins fail after PocketID signing-key
|
||||
rotation until the pod restarts.
|
||||
|
||||
Missing test: cache key A, publish key B, present a B-signed token, and verify one refresh succeeds.
|
||||
|
||||
### M-12 — Audit hashes omit actual arguments and unexpected errors vanish
|
||||
|
||||
Affected code: `src/mdmem/mcp_server.py:83-129,167,335-339`
|
||||
Plan: §9 every tool call logged with argument hash, source, latency, IDs
|
||||
|
||||
Search hashes only query and `k`, omitting filters. Flag hashes only ID, omitting reason. Different
|
||||
calls therefore receive identical audit fingerprints. `_guard` catches only `MdmemError`; an
|
||||
injected database/programming exception produced no audit entry at all.
|
||||
|
||||
Missing tests: vary every argument and require a hash change; inject unexpected exceptions and
|
||||
require `ok:false`, latency, request ID, and a bounded error class.
|
||||
|
||||
### M-13 — `memory_flag` is non-idempotent
|
||||
|
||||
Affected code: `src/mdmem/mcp_server.py:328-339`
|
||||
Plan: §9 idempotency
|
||||
|
||||
The mutating flag tool has no `request_id`. A transport retry creates duplicate review items. Search
|
||||
and get also omit the plan's broadly stated optional request ID, though they have no side effect.
|
||||
|
||||
Missing test: replaying one flag request creates exactly one flag.
|
||||
|
||||
### M-14 — Evaluation can recommend a variant with a catastrophic category regression
|
||||
|
||||
Affected code: `src/mdmem/evalharness.py:340-372`
|
||||
Plan: §11 “without a regression in any category beyond noise”
|
||||
|
||||
`pick_default` checks only overall nDCG and confidence intervals. It never examines `per_category`.
|
||||
|
||||
Missing test: a candidate with a clear overall win but severe temporal or update regression must
|
||||
remain unselected.
|
||||
|
||||
### M-15 — Eval output is not standards-compliant JSON
|
||||
|
||||
Affected code: `src/mdmem/evalharness.py:90-117,160-174,395-425`,
|
||||
`eval/results/2026-09-12-synthetic-smoke.json:538-541`
|
||||
Plan: §11 committed JSON result artifact
|
||||
|
||||
Unavailable metrics remain `float("nan")`; `json.dumps` emits raw `NaN`, which RFC 8259 does not
|
||||
permit. The test named as a CLI writer test manually writes a different object and never calls
|
||||
`run_eval_cli`, leaving serialization false-green.
|
||||
|
||||
Missing test: invoke the actual CLI and parse with a strict JSON parser; unavailable values should
|
||||
serialize as `null`.
|
||||
|
||||
### M-16 — Eval latency is order-biased by a shared embedding cache
|
||||
|
||||
Affected code: `src/mdmem/evalharness.py:293-312`, `src/mdmem/service.py:99-115`
|
||||
Plan: §11 fair ablation comparison
|
||||
|
||||
Each variant gets an isolated index but retains the configured `cache_db`. The first real-model
|
||||
variant pays query embedding latency while later variants reuse cached query embeddings, making p95
|
||||
latency incomparable.
|
||||
|
||||
Missing test: instrument backend query calls and prove equivalent cache state or perform an explicit
|
||||
warm-up excluded from every variant's timing.
|
||||
|
||||
### M-17 — Required Prometheus signals are absent
|
||||
|
||||
Affected code: `src/mdmem/ui/app.py:75-145`
|
||||
Plan: §13a.7
|
||||
|
||||
Embedding error count, search-latency histogram, and MCP calls by tool/outcome are not exported.
|
||||
Search/write totals are Gauges rather than Counters. Existing tests only check that generic metrics
|
||||
exist and omit note content.
|
||||
|
||||
Missing test: scrape and assert all required metric families and bounded label cardinality.
|
||||
|
||||
### M-18 — UI state filtering and state transitions do not enforce the displayed workflow
|
||||
|
||||
Affected code: `src/mdmem/ui/app.py:261-300`, `src/mdmem/service.py:638-653`
|
||||
Plan: §2.4, §10 Search filters and review workflow
|
||||
|
||||
Search `state=inbox` includes both active and inbox notes, while `state=archive` includes active,
|
||||
inbox, and archive. In addition, the generic state endpoint can promote an inbox revision or
|
||||
supersede proposal directly to active, bypassing `approve` merge/chain behavior if called manually.
|
||||
|
||||
Missing tests: exact results for every state filter and forbidden lifecycle transitions that must
|
||||
go through approval.
|
||||
|
||||
## Lower-severity plan drift and test-suite gaps
|
||||
|
||||
- Health has no “run eval” action required by §10 (`src/mdmem/ui/templates/health.html`,
|
||||
`src/mdmem/ui/app.py:443-460`).
|
||||
- History compares one snapshot only to current, not any two revisions as required by §10
|
||||
(`src/mdmem/ui/app.py:313-324`).
|
||||
- Browse does not offer sorting by tags as required by §10.
|
||||
- Health audit rows do not display the recorded `source` field.
|
||||
- “Edit & approve” converts an intentionally empty body to `None`, approving the old body instead
|
||||
(`src/mdmem/ui/app.py:235-239`). Empty must be saved or explicitly rejected, not ignored.
|
||||
- `AuditLog.recent` reads persisted history only while its in-memory list is completely empty. After
|
||||
restart and one new call, Health shows only that call rather than the last 100 across restart
|
||||
(`src/mdmem/logging.py:77-92`).
|
||||
- `memory_supersede` omits relations, validity bounds, and source metadata available to
|
||||
`memory_write`; either complete the §9 schema or narrow the contract.
|
||||
- Bearer configuration accepts whitespace/non-ASCII values that clients cannot reliably present;
|
||||
startup should validate a trimmed ASCII token with an explicit minimum length.
|
||||
- `validate_id` is dead code. ID arguments have no schema pattern or maximum length and are echoed
|
||||
in `NOT_FOUND` responses.
|
||||
- The session test named “round trip and expiry” never advances time or tests expiration.
|
||||
- No route-level OIDC test covers discovery, state, PKCE, nonce carriage, callback errors, cookie
|
||||
flags, expiry, logout, or signing-key rotation.
|
||||
- The OIDC HTTP client created in `build_ui_app` has no application shutdown hook calling
|
||||
`OidcClient.aclose`.
|
||||
- CLI tests cover help/version but not serve, doctor, reindex, eval, contradictory mode flags,
|
||||
startup failures, or graceful shutdown.
|
||||
- The note round-trip property test generates a body but does not assert its round trip and does not
|
||||
property-test arbitrary unknown-key order/comments.
|
||||
- The clean manifest template is not directly buildable: `app/kustomization.yaml` references the
|
||||
intentionally absent encrypted `secret.yaml`; image digest and PocketID client ID remain release
|
||||
placeholders. These are documented setup blockers, but there is no CI smoke test for the rendered
|
||||
deployment overlay.
|
||||
- The only committed evaluation is a synthetic fake-embedding smoke run. It proves harness wiring,
|
||||
not the P5 acceptance criterion of 50–100 real labelled queries and a production-model tuning
|
||||
report.
|
||||
|
||||
## Test integrity assessment
|
||||
|
||||
The suite is useful for sequential happy paths but systematically underrepresents interleavings,
|
||||
recovery from canonical Markdown, deployment startup, and strict external-format validation.
|
||||
|
||||
Highest-value missing test groups, in order:
|
||||
|
||||
1. deterministic service concurrency tests around the whole read/check/write operation;
|
||||
2. chunk-hash-guarded embedding races, including background retry and re-embed;
|
||||
3. malformed frontmatter delimiter and filename/frontmatter identity conflict cases;
|
||||
4. startup with a blocked embedding backend plus a mutation during watcher registration;
|
||||
5. concurrent idempotency and crash-between-side-effect-and-record cases;
|
||||
6. secret scanning and audit redaction across every persistent channel;
|
||||
7. full OIDC route tests with strict claims and signing-key rotation;
|
||||
8. actual CLI eval serialization through a strict JSON parser;
|
||||
9. disaster rebuild proving all canonical state survives; and
|
||||
10. deployment render/probe smoke tests.
|
||||
|
||||
Some current tests actively encode plan drift rather than merely omitting it:
|
||||
|
||||
- `tests/test_ui_auth.py` accepts a scalar group claim despite §14.21;
|
||||
- `tests/test_search.py` permits the first result to exceed `max_chars` by up to one snippet;
|
||||
- `tests/test_evalharness.py::test_cli_reports_and_writes_a_json_file` does not invoke the CLI it
|
||||
names.
|
||||
|
||||
## Recommended remediation order
|
||||
|
||||
1. Serialize lifecycle/version operations end-to-end and add deterministic race tests.
|
||||
2. Reject unterminated/mismatched frontmatter without touching the file or last-good index.
|
||||
3. Guard vector writes by chunk ID plus embedding-text hash.
|
||||
4. Make startup bind in lexical mode independently of Bifrost and close the scan/watch gap.
|
||||
5. Make idempotency atomic and reconcile its database key.
|
||||
6. Close all secret-bearing secondary channels and redact audit fields.
|
||||
7. Enforce revision/supersede target version, lifecycle, and exclusivity.
|
||||
8. Validate embeddings, timestamps, ULIDs, OIDC claims, Host, and response bounds.
|
||||
9. Restore the Markdown-only recovery guarantee for flags or explicitly redesign it.
|
||||
10. Repair eval selection/serialization/fairness, observability, and the lower-priority UI drift.
|
||||
|
||||
## Baseline notes
|
||||
|
||||
The following earlier findings are fixed in the reviewed revision and are intentionally not listed
|
||||
as open defects above: full-field note secret scanning, mandatory `expected_version`, archived-note
|
||||
update rejection in the sequential case, approval validation, audit-file wiring, ordinary
|
||||
hard-delete link repair, background retry of pending vectors, safe re-embed on a simple outage,
|
||||
trigram backfill after toggling it off/on, missing-Origin CSRF rejection, nonce/`exp`/`iat` presence
|
||||
checks, eval index isolation, reciprocal synthetic supersede data, connection/cache shutdown, and
|
||||
the direct fusion formula tests.
|
||||
@@ -1,19 +0,0 @@
|
||||
{
|
||||
"_howto": "Add this object to mcp.client_configs in kubernetes/apps/apps/bifrost/bifrost/app/config.json. That file is a configMapGenerator input, so editing it is the whole change — the generated ConfigMap's hash suffix moves and Flux rolls the Deployment. Commit it only AFTER mdmem itself is healthy, so a failure is attributable to one change.",
|
||||
"name": "mdmem",
|
||||
"connection_type": "http",
|
||||
"connection_string": "http://mdmem.apps.svc.cluster.local:8080/mcp",
|
||||
"auth_type": "headers",
|
||||
"headers": { "Authorization": "env.MDMEM_AUTH_HEADER" },
|
||||
"is_ping_available": false,
|
||||
"is_code_mode_client": false,
|
||||
"_comment": "Long-term agent memory. Code Mode deliberately off: memory bodies are untrusted text, and code mode hands tool output to a model that then writes executable code. Revisit only after the prompt-injection behaviour is tested with a local model. tools_to_execute is an explicit allow-list, not '*', so a future mdmem release cannot add a destructive tool that silently becomes reachable.",
|
||||
"tools_to_execute": [
|
||||
"memory_search",
|
||||
"memory_get",
|
||||
"memory_write",
|
||||
"memory_update",
|
||||
"memory_supersede",
|
||||
"memory_flag"
|
||||
]
|
||||
}
|
||||
@@ -1,5 +0,0 @@
|
||||
---
|
||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
resources:
|
||||
- ./mdmem/ks.yaml
|
||||
@@ -1,88 +0,0 @@
|
||||
---
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: mdmem
|
||||
labels:
|
||||
app: mdmem
|
||||
spec:
|
||||
replicas: 1
|
||||
strategy:
|
||||
type: Recreate # SQLite + watchfiles need a single writer
|
||||
selector:
|
||||
matchLabels:
|
||||
app: mdmem
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: mdmem
|
||||
spec:
|
||||
securityContext:
|
||||
runAsUser: 1000
|
||||
runAsGroup: 1000
|
||||
runAsNonRoot: true
|
||||
fsGroup: 1000
|
||||
fsGroupChangePolicy: OnRootMismatch # Longhorn: else /data comes up root-owned
|
||||
containers:
|
||||
- name: mdmem
|
||||
# Pin tag AND digest. Renovate does not track library/*, so the digest
|
||||
# is bumped by hand in the same commit that pushes a build (§13a.2).
|
||||
image: harbor.cloud.traunseenet.com/library/mdmem:0.1.0@sha256:4b0c21bec82f12ae5f6e3b5398fc6af31fecf3bfb5c0d96b44acd85bf638d1f4
|
||||
imagePullPolicy: IfNotPresent
|
||||
args: ["serve"]
|
||||
securityContext:
|
||||
allowPrivilegeEscalation: false
|
||||
readOnlyRootFilesystem: true
|
||||
capabilities:
|
||||
drop: ["ALL"]
|
||||
env:
|
||||
- name: HOME
|
||||
value: /tmp
|
||||
- name: XDG_CACHE_HOME
|
||||
value: /tmp
|
||||
envFrom:
|
||||
- configMapRef:
|
||||
name: mdmem-env
|
||||
- secretRef:
|
||||
name: mdmem # from Vault via the ExternalSecret
|
||||
ports:
|
||||
- name: mcp
|
||||
containerPort: 8080
|
||||
- name: ui
|
||||
containerPort: 8081
|
||||
volumeMounts:
|
||||
- name: data
|
||||
mountPath: /data
|
||||
- name: tmp
|
||||
mountPath: /tmp
|
||||
livenessProbe:
|
||||
httpGet:
|
||||
path: /healthz
|
||||
port: ui
|
||||
initialDelaySeconds: 30
|
||||
periodSeconds: 30
|
||||
startupProbe:
|
||||
httpGet:
|
||||
path: /healthz
|
||||
port: ui
|
||||
periodSeconds: 5
|
||||
failureThreshold: 36
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
path: /readyz # never checks Bifrost (§14.17)
|
||||
port: ui
|
||||
initialDelaySeconds: 5
|
||||
periodSeconds: 10
|
||||
resources:
|
||||
requests:
|
||||
cpu: 100m
|
||||
memory: 256Mi
|
||||
limits:
|
||||
cpu: "1"
|
||||
memory: 1Gi
|
||||
volumes:
|
||||
- name: data
|
||||
persistentVolumeClaim:
|
||||
claimName: mdmem-data
|
||||
- name: tmp
|
||||
emptyDir: {}
|
||||
@@ -1,17 +0,0 @@
|
||||
---
|
||||
apiVersion: external-secrets.io/v1
|
||||
kind: ExternalSecret
|
||||
metadata:
|
||||
name: mdmem
|
||||
spec:
|
||||
refreshInterval: 5m
|
||||
secretStoreRef:
|
||||
name: vault-backend
|
||||
kind: ClusterSecretStore
|
||||
target:
|
||||
name: mdmem
|
||||
dataFrom:
|
||||
# KV-v2 mount is `secret`, so this reads secret/apps/mdmem:
|
||||
# MCP_TOKEN, BIFROST_VK, OIDC_CLIENT_SECRET, SESSION_SECRET
|
||||
- extract:
|
||||
key: apps/mdmem
|
||||
@@ -1,25 +0,0 @@
|
||||
# Only the UI port is routed. :8080 (MCP) gets no HTTPRoute — Bifrost reaches it
|
||||
# at mdmem.apps.svc.cluster.local:8080, same namespace (§13a.5).
|
||||
#
|
||||
# Do not apply this before the OIDC group check is live (§14.19): *.cloud is
|
||||
# split-horizon, not private.
|
||||
---
|
||||
apiVersion: gateway.networking.k8s.io/v1
|
||||
kind: HTTPRoute
|
||||
metadata:
|
||||
name: mdmem
|
||||
spec:
|
||||
parentRefs:
|
||||
- name: traefik
|
||||
namespace: networking
|
||||
sectionName: websecure-cloud
|
||||
hostnames:
|
||||
- memory.cloud.traunseenet.com
|
||||
rules:
|
||||
- matches:
|
||||
- path:
|
||||
type: PathPrefix
|
||||
value: /
|
||||
backendRefs:
|
||||
- name: mdmem
|
||||
port: 8081
|
||||
@@ -1,56 +0,0 @@
|
||||
---
|
||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
namespace: apps
|
||||
resources:
|
||||
- ./externalsecret.yaml
|
||||
- ./secret.yaml # SOPS: restic-mdmem only (§12.1)
|
||||
- ./pvc.yaml
|
||||
- ./deployment.yaml
|
||||
- ./service.yaml
|
||||
- ./httproute.yaml
|
||||
- ./networkpolicy.yaml
|
||||
- ./servicemonitor.yaml
|
||||
configMapGenerator:
|
||||
- name: mdmem-env
|
||||
literals:
|
||||
# --- storage ---------------------------------------------------------
|
||||
- VAULT_DIR=/data/vault
|
||||
- INDEX_DB=/data/index.db
|
||||
- CACHE_DB=/data/emb_cache.db
|
||||
- HISTORY_DIR=/data/.history
|
||||
- AUDIT_LOG=/data/audit.jsonl
|
||||
# --- embeddings ------------------------------------------------------
|
||||
- BIFROST_URL=http://bifrost.apps.svc.cluster.local:8080
|
||||
# provider-prefixed; a bare alias 404s and looks like an mdmem bug (§7, §14.14)
|
||||
- EMBED_MODEL=llamaswap/qwen3-embedding-4b
|
||||
- EMBED_DIM=2560
|
||||
# --- retrieval -------------------------------------------------------
|
||||
- FUSION=rrf
|
||||
- CC_ALPHA=0.5
|
||||
- FTS_TRIGRAM=1
|
||||
- INBOX_WEIGHT=0.8
|
||||
- BM25_WEIGHTS=3,2,1.5,1
|
||||
- INCLUDE_INBOX_DEFAULT=true
|
||||
- MIN_SCORE=
|
||||
- MAX_CHARS_DEFAULT=6000
|
||||
- DUP_THRESHOLD=0.90
|
||||
- MAX_NOTE_CHARS=8000
|
||||
# --- listeners -------------------------------------------------------
|
||||
- MCP_ADDR=0.0.0.0:8080
|
||||
- UI_ADDR=0.0.0.0:8081
|
||||
# the MCP SDK checks the Host header; Bifrost dials the service name (§14.24)
|
||||
- MCP_ALLOWED_HOSTS=mdmem.apps.svc.cluster.local:8080,mdmem:8080,localhost:8080
|
||||
# --- ui / auth -------------------------------------------------------
|
||||
- UI_BASE_URL=https://memory.cloud.traunseenet.com
|
||||
- UI_ALLOWED_ORIGINS=https://memory.cloud.traunseenet.com
|
||||
- OIDC_ISSUER=https://pocketid.cloud.traunseenet.com
|
||||
- OIDC_SCOPE=openid email profile groups
|
||||
- OIDC_CLIENT_ID=dc2aa40e-4386-46cc-bae6-0259a2e7490e
|
||||
- OIDC_GROUPS_CLAIM=groups
|
||||
- OIDC_REQUIRED_GROUP=mdmem_admins
|
||||
- SESSION_TTL_HOURS=12
|
||||
# --- misc ------------------------------------------------------------
|
||||
- LOG_LEVEL=info
|
||||
- LOG_STYLE=json
|
||||
- TZ=Europe/Vienna
|
||||
@@ -1,49 +0,0 @@
|
||||
# Required, not optional (§13a.5a). newt's egress policy allows the entire apps
|
||||
# namespace on every port and reaches pods directly rather than through Traefik,
|
||||
# so without this policy "not published through Pangolin" is a promise that
|
||||
# lives in a remote dashboard instead of in git — and :8080, whose only control
|
||||
# is a bearer token, would be one click from the internet.
|
||||
#
|
||||
# After applying, verify the two things it can silently break: the pod stays
|
||||
# Ready (kubelet probes) and the Prometheus target stays UP (§14.23).
|
||||
---
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
name: mdmem-ingress
|
||||
spec:
|
||||
podSelector:
|
||||
matchLabels:
|
||||
app: mdmem
|
||||
policyTypes: ["Ingress"]
|
||||
ingress:
|
||||
# MCP: Bifrost only, same namespace
|
||||
- from:
|
||||
- namespaceSelector:
|
||||
matchLabels:
|
||||
kubernetes.io/metadata.name: apps
|
||||
podSelector:
|
||||
matchLabels:
|
||||
app: bifrost
|
||||
ports:
|
||||
- protocol: TCP
|
||||
port: 8080
|
||||
# UI: Traefik only — deliberately NOT newt, so Pangolin cannot bypass the gateway
|
||||
- from:
|
||||
- namespaceSelector:
|
||||
matchLabels:
|
||||
kubernetes.io/metadata.name: networking
|
||||
podSelector:
|
||||
matchLabels:
|
||||
app.kubernetes.io/name: traefik
|
||||
ports:
|
||||
- protocol: TCP
|
||||
port: 8081
|
||||
# /metrics scrape
|
||||
- from:
|
||||
- namespaceSelector:
|
||||
matchLabels:
|
||||
kubernetes.io/metadata.name: observability
|
||||
ports:
|
||||
- protocol: TCP
|
||||
port: 8081
|
||||
@@ -1,38 +0,0 @@
|
||||
---
|
||||
apiVersion: volsync.backube/v1alpha1
|
||||
kind: ReplicationDestination
|
||||
metadata:
|
||||
name: mdmem-restore
|
||||
spec:
|
||||
trigger:
|
||||
manual: restore-once
|
||||
restic:
|
||||
repository: restic-mdmem
|
||||
copyMethod: Snapshot
|
||||
storageClassName: longhorn-storage
|
||||
volumeSnapshotClassName: longhorn-snapshot
|
||||
accessModes: ["ReadWriteOnce"]
|
||||
capacity: 5Gi
|
||||
cacheCapacity: 1Gi
|
||||
enableFileDeletion: false
|
||||
moverSecurityContext:
|
||||
runAsUser: 1000
|
||||
runAsGroup: 1000
|
||||
fsGroup: 1000
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: PersistentVolumeClaim
|
||||
metadata:
|
||||
name: mdmem-data
|
||||
spec:
|
||||
accessModes: ["ReadWriteOnce"]
|
||||
storageClassName: longhorn-storage
|
||||
# Do NOT set numberOfReplicas: 1 here. The vault is the primary artefact of
|
||||
# this service; an hourly restic snapshot is not block-level redundancy.
|
||||
dataSourceRef:
|
||||
kind: ReplicationDestination
|
||||
apiGroup: volsync.backube
|
||||
name: mdmem-restore
|
||||
resources:
|
||||
requests:
|
||||
storage: 5Gi
|
||||
@@ -1,17 +0,0 @@
|
||||
# Rename to secret.yaml and SOPS-encrypt before committing. This is the ONE
|
||||
# secret that stays out of Vault (§12.1): the restore path must work when Vault
|
||||
# itself is what is being restored.
|
||||
#
|
||||
# sops --encrypt --age age1q3mu6g0l3kr7spszks69yqrvlxvlgfrqgw9r00qqt8cj9xpjd9mqdkd7um \
|
||||
# --encrypted-regex '^(data|stringData)$' --in-place secret.yaml
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: restic-mdmem
|
||||
type: Opaque
|
||||
stringData:
|
||||
RESTIC_REPOSITORY: s3:https://s3.example/restic-mdmem
|
||||
RESTIC_PASSWORD: REPLACE
|
||||
AWS_ACCESS_KEY_ID: REPLACE
|
||||
AWS_SECRET_ACCESS_KEY: REPLACE
|
||||
@@ -1,17 +0,0 @@
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: mdmem
|
||||
labels:
|
||||
app: mdmem
|
||||
spec:
|
||||
selector:
|
||||
app: mdmem
|
||||
ports:
|
||||
- name: mcp
|
||||
port: 8080
|
||||
targetPort: mcp
|
||||
- name: ui
|
||||
port: 8081
|
||||
targetPort: ui
|
||||
@@ -1,18 +0,0 @@
|
||||
---
|
||||
apiVersion: monitoring.coreos.com/v1
|
||||
kind: ServiceMonitor
|
||||
metadata:
|
||||
name: mdmem
|
||||
labels:
|
||||
app: mdmem
|
||||
spec:
|
||||
endpoints:
|
||||
- port: ui
|
||||
path: /metrics
|
||||
interval: 15s
|
||||
honorLabels: true
|
||||
namespaceSelector:
|
||||
matchNames: ["apps"]
|
||||
selector:
|
||||
matchLabels:
|
||||
app: mdmem
|
||||
@@ -1,6 +0,0 @@
|
||||
---
|
||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
namespace: apps
|
||||
resources:
|
||||
- ./pvc-backup.yaml
|
||||
@@ -1,30 +0,0 @@
|
||||
# Hourly, not six-hourly: the vault accumulates irreplaceable hand-written
|
||||
# content between snapshots, unlike a re-derivable app database (§13a.4).
|
||||
# The whole PVC is backed up — excluding index.db/emb_cache.db would only buy a
|
||||
# full re-embed at restore time.
|
||||
---
|
||||
apiVersion: volsync.backube/v1alpha1
|
||||
kind: ReplicationSource
|
||||
metadata:
|
||||
name: mdmem
|
||||
spec:
|
||||
sourcePVC: mdmem-data
|
||||
trigger:
|
||||
schedule: "0 * * * *"
|
||||
restic:
|
||||
repository: restic-mdmem
|
||||
copyMethod: Snapshot
|
||||
storageClassName: longhorn-storage
|
||||
volumeSnapshotClassName: longhorn-snapshot
|
||||
cacheCapacity: 1Gi
|
||||
pruneIntervalDays: 1
|
||||
retain:
|
||||
hourly: 5
|
||||
daily: 4
|
||||
weekly: 3
|
||||
monthly: 2
|
||||
yearly: 1
|
||||
moverSecurityContext:
|
||||
runAsUser: 1000
|
||||
runAsGroup: 1000
|
||||
fsGroup: 1000
|
||||
@@ -1,58 +0,0 @@
|
||||
---
|
||||
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
||||
kind: Kustomization
|
||||
metadata:
|
||||
name: mdmem-app
|
||||
namespace: flux-system
|
||||
spec:
|
||||
targetNamespace: apps
|
||||
commonMetadata:
|
||||
labels:
|
||||
app.kubernetes.io/name: mdmem
|
||||
dependsOn:
|
||||
- name: traefik-base
|
||||
- name: longhorn-addons
|
||||
- name: cert-manager-addons
|
||||
- name: volsync
|
||||
- name: harbor-app # the image is pulled from in-cluster Harbor
|
||||
- name: hashicorp-vault-app # the ExternalSecret needs Vault reachable
|
||||
- name: external-secrets-app
|
||||
- name: bifrost-app # embeddings + the MCP front door
|
||||
path: ./kubernetes/apps/apps/mdmem/mdmem/app
|
||||
prune: true
|
||||
sourceRef:
|
||||
kind: GitRepository
|
||||
name: fluxcd-kubernetes
|
||||
namespace: flux-system
|
||||
wait: true
|
||||
interval: 15m
|
||||
retryInterval: 15m
|
||||
timeout: 15m
|
||||
healthChecks:
|
||||
- apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
name: mdmem
|
||||
namespace: apps
|
||||
---
|
||||
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
||||
kind: Kustomization
|
||||
metadata:
|
||||
name: mdmem-backup
|
||||
namespace: flux-system
|
||||
spec:
|
||||
targetNamespace: apps
|
||||
commonMetadata:
|
||||
labels:
|
||||
app.kubernetes.io/name: mdmem
|
||||
dependsOn:
|
||||
- name: mdmem-app
|
||||
- name: volsync
|
||||
path: ./kubernetes/apps/apps/mdmem/mdmem/backup
|
||||
prune: true
|
||||
sourceRef:
|
||||
kind: GitRepository
|
||||
name: fluxcd-kubernetes
|
||||
namespace: flux-system
|
||||
interval: 15m
|
||||
retryInterval: 15m
|
||||
timeout: 15m
|
||||
@@ -1,25 +0,0 @@
|
||||
# Add to llama-swap's config on evo-x2 (10.0.0.59:9000) — the box Bifrost's
|
||||
# `llamaswap` provider already points at. Not managed by Flux; llama-swap runs
|
||||
# off-cluster (§7).
|
||||
#
|
||||
# --pooling last is mandatory: Qwen3 embeddings use last-token pooling and the
|
||||
# wrong pooling degrades retrieval silently (§14.1, [R2][R3]).
|
||||
# Never add --reranking to this instance: with both flags on, a reported
|
||||
# llama.cpp bug returns all-zero embeddings (§14.2, [R5]).
|
||||
# Q8_0, not Q4_K: Q4_K measured 0.974 cosine against the reference (§14.3, [R4]).
|
||||
models:
|
||||
qwen3-embedding-4b:
|
||||
cmd: >
|
||||
llama-server --port ${PORT}
|
||||
-m /models/Qwen3-Embedding-4B-Q8_0.gguf
|
||||
--embedding --pooling last
|
||||
-ub 8192 -c 8192 -ngl 99
|
||||
ttl: 0 # keep resident; an embedding call must not evict the chat model
|
||||
|
||||
# Keep it co-resident with the chat model rather than swapping. Check the group
|
||||
# syntax against your llama-swap version:
|
||||
# groups:
|
||||
# resident:
|
||||
# swap: false
|
||||
# exclusive: false
|
||||
# members: ["qwen3-embedding-4b", "<your chat model>"]
|
||||
@@ -1,44 +0,0 @@
|
||||
# Eval harness (§11)
|
||||
|
||||
```bash
|
||||
mdmem eval --queries eval/queries.yaml --config default # one config
|
||||
mdmem eval --queries eval/queries.yaml --ablate # the whole matrix
|
||||
```
|
||||
|
||||
Both write `eval/results/<date>.json` and print a markdown table plus a
|
||||
recommendation. The recommendation only names a new default when its nDCG@10
|
||||
95 % bootstrap CI clears the baseline's — a two-point difference on 50 queries
|
||||
is noise, not a result.
|
||||
|
||||
`--ablate` covers: lex-only, sem-only, RRF, CC with α ∈ {0.3 … 0.9}, trigram
|
||||
off, inbox weight 1.0, flat BM25 weights, and embedding dims 1024 / 512. Every
|
||||
variant runs at the same `MAX_CHARS_DEFAULT` budget, because a retrieval
|
||||
comparison at different context volumes measures the volume [R16]. Dim variants
|
||||
build their own throwaway index and delete it afterwards.
|
||||
|
||||
`calibrate_min_score()` reports the `MIN_SCORE` that maximises abstention F1 on
|
||||
the `category: abstain` queries. It is not applied automatically — set it in the
|
||||
ConfigMap once you believe it.
|
||||
|
||||
## Runs so far
|
||||
|
||||
| file | what it is | conclusion |
|
||||
|---|---|---|
|
||||
| `results/2026-09-13-synthetic-smoke.json` | 18 synthetic queries over the 12 seeded notes in `corpus/`, run with `--fake-embeddings` | **The harness works; nothing about the config was learned.** Vectors are hashed word bags, not Qwen3, and 18 queries cannot separate fusion variants — every CC arm's 95 % CI overlaps the RRF baseline's. Defaults stay as §2.5 specifies: `FUSION=rrf`, `EMBED_DIM=2560`, `MIN_SCORE` unset. |
|
||||
|
||||
`rrf`, `rrf-inbox-1.0` and `rrf-bm25-flat` report the *same* nDCG@10 in that run.
|
||||
That is expected here and is not the shared-state corruption an earlier report had:
|
||||
none of the 12 seeded notes sit in `inbox/`, so the inbox weight has nothing to
|
||||
act on, and the two lexical subchannels are fused by ordinal rank (§8.1), so a
|
||||
BM25 weight change only moves the score if it reorders the hits — on a corpus this
|
||||
small, flat weights do not.
|
||||
|
||||
The abstention F1 in that run is 0 because `MIN_SCORE` is unset by default, so
|
||||
the two `abstain` queries still return their best (wrong) match. The calibration
|
||||
block in the report names the threshold that would maximise F1 — on a real,
|
||||
labelled set it is the number to put in the ConfigMap.
|
||||
|
||||
**Before tuning anything**: label 50–100 real queries in `eval/queries.yaml`,
|
||||
run `mdmem eval --ablate` against the real Bifrost endpoint, and only then move
|
||||
a default. A variant ships only when its nDCG@10 CI clears the baseline's and no
|
||||
category regresses.
|
||||
@@ -1,20 +0,0 @@
|
||||
---
|
||||
id: 01M00000000000000000000BFR
|
||||
title: Bifrost ist das AI-Gateway
|
||||
type: fact
|
||||
summary: Bifrost bündelt alle LLM-Aufrufe und hält die virtuellen Schlüssel.
|
||||
keywords: [bifrost, gateway, virtual key, api gateway, modellrouter]
|
||||
tags: [homelab, ai]
|
||||
relations: {}
|
||||
supersedes: [01M00000000000000000000TEM]
|
||||
superseded_by: null
|
||||
revises: null
|
||||
possible_duplicates: []
|
||||
observed_at: 2026-09-11T08:00:00Z
|
||||
valid_from: null
|
||||
valid_until: null
|
||||
source: {harness: seed}
|
||||
created: 2026-09-11T08:00:00Z
|
||||
updated: 2026-09-11T08:00:00Z
|
||||
---
|
||||
Bifrost ist der einzige Eingang für LLM-Aufrufe und MCP-Clients. Es steht vor llama-swap.
|
||||
-20
@@ -1,20 +0,0 @@
|
||||
---
|
||||
id: 01M00000000000000000000TEM
|
||||
title: Der Modellrouter hieß früher LiteLLM
|
||||
type: fact
|
||||
summary: Vor Bifrost lief LiteLLM als Gateway. Ersetzt.
|
||||
keywords: [litellm, gateway, alt, modellrouter]
|
||||
tags: [homelab, ai]
|
||||
relations: {}
|
||||
supersedes: []
|
||||
superseded_by: 01M00000000000000000000BFR
|
||||
revises: null
|
||||
possible_duplicates: []
|
||||
observed_at: 2026-09-11T08:00:00Z
|
||||
valid_from: null
|
||||
valid_until: null
|
||||
source: {harness: seed}
|
||||
created: 2026-09-11T08:00:00Z
|
||||
updated: 2026-09-11T08:00:00Z
|
||||
---
|
||||
Historisch: LiteLLM war das Gateway bis 2026.
|
||||
@@ -1,20 +0,0 @@
|
||||
---
|
||||
id: 01M00000000000000000000EVX
|
||||
title: evo-x2
|
||||
type: entity
|
||||
summary: Die GPU-Box im Keller mit 128 GB unified memory.
|
||||
keywords: [evo-x2, gpu, hardware, workstation]
|
||||
tags: [homelab]
|
||||
relations: {}
|
||||
supersedes: []
|
||||
superseded_by: null
|
||||
revises: null
|
||||
possible_duplicates: []
|
||||
observed_at: 2026-09-11T08:00:00Z
|
||||
valid_from: null
|
||||
valid_until: null
|
||||
source: {harness: seed}
|
||||
created: 2026-09-11T08:00:00Z
|
||||
updated: 2026-09-11T08:00:00Z
|
||||
---
|
||||
evo-x2 ist die Inferenz-Box unter 10.0.0.59. Beherbergt llama-swap.
|
||||
@@ -1,20 +0,0 @@
|
||||
---
|
||||
id: 01M00000000000000000000FXC
|
||||
title: FluxCD deployt alles
|
||||
type: fact
|
||||
summary: Jede App kommt über FluxCD-Kustomizations aus dem fluxcd-Repo.
|
||||
keywords: [fluxcd, gitops, kustomization, deployment, ausrollen]
|
||||
tags: [homelab, k8s]
|
||||
relations: {}
|
||||
supersedes: []
|
||||
superseded_by: null
|
||||
revises: null
|
||||
possible_duplicates: []
|
||||
observed_at: 2026-09-11T08:00:00Z
|
||||
valid_from: null
|
||||
valid_until: null
|
||||
source: {harness: seed}
|
||||
created: 2026-09-11T08:00:00Z
|
||||
updated: 2026-09-11T08:00:00Z
|
||||
---
|
||||
Der Cluster ist RKE2 mit FluxCD. Kein manuelles kubectl apply.
|
||||
@@ -1,20 +0,0 @@
|
||||
---
|
||||
id: 01M00000000000000000000HRB
|
||||
title: Harbor spiegelt die Images
|
||||
type: fact
|
||||
summary: harbor.cloud.traunseenet.com ist Proxy-Cache und Registry für eigene Images.
|
||||
keywords: [harbor, registry, images, proxy cache, container]
|
||||
tags: [homelab, k8s]
|
||||
relations: {}
|
||||
supersedes: []
|
||||
superseded_by: null
|
||||
revises: null
|
||||
possible_duplicates: []
|
||||
observed_at: 2026-09-11T08:00:00Z
|
||||
valid_from: null
|
||||
valid_until: null
|
||||
source: {harness: seed}
|
||||
created: 2026-09-11T08:00:00Z
|
||||
updated: 2026-09-11T08:00:00Z
|
||||
---
|
||||
Eigene Images liegen unter library/, gespiegelte unter docker.io/ und ghcr.io/.
|
||||
@@ -1,20 +0,0 @@
|
||||
---
|
||||
id: 01M00000000000000000000KAF
|
||||
title: Kaffeemaschine entkalken
|
||||
type: procedure
|
||||
summary: Alle drei Monate mit Essigessenz entkalken.
|
||||
keywords: [kaffee, entkalken, wartung, descale, maintenance]
|
||||
tags: [haushalt]
|
||||
relations: {}
|
||||
supersedes: []
|
||||
superseded_by: null
|
||||
revises: null
|
||||
possible_duplicates: []
|
||||
observed_at: 2026-09-11T08:00:00Z
|
||||
valid_from: null
|
||||
valid_until: null
|
||||
source: {harness: seed}
|
||||
created: 2026-09-11T08:00:00Z
|
||||
updated: 2026-09-11T08:00:00Z
|
||||
---
|
||||
1. Tank mit 1:4 Essigessenz füllen. 2. Zwei Durchläufe. 3. Zweimal klar spülen.
|
||||
@@ -1,20 +0,0 @@
|
||||
---
|
||||
id: 01M00000000000000000000SWP
|
||||
title: llama-swap routet lokale Modelle
|
||||
type: fact
|
||||
summary: llama-swap tauscht Modelle auf evo-x2; Bifrost steht davor.
|
||||
keywords: [llama-swap, modellrouter, evo-x2, model router, inference]
|
||||
tags: [homelab, ai]
|
||||
relations: {}
|
||||
supersedes: []
|
||||
superseded_by: null
|
||||
revises: null
|
||||
possible_duplicates: []
|
||||
observed_at: 2026-09-11T08:00:00Z
|
||||
valid_from: null
|
||||
valid_until: null
|
||||
source: {harness: seed}
|
||||
created: 2026-09-11T08:00:00Z
|
||||
updated: 2026-09-11T08:00:00Z
|
||||
---
|
||||
llama-swap läuft auf [[evo-x2]] und tauscht llama-server-Modelle. Bifrost adressiert es als llamaswap/<modell>.
|
||||
@@ -1,20 +0,0 @@
|
||||
---
|
||||
id: 01M00000000000000000000STG
|
||||
title: Longhorn liefert die Volumes
|
||||
type: fact
|
||||
summary: Longhorn ist die StorageClass im RKE2-Cluster, nicht democratic-csi.
|
||||
keywords: [longhorn, storage, pvc, storageclass, volumes, speicher]
|
||||
tags: [homelab, k8s]
|
||||
relations: {}
|
||||
supersedes: []
|
||||
superseded_by: null
|
||||
revises: null
|
||||
possible_duplicates: []
|
||||
observed_at: 2026-09-11T08:00:00Z
|
||||
valid_from: null
|
||||
valid_until: null
|
||||
source: {harness: seed}
|
||||
created: 2026-09-11T08:00:00Z
|
||||
updated: 2026-09-11T08:00:00Z
|
||||
---
|
||||
Persistente Volumes kommen von longhorn-storage. RWO, repliziert.
|
||||
@@ -1,20 +0,0 @@
|
||||
---
|
||||
id: 01M00000000000000000000PKT
|
||||
title: PocketID macht das SSO
|
||||
type: fact
|
||||
summary: Alle Web-UIs authentifizieren sich per OIDC gegen PocketID.
|
||||
keywords: [pocketid, oidc, sso, login, authentifizierung]
|
||||
tags: [homelab, k8s]
|
||||
relations: {}
|
||||
supersedes: []
|
||||
superseded_by: null
|
||||
revises: null
|
||||
possible_duplicates: []
|
||||
observed_at: 2026-09-11T08:00:00Z
|
||||
valid_from: null
|
||||
valid_until: null
|
||||
source: {harness: seed}
|
||||
created: 2026-09-11T08:00:00Z
|
||||
updated: 2026-09-11T08:00:00Z
|
||||
---
|
||||
Es gibt kein forward-auth im Cluster. Jede App macht OIDC selbst gegen PocketID.
|
||||
@@ -1,20 +0,0 @@
|
||||
---
|
||||
id: 01M00000000000000000000QWN
|
||||
title: Qwen3-Embedding-4B als Embedder
|
||||
type: decision
|
||||
summary: Q8_0-Quant, last-token pooling, 2560 Dimensionen.
|
||||
keywords: [qwen3, embedding, q8_0, pooling, vektoren, einbettung]
|
||||
tags: [homelab, ai]
|
||||
relations: {}
|
||||
supersedes: []
|
||||
superseded_by: null
|
||||
revises: null
|
||||
possible_duplicates: []
|
||||
observed_at: 2026-09-11T08:00:00Z
|
||||
valid_from: null
|
||||
valid_until: null
|
||||
source: {harness: seed}
|
||||
created: 2026-09-11T08:00:00Z
|
||||
updated: 2026-09-11T08:00:00Z
|
||||
---
|
||||
Entschieden: 4B statt 8B (kleiner Abstand) und Q8_0 statt Q4_K (Drift).
|
||||
@@ -1,20 +0,0 @@
|
||||
---
|
||||
id: 01M00000000000000000000TRN
|
||||
title: Traunsee Wasserstand
|
||||
type: note
|
||||
summary: Pegel Gmunden liegt im Sommer meist um 422 m.
|
||||
keywords: [traunsee, pegel, wasserstand, lake level, gmunden]
|
||||
tags: [region]
|
||||
relations: {}
|
||||
supersedes: []
|
||||
superseded_by: null
|
||||
revises: null
|
||||
possible_duplicates: []
|
||||
observed_at: 2026-09-11T08:00:00Z
|
||||
valid_from: null
|
||||
valid_until: null
|
||||
source: {harness: seed}
|
||||
created: 2026-09-11T08:00:00Z
|
||||
updated: 2026-09-11T08:00:00Z
|
||||
---
|
||||
Der Pegel Gmunden schwankt zwischen 421,5 m und 423 m.
|
||||
-20
@@ -1,20 +0,0 @@
|
||||
---
|
||||
id: 01M00000000000000000000VSY
|
||||
title: VolSync sichert stündlich nach Restic
|
||||
type: decision
|
||||
summary: Backups laufen stündlich per VolSync-ReplicationSource in ein Restic-Repo.
|
||||
keywords: [volsync, restic, backup, snapshot, sicherung]
|
||||
tags: [homelab, k8s]
|
||||
relations: {}
|
||||
supersedes: []
|
||||
superseded_by: null
|
||||
revises: null
|
||||
possible_duplicates: []
|
||||
observed_at: 2026-09-11T08:00:00Z
|
||||
valid_from: null
|
||||
valid_until: null
|
||||
source: {harness: seed}
|
||||
created: 2026-09-11T08:00:00Z
|
||||
updated: 2026-09-11T08:00:00Z
|
||||
---
|
||||
Entschieden am 2026-09-11: stündliche Snapshots statt sechsstündlich, weil der Vault handgeschrieben ist.
|
||||
@@ -1,22 +0,0 @@
|
||||
# Synthetic query set over eval/corpus (12 seeded notes). It exists so the
|
||||
# harness itself can be exercised offline with --fake-embeddings. It is NOT a
|
||||
# tuning set: the vectors are hashed word bags, not Qwen3, and 18 queries over
|
||||
# 12 notes cannot separate fusion variants. Real tuning needs eval/queries.yaml.
|
||||
- {q: "Welcher Router verteilt meine lokalen Modelle?", relevant: ["01M00000000000000000000SWP"], category: single, lang: de}
|
||||
- {q: "llama-swap", relevant: ["01M00000000000000000000SWP"], category: single, lang: de}
|
||||
- {q: "Which gateway sits in front of llama-swap?", relevant: ["01M00000000000000000000BFR"], category: single, lang: en}
|
||||
- {q: "Wo hält Bifrost die virtuellen Schlüssel?", relevant: ["01M00000000000000000000BFR"], category: single, lang: de}
|
||||
- {q: "Wie heißt die GPU-Box?", relevant: ["01M00000000000000000000EVX"], category: single, lang: de}
|
||||
- {q: "storage class for persistent volumes", relevant: ["01M00000000000000000000STG"], category: single, lang: en}
|
||||
- {q: "Welche StorageClass nutzt der Cluster?", relevant: ["01M00000000000000000000STG"], category: single, lang: de}
|
||||
- {q: "Wie oft laufen die Backups?", relevant: ["01M00000000000000000000VSY"], category: single, lang: de}
|
||||
- {q: "restic snapshot schedule", relevant: ["01M00000000000000000000VSY"], category: single, lang: en}
|
||||
- {q: "Wie melde ich mich bei den Web-UIs an?", relevant: ["01M00000000000000000000PKT"], category: single, lang: de}
|
||||
- {q: "Wie entkalke ich die Kaffeemaschine?", relevant: ["01M00000000000000000000KAF"], category: single, lang: de}
|
||||
- {q: "Pegel am Traunsee", relevant: ["01M00000000000000000000TRN"], category: single, lang: de}
|
||||
- {q: "Wie heißt der Modellrouter jetzt?", relevant: ["01M00000000000000000000BFR"], category: update, lang: de}
|
||||
- {q: "Welches Embedding-Modell und welche Quantisierung?", relevant: ["01M00000000000000000000QWN"], category: multi, lang: de}
|
||||
- {q: "Woher kommen die Container-Images und wer rollt sie aus?", relevant: ["01M00000000000000000000HRB", "01M00000000000000000000FXC"], category: multi, lang: de}
|
||||
- {q: "Was wurde am 11. September 2026 entschieden?", relevant: ["01M00000000000000000000VSY"], category: temporal, lang: de, since: "2026-09-01T00:00:00Z", until: "2026-12-31T00:00:00Z"}
|
||||
- {q: "Wie ist die Telefonnummer meines Zahnarztes?", relevant: [], category: abstain, lang: de}
|
||||
- {q: "Wann fährt der letzte Zug nach Wien?", relevant: [], category: abstain, lang: de}
|
||||
File diff suppressed because it is too large.
Load diff
@@ -1,6 +1,6 @@
|
||||
# Query set for `mdmem eval` (§11).
|
||||
#
|
||||
# Copy to eval/queries.yaml and label 50–100 REAL queries — the ones you would
|
||||
# Copy to queries.yaml and label 50–100 REAL queries — the ones you would
|
||||
# actually ask your assistant. The harness is only as meaningful as this file.
|
||||
#
|
||||
# Coverage the plan asks for (≥10 each):
|
||||
+1
-1
@@ -32,7 +32,7 @@ def build_parser() -> argparse.ArgumentParser:
|
||||
reindex.add_argument("--no-embed", action="store_true", help="lexical index only")
|
||||
|
||||
ev = sub.add_parser("eval", help="run the retrieval eval harness (§11)")
|
||||
ev.add_argument("--queries", default="eval/queries.yaml")
|
||||
ev.add_argument("--queries", default="queries.yaml")
|
||||
ev.add_argument("--config", default="default", help="name for this run's config")
|
||||
ev.add_argument("--out", default=None, help="write the JSON report here")
|
||||
ev.add_argument("--ablate", action="store_true", help="run the ablation matrix")
|
||||
|
||||
@@ -404,7 +404,7 @@ def run_eval_cli(settings: Settings, args: Any) -> int:
|
||||
queries_path = Path(args.queries)
|
||||
if not queries_path.exists():
|
||||
print(
|
||||
f"no query set at {queries_path}. Copy eval/queries.example.yaml and label "
|
||||
f"no query set at {queries_path}. Copy queries.example.yaml and label "
|
||||
"50–100 real queries first (§11)."
|
||||
)
|
||||
return 2
|
||||
@@ -435,7 +435,7 @@ def run_eval_cli(settings: Settings, args: Any) -> int:
|
||||
print(table)
|
||||
print()
|
||||
print(dumps_json({"decision": decision, "min_score": min_score}, indent=2, ensure_ascii=False))
|
||||
out = Path(args.out) if args.out else Path("eval/results") / f"{time.strftime('%Y-%m-%d')}.json"
|
||||
out = Path(args.out) if args.out else Path("results") / f"{time.strftime('%Y-%m-%d')}.json"
|
||||
out.parent.mkdir(parents=True, exist_ok=True)
|
||||
out.write_text(
|
||||
dumps_json(
|
||||
|
||||
+1
-1
@@ -377,7 +377,7 @@ def _merge_lexical_hits(
|
||||
can. The trigram pass keeps its 0.5 discount as a rank weight: substring
|
||||
matching is the fuzzier signal, so a trigram hit has to be clearly better
|
||||
ranked than a whole-word hit to displace it, rather than winning a tie.
|
||||
Measured on eval/corpus, dropping the discount cost the rrf baseline about
|
||||
Measured on a synthetic corpus, dropping the discount cost the rrf baseline about
|
||||
0.03 nDCG@10 by promoting trigram hits over better whole-word ones.
|
||||
"""
|
||||
weight = 1.0 if source == "fts" else TRIGRAM_RANK_WEIGHT
|
||||
|
||||
@@ -90,7 +90,7 @@ class MemoryService:
|
||||
# retry loop, embed_pending/_embed_items). Cancelling the asyncio task
|
||||
# that awaits one of these does NOT stop the worker thread, so stop()
|
||||
# must wait for them to actually finish before closing the db out from
|
||||
# under them (§ SECURITY_ANALYSIS.md, segfault in Indexer.pending_embeds).
|
||||
# under them (segfault risk in Indexer.pending_embeds).
|
||||
self._inflight: set[asyncio.Future[Any]] = set()
|
||||
self._secret_warnings: list[str] = []
|
||||
self.metrics = {"search": 0, "writes": 0, "embed_calls": 0}
|
||||
|
||||
+4
-4
@@ -620,20 +620,20 @@ def build_ui_app(svc: MemoryService) -> FastAPI: # noqa: C901 - one place, many
|
||||
`asyncio.run` from and keeps the response fast.
|
||||
|
||||
Both the query set and the report live on the data volume, not in the
|
||||
working directory: the pod runs with `readOnlyRootFilesystem: true`
|
||||
(§13a.3), so a relative `eval/results/...` path would fail at the write
|
||||
working directory: the pod runs with `readOnlyRootFilesystem: true`,
|
||||
so a relative `results/...` path would fail at the write
|
||||
with nothing but a log line to show for it.
|
||||
"""
|
||||
from mdmem.evalharness import run_eval_cli
|
||||
|
||||
data_dir = settings.index_db.parent
|
||||
queries = data_dir / "eval" / "queries.yaml"
|
||||
queries = data_dir / "results" / "queries.yaml"
|
||||
if not queries.exists():
|
||||
# §11: a run needs 50-100 labelled queries. Say so on the page
|
||||
# instead of starting a job that prints a hint into the log and
|
||||
# exits 2.
|
||||
return RedirectResponse(f"/ui/health?eval=missing:{queries}", status_code=303)
|
||||
out = data_dir / "eval" / f"{time.strftime('%Y-%m-%dT%H-%M-%SZ', time.gmtime())}.json"
|
||||
out = data_dir / "results" / f"{time.strftime('%Y-%m-%dT%H-%M-%SZ', time.gmtime())}.json"
|
||||
args = SimpleNamespace(
|
||||
queries=str(queries), config="ui", out=str(out), ablate=False, fake_embeddings=False
|
||||
)
|
||||
|
||||
@@ -1,184 +0,0 @@
|
||||
"""Smoke tests for the committed Kubernetes overlay (PLAN §13a).
|
||||
|
||||
The overlay is not directly buildable: `secret.yaml` is SOPS-encrypted and kept out
|
||||
of the repo. The image digest and the PocketID client id are filled in, but both are
|
||||
release-time values — the digest is bumped by hand on every build (§13a.2) — so their
|
||||
placeholder spellings stay on the allowlist below rather than being treated as a
|
||||
defect if they reappear.
|
||||
|
||||
Nothing else checked that the overlay stayed coherent: a renamed environment
|
||||
variable, a probe pointing at a port the container does not expose, or a resource
|
||||
dropped from the kustomization would only surface on a cluster.
|
||||
"""
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
import yaml
|
||||
|
||||
APP = Path("deploy/kubernetes/apps/apps/mdmem/mdmem/app")
|
||||
DEPLOYMENT = APP / "deployment.yaml"
|
||||
|
||||
# Documented, intentional gaps: the overlay cannot render as committed.
|
||||
ABSENT_BY_DESIGN = {"./secret.yaml"}
|
||||
RELEASE_PLACEHOLDERS = ("REPLACE_WITH_DIGEST", "REPLACE-WITH-THE-POCKETID-CLIENT-UUID")
|
||||
|
||||
|
||||
def _load(path: Path) -> dict:
|
||||
"""The overlay has multi-document files; take the first real document."""
|
||||
docs = [d for d in yaml.safe_load_all(path.read_text(encoding="utf-8")) if d]
|
||||
return docs[0]
|
||||
|
||||
|
||||
def _load_all(path: Path) -> list[dict]:
|
||||
return [d for d in yaml.safe_load_all(path.read_text(encoding="utf-8")) if d]
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def deployment() -> dict:
|
||||
return _load(DEPLOYMENT)
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def container(deployment) -> dict:
|
||||
return deployment["spec"]["template"]["spec"]["containers"][0]
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def kustomization() -> dict:
|
||||
return _load(APP / "kustomization.yaml")
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def env_literals(kustomization) -> dict[str, str]:
|
||||
generator = next(g for g in kustomization["configMapGenerator"] if g["name"] == "mdmem-env")
|
||||
out = {}
|
||||
for literal in generator["literals"]:
|
||||
key, _, value = literal.partition("=")
|
||||
out[key] = value
|
||||
return out
|
||||
|
||||
|
||||
def test_every_manifest_in_the_overlay_parses():
|
||||
for path in sorted(APP.glob("*.yaml")):
|
||||
docs = _load_all(path)
|
||||
assert docs, f"{path} is empty or unparseable"
|
||||
for doc in docs:
|
||||
assert doc.get("kind") or "kustomize" in str(doc.get("apiVersion", "")), path
|
||||
|
||||
|
||||
def test_kustomization_resources_exist_apart_from_the_sops_secret(kustomization):
|
||||
for resource in kustomization["resources"]:
|
||||
if resource in ABSENT_BY_DESIGN:
|
||||
# ...but its template must be committed so the gap is fixable
|
||||
assert (APP / "secret.example.yaml").exists()
|
||||
continue
|
||||
assert (APP / resource).exists(), f"{resource} is referenced but missing"
|
||||
|
||||
|
||||
def test_the_only_unrendered_placeholders_are_the_documented_release_values():
|
||||
"""If a new placeholder creeps in, it must be declared here, not discovered
|
||||
during a deploy."""
|
||||
found = set()
|
||||
for path in sorted(APP.glob("*.yaml")):
|
||||
text = path.read_text(encoding="utf-8")
|
||||
for token in ("REPLACE_WITH", "REPLACE-WITH", "CHANGEME", "TODO:"):
|
||||
if token in text:
|
||||
found.update(w for w in text.split() if token in w)
|
||||
unexpected = {f for f in found if not any(p in f for p in RELEASE_PLACEHOLDERS)}
|
||||
assert not unexpected, f"undocumented placeholders in the overlay: {sorted(unexpected)}"
|
||||
|
||||
|
||||
def test_configmap_env_keys_are_all_read_by_the_application(env_literals):
|
||||
"""Catches a variable renamed in config.py but not in the overlay (or vice versa):
|
||||
a silently ignored EMBED_MODEL would serve lexical-only in production."""
|
||||
src = "\n".join(p.read_text(encoding="utf-8") for p in Path("src/mdmem").rglob("*.py"))
|
||||
unread = [key for key in env_literals if f'"{key}"' not in src]
|
||||
assert not unread, f"the overlay sets variables the app never reads: {unread}"
|
||||
|
||||
|
||||
def test_secret_supplied_variables_are_not_duplicated_in_the_configmap(env_literals):
|
||||
"""MCP_TOKEN and friends come from Vault via the ExternalSecret; a plaintext
|
||||
copy in the ConfigMap would both leak and win or lose unpredictably."""
|
||||
secret_keys = {"MCP_TOKEN", "BIFROST_VK", "OIDC_CLIENT_SECRET", "SESSION_SECRET"}
|
||||
assert not (secret_keys & set(env_literals))
|
||||
|
||||
|
||||
def test_embed_model_carries_its_provider_prefix(env_literals):
|
||||
"""A bare alias 404s at Bifrost and looks like an mdmem bug (§7, §14.14)."""
|
||||
assert "/" in env_literals["EMBED_MODEL"]
|
||||
|
||||
|
||||
def test_listener_addresses_match_the_declared_container_ports(env_literals, container):
|
||||
ports = {p["name"]: p["containerPort"] for p in container["ports"]}
|
||||
assert env_literals["MCP_ADDR"].endswith(f":{ports['mcp']}")
|
||||
assert env_literals["UI_ADDR"].endswith(f":{ports['ui']}")
|
||||
# the MCP SDK checks Host, so every allowed host must name the MCP port
|
||||
for host in env_literals["MCP_ALLOWED_HOSTS"].split(","):
|
||||
assert host.strip().endswith(f":{ports['mcp']}"), host
|
||||
|
||||
|
||||
def test_probes_point_at_ports_the_container_actually_exposes(container):
|
||||
names = {p["name"] for p in container["ports"]}
|
||||
for probe in ("livenessProbe", "readinessProbe", "startupProbe"):
|
||||
assert container[probe]["httpGet"]["port"] in names
|
||||
|
||||
|
||||
def test_mdmem_deployment_has_startup_probe_budget(container):
|
||||
"""§14.17: the pod must be allowed to come up in lexical-only mode while the
|
||||
embedding backend is unreachable, instead of being restarted mid-startup."""
|
||||
probe = container["startupProbe"]
|
||||
assert probe["httpGet"] == {"path": "/healthz", "port": "ui"}
|
||||
assert probe["periodSeconds"] * probe["failureThreshold"] >= 180
|
||||
|
||||
|
||||
def test_liveness_is_dumb_and_readiness_never_depends_on_bifrost(container):
|
||||
assert container["livenessProbe"]["httpGet"]["path"] == "/healthz"
|
||||
assert container["readinessProbe"]["httpGet"]["path"] == "/readyz"
|
||||
|
||||
|
||||
def test_the_pod_stays_locked_down(deployment, container):
|
||||
pod_sc = deployment["spec"]["template"]["spec"]["securityContext"]
|
||||
assert pod_sc["runAsNonRoot"] is True
|
||||
sc = container["securityContext"]
|
||||
assert sc["readOnlyRootFilesystem"] is True
|
||||
assert sc["allowPrivilegeEscalation"] is False
|
||||
assert sc["capabilities"]["drop"] == ["ALL"]
|
||||
|
||||
|
||||
def test_read_only_root_filesystem_has_somewhere_to_write(deployment, container):
|
||||
"""readOnlyRootFilesystem only holds if HOME/XDG_CACHE_HOME point at a mounted
|
||||
writable path."""
|
||||
mounts = {m["name"]: m["mountPath"] for m in container["volumeMounts"]}
|
||||
env = {e["name"]: e["value"] for e in container["env"]}
|
||||
volumes = {v["name"] for v in deployment["spec"]["template"]["spec"]["volumes"]}
|
||||
assert mounts["tmp"] == "/tmp" and "tmp" in volumes
|
||||
assert env["HOME"] == "/tmp" and env["XDG_CACHE_HOME"] == "/tmp"
|
||||
assert mounts["data"] == "/data" and "data" in volumes
|
||||
|
||||
|
||||
def test_every_persistent_path_lives_on_the_data_volume(env_literals, container):
|
||||
mounts = {m["name"]: m["mountPath"] for m in container["volumeMounts"]}
|
||||
for key in ("VAULT_DIR", "INDEX_DB", "CACHE_DB", "HISTORY_DIR", "AUDIT_LOG"):
|
||||
assert env_literals[key].startswith(mounts["data"] + "/"), key
|
||||
|
||||
|
||||
def test_sqlite_requires_a_single_writer(deployment):
|
||||
assert deployment["spec"]["replicas"] == 1
|
||||
assert deployment["spec"]["strategy"]["type"] == "Recreate"
|
||||
|
||||
|
||||
def test_the_servicemonitor_scrapes_the_ui_port_where_metrics_live():
|
||||
monitor = _load(APP / "servicemonitor.yaml")
|
||||
service = _load(APP / "service.yaml")
|
||||
endpoint = monitor["spec"]["endpoints"][0]
|
||||
assert endpoint["path"] == "/metrics"
|
||||
assert endpoint["port"] in {p["name"] for p in service["spec"]["ports"]}
|
||||
assert monitor["spec"]["selector"]["matchLabels"] == service["metadata"]["labels"]
|
||||
|
||||
|
||||
def test_the_service_targets_the_containers_named_ports(container):
|
||||
service = _load(APP / "service.yaml")
|
||||
names = {p["name"] for p in container["ports"]}
|
||||
for port in service["spec"]["ports"]:
|
||||
assert port["targetPort"] in names
|
||||
@@ -62,7 +62,7 @@ def test_load_queries_validates(tmp_path):
|
||||
|
||||
|
||||
def test_the_shipped_example_query_set_parses():
|
||||
qs = load_queries("eval/queries.example.yaml")
|
||||
qs = load_queries("queries.example.yaml")
|
||||
assert len(qs) >= 5
|
||||
assert {q.category for q in qs} >= {"single", "temporal", "update", "abstain"}
|
||||
|
||||
|
||||
@@ -6,7 +6,7 @@ from mdmem.logging import AuditLog
|
||||
def test_recent_merges_persisted_history_across_restart_without_duplicates(tmp_path):
|
||||
"""`AuditLog.recent` must return the merged tail of persisted history plus
|
||||
in-memory entries across a restart, newest-first, without duplicating the
|
||||
entries that show up in both (SECURITY_ANALYSIS.md, logging.py:77-92)."""
|
||||
entries that show up in both (logging.py:77-92)."""
|
||||
path = tmp_path / "audit.jsonl"
|
||||
first = AuditLog(path)
|
||||
for i in range(5):
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
"""Regression tests for service-layer write-path and security findings (PLAN.md §2.4, §9, §14.9-14.12)."""
|
||||
"""Regression tests for service-layer write-path and security findings."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
@@ -361,7 +361,7 @@ async def test_flag_reason_carrying_a_secret_is_rejected_before_it_is_stored(svc
|
||||
|
||||
# -- C-01: the whole read/check/write operation is one serialized step -------
|
||||
#
|
||||
# These are the deterministic barrier tests SECURITY_ANALYSIS.md asked for. Every
|
||||
# These are the deterministic barrier tests for concurrency-critical paths.
|
||||
# other "race" test in this file is sequential and would still pass if the write
|
||||
# lock were narrowed back to just the persist call, which is exactly the defect
|
||||
# C-01 described. The barrier below wedges the first operation *inside* its
|
||||
|
||||
@@ -428,9 +428,9 @@ async def test_run_eval_is_auth_and_csrf_protected_like_the_other_health_actions
|
||||
|
||||
|
||||
async def test_run_eval_keeps_its_files_on_the_data_volume_and_reports_status(ui, monkeypatch):
|
||||
"""The pod runs with readOnlyRootFilesystem, so a relative eval/results path
|
||||
"""The pod runs with readOnlyRootFilesystem, so a relative results path
|
||||
would fail at the write with only a log line. Both sides of the run must sit
|
||||
next to index.db on the mounted volume (§13a.3, §11)."""
|
||||
next to index.db on the mounted volume."""
|
||||
client, svc = ui
|
||||
data_dir = svc.settings.index_db.parent
|
||||
|
||||
|
||||
Reference in new issue
Block a user