Mdmem Agent
8fd3e37b67
- Delete PLAN.md (LLM agent development brief) - Delete SECURITY_ANALYSIS.md (internal audit report) - Delete deploy/ (environment-specific K8s manifests with hardcoded infra) - Remove eval/corpus/, eval/results/, eval/queries.synthetic.yaml, eval/README.md - Move queries.example.yaml to root, update all path references
44 lines
1.5 KiB
YAML
44 lines
1.5 KiB
YAML
# Query set for `mdmem eval` (§11).
|
||
#
|
||
# Copy to queries.yaml and label 50–100 REAL queries — the ones you would
|
||
# actually ask your assistant. The harness is only as meaningful as this file.
|
||
#
|
||
# Coverage the plan asks for (≥10 each):
|
||
# lang: de · lang: en · cross-language (German query, English note)
|
||
# category: temporal — needs since/until/valid_at to answer correctly
|
||
# category: update — a superseded fact; only the successor may rank
|
||
# category: abstain — nothing in memory answers it; `relevant: []`
|
||
#
|
||
# `relevant` holds note ULIDs. Find them in the UI (the id is on the note page)
|
||
# or with: grep -l "Suchbegriff" /data/vault/active/*.md
|
||
#
|
||
# Fields: q (required), relevant, category, lang, types, tags,
|
||
# since, until, valid_at, include_superseded.
|
||
|
||
- q: "Welcher Router verteilt meine lokalen Modelle?"
|
||
relevant: [01J9Z4K8X2M3N5P7Q9R1S3T5V7]
|
||
category: single
|
||
lang: de
|
||
|
||
- q: "Which gateway sits in front of llama-swap?"
|
||
relevant: [01J9Z4K8X2M3N5P7Q9R1S3T5V7]
|
||
category: single
|
||
lang: en # cross-language: the note is German, the query is English
|
||
|
||
- q: "Was habe ich im September 2026 über Backups entschieden?"
|
||
relevant: []
|
||
category: temporal
|
||
lang: de
|
||
since: "2026-09-01T00:00:00Z"
|
||
until: "2026-10-01T00:00:00Z"
|
||
|
||
- q: "Wie heißt der Router jetzt?"
|
||
relevant: [] # only the successor note may rank; list its id here
|
||
category: update
|
||
lang: de
|
||
|
||
- q: "Wie ist die Telefonnummer meines Zahnarztes?"
|
||
relevant: []
|
||
category: abstain
|
||
lang: de
|