feat(p6): Kubernetes manifests, Bifrost MCP client entry, README

Manifests mirror the fluxcd repo's apps/apps/<name>/<name>/{app,backup} nesting and build clean with kubectl kustomize; they are unverified against a live cluster. README documents the build, the deploy order, MCP_TOKEN rotation across its two Vault paths, harness wiring, and every deviation from PLAN.md.
2026-09-12 08:15:03 +00:00
1 parent 8e5e3e6309
commit 48e8d9bc26
@@ -1,3 +1,163 @@
 # mdmem

-Markdown-canonical agent memory service. See `PLAN.md` for the full design.
+Markdown-canonical agent memory service. One self-hosted process that gives every
+harness (Claude Code, claude.ai, Open WebUI, Hermes, opencode…) a single shared
+long-term memory over MCP, with markdown files as the only source of truth.
+
+Full design and its evidence: [`PLAN.md`](PLAN.md). Section numbers below refer to it.
+
+- **Markdown is canonical.** `/data/vault/{inbox,active,archive}/*.md`. The SQLite
+  index is derived and rebuildable; delete it and the service rebuilds from the files.
+- **Hybrid retrieval.** FTS5 (BM25, `unicode61 remove_diacritics 2`, optional trigram)
+  + sqlite-vec KNN, fused with RRF (default) or convex combination. No graph traversal
+  in the retrieval path (§2.7).
+- **Agents propose, the owner disposes.** Agent writes land in `inbox/`; an agent
+  editing an approved note creates a *proposed revision*, never an in-place change.
+  There is no delete tool (§2.4).
+- **Memory content is untrusted data.** Tool descriptions and the MCP `instructions`
+  say so; the UI renders markdown with raw HTML disabled.
+
+## Layout
+
+```
+src/mdmem/
+  config.py      settings from the environment (§12)
+  notes.py       frontmatter round-trip (ruamel): unknown keys, order and comments survive
+  vaultio.py     atomic write → fsync → replace, pre-write history snapshot
+  chunker.py     atomic notes are one chunk; long ones split on H2/H3 (§6)
+  db.py          schema, sqlite-vec, single-writer rule (§5)
+  index.py       vault → rows, links, duplicate hints
+  embed.py       Bifrost client, content-hash cache, FakeEmbedder for tests (§7)
+  search.py      the retrieval algorithm (§8)
+  service.py     the one code path every write goes through — MCP and UI alike
+  mcp_server.py  six memory_* tools, bearer auth, idempotency, audit log (§9)
+  ui/            FastAPI + Jinja + HTMX, PocketID OIDC with a group check (§10)
+  evalharness.py metrics, bootstrap CIs, ablation matrix (§11)
+deploy/          Kubernetes manifests, laid out for the fluxcd repo (§13a)
+eval/            query-set template, synthetic corpus, results
+```
+
+## Local development
+
+```bash
+uv sync --extra dev
+uv run pytest                       # 190+ tests, no network, no GPU
+uv run pytest -m live               # only the tests that hit the real Bifrost endpoint
+uv run ruff check src tests && uv run ruff format --check src tests
+```
+
+Run it against a scratch vault with no embedding backend and no login:
+
+```bash
+VAULT_DIR=/tmp/mdmem/vault INDEX_DB=/tmp/mdmem/index.db CACHE_DB=/tmp/mdmem/cache.db \
+HISTORY_DIR=/tmp/mdmem/.history MCP_TOKEN=dev SESSION_SECRET=$(openssl rand -hex 32) \
+UI_AUTH_DISABLED=1 LOG_STYLE=text uv run mdmem serve
+```
+
+`UI_AUTH_DISABLED=1` is for a port-forwarded local UI only. It logs an error on
+startup, and **an HTTPRoute must never point at a listener running with it** (§14.19).
+
+Other commands: `mdmem reindex [--full] [--no-embed]`, `mdmem doctor`,
+`mdmem eval --queries eval/queries.yaml [--ablate]`.
+
+## Build and push
+
+There is no CI runner in the cluster, so builds are manual from the workstation:
+
+```bash
+podman build -t harbor.cloud.traunseenet.com/library/mdmem:0.1.0 -f Containerfile .
+podman push harbor.cloud.traunseenet.com/library/mdmem:0.1.0
+podman inspect --format '{{index .RepoDigests 0}}' harbor.cloud.traunseenet.com/library/mdmem:0.1.0
+```
+
+Put tag **and** digest in `deployment.yaml`. Never `:latest` + `imagePullPolicy: Always`
+for this service: without a runner, an unpinned tag means the running code is not in
+git, and a pod restart becomes an unplanned upgrade of your memory store (§13a.2, §14.20).
+
+## Deploy (first install)
+
+`deploy/kubernetes/apps/apps/mdmem/**` mirrors the fluxcd repo's nesting; copy it in.
+The manifests were written from §13a and validated with `kubectl kustomize`, but **not**
+against a live cluster — verify each step.
+
+1. Build and push the image; record the digest and put it in `deployment.yaml`.
+2. Seed Vault:
+   ```bash
+   vault kv put secret/apps/mdmem \
+     MCP_TOKEN="$(openssl rand -hex 32)" \
+     BIFROST_VK="<vk from the Bifrost UI>" \
+     OIDC_CLIENT_SECRET="<from PocketID>" \
+     SESSION_SECRET="$(openssl rand -hex 32)"
+   vault kv patch secret/apps/bifrost MDMEM_AUTH_HEADER="Bearer <the same MCP_TOKEN>"
+   ```
+3. In PocketID: create the `mdmem_admins` group, add your account, create the OIDC
+   client with redirect URI `https://memory.cloud.traunseenet.com/ui/auth/callback`,
+   and confirm the client emits `groups` in the ID token — decode a real token, don't
+   assume. Client id → `mdmem-env`, client secret → Vault.
+4. Create the restic repo for `restic-mdmem`; copy `secret.example.yaml` to
+   `secret.yaml`, fill it in, SOPS-encrypt it.
+5. Commit `kubernetes/apps/apps/mdmem/**`. Check: ExternalSecret `Ready`, PVC `Bound`,
+   Deployment healthy, `/readyz` 200.
+6. Verify embeddings end to end from inside the cluster:
+   ```bash
+   kubectl -n apps run curl --rm -it --restart=Never \
+     --image=harbor.cloud.traunseenet.com/docker.io/curlimages/curl -- \
+     sh -c 'curl -s http://bifrost.apps.svc.cluster.local:8080/v1/embeddings \
+       -H "Authorization: Bearer $VK" -H "content-type: application/json" \
+       -d "{\"model\":\"llamaswap/qwen3-embedding-4b\",\"input\":[\"test\"]}"' \
+     | jq '.data[0].embedding|length'    # expect 2560
+   ```
+   A 404/400 means the alias or the provider prefix is wrong, not that mdmem is broken.
+7. Only then add `deploy/bifrost-mcp-client.json` to Bifrost's `config.json` and commit,
+   so a failure is attributable to one change at a time. Create a VK scoped to the six
+   `memory_*` tools.
+8. Apply the NetworkPolicy and re-check the two things it can silently break: the pod
+   stays Ready (kubelet probes) and the Prometheus target stays `UP` (§14.23).
+9. Run a `ReplicationSource` and **test a restore** into a throwaway PVC before the
+   vault holds anything you would miss.
+
+### Rotating MCP_TOKEN
+
+The token lives in two Vault paths. Rotating one and not the other gives every agent a
+401 that looks like an mdmem bug (§14.15):
+
+```bash
+NEW=$(openssl rand -hex 32)
+vault kv patch secret/apps/mdmem   MCP_TOKEN="$NEW"
+vault kv patch secret/apps/bifrost MDMEM_AUTH_HEADER="Bearer $NEW"
+# ESO refreshes both within 5 min; restart both pods to pick them up at once:
+kubectl -n apps rollout restart deploy/mdmem deploy/bifrost
+```
+
+## Connecting a harness
+
+All harnesses go through Bifrost — mdmem's own port is never exposed.
+
+- **Claude Code**
+  `claude mcp add --transport http mdmem https://bifrost.cloud.traunseenet.com/mcp --header "Authorization: Bearer <vk>"`
+- **Open WebUI** — External Tools → MCP (Streamable HTTP), same URL + VK header.
+- **claude.ai** — custom connector against Bifrost's OAuth 2.1 endpoint. `client.allowed_origins`
+  in Bifrost's `config.json` currently lists only `https://bifrost.cloud.traunseenet.com`;
+  a browser-side connector flow may need `https://claude.ai` added.
+- **Hermes / opencode** — remote MCP config, same URL + VK header.
+
+## Where the implementation deviates from PLAN.md
+
+Each of these is a decision made while building, with the reason:
+
+| § | Plan said | Built | Why |
+|---|---|---|---|
+| 3, 9 | `mcp` SDK, `FastMCP` | `mcp==2.2.0`, `MCPServer` | FastMCP was renamed in SDK 2.x; 1.2.0 has no Streamable HTTP at all. Same API shape, current SDK. |
+| 5 | `chunks_fts`/`chunks_tri` are `content=''` | ordinary FTS5 tables | Contentless FTS5 cannot delete a row before SQLite 3.43 (bookworm ships 3.40) and cannot `snippet()`. The index is rewritten on every edit, so deletes are the hot path. Costs a few MB of duplicated text. |
+| 6 | "one asyncio task owns all SQLite writes (queue)" | one write connection behind a lock; async paths use `asyncio.to_thread` | Same invariant (exactly one writer), and it also holds for the CLI and the tests, which have no event loop of the service's. |
+| 8 | vec0 cosine | vec0 L2 over L2-normalised vectors, `cos = 1 − d²/2` | Monotone-equivalent and does not depend on a `distance_metric=` option in a pre-v1 extension. |
+| 9 | — | `MCP_ALLOWED_HOSTS` (new setting) | The SDK's DNS-rebinding protection checks the `Host` header. Without the in-cluster service name on the allow-list, every Bifrost call returns 421. Added to §14 as pitfall 24. |
+| 10 | — | `UI_AUTH_DISABLED` (new setting) | Makes §10's "P4 may ship without OIDC, but then without an HTTPRoute" explicit and loud instead of implicit. |
+| 11 | `eval/queries.yaml` with 50–100 real queries | `eval/queries.example.yaml` + a synthetic corpus and `--fake-embeddings` | Real queries are Markus's to write, and the real model is not reachable from the build environment. The harness is complete and tested; the committed run is a machinery check, and the defaults stay as §2.5 specifies. |
+
+## Status
+
+P0–P5 are implemented and tested. P6 is written but unverified: no cluster, registry,
+Vault or PocketID was reachable from the build environment, so the image was never
+built, the manifests were never applied, and the live checks in the deploy list above
+are still to do.
@@ -0,0 +1,19 @@
+{
+  "_howto": "Add this object to mcp.client_configs in kubernetes/apps/apps/bifrost/bifrost/app/config.json. That file is a configMapGenerator input, so editing it is the whole change — the generated ConfigMap's hash suffix moves and Flux rolls the Deployment. Commit it only AFTER mdmem itself is healthy, so a failure is attributable to one change.",
+  "name": "mdmem",
+  "connection_type": "http",
+  "connection_string": "http://mdmem.apps.svc.cluster.local:8080/mcp",
+  "auth_type": "headers",
+  "headers": { "Authorization": "env.MDMEM_AUTH_HEADER" },
+  "is_ping_available": false,
+  "is_code_mode_client": false,
+  "_comment": "Long-term agent memory. Code Mode deliberately off: memory bodies are untrusted text, and code mode hands tool output to a model that then writes executable code. Revisit only after the prompt-injection behaviour is tested with a local model. tools_to_execute is an explicit allow-list, not '*', so a future mdmem release cannot add a destructive tool that silently becomes reachable.",
+  "tools_to_execute": [
+    "memory_search",
+    "memory_get",
+    "memory_write",
+    "memory_update",
+    "memory_supersede",
+    "memory_flag"
+  ]
+}
@@ -0,0 +1,5 @@
+---
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+resources:
+  - ./mdmem/ks.yaml
@@ -0,0 +1,82 @@
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: mdmem
+  labels:
+    app: mdmem
+spec:
+  replicas: 1
+  strategy:
+    type: Recreate            # SQLite + watchfiles need a single writer
+  selector:
+    matchLabels:
+      app: mdmem
+  template:
+    metadata:
+      labels:
+        app: mdmem
+    spec:
+      securityContext:
+        runAsUser: 1000
+        runAsGroup: 1000
+        runAsNonRoot: true
+        fsGroup: 1000
+        fsGroupChangePolicy: OnRootMismatch   # Longhorn: else /data comes up root-owned
+      containers:
+        - name: mdmem
+          # Pin tag AND digest. Renovate does not track library/*, so the digest
+          # is bumped by hand in the same commit that pushes a build (§13a.2).
+          image: harbor.cloud.traunseenet.com/library/mdmem:0.1.0@sha256:REPLACE_WITH_DIGEST
+          imagePullPolicy: IfNotPresent
+          args: ["serve"]
+          securityContext:
+            allowPrivilegeEscalation: false
+            readOnlyRootFilesystem: true
+            capabilities:
+              drop: ["ALL"]
+          env:
+            - name: HOME
+              value: /tmp
+            - name: XDG_CACHE_HOME
+              value: /tmp
+          envFrom:
+            - configMapRef:
+                name: mdmem-env
+            - secretRef:
+                name: mdmem          # from Vault via the ExternalSecret
+          ports:
+            - name: mcp
+              containerPort: 8080
+            - name: ui
+              containerPort: 8081
+          volumeMounts:
+            - name: data
+              mountPath: /data
+            - name: tmp
+              mountPath: /tmp
+          livenessProbe:
+            httpGet:
+              path: /healthz
+              port: ui
+            initialDelaySeconds: 30
+            periodSeconds: 30
+          readinessProbe:
+            httpGet:
+              path: /readyz          # never checks Bifrost (§14.17)
+              port: ui
+            initialDelaySeconds: 5
+            periodSeconds: 10
+          resources:
+            requests:
+              cpu: 100m
+              memory: 256Mi
+            limits:
+              cpu: "1"
+              memory: 1Gi
+      volumes:
+        - name: data
+          persistentVolumeClaim:
+            claimName: mdmem-data
+        - name: tmp
+          emptyDir: {}
@@ -0,0 +1,17 @@
+---
+apiVersion: external-secrets.io/v1
+kind: ExternalSecret
+metadata:
+  name: mdmem
+spec:
+  refreshInterval: 5m
+  secretStoreRef:
+    name: vault-backend
+    kind: ClusterSecretStore
+  target:
+    name: mdmem
+  dataFrom:
+    # KV-v2 mount is `secret`, so this reads secret/apps/mdmem:
+    #   MCP_TOKEN, BIFROST_VK, OIDC_CLIENT_SECRET, SESSION_SECRET
+    - extract:
+        key: apps/mdmem
@@ -0,0 +1,25 @@
+# Only the UI port is routed. :8080 (MCP) gets no HTTPRoute — Bifrost reaches it
+# at mdmem.apps.svc.cluster.local:8080, same namespace (§13a.5).
+#
+# Do not apply this before the OIDC group check is live (§14.19): *.cloud is
+# split-horizon, not private.
+---
+apiVersion: gateway.networking.k8s.io/v1
+kind: HTTPRoute
+metadata:
+  name: mdmem
+spec:
+  parentRefs:
+    - name: traefik
+      namespace: networking
+      sectionName: websecure-cloud
+  hostnames:
+    - memory.cloud.traunseenet.com
+  rules:
+    - matches:
+        - path:
+            type: PathPrefix
+            value: /
+      backendRefs:
+        - name: mdmem
+          port: 8081
@@ -0,0 +1,56 @@
+---
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+namespace: apps
+resources:
+  - ./externalsecret.yaml
+  - ./secret.yaml            # SOPS: restic-mdmem only (§12.1)
+  - ./pvc.yaml
+  - ./deployment.yaml
+  - ./service.yaml
+  - ./httproute.yaml
+  - ./networkpolicy.yaml
+  - ./servicemonitor.yaml
+configMapGenerator:
+  - name: mdmem-env
+    literals:
+      # --- storage ---------------------------------------------------------
+      - VAULT_DIR=/data/vault
+      - INDEX_DB=/data/index.db
+      - CACHE_DB=/data/emb_cache.db
+      - HISTORY_DIR=/data/.history
+      - AUDIT_LOG=/data/audit.jsonl
+      # --- embeddings ------------------------------------------------------
+      - BIFROST_URL=http://bifrost.apps.svc.cluster.local:8080
+      # provider-prefixed; a bare alias 404s and looks like an mdmem bug (§7, §14.14)
+      - EMBED_MODEL=llamaswap/qwen3-embedding-4b
+      - EMBED_DIM=2560
+      # --- retrieval -------------------------------------------------------
+      - FUSION=rrf
+      - CC_ALPHA=0.5
+      - FTS_TRIGRAM=1
+      - INBOX_WEIGHT=0.8
+      - BM25_WEIGHTS=3,2,1.5,1
+      - INCLUDE_INBOX_DEFAULT=true
+      - MIN_SCORE=
+      - MAX_CHARS_DEFAULT=6000
+      - DUP_THRESHOLD=0.90
+      - MAX_NOTE_CHARS=8000
+      # --- listeners -------------------------------------------------------
+      - MCP_ADDR=0.0.0.0:8080
+      - UI_ADDR=0.0.0.0:8081
+      # the MCP SDK checks the Host header; Bifrost dials the service name (§14.24)
+      - MCP_ALLOWED_HOSTS=mdmem.apps.svc.cluster.local:8080,mdmem:8080,localhost:8080
+      # --- ui / auth -------------------------------------------------------
+      - UI_BASE_URL=https://memory.cloud.traunseenet.com
+      - UI_ALLOWED_ORIGINS=https://memory.cloud.traunseenet.com
+      - OIDC_ISSUER=https://pocketid.cloud.traunseenet.com
+      - OIDC_SCOPE=openid email profile groups
+      - OIDC_CLIENT_ID=REPLACE-WITH-THE-POCKETID-CLIENT-UUID
+      - OIDC_GROUPS_CLAIM=groups
+      - OIDC_REQUIRED_GROUP=mdmem_admins
+      - SESSION_TTL_HOURS=12
+      # --- misc ------------------------------------------------------------
+      - LOG_LEVEL=info
+      - LOG_STYLE=json
+      - TZ=Europe/Vienna
@@ -0,0 +1,49 @@
+# Required, not optional (§13a.5a). newt's egress policy allows the entire apps
+# namespace on every port and reaches pods directly rather than through Traefik,
+# so without this policy "not published through Pangolin" is a promise that
+# lives in a remote dashboard instead of in git — and :8080, whose only control
+# is a bearer token, would be one click from the internet.
+#
+# After applying, verify the two things it can silently break: the pod stays
+# Ready (kubelet probes) and the Prometheus target stays UP (§14.23).
+---
+apiVersion: networking.k8s.io/v1
+kind: NetworkPolicy
+metadata:
+  name: mdmem-ingress
+spec:
+  podSelector:
+    matchLabels:
+      app: mdmem
+  policyTypes: ["Ingress"]
+  ingress:
+    # MCP: Bifrost only, same namespace
+    - from:
+        - namespaceSelector:
+            matchLabels:
+              kubernetes.io/metadata.name: apps
+          podSelector:
+            matchLabels:
+              app: bifrost
+      ports:
+        - protocol: TCP
+          port: 8080
+    # UI: Traefik only — deliberately NOT newt, so Pangolin cannot bypass the gateway
+    - from:
+        - namespaceSelector:
+            matchLabels:
+              kubernetes.io/metadata.name: networking
+          podSelector:
+            matchLabels:
+              app.kubernetes.io/name: traefik
+      ports:
+        - protocol: TCP
+          port: 8081
+    # /metrics scrape
+    - from:
+        - namespaceSelector:
+            matchLabels:
+              kubernetes.io/metadata.name: observability
+      ports:
+        - protocol: TCP
+          port: 8081
@@ -0,0 +1,38 @@
+---
+apiVersion: volsync.backube/v1alpha1
+kind: ReplicationDestination
+metadata:
+  name: mdmem-restore
+spec:
+  trigger:
+    manual: restore-once
+  restic:
+    repository: restic-mdmem
+    copyMethod: Snapshot
+    storageClassName: longhorn-storage
+    volumeSnapshotClassName: longhorn-snapshot
+    accessModes: ["ReadWriteOnce"]
+    capacity: 5Gi
+    cacheCapacity: 1Gi
+    enableFileDeletion: false
+    moverSecurityContext:
+      runAsUser: 1000
+      runAsGroup: 1000
+      fsGroup: 1000
+---
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+  name: mdmem-data
+spec:
+  accessModes: ["ReadWriteOnce"]
+  storageClassName: longhorn-storage
+  # Do NOT set numberOfReplicas: 1 here. The vault is the primary artefact of
+  # this service; an hourly restic snapshot is not block-level redundancy.
+  dataSourceRef:
+    kind: ReplicationDestination
+    apiGroup: volsync.backube
+    name: mdmem-restore
+  resources:
+    requests:
+      storage: 5Gi
@@ -0,0 +1,17 @@
+# Rename to secret.yaml and SOPS-encrypt before committing. This is the ONE
+# secret that stays out of Vault (§12.1): the restore path must work when Vault
+# itself is what is being restored.
+#
+#   sops --encrypt --age age1q3mu6g0l3kr7spszks69yqrvlxvlgfrqgw9r00qqt8cj9xpjd9mqdkd7um \
+#        --encrypted-regex '^(data|stringData)$' --in-place secret.yaml
+---
+apiVersion: v1
+kind: Secret
+metadata:
+  name: restic-mdmem
+type: Opaque
+stringData:
+  RESTIC_REPOSITORY: s3:https://s3.example/restic-mdmem
+  RESTIC_PASSWORD: REPLACE
+  AWS_ACCESS_KEY_ID: REPLACE
+  AWS_SECRET_ACCESS_KEY: REPLACE
@@ -0,0 +1,17 @@
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: mdmem
+  labels:
+    app: mdmem
+spec:
+  selector:
+    app: mdmem
+  ports:
+    - name: mcp
+      port: 8080
+      targetPort: mcp
+    - name: ui
+      port: 8081
+      targetPort: ui
@@ -0,0 +1,18 @@
+---
+apiVersion: monitoring.coreos.com/v1
+kind: ServiceMonitor
+metadata:
+  name: mdmem
+  labels:
+    app: mdmem
+spec:
+  endpoints:
+    - port: ui
+      path: /metrics
+      interval: 15s
+      honorLabels: true
+  namespaceSelector:
+    matchNames: ["apps"]
+  selector:
+    matchLabels:
+      app: mdmem
@@ -0,0 +1,6 @@
+---
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+namespace: apps
+resources:
+  - ./pvc-backup.yaml
@@ -0,0 +1,30 @@
+# Hourly, not six-hourly: the vault accumulates irreplaceable hand-written
+# content between snapshots, unlike a re-derivable app database (§13a.4).
+# The whole PVC is backed up — excluding index.db/emb_cache.db would only buy a
+# full re-embed at restore time.
+---
+apiVersion: volsync.backube/v1alpha1
+kind: ReplicationSource
+metadata:
+  name: mdmem
+spec:
+  sourcePVC: mdmem-data
+  trigger:
+    schedule: "0 * * * *"
+  restic:
+    repository: restic-mdmem
+    copyMethod: Snapshot
+    storageClassName: longhorn-storage
+    volumeSnapshotClassName: longhorn-snapshot
+    cacheCapacity: 1Gi
+    pruneIntervalDays: 1
+    retain:
+      hourly: 5
+      daily: 4
+      weekly: 3
+      monthly: 2
+      yearly: 1
+    moverSecurityContext:
+      runAsUser: 1000
+      runAsGroup: 1000
+      fsGroup: 1000
@@ -0,0 +1,58 @@
+---
+apiVersion: kustomize.toolkit.fluxcd.io/v1
+kind: Kustomization
+metadata:
+  name: mdmem-app
+  namespace: flux-system
+spec:
+  targetNamespace: apps
+  commonMetadata:
+    labels:
+      app.kubernetes.io/name: mdmem
+  dependsOn:
+    - name: traefik-base
+    - name: longhorn-addons
+    - name: cert-manager-addons
+    - name: volsync
+    - name: harbor-app                # the image is pulled from in-cluster Harbor
+    - name: hashicorp-vault-app       # the ExternalSecret needs Vault reachable
+    - name: external-secrets-app
+    - name: bifrost-app               # embeddings + the MCP front door
+  path: ./kubernetes/apps/apps/mdmem/mdmem/app
+  prune: true
+  sourceRef:
+    kind: GitRepository
+    name: fluxcd-kubernetes
+    namespace: flux-system
+  wait: true
+  interval: 15m
+  retryInterval: 15m
+  timeout: 15m
+  healthChecks:
+    - apiVersion: apps/v1
+      kind: Deployment
+      name: mdmem
+      namespace: apps
+---
+apiVersion: kustomize.toolkit.fluxcd.io/v1
+kind: Kustomization
+metadata:
+  name: mdmem-backup
+  namespace: flux-system
+spec:
+  targetNamespace: apps
+  commonMetadata:
+    labels:
+      app.kubernetes.io/name: mdmem
+  dependsOn:
+    - name: mdmem-app
+    - name: volsync
+  path: ./kubernetes/apps/apps/mdmem/mdmem/backup
+  prune: true
+  sourceRef:
+    kind: GitRepository
+    name: fluxcd-kubernetes
+    namespace: flux-system
+  interval: 15m
+  retryInterval: 15m
+  timeout: 15m