Markus
c79e55f3e7
Found by running the service for real: creating a file in the vault while mdmem is up only logged 'reindex of changed file failed' until the next scan, which is exactly the workflow the vault is meant to support. Adds the live test suite (-m live) and the llama-swap model entry.
26 lines
1.0 KiB
YAML
26 lines
1.0 KiB
YAML
# Add to llama-swap's config on evo-x2 (10.0.0.59:9000) — the box Bifrost's
|
|
# `llamaswap` provider already points at. Not managed by Flux; llama-swap runs
|
|
# off-cluster (§7).
|
|
#
|
|
# --pooling last is mandatory: Qwen3 embeddings use last-token pooling and the
|
|
# wrong pooling degrades retrieval silently (§14.1, [R2][R3]).
|
|
# Never add --reranking to this instance: with both flags on, a reported
|
|
# llama.cpp bug returns all-zero embeddings (§14.2, [R5]).
|
|
# Q8_0, not Q4_K: Q4_K measured 0.974 cosine against the reference (§14.3, [R4]).
|
|
models:
|
|
qwen3-embedding-4b:
|
|
cmd: >
|
|
llama-server --port ${PORT}
|
|
-m /models/Qwen3-Embedding-4B-Q8_0.gguf
|
|
--embedding --pooling last
|
|
-ub 8192 -c 8192 -ngl 99
|
|
ttl: 0 # keep resident; an embedding call must not evict the chat model
|
|
|
|
# Keep it co-resident with the chat model rather than swapping. Check the group
|
|
# syntax against your llama-swap version:
|
|
# groups:
|
|
# resident:
|
|
# swap: false
|
|
# exclusive: false
|
|
# members: ["qwen3-embedding-4b", "<your chat model>"]
|