fin/.claude/skills/fin-diagnostics-and-tooling/SKILL.md

name, description

name

description

fin-diagnostics-and-tooling

Read-only diagnostic scripts and API probes for the fin app. Load when you need to verify data integrity, quantify a suspected bug with real numbers, audit the ledger (linked transfer/settlement legs, holdings vs stock transactions, account balances), check price_cache or net-worth-snapshot freshness, independently recompute a month's spending, or gather before/after evidence around a change. The measurement layer — measure, don't eyeball.

fin — diagnostics and tooling

Five tested, stdlib-only, READ-ONLY Python scripts plus API probes that turn "the numbers look wrong" into a precise, diffable anomaly list. Every script:

• takes the SQLite DB path as argv[1] and opens it with sqlite3.connect("file:...?mode=ro", uri=True) — it cannot write
• exits 0 = clean, 1 = anomalies found (exception: spending_recompute.py is a recompute tool, exit 0 on success)
• supports --json for machine-diffable output
• needs only /usr/bin/python3 (3.11+), no pip, no venv, no docker

The prod DB is sacred. These scripts are safe to point at a live copy, but prefer a copy anyway (below). Never run anything that writes against prod.

Get a safe DB copy first

# from the homelab host running the stack (full runbook: fin-run-and-operate)
docker compose exec api python3 -c "import sqlite3; s=sqlite3.connect('file:/data/fin.db?mode=ro', uri=True); d=sqlite3.connect('/tmp/fin-copy.db'); s.backup(d); d.close()"
docker compose cp api:/tmp/fin-copy.db ./fin-copy.db

sqlite3 .backup snapshots WAL databases consistently. Note: a mode=ro open of a live WAL DB can fail if the -wal/-shm files need recovery and aren't writable — if that happens, use the backup copy, never loosen the mode. Backup/restore discipline and volume paths: see fin-run-and-operate.

Diagnostic decision table

Question	Tool	Healthy looks like
Is the API up / authed?	curl -s $FIN_API_URL/api/health (always open)	{"status":"ok"...}
Is net worth plausible right now?	fin networth --json | jq '.total_eur, .eur_usd_rate'	total matches expectation; rate ≈ market EUR/USD, not 1.0
Does a transfer/settlement have both legs?	scripts/audit_linked_legs.py	exit 0
Do holdings match the trade log?	scripts/audit_holdings.py	exit 0
What is each account's computed balance? Any masking overrides?	scripts/audit_balances.py	exit 0; computed values match the UI
Are prices/FX fresh? Snapshot gaps?	scripts/check_freshness.py	exit 0; ticker ages are minutes (5-min scheduler job)
Is the spending total right for month X?	scripts/spending_recompute.py vs fin spending month X --json	identical totals per category
Prices stale in prod, why?	check_freshness.py on a copy + api logs (see fin-run-and-operate)	—
Is the cause a known bug pattern?	not this skill — fin-debugging-playbook / fin-failure-archaeology	—

DB scripts answer "is the stored data consistent with the code's rules?". API probes answer "what is the running system reporting?". A bug report needs both: if the DB audits are clean but the API number is wrong, the bug is in a service/router; if an audit fails, the data itself is damaged.

Scripts

All paths below relative to this skill's scripts/ directory. All five were tested (as of 2026-07-05) against a synthetic SQLite DB built from the api/app/models.py schema, seeded with healthy rows and one deliberately broken case per check — each script flagged exactly the broken cases and exited 0 on the clean DB (30/30 assertions passed).

audit_linked_legs.py — two-leg integrity

python3 scripts/audit_linked_legs.py fin-copy.db [--allow-legacy] [--json]

Checks every transaction against the pairing rules in crud.py (create_transfer, create_settlement, create_split_expense, create_splitwise_i_paid, create_funded_buy, delete_transaction).

Interpretation:

• CLEAN + counts line — healthy. The counts tell you how many same-currency pairs were amount-checked, how many cross-currency pairs and funded-buy single legs were (correctly) skipped.
• orphan-link — linked_transaction_id points at a deleted row. One leg of a pair was removed without the other: the surviving leg inflates/deflates one account's balance. This is invariant #1 damage (see CLAUDE.md).
• self-link / self-account-link — a row references itself; corrupt write, never produced by crud.py.
• amounts-dont-cancel — a same-currency transfer/settlement pair whose amounts don't sum to 0 (tolerance 0.005; fees live in fee and are excluded). One side was edited after creation. Cross-currency pairs are exempt: create_transfer writes to_amount on the destination leg.
• non-reciprocal — A links to B but B doesn't link back. Deleting B via the API would still find A (delete follows links both ways), but the pair is malformed; expect an edit went through PATCH /api/transactions.
• unlinked-leg — transfer/settlement with NULL linked_transaction_id. Rows written before that column existed are "legacy"; rerun with --allow-legacy to accept legacy rows that have a crud-style fallback match (reversed accounts + same date + same type). A leg that's still flagged with --allow-legacy is a genuine orphan.
• missing-account / missing-linked-account — leg points at a deleted account id (accounts are normally deactivated, never deleted).
• Never flagged, by design: funded-buy cash legs (single transfer linked to an investment/rsu account — create_funded_buy writes no partner row); splitwise_pay pair amounts (bank leg = −total, splitwise leg = +partner_share; they must NOT cancel); the split-expense expense row's one-way link into its transfer pair.

audit_holdings.py — holdings vs stock_transactions

python3 scripts/audit_holdings.py fin-copy.db [--json]

Faithful reimplementation of crud.recompute_holding_from_stock_transactions (date-ASC/id-ASC replay; buy/rsu_vest add shares*price + fee to cost; sell removes at running average cost, clamped at 0; fractional shares supported), diffed against the holdings table.

Interpretation:

• shares-mismatch / cost-basis-mismatch (tolerance 1e-6 shares / 0.005 cost) — the holding was mutated outside the recompute path, or a recompute was skipped (invariant #2). Fix by re-running the recompute via the API (delete+recreate a stock txn) — never by editing the holding row.
• missing-holding — stock transactions exist but no holding row; the position is invisible to net worth.
• stale-holding — holding row with zero stock transactions; recompute would delete it; it inflates net worth.
• invalid-history — a sell exceeds shares held at that point in the replay. crud rejects this on write (422), so its presence means the history was edited/imported around validation.
• unknown-stock-txn-type — type outside buy/sell/rsu_vest; crud silently ignores such rows, so the holding may still "match" while the ledger carries a dead row.
• WARNING currency-drift (not exit-affecting) — holding currency differs from the first transaction's; crud never updates currency on existing holdings, so this can drift legitimately after deleting the first txn.

audit_balances.py — computed balance per account

python3 scripts/audit_balances.py fin-copy.db [--allow-overrides] [--json]

Prints SUM(amount - fee) per account (invariant #5 — never SUM(amount)), with type, currency, txn count, and any manual_override_balance.

Interpretation:

• The table itself is the evidence: capture it before/after any change that touches transactions.
• manual-override — the override silently replaces the computed value in services/net_worth.py. The printed delta is how much net worth differs from what the ledger says. Overrides may be intentional (e.g. a hand-set Splitwise balance) — use --allow-overrides once confirmed, but a large drifting delta means the ledger under the override has rotted.
• orphan-transaction — txn whose account_id no longer exists; it is invisible to every balance and net worth calculation.
• Investment/rsu accounts show computed ≈ 0 with 0 txns — normal; their value comes from holdings × price_cache, not transactions.

check_freshness.py — prices, FX, snapshot gaps

python3 scripts/check_freshness.py fin-copy.db [--hours-stale 26] [--days 30] [--json]

Interpretation (thresholds as of 2026-07-05):

• Age column — the scheduler refreshes every 5 minutes plus a midnight job, so healthy ages are minutes while the api container is up. Hours old = container was down or yfinance/frankfurter kept failing (both fall back to the stale cache silently — check api logs, see fin-run-and-operate). The default 26 h threshold only catches hard outages; tighten with --hours-stale 1 when investigating.
• missing-fx EUR/USD — worst case: calculate_net_worth falls back to rate 1.0 and produces confidently wrong totals.
• missing-fx EUR/GBP — flagged only when GBP data exists; then _to_eur returns 0 and GBP amounts vanish from net worth (deliberate — see fin-failure-archaeology before "fixing").
• missing-ticker-price — a tracked ticker (holding with shares > 0, or a pending RSU grant's ticker) has no cache row: valued at €0. Fix: fin prices fetch TICKER or POST /api/prices/refresh.
• zero-price — cached price ≤ 0; same €0 effect.
• missing-snapshot — gap in the daily net-worth series (window ends today, container-local date). Today missing usually means the api container was down at midnight. Gaps make History charts lie by interpolation; the midnight job upserts, so a restart today backfills only today.

spending_recompute.py — independent spending recompute

python3 scripts/spending_recompute.py fin-copy.db 2026-06 [--account N] [--json]

Line-by-line replication of crud.get_monthly_spending: expense + splitwise_receive at raw amount; splitwise_pay at amount + splitwise_partner_share (user's net share — intentional, do not "fix"); receivable accounts excluded from spending; income excludes receivable/payable accounts, Opening balance rows, and rows with NULL description (SQL <> semantics — replicated deliberately). No is_active filter — inactive accounts count, same as the API.

Interpretation — diff against the API:

python3 scripts/spending_recompute.py fin-copy.db 2026-06 --json | jq -S '{total, income_total, categories: [.categories[] | {category_id, total}] | sort_by(.category_id)}' > /tmp/recomputed.json
fin spending month 2026-06 --json                | jq -S '{total, income_total, categories: [.categories[] | {category_id, total}] | sort_by(.category_id)}' > /tmp/api.json
diff /tmp/recomputed.json /tmp/api.json

• Identical — spending math is consistent between code and this spec; if the user still disputes the number, the disagreement is about semantics (load fin-accounting-reference), not a bug.
• Differs — either the DB copy is older than the live DB (recopy!), or get_monthly_spending changed since 2026-07-05 (see Provenance), or you found a real bug — capture both JSONs as evidence.
• Category order may differ (script sorts by total DESC; API preserves dict order) — always diff sorted by category_id as above.

API-level probes

Full CLI surface: the fin skill. Always --json | jq. Curl equivalents (add -H "Authorization: Bearer $FIN_API_TOKEN" only if the API enforces it):

fin networth --json | jq '{total: .total_eur, rate: .eur_usd_rate}'   # rate==1.0 => EUR/USD cache row missing
curl -s $FIN_API_URL/api/net-worth/current | jq '.total_eur'
fin networth history --range 1M --json | jq 'length'                  # ~30 => no snapshot gaps
curl -s "$FIN_API_URL/api/net-worth/history?range=1M" | jq 'length'
fin spending month 2026-06 --json | jq '{total, income_total}'
curl -s "$FIN_API_URL/api/spending/monthly?month=2026-06" | jq .
fin prices fx --json                                                    # cached EUR/USD
curl -s $FIN_API_URL/api/prices/fx-rate | jq .
curl -s -X POST $FIN_API_URL/api/prices/refresh                        # on-demand refresh (202)
fin accounts list --json | jq '[.[] | {id, name, type, balance}]'      # cross-check vs audit_balances.py

Cross-check pattern: API balance ≠ audit_balances.py computed balance for the same account → look for a manual_override_balance before suspecting crud.

Evidence workflow — before/after a change

The project discipline (see fin-validation-and-qa for the evidence hierarchy): predict the numbers, then measure them.

EV=/tmp/evidence-$(date +%Y%m%d-%H%M%S); mkdir -p $EV
S=.claude/skills/fin-diagnostics-and-tooling/scripts
for a in audit_linked_legs audit_holdings audit_balances check_freshness; do
  python3 $S/$a.py fin-copy.db --json > $EV/before-$a.json; echo "$a: exit $?"
done
python3 $S/spending_recompute.py fin-copy.db 2026-06 --json > $EV/before-spending.json
fin networth --json > $EV/before-networth.json

# ... apply the change, re-copy the DB ...

for a in audit_linked_legs audit_holdings audit_balances check_freshness; do
  python3 $S/$a.py fin-copy.db --json > $EV/after-$a.json
done
python3 $S/spending_recompute.py fin-copy.db 2026-06 --json > $EV/after-spending.json
fin networth --json > $EV/after-networth.json
for f in $EV/before-*.json; do diff "$f" "${f/before-/after-}" && echo "unchanged: $f"; done

Rules: state which numbers the change should move and by how much before running the after-pass; every diff line must be either predicted or explained; audits that were clean before must be clean after. Attach $EV contents to the change record (fin-change-control).

When NOT to use this skill

• Recording transactions / querying finances for the user → fin
• What a formula should be (spending semantics, FX, RSU math) → fin-accounting-reference; proving ledger properties → fin-ledger-analysis-toolkit
• Symptom→cause triage when you don't yet know what to measure → fin-debugging-playbook; historical bug context → fin-failure-archaeology
• Backup/restore, env vars, container ops, prod logs → fin-run-and-operate
• Deciding what evidence a change needs / writing tests → fin-validation-and-qa, fin-test-coverage-campaign
• Design/invariant questions before changing code → fin-architecture-contract, fin-change-control

Provenance and maintenance

• Ground truth (read, not guessed): api/app/models.py (all tables/columns), api/app/crud.py (pairing rules, holding recompute, spending math), api/app/services/price_fetcher.py (EUR/USD + EUR/GBP + yfinance, tracked tickers), api/app/services/net_worth.py (balance formula, fallbacks), api/app/services/scheduler.py (5-min price job, midnight snapshot job). Written against repo HEAD 082b608, 2026-07-05.
• All five scripts tested 2026-07-05 on Python 3.13 against a synthetic DB (schema derived from models.py; healthy rows + one seeded defect per check; each script flagged exactly the seeded defects, exit codes verified).
• Schema-drift check — run before trusting the scripts; any output means the ground truth moved and scripts/SKILL need re-verification:

git -C /home/markuseisl/Documents/git/fin diff --stat 082b608 -- api/app/models.py api/app/crud.py api/app/services/price_fetcher.py api/app/services/net_worth.py api/app/services/scheduler.py

• When drift is found: re-read the changed function, update the affected script and its interpretation section, re-run the synthetic-DB test (build a tiny DB with one broken case per check), and bump the commit hash and dates in this section.