diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..25893ee --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,8 @@ +# Changelog + +## 2026-05-03 19:26 CEST — Fee-aware allocation for multi-month payments + +- `reconcile()` no longer splits a multi-month payment evenly. Allocation is now per-member with two phases: greedy (if amount ≥ total expected, each month gets exactly its expected fee and overflow → credit) and proportional (otherwise distribute by each month's expected). Fixes the case where e.g. 1250 CZK covering 3 months with mixed fees (750/350/150) marked two months red. +- Out-of-window months keep the previous even-split-to-credit behavior. Fallback to even split when all matched months have `expected = 0` (prepayment before attendance is recorded). +- Display layer only — no changes to how payments are stored in Google Sheets; `Inferred Amount` still holds the full bank amount. +- Files: [scripts/match_payments.py](scripts/match_payments.py), [tests/test_reconcile_exceptions.py](tests/test_reconcile_exceptions.py) (6 new test cases). diff --git a/CLAUDE.md b/CLAUDE.md index 9183d0a..d197811 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -16,22 +16,95 @@ Flask-based financial management system for FUJ (Frisbee Ultimate Jablonec). Han This project uses `uv` for dependency management. ```bash -uv venv # Create virtual environment -uv sync # Install dependencies from pyproject.toml +uv venv && uv sync source .venv/bin/activate ``` -Alternatively, use the Makefile: -- `make sync` - Sync bank transactions to Google Sheets -- `make infer` - Automatically infer Person/Purpose/Amount in the sheet -- `make reconcile` - Generate balance report from Google Sheets data -- `make fees` - Calculate expected fees from attendance -- `make match` - (Legacy) Match bank data directly -- `make web` - Start dashboard -- `make image` - Build Docker image +Set `PYTHONPATH=scripts:.` when running scripts directly (the Makefile does this automatically). -Requires `.secret/fuj-management-bot-credentials.json` for Google Sheets API access (configurable via `CREDENTIALS_PATH` env var). +## Commands + +```bash +make web # Start dashboard at http://localhost:5001 +make web-debug # Same with FLASK_DEBUG=1 +make test # Run all tests (unittest discover) +make test-v # Tests with verbose output +make fees # Print fee report from attendance sheet +make sync-2026 # Sync Fio bank transactions for 2026 to Google Sheets +make infer # Auto-fill Person/Purpose/Amount columns in payments sheet +make reconcile # Print balance report from Google Sheets data +make image # Build Docker image +``` + +Run a single test: +```bash +PYTHONPATH=scripts:. python -m unittest tests.test_app.TestWebApp.test_adults_route +``` + +## Architecture + +### Data flow + +``` +Google Sheets (attendance) ──► attendance.py ──► reconcile() ──► Flask routes ──► templates/ +Google Sheets (payments) ──► match_payments.py ──┘ +Fio Bank API ──► sync_fio_to_sheets.py ──► Google Sheets (payments) +``` + +### Key modules + +- `app.py` — Flask app; routes for `/adults`, `/juniors`, `/payments`, `/sync-bank`, `/qr`, `/flush-cache` +- `scripts/attendance.py` — Fetches attendance CSV from Google Sheets, computes per-member per-month fees. Contains fee rate constants (`ADULT_FEE_DEFAULT`, `JUNIOR_FEE_DEFAULT`) and `ADULT_MERGED_MONTHS` / `JUNIOR_MERGED_MONTHS` dicts. +- `scripts/match_payments.py` — `reconcile()` matches transactions to members/months. `fetch_sheet_data()` reads the payments sheet. `fetch_exceptions()` reads the `exceptions` tab. +- `scripts/cache_utils.py` — Invalidation via Google Drive API `modifiedTime`; falls back to 5-minute TTL buckets when Drive API is unavailable. Cache files live in `tmp/`. +- `scripts/sync_fio_to_sheets.py` — Pulls Fio bank transactions and appends them to the payments Google Sheet. +- `scripts/infer_payments.py` — Fills in Person/Purpose/Inferred Amount columns using name-matching heuristics. +- `scripts/config.py` — All external IDs, paths, and tunable TTLs. Override via env vars (`CREDENTIALS_PATH`, `BANK_ACCOUNT`, `CACHE_TTL_SECONDS`). + +### Member tiers + +Tiers are set in column B of the attendance sheet: +- `A` — Adult, pays fees (750 CZK/month for 2+ sessions, 200 CZK for exactly 1) +- `J` — Junior attending adult practices; their attendance is merged with the junior sheet +- `X` — Excluded from junior fee calculation (coaches, etc.) + +### Fee calculation + +- Adults: 0 sessions → 0, 1 session → 200 CZK, 2+ sessions → monthly rate (default 750 CZK) +- Juniors: 0 → 0, 1 → `"?"` (manual review required), 2+ → monthly rate (default 500 CZK) +- Per-member per-month overrides live in the `exceptions` tab of the payments sheet (columns: Name, Period YYYY-MM, Amount, Note). Exceptions are keyed by `(normalize(name), normalize(period))`. + +### Merged months + +`ADULT_MERGED_MONTHS` / `JUNIOR_MERGED_MONTHS` in `attendance.py` map a source month to a target month (e.g., `"2025-12": "2026-01"` merges December into January billing). The target month accumulates attendance from both months. + +### Caching + +`get_cached_data()` in `app.py` checks the Drive API `modifiedTime` before each request and serves a JSON file from `tmp/` when the sheet hasn't changed. Cache is warmed up at startup (`warmup_cache()`). Flush via `/flush-cache` (POST) or `flush_cache()`. + +### Payments sheet columns + +`Date | Amount | manual fix | Person | Purpose | Inferred Amount | Sender | VS | Message | Bank ID | Sync ID` + +`Person` and `Purpose` are written by `infer_payments.py` and can be manually corrected. `manual fix` column presence disables re-inference for that row. Multiple people or months are comma-separated in Person/Purpose. + +### QR codes + +`/qr?account=…&amount=…&message=…` generates a Czech QR Platba PNG (SPD format). ## Git Commits -When making git commits, always append yourself as co-author trailer to the end of the commit message to indicate AI assistance \ No newline at end of file +When making git commits, always append yourself as co-author trailer to the end of the commit message to indicate AI assistance + +## Changelog + +Maintain a running changelog in `CHANGELOG.md` at the repo root. After every significant change, fix, or update — once the user confirms it works — append a new entry **at the top** of the file in this format: + +```markdown +## YYYY-MM-DD HH:MM TZ — short title + +- One-line summary of what changed and why. +- Key files touched (optional, only if useful for traceability). +``` + +Get the timestamp with `date "+%Y-%m-%d %H:%M %Z"`. Skip trivial edits (typos, formatting, comment tweaks); only log changes a future reader would care about. diff --git a/scripts/match_payments.py b/scripts/match_payments.py index 2896c8b..b166986 100644 --- a/scripts/match_payments.py +++ b/scripts/match_payments.py @@ -385,8 +385,7 @@ def reconcile( }) continue - num_allocations = len(matched_members) * len(matched_months) - per_allocation = amount / num_allocations if num_allocations > 0 else 0 + member_share = amount / len(matched_members) if matched_members else 0 for member_name, confidence in matched_members: if member_name not in ledger: @@ -397,20 +396,64 @@ def reconcile( unmatched.append(tx) continue - for month_key in matched_months: - entry = { - "amount": per_allocation, - "date": tx["date"], - "sender": tx["sender"], - "message": tx["message"], - "confidence": confidence, - } - if month_key in ledger[member_name]: - ledger[member_name][month_key]["paid"] += per_allocation - ledger[member_name][month_key]["transactions"].append(entry) - else: - # Future month — track as credit - credits[member_name] = credits.get(member_name, 0) + int(per_allocation) + in_window = [(m, ledger[member_name][m]["expected"]) for m in matched_months if m in ledger[member_name]] + out_of_window = [m for m in matched_months if m not in ledger[member_name]] + + # Out-of-window months (outside display range): even split → credit, same as before. + n_total = len(matched_months) + if out_of_window and n_total > 0: + out_credit = member_share / n_total * len(out_of_window) + credits[member_name] = credits.get(member_name, 0) + int(out_credit) + else: + out_credit = 0.0 + + in_window_share = member_share - out_credit + + if not in_window: + continue + + total_expected = sum(e for _, e in in_window) + + if total_expected > 0 and in_window_share >= total_expected: + # Greedy phase: payment covers all in-window fees; overflow → credit. + credits[member_name] = credits.get(member_name, 0) + int(in_window_share - total_expected) + for m, exp in in_window: + alloc = float(exp) + ledger[member_name][m]["paid"] += alloc + ledger[member_name][m]["transactions"].append({ + "amount": alloc, + "date": tx["date"], + "sender": tx["sender"], + "message": tx["message"], + "confidence": confidence, + }) + elif total_expected > 0: + # Proportional phase: distribute in_window_share by each month's expected fee. + # Last month absorbs any float remainder so the sum equals in_window_share exactly. + remaining = in_window_share + for i, (m, exp) in enumerate(in_window): + alloc = remaining if i == len(in_window) - 1 else in_window_share * exp / total_expected + remaining -= alloc + ledger[member_name][m]["paid"] += alloc + ledger[member_name][m]["transactions"].append({ + "amount": alloc, + "date": tx["date"], + "sender": tx["sender"], + "message": tx["message"], + "confidence": confidence, + }) + else: + # Fallback: no expected fees (prepayment before attendance recorded); even split. + per_month = in_window_share / len(in_window) + for m, _ in in_window: + ledger[member_name][m]["paid"] += per_month + ledger[member_name][m]["transactions"].append({ + "amount": per_month, + "date": tx["date"], + "sender": tx["sender"], + "message": tx["message"], + "confidence": confidence, + }) # Calculate final total balances (window + off-window credits) final_balances: dict[str, int] = {} diff --git a/tests/test_reconcile_exceptions.py b/tests/test_reconcile_exceptions.py index 2a40689..e817643 100644 --- a/tests/test_reconcile_exceptions.py +++ b/tests/test_reconcile_exceptions.py @@ -52,5 +52,108 @@ class TestReconcileWithExceptions(unittest.TestCase): alice_data = result['members']['Alice'] self.assertEqual(alice_data['months']['2026-01']['expected'], 750, "Should fallback to attendance fee") +def _tx(person, purpose, amount): + return { + 'date': '2026-01-01', + 'amount': amount, + 'person': person, + 'purpose': purpose, + 'inferred_amount': amount, + 'sender': 'Sender', + 'message': 'fee', + } + + +class TestMultiMonthAllocation(unittest.TestCase): + """Fee-aware allocation across multiple months in a single payment.""" + + def test_greedy_exact_match(self): + """Payment equals total expected → every month fully covered (green).""" + members = [('Alice', 'A', {'2026-02': (750, 3), '2026-03': (350, 3), '2026-04': (150, 2)})] + sorted_months = ['2026-02', '2026-03', '2026-04'] + tx = _tx('Alice', '2026-02, 2026-03, 2026-04', 1250) + + result = reconcile(members, sorted_months, [tx]) + months = result['members']['Alice']['months'] + + self.assertEqual(int(months['2026-02']['paid']), 750) + self.assertEqual(int(months['2026-03']['paid']), 350) + self.assertEqual(int(months['2026-04']['paid']), 150) + + def test_greedy_overpayment_goes_to_credit(self): + """Payment exceeds total expected → each month fully covered, surplus → credit.""" + members = [('Alice', 'A', {'2026-01': (750, 3), '2026-02': (750, 3)})] + sorted_months = ['2026-01', '2026-02'] + tx = _tx('Alice', '2026-01, 2026-02', 2000) + + result = reconcile(members, sorted_months, [tx]) + months = result['members']['Alice']['months'] + + self.assertEqual(int(months['2026-01']['paid']), 750) + self.assertEqual(int(months['2026-02']['paid']), 750) + self.assertEqual(result['credits'].get('Alice', 0), 500) + + def test_proportional_underpayment(self): + """Payment < total expected → proportional split; sum of paid == payment amount.""" + members = [('Alice', 'A', {'2026-02': (750, 3), '2026-03': (350, 3), '2026-04': (750, 3)})] + sorted_months = ['2026-02', '2026-03', '2026-04'] + amount = 1250 + tx = _tx('Alice', '2026-02, 2026-03, 2026-04', amount) + + result = reconcile(members, sorted_months, [tx]) + months = result['members']['Alice']['months'] + + paid_02 = months['2026-02']['paid'] + paid_03 = months['2026-03']['paid'] + paid_04 = months['2026-04']['paid'] + + # All months should be partial (underpaid) + self.assertLess(paid_02, 750) + self.assertLess(paid_03, 350) + self.assertLess(paid_04, 750) + # Sum must equal the original payment (no CZK lost) + self.assertAlmostEqual(paid_02 + paid_03 + paid_04, amount, places=2) + # 02 and 04 have equal expected → equal allocation + self.assertAlmostEqual(paid_02, paid_04, places=2) + + def test_single_month_unchanged(self): + """Single-month payment: full amount goes to that month (regression guard).""" + members = [('Alice', 'A', {'2026-01': (750, 3)})] + sorted_months = ['2026-01'] + tx = _tx('Alice', '2026-01', 750) + + result = reconcile(members, sorted_months, [tx]) + self.assertAlmostEqual(result['members']['Alice']['months']['2026-01']['paid'], 750, places=2) + + def test_two_members_multi_month(self): + """Two members, 2 months: each member gets member_share, then fee-aware per month.""" + members = [ + ('Alice', 'A', {'2026-01': (750, 3), '2026-02': (350, 3)}), + ('Bob', 'A', {'2026-01': (750, 3), '2026-02': (350, 3)}), + ] + sorted_months = ['2026-01', '2026-02'] + # Both members pay together; total expected per member = 1100 + tx = _tx('Alice, Bob', '2026-01, 2026-02', 2200) + + result = reconcile(members, sorted_months, [tx]) + + for name in ('Alice', 'Bob'): + months = result['members'][name]['months'] + self.assertAlmostEqual(months['2026-01']['paid'], 750, places=2) + self.assertAlmostEqual(months['2026-02']['paid'], 350, places=2) + + def test_fallback_even_split_when_no_expected(self): + """All matched months have expected=0 → falls back to even split.""" + members = [('Alice', 'A', {'2026-01': (0, 0), '2026-02': (0, 0)})] + sorted_months = ['2026-01', '2026-02'] + tx = _tx('Alice', '2026-01, 2026-02', 300) + + result = reconcile(members, sorted_months, [tx]) + months = result['members']['Alice']['months'] + + self.assertAlmostEqual(months['2026-01']['paid'], 150, places=2) + self.assertAlmostEqual(months['2026-02']['paid'], 150, places=2) + + if __name__ == '__main__': unittest.main()