fix: Distribute multi-month payments by per-month expected fee
All checks were successful
Build and Push / build (push) Successful in 33s
Deploy to K8s / deploy (push) Successful in 12s

reconcile() previously split a multi-month payment evenly across months,
which falsely flagged months as underpaid when their expected fees
differed (e.g. 1250 CZK for 02+03+04 2026 with rates 750/350/150 was
shown as 416/month with two months red).

The allocation now runs per matched member: greedy when the share covers
the total expected (each month gets its expected fee, surplus -> credit),
proportional by expected fee otherwise. Out-of-window months keep the
previous even-split-to-credit behavior. 6 new test cases.

Also adds CHANGELOG.md and a changelog convention in CLAUDE.md.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
2026-05-03 19:38:10 +02:00
parent ced238385e
commit dfdf2aacb8
4 changed files with 255 additions and 28 deletions

8
CHANGELOG.md Normal file
View File

@@ -0,0 +1,8 @@
# Changelog
## 2026-05-03 19:26 CEST — Fee-aware allocation for multi-month payments
- `reconcile()` no longer splits a multi-month payment evenly. Allocation is now per-member with two phases: greedy (if amount ≥ total expected, each month gets exactly its expected fee and overflow → credit) and proportional (otherwise distribute by each month's expected). Fixes the case where e.g. 1250 CZK covering 3 months with mixed fees (750/350/150) marked two months red.
- Out-of-window months keep the previous even-split-to-credit behavior. Fallback to even split when all matched months have `expected = 0` (prepayment before attendance is recorded).
- Display layer only — no changes to how payments are stored in Google Sheets; `Inferred Amount` still holds the full bank amount.
- Files: [scripts/match_payments.py](scripts/match_payments.py), [tests/test_reconcile_exceptions.py](tests/test_reconcile_exceptions.py) (6 new test cases).

View File

@@ -16,22 +16,95 @@ Flask-based financial management system for FUJ (Frisbee Ultimate Jablonec). Han
This project uses `uv` for dependency management. This project uses `uv` for dependency management.
```bash ```bash
uv venv # Create virtual environment uv venv && uv sync
uv sync # Install dependencies from pyproject.toml
source .venv/bin/activate source .venv/bin/activate
``` ```
Alternatively, use the Makefile: Set `PYTHONPATH=scripts:.` when running scripts directly (the Makefile does this automatically).
- `make sync` - Sync bank transactions to Google Sheets
- `make infer` - Automatically infer Person/Purpose/Amount in the sheet
- `make reconcile` - Generate balance report from Google Sheets data
- `make fees` - Calculate expected fees from attendance
- `make match` - (Legacy) Match bank data directly
- `make web` - Start dashboard
- `make image` - Build Docker image
Requires `.secret/fuj-management-bot-credentials.json` for Google Sheets API access (configurable via `CREDENTIALS_PATH` env var). ## Commands
```bash
make web # Start dashboard at http://localhost:5001
make web-debug # Same with FLASK_DEBUG=1
make test # Run all tests (unittest discover)
make test-v # Tests with verbose output
make fees # Print fee report from attendance sheet
make sync-2026 # Sync Fio bank transactions for 2026 to Google Sheets
make infer # Auto-fill Person/Purpose/Amount columns in payments sheet
make reconcile # Print balance report from Google Sheets data
make image # Build Docker image
```
Run a single test:
```bash
PYTHONPATH=scripts:. python -m unittest tests.test_app.TestWebApp.test_adults_route
```
## Architecture
### Data flow
```
Google Sheets (attendance) ──► attendance.py ──► reconcile() ──► Flask routes ──► templates/
Google Sheets (payments) ──► match_payments.py ──┘
Fio Bank API ──► sync_fio_to_sheets.py ──► Google Sheets (payments)
```
### Key modules
- `app.py` — Flask app; routes for `/adults`, `/juniors`, `/payments`, `/sync-bank`, `/qr`, `/flush-cache`
- `scripts/attendance.py` — Fetches attendance CSV from Google Sheets, computes per-member per-month fees. Contains fee rate constants (`ADULT_FEE_DEFAULT`, `JUNIOR_FEE_DEFAULT`) and `ADULT_MERGED_MONTHS` / `JUNIOR_MERGED_MONTHS` dicts.
- `scripts/match_payments.py``reconcile()` matches transactions to members/months. `fetch_sheet_data()` reads the payments sheet. `fetch_exceptions()` reads the `exceptions` tab.
- `scripts/cache_utils.py` — Invalidation via Google Drive API `modifiedTime`; falls back to 5-minute TTL buckets when Drive API is unavailable. Cache files live in `tmp/`.
- `scripts/sync_fio_to_sheets.py` — Pulls Fio bank transactions and appends them to the payments Google Sheet.
- `scripts/infer_payments.py` — Fills in Person/Purpose/Inferred Amount columns using name-matching heuristics.
- `scripts/config.py` — All external IDs, paths, and tunable TTLs. Override via env vars (`CREDENTIALS_PATH`, `BANK_ACCOUNT`, `CACHE_TTL_SECONDS`).
### Member tiers
Tiers are set in column B of the attendance sheet:
- `A` — Adult, pays fees (750 CZK/month for 2+ sessions, 200 CZK for exactly 1)
- `J` — Junior attending adult practices; their attendance is merged with the junior sheet
- `X` — Excluded from junior fee calculation (coaches, etc.)
### Fee calculation
- Adults: 0 sessions → 0, 1 session → 200 CZK, 2+ sessions → monthly rate (default 750 CZK)
- Juniors: 0 → 0, 1 → `"?"` (manual review required), 2+ → monthly rate (default 500 CZK)
- Per-member per-month overrides live in the `exceptions` tab of the payments sheet (columns: Name, Period YYYY-MM, Amount, Note). Exceptions are keyed by `(normalize(name), normalize(period))`.
### Merged months
`ADULT_MERGED_MONTHS` / `JUNIOR_MERGED_MONTHS` in `attendance.py` map a source month to a target month (e.g., `"2025-12": "2026-01"` merges December into January billing). The target month accumulates attendance from both months.
### Caching
`get_cached_data()` in `app.py` checks the Drive API `modifiedTime` before each request and serves a JSON file from `tmp/` when the sheet hasn't changed. Cache is warmed up at startup (`warmup_cache()`). Flush via `/flush-cache` (POST) or `flush_cache()`.
### Payments sheet columns
`Date | Amount | manual fix | Person | Purpose | Inferred Amount | Sender | VS | Message | Bank ID | Sync ID`
`Person` and `Purpose` are written by `infer_payments.py` and can be manually corrected. `manual fix` column presence disables re-inference for that row. Multiple people or months are comma-separated in Person/Purpose.
### QR codes
`/qr?account=…&amount=…&message=…` generates a Czech QR Platba PNG (SPD format).
## Git Commits ## Git Commits
When making git commits, always append yourself as co-author trailer to the end of the commit message to indicate AI assistance When making git commits, always append yourself as co-author trailer to the end of the commit message to indicate AI assistance
## Changelog
Maintain a running changelog in `CHANGELOG.md` at the repo root. After every significant change, fix, or update — once the user confirms it works — append a new entry **at the top** of the file in this format:
```markdown
## YYYY-MM-DD HH:MM TZ — short title
- One-line summary of what changed and why.
- Key files touched (optional, only if useful for traceability).
```
Get the timestamp with `date "+%Y-%m-%d %H:%M %Z"`. Skip trivial edits (typos, formatting, comment tweaks); only log changes a future reader would care about.

View File

@@ -385,8 +385,7 @@ def reconcile(
}) })
continue continue
num_allocations = len(matched_members) * len(matched_months) member_share = amount / len(matched_members) if matched_members else 0
per_allocation = amount / num_allocations if num_allocations > 0 else 0
for member_name, confidence in matched_members: for member_name, confidence in matched_members:
if member_name not in ledger: if member_name not in ledger:
@@ -397,20 +396,64 @@ def reconcile(
unmatched.append(tx) unmatched.append(tx)
continue continue
for month_key in matched_months: in_window = [(m, ledger[member_name][m]["expected"]) for m in matched_months if m in ledger[member_name]]
entry = { out_of_window = [m for m in matched_months if m not in ledger[member_name]]
"amount": per_allocation,
# Out-of-window months (outside display range): even split → credit, same as before.
n_total = len(matched_months)
if out_of_window and n_total > 0:
out_credit = member_share / n_total * len(out_of_window)
credits[member_name] = credits.get(member_name, 0) + int(out_credit)
else:
out_credit = 0.0
in_window_share = member_share - out_credit
if not in_window:
continue
total_expected = sum(e for _, e in in_window)
if total_expected > 0 and in_window_share >= total_expected:
# Greedy phase: payment covers all in-window fees; overflow → credit.
credits[member_name] = credits.get(member_name, 0) + int(in_window_share - total_expected)
for m, exp in in_window:
alloc = float(exp)
ledger[member_name][m]["paid"] += alloc
ledger[member_name][m]["transactions"].append({
"amount": alloc,
"date": tx["date"], "date": tx["date"],
"sender": tx["sender"], "sender": tx["sender"],
"message": tx["message"], "message": tx["message"],
"confidence": confidence, "confidence": confidence,
} })
if month_key in ledger[member_name]: elif total_expected > 0:
ledger[member_name][month_key]["paid"] += per_allocation # Proportional phase: distribute in_window_share by each month's expected fee.
ledger[member_name][month_key]["transactions"].append(entry) # Last month absorbs any float remainder so the sum equals in_window_share exactly.
remaining = in_window_share
for i, (m, exp) in enumerate(in_window):
alloc = remaining if i == len(in_window) - 1 else in_window_share * exp / total_expected
remaining -= alloc
ledger[member_name][m]["paid"] += alloc
ledger[member_name][m]["transactions"].append({
"amount": alloc,
"date": tx["date"],
"sender": tx["sender"],
"message": tx["message"],
"confidence": confidence,
})
else: else:
# Future month — track as credit # Fallback: no expected fees (prepayment before attendance recorded); even split.
credits[member_name] = credits.get(member_name, 0) + int(per_allocation) per_month = in_window_share / len(in_window)
for m, _ in in_window:
ledger[member_name][m]["paid"] += per_month
ledger[member_name][m]["transactions"].append({
"amount": per_month,
"date": tx["date"],
"sender": tx["sender"],
"message": tx["message"],
"confidence": confidence,
})
# Calculate final total balances (window + off-window credits) # Calculate final total balances (window + off-window credits)
final_balances: dict[str, int] = {} final_balances: dict[str, int] = {}

View File

@@ -52,5 +52,108 @@ class TestReconcileWithExceptions(unittest.TestCase):
alice_data = result['members']['Alice'] alice_data = result['members']['Alice']
self.assertEqual(alice_data['months']['2026-01']['expected'], 750, "Should fallback to attendance fee") self.assertEqual(alice_data['months']['2026-01']['expected'], 750, "Should fallback to attendance fee")
def _tx(person, purpose, amount):
return {
'date': '2026-01-01',
'amount': amount,
'person': person,
'purpose': purpose,
'inferred_amount': amount,
'sender': 'Sender',
'message': 'fee',
}
class TestMultiMonthAllocation(unittest.TestCase):
"""Fee-aware allocation across multiple months in a single payment."""
def test_greedy_exact_match(self):
"""Payment equals total expected → every month fully covered (green)."""
members = [('Alice', 'A', {'2026-02': (750, 3), '2026-03': (350, 3), '2026-04': (150, 2)})]
sorted_months = ['2026-02', '2026-03', '2026-04']
tx = _tx('Alice', '2026-02, 2026-03, 2026-04', 1250)
result = reconcile(members, sorted_months, [tx])
months = result['members']['Alice']['months']
self.assertEqual(int(months['2026-02']['paid']), 750)
self.assertEqual(int(months['2026-03']['paid']), 350)
self.assertEqual(int(months['2026-04']['paid']), 150)
def test_greedy_overpayment_goes_to_credit(self):
"""Payment exceeds total expected → each month fully covered, surplus → credit."""
members = [('Alice', 'A', {'2026-01': (750, 3), '2026-02': (750, 3)})]
sorted_months = ['2026-01', '2026-02']
tx = _tx('Alice', '2026-01, 2026-02', 2000)
result = reconcile(members, sorted_months, [tx])
months = result['members']['Alice']['months']
self.assertEqual(int(months['2026-01']['paid']), 750)
self.assertEqual(int(months['2026-02']['paid']), 750)
self.assertEqual(result['credits'].get('Alice', 0), 500)
def test_proportional_underpayment(self):
"""Payment < total expected → proportional split; sum of paid == payment amount."""
members = [('Alice', 'A', {'2026-02': (750, 3), '2026-03': (350, 3), '2026-04': (750, 3)})]
sorted_months = ['2026-02', '2026-03', '2026-04']
amount = 1250
tx = _tx('Alice', '2026-02, 2026-03, 2026-04', amount)
result = reconcile(members, sorted_months, [tx])
months = result['members']['Alice']['months']
paid_02 = months['2026-02']['paid']
paid_03 = months['2026-03']['paid']
paid_04 = months['2026-04']['paid']
# All months should be partial (underpaid)
self.assertLess(paid_02, 750)
self.assertLess(paid_03, 350)
self.assertLess(paid_04, 750)
# Sum must equal the original payment (no CZK lost)
self.assertAlmostEqual(paid_02 + paid_03 + paid_04, amount, places=2)
# 02 and 04 have equal expected → equal allocation
self.assertAlmostEqual(paid_02, paid_04, places=2)
def test_single_month_unchanged(self):
"""Single-month payment: full amount goes to that month (regression guard)."""
members = [('Alice', 'A', {'2026-01': (750, 3)})]
sorted_months = ['2026-01']
tx = _tx('Alice', '2026-01', 750)
result = reconcile(members, sorted_months, [tx])
self.assertAlmostEqual(result['members']['Alice']['months']['2026-01']['paid'], 750, places=2)
def test_two_members_multi_month(self):
"""Two members, 2 months: each member gets member_share, then fee-aware per month."""
members = [
('Alice', 'A', {'2026-01': (750, 3), '2026-02': (350, 3)}),
('Bob', 'A', {'2026-01': (750, 3), '2026-02': (350, 3)}),
]
sorted_months = ['2026-01', '2026-02']
# Both members pay together; total expected per member = 1100
tx = _tx('Alice, Bob', '2026-01, 2026-02', 2200)
result = reconcile(members, sorted_months, [tx])
for name in ('Alice', 'Bob'):
months = result['members'][name]['months']
self.assertAlmostEqual(months['2026-01']['paid'], 750, places=2)
self.assertAlmostEqual(months['2026-02']['paid'], 350, places=2)
def test_fallback_even_split_when_no_expected(self):
"""All matched months have expected=0 → falls back to even split."""
members = [('Alice', 'A', {'2026-01': (0, 0), '2026-02': (0, 0)})]
sorted_months = ['2026-01', '2026-02']
tx = _tx('Alice', '2026-01, 2026-02', 300)
result = reconcile(members, sorted_months, [tx])
months = result['members']['Alice']['months']
self.assertAlmostEqual(months['2026-01']['paid'], 150, places=2)
self.assertAlmostEqual(months['2026-02']['paid'], 150, places=2)
if __name__ == '__main__': if __name__ == '__main__':
unittest.main() unittest.main()