fix: Distribute multi-month payments by per-month expected fee
reconcile() previously split a multi-month payment evenly across months, which falsely flagged months as underpaid when their expected fees differed (e.g. 1250 CZK for 02+03+04 2026 with rates 750/350/150 was shown as 416/month with two months red). The allocation now runs per matched member: greedy when the share covers the total expected (each month gets its expected fee, surplus -> credit), proportional by expected fee otherwise. Out-of-window months keep the previous even-split-to-credit behavior. 6 new test cases. Also adds CHANGELOG.md and a changelog convention in CLAUDE.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
8
CHANGELOG.md
Normal file
8
CHANGELOG.md
Normal file
@@ -0,0 +1,8 @@
|
||||
# Changelog
|
||||
|
||||
## 2026-05-03 19:26 CEST — Fee-aware allocation for multi-month payments
|
||||
|
||||
- `reconcile()` no longer splits a multi-month payment evenly. Allocation is now per-member with two phases: greedy (if amount ≥ total expected, each month gets exactly its expected fee and overflow → credit) and proportional (otherwise distribute by each month's expected). Fixes the case where e.g. 1250 CZK covering 3 months with mixed fees (750/350/150) marked two months red.
|
||||
- Out-of-window months keep the previous even-split-to-credit behavior. Fallback to even split when all matched months have `expected = 0` (prepayment before attendance is recorded).
|
||||
- Display layer only — no changes to how payments are stored in Google Sheets; `Inferred Amount` still holds the full bank amount.
|
||||
- Files: [scripts/match_payments.py](scripts/match_payments.py), [tests/test_reconcile_exceptions.py](tests/test_reconcile_exceptions.py) (6 new test cases).
|
||||
97
CLAUDE.md
97
CLAUDE.md
@@ -16,22 +16,95 @@ Flask-based financial management system for FUJ (Frisbee Ultimate Jablonec). Han
|
||||
This project uses `uv` for dependency management.
|
||||
|
||||
```bash
|
||||
uv venv # Create virtual environment
|
||||
uv sync # Install dependencies from pyproject.toml
|
||||
uv venv && uv sync
|
||||
source .venv/bin/activate
|
||||
```
|
||||
|
||||
Alternatively, use the Makefile:
|
||||
- `make sync` - Sync bank transactions to Google Sheets
|
||||
- `make infer` - Automatically infer Person/Purpose/Amount in the sheet
|
||||
- `make reconcile` - Generate balance report from Google Sheets data
|
||||
- `make fees` - Calculate expected fees from attendance
|
||||
- `make match` - (Legacy) Match bank data directly
|
||||
- `make web` - Start dashboard
|
||||
- `make image` - Build Docker image
|
||||
Set `PYTHONPATH=scripts:.` when running scripts directly (the Makefile does this automatically).
|
||||
|
||||
Requires `.secret/fuj-management-bot-credentials.json` for Google Sheets API access (configurable via `CREDENTIALS_PATH` env var).
|
||||
## Commands
|
||||
|
||||
```bash
|
||||
make web # Start dashboard at http://localhost:5001
|
||||
make web-debug # Same with FLASK_DEBUG=1
|
||||
make test # Run all tests (unittest discover)
|
||||
make test-v # Tests with verbose output
|
||||
make fees # Print fee report from attendance sheet
|
||||
make sync-2026 # Sync Fio bank transactions for 2026 to Google Sheets
|
||||
make infer # Auto-fill Person/Purpose/Amount columns in payments sheet
|
||||
make reconcile # Print balance report from Google Sheets data
|
||||
make image # Build Docker image
|
||||
```
|
||||
|
||||
Run a single test:
|
||||
```bash
|
||||
PYTHONPATH=scripts:. python -m unittest tests.test_app.TestWebApp.test_adults_route
|
||||
```
|
||||
|
||||
## Architecture
|
||||
|
||||
### Data flow
|
||||
|
||||
```
|
||||
Google Sheets (attendance) ──► attendance.py ──► reconcile() ──► Flask routes ──► templates/
|
||||
Google Sheets (payments) ──► match_payments.py ──┘
|
||||
Fio Bank API ──► sync_fio_to_sheets.py ──► Google Sheets (payments)
|
||||
```
|
||||
|
||||
### Key modules
|
||||
|
||||
- `app.py` — Flask app; routes for `/adults`, `/juniors`, `/payments`, `/sync-bank`, `/qr`, `/flush-cache`
|
||||
- `scripts/attendance.py` — Fetches attendance CSV from Google Sheets, computes per-member per-month fees. Contains fee rate constants (`ADULT_FEE_DEFAULT`, `JUNIOR_FEE_DEFAULT`) and `ADULT_MERGED_MONTHS` / `JUNIOR_MERGED_MONTHS` dicts.
|
||||
- `scripts/match_payments.py` — `reconcile()` matches transactions to members/months. `fetch_sheet_data()` reads the payments sheet. `fetch_exceptions()` reads the `exceptions` tab.
|
||||
- `scripts/cache_utils.py` — Invalidation via Google Drive API `modifiedTime`; falls back to 5-minute TTL buckets when Drive API is unavailable. Cache files live in `tmp/`.
|
||||
- `scripts/sync_fio_to_sheets.py` — Pulls Fio bank transactions and appends them to the payments Google Sheet.
|
||||
- `scripts/infer_payments.py` — Fills in Person/Purpose/Inferred Amount columns using name-matching heuristics.
|
||||
- `scripts/config.py` — All external IDs, paths, and tunable TTLs. Override via env vars (`CREDENTIALS_PATH`, `BANK_ACCOUNT`, `CACHE_TTL_SECONDS`).
|
||||
|
||||
### Member tiers
|
||||
|
||||
Tiers are set in column B of the attendance sheet:
|
||||
- `A` — Adult, pays fees (750 CZK/month for 2+ sessions, 200 CZK for exactly 1)
|
||||
- `J` — Junior attending adult practices; their attendance is merged with the junior sheet
|
||||
- `X` — Excluded from junior fee calculation (coaches, etc.)
|
||||
|
||||
### Fee calculation
|
||||
|
||||
- Adults: 0 sessions → 0, 1 session → 200 CZK, 2+ sessions → monthly rate (default 750 CZK)
|
||||
- Juniors: 0 → 0, 1 → `"?"` (manual review required), 2+ → monthly rate (default 500 CZK)
|
||||
- Per-member per-month overrides live in the `exceptions` tab of the payments sheet (columns: Name, Period YYYY-MM, Amount, Note). Exceptions are keyed by `(normalize(name), normalize(period))`.
|
||||
|
||||
### Merged months
|
||||
|
||||
`ADULT_MERGED_MONTHS` / `JUNIOR_MERGED_MONTHS` in `attendance.py` map a source month to a target month (e.g., `"2025-12": "2026-01"` merges December into January billing). The target month accumulates attendance from both months.
|
||||
|
||||
### Caching
|
||||
|
||||
`get_cached_data()` in `app.py` checks the Drive API `modifiedTime` before each request and serves a JSON file from `tmp/` when the sheet hasn't changed. Cache is warmed up at startup (`warmup_cache()`). Flush via `/flush-cache` (POST) or `flush_cache()`.
|
||||
|
||||
### Payments sheet columns
|
||||
|
||||
`Date | Amount | manual fix | Person | Purpose | Inferred Amount | Sender | VS | Message | Bank ID | Sync ID`
|
||||
|
||||
`Person` and `Purpose` are written by `infer_payments.py` and can be manually corrected. `manual fix` column presence disables re-inference for that row. Multiple people or months are comma-separated in Person/Purpose.
|
||||
|
||||
### QR codes
|
||||
|
||||
`/qr?account=…&amount=…&message=…` generates a Czech QR Platba PNG (SPD format).
|
||||
|
||||
## Git Commits
|
||||
|
||||
When making git commits, always append yourself as co-author trailer to the end of the commit message to indicate AI assistance
|
||||
When making git commits, always append yourself as co-author trailer to the end of the commit message to indicate AI assistance
|
||||
|
||||
## Changelog
|
||||
|
||||
Maintain a running changelog in `CHANGELOG.md` at the repo root. After every significant change, fix, or update — once the user confirms it works — append a new entry **at the top** of the file in this format:
|
||||
|
||||
```markdown
|
||||
## YYYY-MM-DD HH:MM TZ — short title
|
||||
|
||||
- One-line summary of what changed and why.
|
||||
- Key files touched (optional, only if useful for traceability).
|
||||
```
|
||||
|
||||
Get the timestamp with `date "+%Y-%m-%d %H:%M %Z"`. Skip trivial edits (typos, formatting, comment tweaks); only log changes a future reader would care about.
|
||||
|
||||
@@ -385,8 +385,7 @@ def reconcile(
|
||||
})
|
||||
continue
|
||||
|
||||
num_allocations = len(matched_members) * len(matched_months)
|
||||
per_allocation = amount / num_allocations if num_allocations > 0 else 0
|
||||
member_share = amount / len(matched_members) if matched_members else 0
|
||||
|
||||
for member_name, confidence in matched_members:
|
||||
if member_name not in ledger:
|
||||
@@ -397,20 +396,64 @@ def reconcile(
|
||||
unmatched.append(tx)
|
||||
continue
|
||||
|
||||
for month_key in matched_months:
|
||||
entry = {
|
||||
"amount": per_allocation,
|
||||
"date": tx["date"],
|
||||
"sender": tx["sender"],
|
||||
"message": tx["message"],
|
||||
"confidence": confidence,
|
||||
}
|
||||
if month_key in ledger[member_name]:
|
||||
ledger[member_name][month_key]["paid"] += per_allocation
|
||||
ledger[member_name][month_key]["transactions"].append(entry)
|
||||
else:
|
||||
# Future month — track as credit
|
||||
credits[member_name] = credits.get(member_name, 0) + int(per_allocation)
|
||||
in_window = [(m, ledger[member_name][m]["expected"]) for m in matched_months if m in ledger[member_name]]
|
||||
out_of_window = [m for m in matched_months if m not in ledger[member_name]]
|
||||
|
||||
# Out-of-window months (outside display range): even split → credit, same as before.
|
||||
n_total = len(matched_months)
|
||||
if out_of_window and n_total > 0:
|
||||
out_credit = member_share / n_total * len(out_of_window)
|
||||
credits[member_name] = credits.get(member_name, 0) + int(out_credit)
|
||||
else:
|
||||
out_credit = 0.0
|
||||
|
||||
in_window_share = member_share - out_credit
|
||||
|
||||
if not in_window:
|
||||
continue
|
||||
|
||||
total_expected = sum(e for _, e in in_window)
|
||||
|
||||
if total_expected > 0 and in_window_share >= total_expected:
|
||||
# Greedy phase: payment covers all in-window fees; overflow → credit.
|
||||
credits[member_name] = credits.get(member_name, 0) + int(in_window_share - total_expected)
|
||||
for m, exp in in_window:
|
||||
alloc = float(exp)
|
||||
ledger[member_name][m]["paid"] += alloc
|
||||
ledger[member_name][m]["transactions"].append({
|
||||
"amount": alloc,
|
||||
"date": tx["date"],
|
||||
"sender": tx["sender"],
|
||||
"message": tx["message"],
|
||||
"confidence": confidence,
|
||||
})
|
||||
elif total_expected > 0:
|
||||
# Proportional phase: distribute in_window_share by each month's expected fee.
|
||||
# Last month absorbs any float remainder so the sum equals in_window_share exactly.
|
||||
remaining = in_window_share
|
||||
for i, (m, exp) in enumerate(in_window):
|
||||
alloc = remaining if i == len(in_window) - 1 else in_window_share * exp / total_expected
|
||||
remaining -= alloc
|
||||
ledger[member_name][m]["paid"] += alloc
|
||||
ledger[member_name][m]["transactions"].append({
|
||||
"amount": alloc,
|
||||
"date": tx["date"],
|
||||
"sender": tx["sender"],
|
||||
"message": tx["message"],
|
||||
"confidence": confidence,
|
||||
})
|
||||
else:
|
||||
# Fallback: no expected fees (prepayment before attendance recorded); even split.
|
||||
per_month = in_window_share / len(in_window)
|
||||
for m, _ in in_window:
|
||||
ledger[member_name][m]["paid"] += per_month
|
||||
ledger[member_name][m]["transactions"].append({
|
||||
"amount": per_month,
|
||||
"date": tx["date"],
|
||||
"sender": tx["sender"],
|
||||
"message": tx["message"],
|
||||
"confidence": confidence,
|
||||
})
|
||||
|
||||
# Calculate final total balances (window + off-window credits)
|
||||
final_balances: dict[str, int] = {}
|
||||
|
||||
@@ -52,5 +52,108 @@ class TestReconcileWithExceptions(unittest.TestCase):
|
||||
alice_data = result['members']['Alice']
|
||||
self.assertEqual(alice_data['months']['2026-01']['expected'], 750, "Should fallback to attendance fee")
|
||||
|
||||
def _tx(person, purpose, amount):
|
||||
return {
|
||||
'date': '2026-01-01',
|
||||
'amount': amount,
|
||||
'person': person,
|
||||
'purpose': purpose,
|
||||
'inferred_amount': amount,
|
||||
'sender': 'Sender',
|
||||
'message': 'fee',
|
||||
}
|
||||
|
||||
|
||||
class TestMultiMonthAllocation(unittest.TestCase):
|
||||
"""Fee-aware allocation across multiple months in a single payment."""
|
||||
|
||||
def test_greedy_exact_match(self):
|
||||
"""Payment equals total expected → every month fully covered (green)."""
|
||||
members = [('Alice', 'A', {'2026-02': (750, 3), '2026-03': (350, 3), '2026-04': (150, 2)})]
|
||||
sorted_months = ['2026-02', '2026-03', '2026-04']
|
||||
tx = _tx('Alice', '2026-02, 2026-03, 2026-04', 1250)
|
||||
|
||||
result = reconcile(members, sorted_months, [tx])
|
||||
months = result['members']['Alice']['months']
|
||||
|
||||
self.assertEqual(int(months['2026-02']['paid']), 750)
|
||||
self.assertEqual(int(months['2026-03']['paid']), 350)
|
||||
self.assertEqual(int(months['2026-04']['paid']), 150)
|
||||
|
||||
def test_greedy_overpayment_goes_to_credit(self):
|
||||
"""Payment exceeds total expected → each month fully covered, surplus → credit."""
|
||||
members = [('Alice', 'A', {'2026-01': (750, 3), '2026-02': (750, 3)})]
|
||||
sorted_months = ['2026-01', '2026-02']
|
||||
tx = _tx('Alice', '2026-01, 2026-02', 2000)
|
||||
|
||||
result = reconcile(members, sorted_months, [tx])
|
||||
months = result['members']['Alice']['months']
|
||||
|
||||
self.assertEqual(int(months['2026-01']['paid']), 750)
|
||||
self.assertEqual(int(months['2026-02']['paid']), 750)
|
||||
self.assertEqual(result['credits'].get('Alice', 0), 500)
|
||||
|
||||
def test_proportional_underpayment(self):
|
||||
"""Payment < total expected → proportional split; sum of paid == payment amount."""
|
||||
members = [('Alice', 'A', {'2026-02': (750, 3), '2026-03': (350, 3), '2026-04': (750, 3)})]
|
||||
sorted_months = ['2026-02', '2026-03', '2026-04']
|
||||
amount = 1250
|
||||
tx = _tx('Alice', '2026-02, 2026-03, 2026-04', amount)
|
||||
|
||||
result = reconcile(members, sorted_months, [tx])
|
||||
months = result['members']['Alice']['months']
|
||||
|
||||
paid_02 = months['2026-02']['paid']
|
||||
paid_03 = months['2026-03']['paid']
|
||||
paid_04 = months['2026-04']['paid']
|
||||
|
||||
# All months should be partial (underpaid)
|
||||
self.assertLess(paid_02, 750)
|
||||
self.assertLess(paid_03, 350)
|
||||
self.assertLess(paid_04, 750)
|
||||
# Sum must equal the original payment (no CZK lost)
|
||||
self.assertAlmostEqual(paid_02 + paid_03 + paid_04, amount, places=2)
|
||||
# 02 and 04 have equal expected → equal allocation
|
||||
self.assertAlmostEqual(paid_02, paid_04, places=2)
|
||||
|
||||
def test_single_month_unchanged(self):
|
||||
"""Single-month payment: full amount goes to that month (regression guard)."""
|
||||
members = [('Alice', 'A', {'2026-01': (750, 3)})]
|
||||
sorted_months = ['2026-01']
|
||||
tx = _tx('Alice', '2026-01', 750)
|
||||
|
||||
result = reconcile(members, sorted_months, [tx])
|
||||
self.assertAlmostEqual(result['members']['Alice']['months']['2026-01']['paid'], 750, places=2)
|
||||
|
||||
def test_two_members_multi_month(self):
|
||||
"""Two members, 2 months: each member gets member_share, then fee-aware per month."""
|
||||
members = [
|
||||
('Alice', 'A', {'2026-01': (750, 3), '2026-02': (350, 3)}),
|
||||
('Bob', 'A', {'2026-01': (750, 3), '2026-02': (350, 3)}),
|
||||
]
|
||||
sorted_months = ['2026-01', '2026-02']
|
||||
# Both members pay together; total expected per member = 1100
|
||||
tx = _tx('Alice, Bob', '2026-01, 2026-02', 2200)
|
||||
|
||||
result = reconcile(members, sorted_months, [tx])
|
||||
|
||||
for name in ('Alice', 'Bob'):
|
||||
months = result['members'][name]['months']
|
||||
self.assertAlmostEqual(months['2026-01']['paid'], 750, places=2)
|
||||
self.assertAlmostEqual(months['2026-02']['paid'], 350, places=2)
|
||||
|
||||
def test_fallback_even_split_when_no_expected(self):
|
||||
"""All matched months have expected=0 → falls back to even split."""
|
||||
members = [('Alice', 'A', {'2026-01': (0, 0), '2026-02': (0, 0)})]
|
||||
sorted_months = ['2026-01', '2026-02']
|
||||
tx = _tx('Alice', '2026-01, 2026-02', 300)
|
||||
|
||||
result = reconcile(members, sorted_months, [tx])
|
||||
months = result['members']['Alice']['months']
|
||||
|
||||
self.assertAlmostEqual(months['2026-01']['paid'], 150, places=2)
|
||||
self.assertAlmostEqual(months['2026-02']['paid'], 150, places=2)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
|
||||
Reference in New Issue
Block a user