Compare commits

...

2 Commits
0.28 ... 0.30

Author SHA1 Message Date
5a41cdae83 fix: Balance now sums past-month (paid - expected) directly, ignoring current/future months
All checks were successful
Deploy to K8s / deploy (push) Successful in 11s
Build and Push / build (push) Successful in 6s
The previous calculation derived balance from total_balance (which includes
current/future-month activity and out-of-window credits) plus a one-sided
debt-only adjustment. Current-month surplus leaked through, making the balance
appear less negative than actual past-month debt (e.g. Mauric Daniel -1250 vs
correct -1750). Pay-All is now max(0, -balance) so the two values share a
single source and cannot disagree.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-03 20:57:13 +02:00
dfdf2aacb8 fix: Distribute multi-month payments by per-month expected fee
All checks were successful
Build and Push / build (push) Successful in 33s
Deploy to K8s / deploy (push) Successful in 12s
reconcile() previously split a multi-month payment evenly across months,
which falsely flagged months as underpaid when their expected fees
differed (e.g. 1250 CZK for 02+03+04 2026 with rates 750/350/150 was
shown as 416/month with two months red).

The allocation now runs per matched member: greedy when the share covers
the total expected (each month gets its expected fee, surplus -> credit),
proportional by expected fee otherwise. Out-of-window months keep the
previous even-split-to-credit behavior. 6 new test cases.

Also adds CHANGELOG.md and a changelog convention in CLAUDE.md.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-03 19:38:10 +02:00
5 changed files with 295 additions and 62 deletions

15
CHANGELOG.md Normal file
View File

@@ -0,0 +1,15 @@
# Changelog
## 2026-05-03 20:37 CEST — Fix Balance column to correctly reflect past-month debt
- Balance (and Pay-All) are now computed as `sum(paid expected)` over past months only, iterating directly over the ledger entries from `reconcile()`.
- Previously the balance used `total_balance` (which includes current/future-month activity and out-of-window credits) plus a one-sided current-month debt adjustment. Current-month *surplus* leaked through, making the balance appear less negative than the actual past-month debt.
- Pay-All is now `max(0, balance)` so the two values are derived from a single source and can never disagree.
- Affected: `adults_view()` and `juniors_view()` in `app.py`.
## 2026-05-03 19:26 CEST — Fee-aware allocation for multi-month payments
- `reconcile()` no longer splits a multi-month payment evenly. Allocation is now per-member with two phases: greedy (if amount ≥ total expected, each month gets exactly its expected fee and overflow → credit) and proportional (otherwise distribute by each month's expected). Fixes the case where e.g. 1250 CZK covering 3 months with mixed fees (750/350/150) marked two months red.
- Out-of-window months keep the previous even-split-to-credit behavior. Fallback to even split when all matched months have `expected = 0` (prepayment before attendance is recorded).
- Display layer only — no changes to how payments are stored in Google Sheets; `Inferred Amount` still holds the full bank amount.
- Files: [scripts/match_payments.py](scripts/match_payments.py), [tests/test_reconcile_exceptions.py](tests/test_reconcile_exceptions.py) (6 new test cases).

View File

@@ -16,22 +16,95 @@ Flask-based financial management system for FUJ (Frisbee Ultimate Jablonec). Han
This project uses `uv` for dependency management.
```bash
uv venv # Create virtual environment
uv sync # Install dependencies from pyproject.toml
uv venv && uv sync
source .venv/bin/activate
```
Alternatively, use the Makefile:
- `make sync` - Sync bank transactions to Google Sheets
- `make infer` - Automatically infer Person/Purpose/Amount in the sheet
- `make reconcile` - Generate balance report from Google Sheets data
- `make fees` - Calculate expected fees from attendance
- `make match` - (Legacy) Match bank data directly
- `make web` - Start dashboard
- `make image` - Build Docker image
Set `PYTHONPATH=scripts:.` when running scripts directly (the Makefile does this automatically).
Requires `.secret/fuj-management-bot-credentials.json` for Google Sheets API access (configurable via `CREDENTIALS_PATH` env var).
## Commands
```bash
make web # Start dashboard at http://localhost:5001
make web-debug # Same with FLASK_DEBUG=1
make test # Run all tests (unittest discover)
make test-v # Tests with verbose output
make fees # Print fee report from attendance sheet
make sync-2026 # Sync Fio bank transactions for 2026 to Google Sheets
make infer # Auto-fill Person/Purpose/Amount columns in payments sheet
make reconcile # Print balance report from Google Sheets data
make image # Build Docker image
```
Run a single test:
```bash
PYTHONPATH=scripts:. python -m unittest tests.test_app.TestWebApp.test_adults_route
```
## Architecture
### Data flow
```
Google Sheets (attendance) ──► attendance.py ──► reconcile() ──► Flask routes ──► templates/
Google Sheets (payments) ──► match_payments.py ──┘
Fio Bank API ──► sync_fio_to_sheets.py ──► Google Sheets (payments)
```
### Key modules
- `app.py` — Flask app; routes for `/adults`, `/juniors`, `/payments`, `/sync-bank`, `/qr`, `/flush-cache`
- `scripts/attendance.py` — Fetches attendance CSV from Google Sheets, computes per-member per-month fees. Contains fee rate constants (`ADULT_FEE_DEFAULT`, `JUNIOR_FEE_DEFAULT`) and `ADULT_MERGED_MONTHS` / `JUNIOR_MERGED_MONTHS` dicts.
- `scripts/match_payments.py``reconcile()` matches transactions to members/months. `fetch_sheet_data()` reads the payments sheet. `fetch_exceptions()` reads the `exceptions` tab.
- `scripts/cache_utils.py` — Invalidation via Google Drive API `modifiedTime`; falls back to 5-minute TTL buckets when Drive API is unavailable. Cache files live in `tmp/`.
- `scripts/sync_fio_to_sheets.py` — Pulls Fio bank transactions and appends them to the payments Google Sheet.
- `scripts/infer_payments.py` — Fills in Person/Purpose/Inferred Amount columns using name-matching heuristics.
- `scripts/config.py` — All external IDs, paths, and tunable TTLs. Override via env vars (`CREDENTIALS_PATH`, `BANK_ACCOUNT`, `CACHE_TTL_SECONDS`).
### Member tiers
Tiers are set in column B of the attendance sheet:
- `A` — Adult, pays fees (750 CZK/month for 2+ sessions, 200 CZK for exactly 1)
- `J` — Junior attending adult practices; their attendance is merged with the junior sheet
- `X` — Excluded from junior fee calculation (coaches, etc.)
### Fee calculation
- Adults: 0 sessions → 0, 1 session → 200 CZK, 2+ sessions → monthly rate (default 750 CZK)
- Juniors: 0 → 0, 1 → `"?"` (manual review required), 2+ → monthly rate (default 500 CZK)
- Per-member per-month overrides live in the `exceptions` tab of the payments sheet (columns: Name, Period YYYY-MM, Amount, Note). Exceptions are keyed by `(normalize(name), normalize(period))`.
### Merged months
`ADULT_MERGED_MONTHS` / `JUNIOR_MERGED_MONTHS` in `attendance.py` map a source month to a target month (e.g., `"2025-12": "2026-01"` merges December into January billing). The target month accumulates attendance from both months.
### Caching
`get_cached_data()` in `app.py` checks the Drive API `modifiedTime` before each request and serves a JSON file from `tmp/` when the sheet hasn't changed. Cache is warmed up at startup (`warmup_cache()`). Flush via `/flush-cache` (POST) or `flush_cache()`.
### Payments sheet columns
`Date | Amount | manual fix | Person | Purpose | Inferred Amount | Sender | VS | Message | Bank ID | Sync ID`
`Person` and `Purpose` are written by `infer_payments.py` and can be manually corrected. `manual fix` column presence disables re-inference for that row. Multiple people or months are comma-separated in Person/Purpose.
### QR codes
`/qr?account=…&amount=…&message=…` generates a Czech QR Platba PNG (SPD format).
## Git Commits
When making git commits, always append yourself as co-author trailer to the end of the commit message to indicate AI assistance
## Changelog
Maintain a running changelog in `CHANGELOG.md` at the repo root. After every significant change, fix, or update — once the user confirms it works — append a new entry **at the top** of the file in this format:
```markdown
## YYYY-MM-DD HH:MM TZ — short title
- One-line summary of what changed and why.
- Key files touched (optional, only if useful for traceability).
```
Get the timestamp with `date "+%Y-%m-%d %H:%M %Z"`. Skip trivial edits (typos, formatting, comment tweaks); only log changes a future reader would care about.

67
app.py
View File

@@ -192,7 +192,6 @@ def adults_view():
row = {"name": name, "months": [], "balance": data["total_balance"], "unpaid_periods": "", "raw_unpaid_periods": ""}
unpaid_months = []
raw_unpaid_months = []
payable_amount = 0
for m in sorted_months:
mdata = data["months"].get(m, {"expected": 0, "original_expected": 0, "attendance_count": 0, "paid": 0, "exception": None})
expected = mdata.get("expected", 0)
@@ -228,14 +227,12 @@ def adults_view():
if m < current_month:
unpaid_months.append(month_labels[m])
raw_unpaid_months.append(datetime.strptime(m, "%Y-%m").strftime("%m/%Y"))
payable_amount += amount_to_pay
else:
status = "unpaid"
cell_text = f"0/{fee_display}"
if m < current_month:
unpaid_months.append(month_labels[m])
raw_unpaid_months.append(datetime.strptime(m, "%Y-%m").strftime("%m/%Y"))
payable_amount += amount_to_pay
elif paid > 0:
status = "surplus"
cell_text = f"PAID {paid}"
@@ -258,17 +255,17 @@ def adults_view():
"tooltip": tooltip
})
# Compute balance excluding current/future months
current_month_debt = 0
for m in sorted_months:
# Balance = sum of (paid - expected) for past months only; current/future months ignored.
settled_balance = 0
for m, mdata in data["months"].items():
if m >= current_month:
mdata = data["months"].get(m, {"expected": 0, "paid": 0})
exp = mdata.get("expected", 0)
pd = int(mdata.get("paid", 0))
current_month_debt += max(0, exp - pd)
settled_balance = data["total_balance"] + current_month_debt
continue
exp = mdata.get("expected", 0)
if isinstance(exp, int):
settled_balance += int(mdata.get("paid", 0)) - exp
row["unpaid_periods"] = ", ".join(unpaid_months) if unpaid_months else ("Older debt" if settled_balance < 0 and payable_amount == 0 else "")
payable_amount = max(0, -settled_balance)
row["unpaid_periods"] = ", ".join(unpaid_months)
row["raw_unpaid_periods"] = "+".join(raw_unpaid_months)
row["balance"] = settled_balance
row["payable_amount"] = payable_amount
@@ -293,8 +290,14 @@ def adults_view():
def settled_balance(name):
data = result["members"][name]
debt = sum(max(0, data["months"].get(m, {"expected": 0, "paid": 0}).get("expected", 0) - int(data["months"].get(m, {"expected": 0, "paid": 0}).get("paid", 0))) for m in sorted_months if m >= current_month)
return data["total_balance"] + debt
total = 0
for m, mdata in data["months"].items():
if m >= current_month:
continue
exp = mdata.get("expected", 0)
if isinstance(exp, int):
total += int(mdata.get("paid", 0)) - exp
return total
credits = sorted([{"name": n, "amount": settled_balance(n)} for n in adult_names if settled_balance(n) > 0], key=lambda x: x["name"])
debts = sorted([{"name": n, "amount": abs(settled_balance(n))} for n in adult_names if settled_balance(n) < 0], key=lambda x: x["name"])
@@ -373,7 +376,6 @@ def juniors_view():
row = {"name": name, "months": [], "balance": data["total_balance"], "unpaid_periods": "", "raw_unpaid_periods": ""}
unpaid_months = []
raw_unpaid_months = []
payable_amount = 0
for m in sorted_months:
mdata = data["months"].get(m, {"expected": 0, "original_expected": 0, "attendance_count": 0, "paid": 0, "exception": None})
expected = mdata.get("expected", 0)
@@ -429,7 +431,6 @@ def juniors_view():
if m < current_month:
unpaid_months.append(month_labels[m])
raw_unpaid_months.append(datetime.strptime(m, "%Y-%m").strftime("%m/%Y"))
payable_amount += amount_to_pay
else:
status = "unpaid"
cell_text = f"0/{fee_display}"
@@ -437,7 +438,6 @@ def juniors_view():
if m < current_month:
unpaid_months.append(month_labels[m])
raw_unpaid_months.append(datetime.strptime(m, "%Y-%m").strftime("%m/%Y"))
payable_amount += amount_to_pay
elif paid > 0:
status = "surplus"
cell_text = f"PAID {paid}"
@@ -457,18 +457,17 @@ def juniors_view():
"tooltip": tooltip
})
# Compute balance excluding current/future months
current_month_debt = 0
for m in sorted_months:
# Balance = sum of (paid - expected) for past months only; current/future months ignored.
settled_balance = 0
for m, mdata in data["months"].items():
if m >= current_month:
mdata = data["months"].get(m, {"expected": 0, "paid": 0})
exp = mdata.get("expected", 0)
if isinstance(exp, int):
pd = int(mdata.get("paid", 0))
current_month_debt += max(0, exp - pd)
settled_balance = data["total_balance"] + current_month_debt
continue
exp = mdata.get("expected", 0)
if isinstance(exp, int):
settled_balance += int(mdata.get("paid", 0)) - exp
row["unpaid_periods"] = ", ".join(unpaid_months) if unpaid_months else ("Older debt" if settled_balance < 0 and payable_amount == 0 else "")
payable_amount = max(0, -settled_balance)
row["unpaid_periods"] = ", ".join(unpaid_months)
row["raw_unpaid_periods"] = "+".join(raw_unpaid_months)
row["balance"] = settled_balance
row["payable_amount"] = payable_amount
@@ -494,14 +493,14 @@ def juniors_view():
# Format credits and debts
def junior_settled_balance(name):
data = result["members"][name]
debt = 0
for m in sorted_months:
total = 0
for m, mdata in data["months"].items():
if m >= current_month:
mdata = data["months"].get(m, {"expected": 0, "paid": 0})
exp = mdata.get("expected", 0)
if isinstance(exp, int):
debt += max(0, exp - int(mdata.get("paid", 0)))
return data["total_balance"] + debt
continue
exp = mdata.get("expected", 0)
if isinstance(exp, int):
total += int(mdata.get("paid", 0)) - exp
return total
junior_all_names = [name for name, _, _ in adapted_members]
credits = sorted([{"name": n, "amount": junior_settled_balance(n)} for n in junior_all_names if junior_settled_balance(n) > 0], key=lambda x: x["name"])

View File

@@ -385,8 +385,7 @@ def reconcile(
})
continue
num_allocations = len(matched_members) * len(matched_months)
per_allocation = amount / num_allocations if num_allocations > 0 else 0
member_share = amount / len(matched_members) if matched_members else 0
for member_name, confidence in matched_members:
if member_name not in ledger:
@@ -397,20 +396,64 @@ def reconcile(
unmatched.append(tx)
continue
for month_key in matched_months:
entry = {
"amount": per_allocation,
"date": tx["date"],
"sender": tx["sender"],
"message": tx["message"],
"confidence": confidence,
}
if month_key in ledger[member_name]:
ledger[member_name][month_key]["paid"] += per_allocation
ledger[member_name][month_key]["transactions"].append(entry)
else:
# Future month — track as credit
credits[member_name] = credits.get(member_name, 0) + int(per_allocation)
in_window = [(m, ledger[member_name][m]["expected"]) for m in matched_months if m in ledger[member_name]]
out_of_window = [m for m in matched_months if m not in ledger[member_name]]
# Out-of-window months (outside display range): even split → credit, same as before.
n_total = len(matched_months)
if out_of_window and n_total > 0:
out_credit = member_share / n_total * len(out_of_window)
credits[member_name] = credits.get(member_name, 0) + int(out_credit)
else:
out_credit = 0.0
in_window_share = member_share - out_credit
if not in_window:
continue
total_expected = sum(e for _, e in in_window)
if total_expected > 0 and in_window_share >= total_expected:
# Greedy phase: payment covers all in-window fees; overflow → credit.
credits[member_name] = credits.get(member_name, 0) + int(in_window_share - total_expected)
for m, exp in in_window:
alloc = float(exp)
ledger[member_name][m]["paid"] += alloc
ledger[member_name][m]["transactions"].append({
"amount": alloc,
"date": tx["date"],
"sender": tx["sender"],
"message": tx["message"],
"confidence": confidence,
})
elif total_expected > 0:
# Proportional phase: distribute in_window_share by each month's expected fee.
# Last month absorbs any float remainder so the sum equals in_window_share exactly.
remaining = in_window_share
for i, (m, exp) in enumerate(in_window):
alloc = remaining if i == len(in_window) - 1 else in_window_share * exp / total_expected
remaining -= alloc
ledger[member_name][m]["paid"] += alloc
ledger[member_name][m]["transactions"].append({
"amount": alloc,
"date": tx["date"],
"sender": tx["sender"],
"message": tx["message"],
"confidence": confidence,
})
else:
# Fallback: no expected fees (prepayment before attendance recorded); even split.
per_month = in_window_share / len(in_window)
for m, _ in in_window:
ledger[member_name][m]["paid"] += per_month
ledger[member_name][m]["transactions"].append({
"amount": per_month,
"date": tx["date"],
"sender": tx["sender"],
"message": tx["message"],
"confidence": confidence,
})
# Calculate final total balances (window + off-window credits)
final_balances: dict[str, int] = {}

View File

@@ -52,5 +52,108 @@ class TestReconcileWithExceptions(unittest.TestCase):
alice_data = result['members']['Alice']
self.assertEqual(alice_data['months']['2026-01']['expected'], 750, "Should fallback to attendance fee")
def _tx(person, purpose, amount):
return {
'date': '2026-01-01',
'amount': amount,
'person': person,
'purpose': purpose,
'inferred_amount': amount,
'sender': 'Sender',
'message': 'fee',
}
class TestMultiMonthAllocation(unittest.TestCase):
"""Fee-aware allocation across multiple months in a single payment."""
def test_greedy_exact_match(self):
"""Payment equals total expected → every month fully covered (green)."""
members = [('Alice', 'A', {'2026-02': (750, 3), '2026-03': (350, 3), '2026-04': (150, 2)})]
sorted_months = ['2026-02', '2026-03', '2026-04']
tx = _tx('Alice', '2026-02, 2026-03, 2026-04', 1250)
result = reconcile(members, sorted_months, [tx])
months = result['members']['Alice']['months']
self.assertEqual(int(months['2026-02']['paid']), 750)
self.assertEqual(int(months['2026-03']['paid']), 350)
self.assertEqual(int(months['2026-04']['paid']), 150)
def test_greedy_overpayment_goes_to_credit(self):
"""Payment exceeds total expected → each month fully covered, surplus → credit."""
members = [('Alice', 'A', {'2026-01': (750, 3), '2026-02': (750, 3)})]
sorted_months = ['2026-01', '2026-02']
tx = _tx('Alice', '2026-01, 2026-02', 2000)
result = reconcile(members, sorted_months, [tx])
months = result['members']['Alice']['months']
self.assertEqual(int(months['2026-01']['paid']), 750)
self.assertEqual(int(months['2026-02']['paid']), 750)
self.assertEqual(result['credits'].get('Alice', 0), 500)
def test_proportional_underpayment(self):
"""Payment < total expected → proportional split; sum of paid == payment amount."""
members = [('Alice', 'A', {'2026-02': (750, 3), '2026-03': (350, 3), '2026-04': (750, 3)})]
sorted_months = ['2026-02', '2026-03', '2026-04']
amount = 1250
tx = _tx('Alice', '2026-02, 2026-03, 2026-04', amount)
result = reconcile(members, sorted_months, [tx])
months = result['members']['Alice']['months']
paid_02 = months['2026-02']['paid']
paid_03 = months['2026-03']['paid']
paid_04 = months['2026-04']['paid']
# All months should be partial (underpaid)
self.assertLess(paid_02, 750)
self.assertLess(paid_03, 350)
self.assertLess(paid_04, 750)
# Sum must equal the original payment (no CZK lost)
self.assertAlmostEqual(paid_02 + paid_03 + paid_04, amount, places=2)
# 02 and 04 have equal expected → equal allocation
self.assertAlmostEqual(paid_02, paid_04, places=2)
def test_single_month_unchanged(self):
"""Single-month payment: full amount goes to that month (regression guard)."""
members = [('Alice', 'A', {'2026-01': (750, 3)})]
sorted_months = ['2026-01']
tx = _tx('Alice', '2026-01', 750)
result = reconcile(members, sorted_months, [tx])
self.assertAlmostEqual(result['members']['Alice']['months']['2026-01']['paid'], 750, places=2)
def test_two_members_multi_month(self):
"""Two members, 2 months: each member gets member_share, then fee-aware per month."""
members = [
('Alice', 'A', {'2026-01': (750, 3), '2026-02': (350, 3)}),
('Bob', 'A', {'2026-01': (750, 3), '2026-02': (350, 3)}),
]
sorted_months = ['2026-01', '2026-02']
# Both members pay together; total expected per member = 1100
tx = _tx('Alice, Bob', '2026-01, 2026-02', 2200)
result = reconcile(members, sorted_months, [tx])
for name in ('Alice', 'Bob'):
months = result['members'][name]['months']
self.assertAlmostEqual(months['2026-01']['paid'], 750, places=2)
self.assertAlmostEqual(months['2026-02']['paid'], 350, places=2)
def test_fallback_even_split_when_no_expected(self):
"""All matched months have expected=0 → falls back to even split."""
members = [('Alice', 'A', {'2026-01': (0, 0), '2026-02': (0, 0)})]
sorted_months = ['2026-01', '2026-02']
tx = _tx('Alice', '2026-01, 2026-02', 300)
result = reconcile(members, sorted_months, [tx])
months = result['members']['Alice']['months']
self.assertAlmostEqual(months['2026-01']['paid'], 150, places=2)
self.assertAlmostEqual(months['2026-02']['paid'], 150, places=2)
if __name__ == '__main__':
unittest.main()