Merge pull request 'fix(reconcile): fill earliest month deficit first in multi-month allocations' (#35) from fix/fill-first-multi-month-allocation into main

Reviewed-on: #35

CHANGELOG.md

@@ -1,5 +1,20 @@
 # Changelog
 
+## 2026-05-11 23:58 CEST — fix(reconcile): fill earliest month deficit first in multi-month allocations
+
+- Multi-month payment allocation now fills the earliest in-window deficit first and spills
+  any remainder to later months, accounting for prior transactions' contributions to each month.
+  Previously a single transaction was split proportionally to each month's total expected fee,
+  ignoring what earlier transactions had already paid — surfaced by Matyáš Thér's 200+550 case
+  showing 566/183 instead of 500/250.
+- Files: `scripts/match_payments.py`, `go/internal/domain/reconcile/reconcile.go`, tests, parity fixtures.
+
+## 2026-05-11 22:56 CEST — fix(python): parse Fio 2-digit-year dates + add `make sync-debug` dry-run tool
+
+- Fix: `scripts/fio_utils.py` `parse_czech_date` now accepts `DD.MM.YY` / `D.M.YY` in addition to the 4-digit-year variants. Fio's transparent page now mixes both forms in the same response; the 2-digit rows were being silently dropped, which caused `make sync-2026` to miss every recent transfer. Mirrors the Go-side fix from 2026-05-07 (CHANGELOG entry below).
+- Added `--dry-run` and `--print-fio-table` flags to `scripts/sync_fio_to_sheets.py`, plus a `make sync-debug [DAYS=N]` Makefile target. Mirrors `make go-sync-debug`: fetches from Fio and dedupes against the sheet, prints `STATUS=NEW/DUP` per transaction, and prints per-row `Dry run: would append …` lines + `would sort by date` instead of touching the sheet.
+- Added always-on stderr diagnostics in `scripts/fio_utils.py`: which fetcher was selected (authenticated API vs. transparent-page scraper with `FIO_API_TOKEN`-unset warning), and raw-vs-after-filter transaction counts on both paths — so this class of "scraper drops everything" bug surfaces immediately.
+
 ## 2026-05-08 14:55 CEST — feat(go): M6.6.1 — Pay-button QR popup modal
 
 - Restored the Python `showPayQR` in-page modal UX that was lost in M6.6 (Pay buttons were navigating the tab to the raw `/qr` PNG).

Makefile

@@ -35,6 +35,7 @@ help:
 	@echo "  make sync        - Sync Fio transactions to Google Sheets"
 	@echo "  make sync-2025   - Sync Fio transactions for Q4 2025 (Oct-Dec)"
 	@echo "  make sync-2026   - Sync Fio transactions for the whole year of 2026"
+	@echo "  make sync-debug [DAYS=N] - Dry-run Python sync with Fio diagnostics and txn table (default DAYS=30)"
 	@echo "  make infer       - Infer payment details (Person, Purpose, Amount) in the sheet"
 	@echo "  make reconcile   - Show balance report using Google Sheets data"
 	@echo "  make venv        - Sync virtual environment with pyproject.toml"
@@ -125,6 +126,9 @@ sync-2025: $(PYTHON)
 sync-2026: $(PYTHON)
 	$(PYTHON) scripts/sync_fio_to_sheets.py --credentials .secret/fuj-management-bot-credentials.json --from 2026-01-01 --to 2026-12-31 --sort-by-date
 
+sync-debug: $(PYTHON)  ## Dry-run Python sync with Fio diagnostics and txn table (default DAYS=30)
+	$(PYTHON) scripts/sync_fio_to_sheets.py --credentials .secret/fuj-management-bot-credentials.json --days $(DAYS) --dry-run --print-fio-table
+
 infer: $(PYTHON)
 	$(PYTHON) scripts/infer_payments.py --credentials $(CREDENTIALS)
 

docs/plans/2026-05-11-2353-fill-first-multi-month-allocation.md

@@ -0,0 +1,184 @@
+# Fill-first multi-month payment allocation
+
+## Context
+
+Matyáš Thér paid in two transactions:
+
+| # | Amount | Purpose             |
+|---|--------|---------------------|
+| 1 | 200    | `2026-02`           |
+| 2 | 550    | `2026-02, 2026-03`  |
+
+Total 750 = his expected fee for the two months (likely 2026-02 = 500, 2026-03 = 250 — junior tier or exception-adjusted). The app currently shows 2026-02 = **566** paid, 2026-03 = **183** paid. The user wants:
+
+> First use the second payment for the rest of 2026-02 (no more, no less), then put the remainder toward 2026-03.
+
+Both Python ([scripts/match_payments.py](scripts/match_payments.py)) and Go ([go/internal/domain/reconcile/reconcile.go](go/internal/domain/reconcile/reconcile.go)) have the same bug: when a single transaction's `in_window_share` is less than the sum of in-window expected fees, both fall into a **proportional** branch that splits the new transaction across months in proportion to each month's *total* expected fee — never consulting what prior transactions already paid into earlier months.
+
+Trace for txn 2:
+
+- `in_window = [(2026-02, 500), (2026-03, 250)]`, `total_expected = 750`, `in_window_share = 550`.
+- `550 < 750` → proportional:
+  - 02 alloc = `550 × 500 / 750 = 366.67`
+  - 03 alloc = `550 − 366.67 = 183.33`
+- Combined with txn 1's 200 → 02 = 566.67, 03 = 183.33 → display `566 / 183`. ✓ matches reported numbers.
+
+The fix: replace the greedy + proportional branches with a single **fill-first** loop that iterates `in_window` in user-supplied order (already chronological by convention from [scripts/infer_payments.py:151](scripts/infer_payments.py#L151)) and allocates `min(remaining, max(0, expected − paid_so_far))` to each month, with any final surplus going to the credit bucket. This collapses three cases (greedy / proportional / pure overflow) into one and naturally consults the ledger's `paid` field which is already updated by prior transactions in the same reconcile pass.
+
+The even-split branch (`total_expected == 0`, prepayment before fees known) stays untouched — different semantic, folding it in would silently change behavior.
+
+## Changes
+
+### Python — [scripts/match_payments.py:471-498](scripts/match_payments.py#L471-L498)
+
+Replace both `total_expected > 0` branches (current lines 471–498) with a single unified loop. Keep lines 466–469 above and the even-split fallback below (lines 499–510) as-is.
+
+```python
+            if total_expected > 0:
+                # Fill-first: iterate in_window in matched_months order (chronological by
+                # convention), allocate min(remaining, deficit) to each month, where
+                # deficit accounts for prior transactions already credited to that month.
+                # Any surplus after all in-window deficits are covered → credit bucket.
+                remaining = in_window_share
+                for m, exp in in_window:
+                    paid_so_far = ledger[member_name][m]["paid"]
+                    deficit = max(0.0, float(exp) - paid_so_far)
+                    alloc = min(remaining, deficit)
+                    if alloc <= 0:
+                        continue
+                    ledger[member_name][m]["paid"] += alloc
+                    ledger[member_name][m]["transactions"].append({
+                        "amount": alloc,
+                        "date": tx["date"],
+                        "sender": tx["sender"],
+                        "message": tx["message"],
+                        "confidence": confidence,
+                    })
+                    remaining -= alloc
+                if remaining > 0:
+                    credits[member_name] = credits.get(member_name, 0) + int(remaining)
+            else:
+                # … existing even-split branch (lines 499–510) unchanged …
+```
+
+Note: skipping the `transactions.append` when `alloc <= 0` (e.g. month already fully paid by a prior txn) avoids zero-amount ghost rows in the per-month transaction list. This is a small UI-visible side effect; before committing, grep tests for assertions on `len(transactions)` per month to confirm nothing relies on the current "one row per (txn, month) regardless of alloc" behavior.
+
+### Go — [go/internal/domain/reconcile/reconcile.go:320-357](go/internal/domain/reconcile/reconcile.go#L320-L357)
+
+Same shape — replace both `totalExpected > 0` branches. Even-split branch (lines 358–372) stays.
+
+```go
+            if totalExpected > 0 {
+                // Fill-first; see Python reconcile() for rationale.
+                remaining := inWindowShare
+                for _, mw := range inWindow {
+                    md := ledger[memberName][mw.month]
+                    deficit := float64(mw.expected) - md.Paid
+                    if deficit < 0 {
+                        deficit = 0
+                    }
+                    alloc := remaining
+                    if deficit < alloc {
+                        alloc = deficit
+                    }
+                    if alloc <= 0 {
+                        continue
+                    }
+                    md.Paid += alloc
+                    md.Transactions = append(md.Transactions, TxEntry{
+                        Amount:     alloc,
+                        Date:       tx.Date,
+                        Sender:     tx.Sender,
+                        Message:    tx.Message,
+                        Confidence: string(m.Confidence),
+                    })
+                    ledger[memberName][mw.month] = md
+                    remaining -= alloc
+                }
+                if remaining > 0 {
+                    credits[memberName] += int(remaining)
+                }
+            } else {
+                // … existing even-split branch (lines 358–372) unchanged …
+            }
+```
+
+### Tests
+
+#### Python — [tests/test_reconcile_exceptions.py](tests/test_reconcile_exceptions.py)
+
+1. **Rewrite `test_proportional_underpayment`** ([line 96](tests/test_reconcile_exceptions.py#L96)) — its current assertions (`paid_02 < 750`, `paid_03 < 350`, `paid_04 < 750`, and 02/04 equal allocation) are incompatible with fill-first. Under fill-first with the same fixture (1250 across `02:750, 03:350, 04:750`):
+   - 02: `min(1250, 750) = 750` (full) → remaining 500
+   - 03: `min(500, 350) = 350` (full) → remaining 150
+   - 04: `min(150, 750) = 150` (partial) → remaining 0
+   
+   Replace assertions with these exact expected values, rename to `test_underpayment_fills_earliest_first`.
+
+2. **Add `test_fill_first_across_two_transactions`** — the Matyáš regression:
+
+   ```python
+   def test_fill_first_across_two_transactions(self):
+       """Prior txn fills 02 partially; later txn finishes 02 then spills to 03."""
+       members = [('Matyáš', 'A', {'2026-02': (500, 2), '2026-03': (250, 1)})]
+       sorted_months = ['2026-02', '2026-03']
+       tx1 = _tx('Matyáš', '2026-02', 200)
+       tx2 = _tx('Matyáš', '2026-02, 2026-03', 550)
+
+       result = reconcile(members, sorted_months, [tx1, tx2])
+       months = result['members']['Matyáš']['months']
+
+       self.assertAlmostEqual(months['2026-02']['paid'], 500, places=2)
+       self.assertAlmostEqual(months['2026-03']['paid'], 250, places=2)
+   ```
+
+3. `test_greedy_exact_match`, `test_greedy_overpayment_goes_to_credit`, `test_single_month_unchanged`, `test_two_members_multi_month` should pass unchanged — fill-first agrees with greedy when payment ≥ total expected.
+
+#### Go — [go/internal/domain/reconcile/reconcile_test.go](go/internal/domain/reconcile/reconcile_test.go)
+
+- Add `TestUnderpaymentFillsEarliestFirst` mirroring the rewritten Python test.
+- Add `TestFillFirstAcrossTwoTransactions` mirroring the Matyáš scenario.
+
+#### Parity — [go/tests/parity/reconcile/reconcile_parity_test.go](go/tests/parity/reconcile/reconcile_parity_test.go)
+
+Add a fixture for the Matyáš two-transaction case. Since both implementations change together and Python remains canonical, existing parity fixtures should continue to pass; verify after edits.
+
+### Changelog — [CHANGELOG.md](CHANGELOG.md)
+
+Append top entry (use `date "+%Y-%m-%d %H:%M %Z"` at commit time):
+
+```markdown
+## 2026-05-11 23:55 CET — fill-first multi-month payment allocation
+
+- Multi-month payment allocation now fills the earliest in-window deficit first
+  and spills the remainder to later months, accounting for prior transactions'
+  contributions. Previously a single transaction was split proportionally to
+  each month's total expected fee, ignoring earlier payments — surfaced by
+  Matyáš Thér's two-payment 200+550 case showing 566/183 instead of 500/250.
+- scripts/match_payments.py, go/internal/domain/reconcile/reconcile.go, tests.
+```
+
+## Critical files
+
+- [scripts/match_payments.py](scripts/match_payments.py) — Python reconcile (canonical)
+- [go/internal/domain/reconcile/reconcile.go](go/internal/domain/reconcile/reconcile.go) — Go reconcile (mirrors Python)
+- [tests/test_reconcile_exceptions.py](tests/test_reconcile_exceptions.py) — rewrite `test_proportional_underpayment` + add Matyáš test
+- [go/internal/domain/reconcile/reconcile_test.go](go/internal/domain/reconcile/reconcile_test.go) — add Go tests
+- [go/tests/parity/reconcile/reconcile_parity_test.go](go/tests/parity/reconcile/reconcile_parity_test.go) — add parity fixture
+- [CHANGELOG.md](CHANGELOG.md) — top entry
+
+## Verification
+
+1. `make test` — Python unit tests including the rewritten + new fill-first tests.
+2. `cd go && go test ./internal/domain/reconcile/... ./tests/parity/reconcile/...` — Go unit + parity tests.
+3. `make web` → load `/adults` or `/juniors` (whichever lists Matyáš Thér) → his 2026-02 row should be fully paid (no shortfall), 2026-03 fully paid, no leftover credit.
+4. Spot-check one other multi-month-purpose member to make sure fully-covered cases still look right.
+
+## Branch & MR (per CLAUDE.md)
+
+1. `git checkout -b fix/fill-first-multi-month-allocation`
+2. Apply edits + tests + CHANGELOG entry.
+3. `make test && (cd go && go test ./...)` — both green.
+4. Commit: `fix(reconcile): fill earliest month deficit first in multi-month allocations` with `Co-Authored-By` trailer.
+5. `git push -u origin fix/fill-first-multi-month-allocation`
+6. `tea pr create --title "fix(reconcile): fill earliest month deficit first" --description "<body>" --base main --head fix/fill-first-multi-month-allocation`
+7. Print MR URL. Do not merge from CLI.

go/internal/domain/reconcile/reconcile.go

@@ -115,10 +115,11 @@ type monthExpected struct {
 	expected int
 }
 
-// Reconcile matches transactions to members and months using three allocation phases:
+// Reconcile matches transactions to members and months using two allocation phases:
-//  1. Greedy: payment ≥ total expected → fill each month exactly; overflow → credit.
+//  1. Fill-first: iterate matched months in user-supplied order, allocating min(remaining,
-//  2. Proportional: payment < total → distribute by each month's share; last absorbs float remainder.
+//     deficit) to each month where deficit = expected − already-paid. Surplus → credit.
-//  3. Even-split fallback: all expected fees are 0 (prepayment) → divide equally.
+//     Handles both the "greedy" (payment covers all) and "partial" cases in one pass.
+//  2. Even-split fallback: all expected fees are 0 (prepayment) → divide equally.
 //
 // defaultYear seeds czech.ParseMonthReferences in the inference fallback.
 // Pass time.Now().Year() in production; pass a fixed year in tests.
@@ -317,34 +318,26 @@ func Reconcile(
 				totalExpected += mw.expected
 			}
 
-			if totalExpected > 0 && inWindowShare >= float64(totalExpected) {
+			if totalExpected > 0 {
-				// Greedy: payment covers all expected fees; overflow → credit
+				// Fill-first: iterate inWindow in matched-months order (chronological by
-				credits[memberName] += int(inWindowShare - float64(totalExpected))
+				// convention), allocating min(remaining, deficit) to each month. Deficit
-				for _, mw := range inWindow {
+				// is net of what prior transactions already paid, so a second payment on
-					alloc := float64(mw.expected)
+				// the same months correctly fills only what remains due. Any surplus after
-					md := ledger[memberName][mw.month]
+				// all deficits are covered goes to the credit bucket.
-					md.Paid += alloc
-					md.Transactions = append(md.Transactions, TxEntry{
-						Amount:     alloc,
-						Date:       tx.Date,
-						Sender:     tx.Sender,
-						Message:    tx.Message,
-						Confidence: string(m.Confidence),
-					})
-					ledger[memberName][mw.month] = md
-				}
-			} else if totalExpected > 0 {
-				// Proportional: distribute by each month's share; last month absorbs float remainder
 				remaining := inWindowShare
-				for i, mw := range inWindow {
+				for _, mw := range inWindow {
-					var alloc float64
-					if i == len(inWindow)-1 {
-						alloc = remaining
-					} else {
-						alloc = inWindowShare * float64(mw.expected) / float64(totalExpected)
-					}
-					remaining -= alloc
 					md := ledger[memberName][mw.month]
+					deficit := float64(mw.expected) - md.Paid
+					if deficit < 0 {
+						deficit = 0
+					}
+					alloc := remaining
+					if deficit < alloc {
+						alloc = deficit
+					}
+					if alloc <= 0 {
+						continue
+					}
 					md.Paid += alloc
 					md.Transactions = append(md.Transactions, TxEntry{
 						Amount:     alloc,
@@ -354,6 +347,10 @@ func Reconcile(
 						Confidence: string(m.Confidence),
 					})
 					ledger[memberName][mw.month] = md
+					remaining -= alloc
+				}
+				if remaining > 0 {
+					credits[memberName] += int(remaining)
 				}
 			} else {
 				// Even-split fallback: prepayment before attendance recorded

go/internal/domain/reconcile/reconcile_test.go

@@ -111,36 +111,26 @@ func TestReconcileGreedyOverpaymentGoesToCredit(t *testing.T) {
 	}
 }
 
-func TestReconcileProportionalUnderpayment(t *testing.T) {
+func TestReconcileUnderpaymentFillsEarliestFirst(t *testing.T) {
 	t.Parallel()
 	members := []Member{{
 		Name: "Alice", Tier: "A",
 		Fees: map[string]FeeData{"2026-02": {Expected: 750, Attendance: 3}, "2026-03": {Expected: 350, Attendance: 3}, "2026-04": {Expected: 750, Attendance: 3}},
 	}}
 	sortedMonths := []string{"2026-02", "2026-03", "2026-04"}
-	amount := 1250.0
 
-	result := Reconcile(members, sortedMonths, []Transaction{tx("Alice", "2026-02, 2026-03, 2026-04", amount)}, nil, defaultYear)
+	result := Reconcile(members, sortedMonths, []Transaction{tx("Alice", "2026-02, 2026-03, 2026-04", 1250)}, nil, defaultYear)
 
 	months := result.Members["Alice"].Months
-	paid02 := months["2026-02"].Paid
+	// 02 filled first (750), then 03 (350), then remainder 150 to 04
-	paid03 := months["2026-03"].Paid
+	if math.Abs(months["2026-02"].Paid-750) > 0.01 {
-	paid04 := months["2026-04"].Paid
+		t.Errorf("02: want 750, got %f", months["2026-02"].Paid)
-
-	if paid02 >= 750 {
-		t.Errorf("2026-02 should be underpaid, got %f", paid02)
 	}
-	if paid03 >= 350 {
+	if math.Abs(months["2026-03"].Paid-350) > 0.01 {
-		t.Errorf("2026-03 should be underpaid, got %f", paid03)
+		t.Errorf("03: want 350, got %f", months["2026-03"].Paid)
 	}
-	if paid04 >= 750 {
+	if math.Abs(months["2026-04"].Paid-150) > 0.01 {
-		t.Errorf("2026-04 should be underpaid, got %f", paid04)
+		t.Errorf("04: want 150, got %f", months["2026-04"].Paid)
-	}
-	if math.Abs(paid02+paid03+paid04-amount) > 0.01 {
-		t.Errorf("sum of paid want %f, got %f", amount, paid02+paid03+paid04)
-	}
-	if math.Abs(paid02-paid04) > 0.01 {
-		t.Errorf("02 and 04 have equal expected, want equal paid: %f vs %f", paid02, paid04)
 	}
 }
 
@@ -374,3 +364,52 @@ func TestReconcileNoTransactionsAllUnpaid(t *testing.T) {
 		t.Errorf("no txs: want empty unmatched, got %v", result.Unmatched)
 	}
 }
+
+// Payment < total expected → fill earliest months first, spill remainder to later.
+func TestUnderpaymentFillsEarliestFirst(t *testing.T) {
+	t.Parallel()
+	members := []Member{{Name: "Alice", Tier: "A", Fees: map[string]FeeData{
+		"2026-02": {Expected: 750, Attendance: 3},
+		"2026-03": {Expected: 350, Attendance: 3},
+		"2026-04": {Expected: 750, Attendance: 3},
+	}}}
+	txs := []Transaction{tx("Alice", "2026-02, 2026-03, 2026-04", 1250)}
+
+	result := Reconcile(members, []string{"2026-02", "2026-03", "2026-04"}, txs, nil, defaultYear)
+	months := result.Members["Alice"].Months
+
+	// 02 filled first (750), then 03 (350), then remainder 150 to 04
+	if math.Abs(months["2026-02"].Paid-750) > 0.01 {
+		t.Errorf("02: want 750, got %f", months["2026-02"].Paid)
+	}
+	if math.Abs(months["2026-03"].Paid-350) > 0.01 {
+		t.Errorf("03: want 350, got %f", months["2026-03"].Paid)
+	}
+	if math.Abs(months["2026-04"].Paid-150) > 0.01 {
+		t.Errorf("04: want 150, got %f", months["2026-04"].Paid)
+	}
+}
+
+// Prior txn fills 02 partially; later txn finishes 02 then spills to 03.
+func TestFillFirstAcrossTwoTransactions(t *testing.T) {
+	t.Parallel()
+	members := []Member{{Name: "Matyáš", Tier: "A", Fees: map[string]FeeData{
+		"2026-02": {Expected: 500, Attendance: 2},
+		"2026-03": {Expected: 250, Attendance: 1},
+	}}}
+	sortedMonths := []string{"2026-02", "2026-03"}
+	txs := []Transaction{
+		tx("Matyáš", "2026-02", 200),
+		tx("Matyáš", "2026-02, 2026-03", 550),
+	}
+
+	result := Reconcile(members, sortedMonths, txs, nil, defaultYear)
+	months := result.Members["Matyáš"].Months
+
+	if math.Abs(months["2026-02"].Paid-500) > 0.01 {
+		t.Errorf("02: want 500, got %f", months["2026-02"].Paid)
+	}
+	if math.Abs(months["2026-03"].Paid-250) > 0.01 {
+		t.Errorf("03: want 250, got %f", months["2026-03"].Paid)
+	}
+}

go/tests/fixtures/reconcile/03_proportional_remainder.json

@@ -1,7 +1,7 @@
 {
   "case": "03_proportional_remainder",
   "func": "scripts.match_payments.reconcile",
-  "captured_at": "2026-05-06",
+  "captured_at": "2026-05-11",
   "input": {
     "members": [
       {
@@ -54,10 +54,10 @@
             "original_expected": 750,
             "attendance_count": 3,
             "exception": null,
-            "paid": 324.3243243243243,
+            "paid": 750.0,
             "transactions": [
               {
-                "amount": 324.3243243243243,
+                "amount": 750.0,
                 "date": "2026-03-10",
                 "sender": "Member_d035d9f9",
                 "message": "",
@@ -70,10 +70,10 @@
             "original_expected": 750,
             "attendance_count": 2,
             "exception": null,
-            "paid": 324.3243243243243,
+            "paid": 50.0,
             "transactions": [
               {
-                "amount": 324.3243243243243,
+                "amount": 50.0,
                 "date": "2026-03-10",
                 "sender": "Member_d035d9f9",
                 "message": "",
@@ -86,25 +86,17 @@
             "original_expected": 350,
             "attendance_count": 2,
             "exception": null,
-            "paid": 151.35135135135135,
+            "paid": 0,
-            "transactions": [
+            "transactions": []
-              {
-                "amount": 151.35135135135135,
-                "date": "2026-03-10",
-                "sender": "Member_d035d9f9",
-                "message": "",
-                "confidence": "auto"
-              }
-            ]
           }
         },
         "other_transactions": [],
-        "total_balance": -1051
+        "total_balance": -1050
       }
     },
     "unmatched": [],
     "credits": {
-      "Member_d035d9f9": -1051
+      "Member_d035d9f9": -1050
     }
   }
 }

go/tests/fixtures/reconcile/09_multiperson_multimonth.json

@@ -1,7 +1,7 @@
 {
   "case": "09_multiperson_multimonth",
   "func": "scripts.match_payments.reconcile",
-  "captured_at": "2026-05-06",
+  "captured_at": "2026-05-11",
   "input": {
     "members": [
       {
@@ -63,10 +63,10 @@
             "original_expected": 750,
             "attendance_count": 3,
             "exception": null,
-            "paid": 500.0,
+            "paid": 750.0,
             "transactions": [
               {
-                "amount": 500.0,
+                "amount": 750.0,
                 "date": "2026-02-15",
                 "sender": "Member_d035d9f9",
                 "message": "",
@@ -79,10 +79,10 @@
             "original_expected": 750,
             "attendance_count": 2,
             "exception": null,
-            "paid": 500.0,
+            "paid": 250.0,
             "transactions": [
               {
-                "amount": 500.0,
+                "amount": 250.0,
                 "date": "2026-02-15",
                 "sender": "Member_d035d9f9",
                 "message": "",
@@ -102,10 +102,10 @@
             "original_expected": 750,
             "attendance_count": 2,
             "exception": null,
-            "paid": 681.8181818181819,
+            "paid": 750.0,
             "transactions": [
               {
-                "amount": 681.8181818181819,
+                "amount": 750.0,
                 "date": "2026-02-15",
                 "sender": "Member_d035d9f9",
                 "message": "",
@@ -118,10 +118,10 @@
             "original_expected": 350,
             "attendance_count": 2,
             "exception": null,
-            "paid": 318.18181818181813,
+            "paid": 250.0,
             "transactions": [
               {
-                "amount": 318.18181818181813,
+                "amount": 250.0,
                 "date": "2026-02-15",
                 "sender": "Member_d035d9f9",
                 "message": "",
@@ -131,13 +131,13 @@
           }
         },
         "other_transactions": [],
-        "total_balance": -101
+        "total_balance": -100
       }
     },
     "unmatched": [],
     "credits": {
       "Member_d035d9f9": -500,
-      "Member_f4a93e46": -101
+      "Member_f4a93e46": -100
     }
   }
 }

scripts/fio_utils.py

@@ -4,6 +4,7 @@
 import json
 import os
 import re
+import sys
 import urllib.request
 from datetime import datetime
 from html.parser import HTMLParser
@@ -89,9 +90,11 @@ def parse_czech_amount(s: str) -> float | None:
 
 
 def parse_czech_date(s: str) -> str | None:
-    """Parse 'DD.MM.YYYY' to 'YYYY-MM-DD'."""
+    """Parse a Czech date to 'YYYY-MM-DD'. Accepts 4-digit and 2-digit years
+    with dot or slash separators; Fio's transparent page mixes 'DD.MM.YYYY'
+    and 'DD.MM.YY' in the same response."""
     s = s.strip()
-    for fmt in ("%d.%m.%Y", "%d/%m/%Y"):
+    for fmt in ("%d.%m.%Y", "%d/%m/%Y", "%d.%m.%y", "%d/%m/%y"):
         try:
             return datetime.strptime(s, fmt).strftime("%Y-%m-%d")
         except ValueError:
@@ -146,6 +149,7 @@ def fetch_transactions_transparent(
             "bank_id": "", # HTML scraping doesn't give stable ID
         })
 
+    print(f"fio: transparent fetched {len(rows)} raw rows, {len(transactions)} transaction(s) after filtering", file=sys.stderr)
     return transactions
 
 
@@ -169,7 +173,8 @@ def fetch_transactions_api(
 
     transactions = []
     tx_list = data.get("accountStatement", {}).get("transactionList", {})
-    for tx in (tx_list.get("transaction") or []):
+    raw_list = tx_list.get("transaction") or []
+    for tx in raw_list:
         # Each field is {"value": ..., "name": ..., "id": ...} or null
         def val(col_id):
             col = tx.get(f"column{col_id}")
@@ -197,6 +202,7 @@ def fetch_transactions_api(
             "currency": str(val(14) or "CZK"), # column14 = Currency
         })
 
+    print(f"fio: api fetched {len(raw_list)} raw transaction(s), {len(transactions)} after filtering", file=sys.stderr)
     return transactions
 
 
@@ -204,8 +210,14 @@ def fetch_transactions(date_from: str, date_to: str) -> list[dict]:
     """Fetch transactions, using API if token available, else transparent page."""
     token = os.environ.get("FIO_API_TOKEN", "").strip()
     if token:
+        print(f"fio: using authenticated API, window {date_from}..{date_to}", file=sys.stderr)
         return fetch_transactions_api(token, date_from, date_to)
 
+    print(
+        f"fio: using transparent page (FIO_API_TOKEN unset — expect publishing lag), "
+        f"window {date_from}..{date_to}, account=2800359168",
+        file=sys.stderr,
+    )
     # Convert YYYY-MM-DD to DD.MM.YYYY for the transparent page URL
     from_dt = datetime.strptime(date_from, "%Y-%m-%d")
     to_dt = datetime.strptime(date_to, "%Y-%m-%d")

scripts/match_payments.py

@@ -468,26 +468,19 @@ def reconcile(
 
             total_expected = sum(e for _, e in in_window)
 
-            if total_expected > 0 and in_window_share >= total_expected:
+            if total_expected > 0:
-                # Greedy phase: payment covers all in-window fees; overflow → credit.
+                # Fill-first: iterate in_window in matched_months order (chronological by
-                credits[member_name] = credits.get(member_name, 0) + int(in_window_share - total_expected)
+                # convention from infer_payments.py), allocating min(remaining, deficit) to
-                for m, exp in in_window:
+                # each month. Deficit is net of what prior transactions already paid, so a
-                    alloc = float(exp)
+                # second payment on the same months correctly fills only what remains due.
-                    ledger[member_name][m]["paid"] += alloc
+                # Any surplus after all deficits are covered goes to the credit bucket.
-                    ledger[member_name][m]["transactions"].append({
-                        "amount": alloc,
-                        "date": tx["date"],
-                        "sender": tx["sender"],
-                        "message": tx["message"],
-                        "confidence": confidence,
-                    })
-            elif total_expected > 0:
-                # Proportional phase: distribute in_window_share by each month's expected fee.
-                # Last month absorbs any float remainder so the sum equals in_window_share exactly.
                 remaining = in_window_share
-                for i, (m, exp) in enumerate(in_window):
+                for m, exp in in_window:
-                    alloc = remaining if i == len(in_window) - 1 else in_window_share * exp / total_expected
+                    paid_so_far = ledger[member_name][m]["paid"]
-                    remaining -= alloc
+                    deficit = max(0.0, float(exp) - paid_so_far)
+                    alloc = min(remaining, deficit)
+                    if alloc <= 0:
+                        continue
                     ledger[member_name][m]["paid"] += alloc
                     ledger[member_name][m]["transactions"].append({
                         "amount": alloc,
@@ -496,6 +489,9 @@ def reconcile(
                         "message": tx["message"],
                         "confidence": confidence,
                     })
+                    remaining -= alloc
+                if remaining > 0:
+                    credits[member_name] = credits.get(member_name, 0) + int(remaining)
             else:
                 # Fallback: no expected fees (prepayment before attendance recorded); even split.
                 per_month = in_window_share / len(in_window)

scripts/sync_fio_to_sheets.py

@@ -77,6 +77,35 @@ def generate_sync_id(tx: dict) -> str:
     return hashlib.sha256(raw_str.encode("utf-8")).hexdigest()
 
 
+def _trunc(s: str, n: int = 40) -> str:
+    s = str(s)
+    return s if len(s) <= n else s[: n - 1] + "…"
+
+
+def _print_fio_table(transactions: list[dict], statuses: list[str]) -> None:
+    headers = ["DATE", "AMOUNT", "SENDER", "VS", "MESSAGE", "BANKID", "STATUS"]
+    rows = [
+        [
+            str(tx.get("date", "")),
+            f"{float(tx.get('amount', 0)):.2f}",
+            str(tx.get("sender", "")),
+            str(tx.get("vs", "")),
+            _trunc(str(tx.get("message", ""))),
+            str(tx.get("bank_id", "")),
+            status,
+        ]
+        for tx, status in zip(transactions, statuses)
+    ]
+    widths = [
+        max(len(headers[i]), max((len(r[i]) for r in rows), default=0))
+        for i in range(len(headers))
+    ]
+    sep = "  "
+    print(sep.join(h.ljust(w) for h, w in zip(headers, widths)))
+    for row in rows:
+        print(sep.join(cell.ljust(w) for cell, w in zip(row, widths)))
+
+
 def sort_sheet_by_date(service, spreadsheet_id):
     """Sort the sheet by the Date column (Column B)."""
     # Get the sheet ID (gid) of the first sheet
@@ -104,12 +133,21 @@ def sort_sheet_by_date(service, spreadsheet_id):
     print("Sheet sorted by date.")
 
 
-def sync_to_sheets(spreadsheet_id: str, credentials_path: str, days: int = None, date_from_str: str = None, date_to_str: str = None, sort_by_date: bool = False):
+def sync_to_sheets(
+    spreadsheet_id: str,
+    credentials_path: str,
+    days: int = None,
+    date_from_str: str = None,
+    date_to_str: str = None,
+    sort_by_date: bool = False,
+    dry_run: bool = False,
+    print_fio_table: bool = False,
+):
     print(f"Connecting to Google Sheets using {credentials_path}...")
     service = get_sheets_service(credentials_path)
     sheet = service.spreadsheets()
 
-    # 1. Fetch existing IDs from Column G (last column in A-G range)
+    # 1. Read existing sync IDs from Column K
     print(f"Reading existing sync IDs from sheet...")
     try:
         result = sheet.values().get(
@@ -120,6 +158,9 @@ def sync_to_sheets(spreadsheet_id: str, credentials_path: str, days: int = None,
 
         # Check and insert labels if missing
         if not values or values[0] != COLUMN_LABELS:
+            if dry_run:
+                print("Dry run: would write header row")
+            else:
                 print("Inserting column labels...")
                 sheet.values().update(
                     spreadsheetId=spreadsheet_id,
@@ -129,7 +170,7 @@ def sync_to_sheets(spreadsheet_id: str, credentials_path: str, days: int = None,
                 ).execute()
             existing_ids = set()
         else:
-            # Sync ID is now the last column (index 10)
+            # Sync ID is the last column (index 10)
             existing_ids = {row[10] for row in values[1:] if len(row) > 10}
     except Exception as e:
         print(f"Error reading sheet (maybe empty?): {e}")
@@ -150,8 +191,12 @@ def sync_to_sheets(spreadsheet_id: str, credentials_path: str, days: int = None,
     transactions = fetch_transactions(df_str, dt_str)
     print(f"Found {len(transactions)} transactions.")
 
-    # 3. Filter for new transactions
+    if dry_run:
+        print(f"Dry run: window {df_str} to {dt_str}, fetched {len(transactions)} transaction(s) from Fio")
+
+    # 3. Determine NEW/DUP for each transaction
     new_rows = []
+    tx_statuses = []
     for tx in transactions:
         sync_id = generate_sync_id(tx)
         if sync_id not in existing_ids:
@@ -169,12 +214,37 @@ def sync_to_sheets(spreadsheet_id: str, credentials_path: str, days: int = None,
                 tx.get("bank_id", ""),
                 sync_id,
             ])
+            tx_statuses.append("NEW")
+        else:
+            tx_statuses.append("DUP")
+
+    # 4. Print table (before early-return so all transactions are shown including DUPs)
+    if print_fio_table and transactions:
+        _print_fio_table(transactions, tx_statuses)
 
     if not new_rows:
+        if dry_run:
+            print("Dry run: would sync 0 new transaction(s).")
+        else:
             print("No new transactions to sync.")
         return
 
-    # 4. Append to sheet
+    # 5. Append to sheet or print dry-run would-write lines
+    if dry_run:
+        for tx, status in zip(transactions, tx_statuses):
+            if status == "NEW":
+                print(
+                    f"Dry run: would append"
+                    f" date={tx.get('date', '')}"
+                    f" amount={tx.get('amount', '')}"
+                    f" sender={tx.get('sender', '')}"
+                    f" vs={tx.get('vs', '')}"
+                    f" message={tx.get('message', '')}"
+                )
+        if sort_by_date:
+            print("Dry run: would sort by date")
+        print(f"Dry run: would sync {len(new_rows)} new transaction(s).")
+    else:
         print(f"Appending {len(new_rows)} new transactions to the sheet...")
         body = {"values": new_rows}
         sheet.values().append(
@@ -184,7 +254,6 @@ def sync_to_sheets(spreadsheet_id: str, credentials_path: str, days: int = None,
             body=body
         ).execute()
         print("Sync completed successfully.")
-
         if sort_by_date:
             sort_sheet_by_date(service, spreadsheet_id)
 
@@ -197,6 +266,8 @@ def main():
     parser.add_argument("--from", dest="date_from", help="Start date YYYY-MM-DD")
     parser.add_argument("--to", dest="date_to", help="End date YYYY-MM-DD")
     parser.add_argument("--sort-by-date", action="store_true", help="Sort the sheet by date after sync")
+    parser.add_argument("--dry-run", action="store_true", help="Fetch and dedup without writing to the sheet")
+    parser.add_argument("--print-fio-table", action="store_true", help="Print aligned table of all fetched transactions with NEW/DUP status (use with --dry-run)")
     args = parser.parse_args()
 
     try:
@@ -206,7 +277,9 @@ def main():
             days=args.days,
             date_from_str=args.date_from,
             date_to_str=args.date_to,
-            sort_by_date=args.sort_by_date
+            sort_by_date=args.sort_by_date,
+            dry_run=args.dry_run,
+            print_fio_table=args.print_fio_table,
         )
     except Exception as e:
         print(f"Sync failed: {e}")

tests/test_reconcile_exceptions.py

@@ -93,8 +93,8 @@ class TestMultiMonthAllocation(unittest.TestCase):
         self.assertEqual(int(months['2026-02']['paid']), 750)
         self.assertEqual(result['credits'].get('Alice', 0), 500)
 
-    def test_proportional_underpayment(self):
+    def test_underpayment_fills_earliest_first(self):
-        """Payment < total expected → proportional split; sum of paid == payment amount."""
+        """Payment < total expected → fill earliest months first, spill remainder to later."""
         members = [('Alice', 'A', {'2026-02': (750, 3), '2026-03': (350, 3), '2026-04': (750, 3)})]
         sorted_months = ['2026-02', '2026-03', '2026-04']
         amount = 1250
@@ -103,18 +103,28 @@ class TestMultiMonthAllocation(unittest.TestCase):
         result = reconcile(members, sorted_months, [tx])
         months = result['members']['Alice']['months']
 
-        paid_02 = months['2026-02']['paid']
+        # 02 filled first (750), then 03 (350), then remainder 150 to 04
-        paid_03 = months['2026-03']['paid']
+        self.assertAlmostEqual(months['2026-02']['paid'], 750, places=2)
-        paid_04 = months['2026-04']['paid']
+        self.assertAlmostEqual(months['2026-03']['paid'], 350, places=2)
+        self.assertAlmostEqual(months['2026-04']['paid'], 150, places=2)
+        # No CZK lost
+        self.assertAlmostEqual(
+            months['2026-02']['paid'] + months['2026-03']['paid'] + months['2026-04']['paid'],
+            amount, places=2,
+        )
 
-        # All months should be partial (underpaid)
+    def test_fill_first_across_two_transactions(self):
-        self.assertLess(paid_02, 750)
+        """Prior txn fills 02 partially; later txn finishes 02 then spills to 03."""
-        self.assertLess(paid_03, 350)
+        members = [('Matyáš', 'A', {'2026-02': (500, 2), '2026-03': (250, 1)})]
-        self.assertLess(paid_04, 750)
+        sorted_months = ['2026-02', '2026-03']
-        # Sum must equal the original payment (no CZK lost)
+        tx1 = _tx('Matyáš', '2026-02', 200)
-        self.assertAlmostEqual(paid_02 + paid_03 + paid_04, amount, places=2)
+        tx2 = _tx('Matyáš', '2026-02, 2026-03', 550)
-        # 02 and 04 have equal expected → equal allocation
+
-        self.assertAlmostEqual(paid_02, paid_04, places=2)
+        result = reconcile(members, sorted_months, [tx1, tx2])
+        months = result['members']['Matyáš']['months']
+
+        self.assertAlmostEqual(months['2026-02']['paid'], 500, places=2)
+        self.assertAlmostEqual(months['2026-03']['paid'], 250, places=2)
 
     def test_single_month_unchanged(self):
         """Single-month payment: full amount goes to that month (regression guard)."""