diff --git a/CHANGELOG.md b/CHANGELOG.md index 51d4a1f..3b3d036 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,12 @@ # Changelog +## 2026-10-09 00:21 CEST — fix(pii): remove real member names tied to payments + +- Three parity fixtures held raw bank messages with real names; the names are now `Member_` pseudonyms, as the scrubber produces. +- The junior regression test, its changelog entry and its plan file used a real junior's and parent's name with paid months; replaced with fictional names. +- `CLAUDE.md` now states the rule: a real name as a bare test input is fine, a real name next to payment details or any junior/parent name is not. +- Git history still contains the old text. + ## 2026-10-08 21:27 CEST — feat(fees): junior rate for 2026-08 is 0 - Junior monthly rate for 2026-08: 500 → 0 CZK, same as adults. @@ -294,7 +301,7 @@ ## 2026-05-06 16:38 CEST — fix: include juniors in payment-inference roster - `scripts/infer_payments.py`: union adults + junior rosters so junior-only members are visible to the matcher. -- Root cause: `get_members_with_fees()` reads only the adults sheet; junior-only kids like Jáchym Kubík were absent from `member_names`, causing the exact-match short-circuit to never fire and a different adult sharing the first name to win via fuzzy review. +- Root cause: `get_members_with_fees()` reads only the adults sheet; junior-only kids like Jáchym Dvořák were absent from `member_names`, causing the exact-match short-circuit to never fire and a different adult sharing the first name to win via fuzzy review. - Two regression tests added to `tests/test_match_members.py`. ## 2026-05-06 16:05 CEST — feat(go/M2.10): port domain/reconcile.Reconcile diff --git a/CLAUDE.md b/CLAUDE.md index 9ebb3b3..2891e7c 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -29,6 +29,9 @@ These exist only in the Go app on purpose. Do not port them to the legacy Python ## Key Constraints - **PII separation**: Member data (names, emails, payment info) must never be committed to git. Enforce config/data separation from day one. + - Tests may use a real member's name as a bare matching input (diacritics, nicknames, word order). + - Never commit a real name together with payment details or raw bank messages, and never commit names of juniors or their parents. Use fictional names there. + - Parity fixtures under `go/tests/fixtures/` must be scrubbed (`Member_`); check new or regenerated fixtures before committing. - **Configuration**: External service IDs, credentials, and tunable parameters are centralized in `go/internal/config/config.go` (mirrored in `legacy/scripts/config.py`). Domain-specific constants (fees, merged months) stay in their respective modules. ## Development Setup diff --git a/docs/plans/2026-05-06-1626-infer-payments-junior-roster.md b/docs/plans/2026-05-06-1626-infer-payments-junior-roster.md index be158a1..c2c096f 100644 --- a/docs/plans/2026-05-06-1626-infer-payments-junior-roster.md +++ b/docs/plans/2026-05-06-1626-infer-payments-junior-roster.md @@ -2,12 +2,12 @@ ## Context -A bank payment from sender `JIŘÍ KUBÍK` with the message -`Jáchym Kubík: 01/2026+03/2026+04/2026` is being inferred as -`[?] Jáchym Hrušák (G)` instead of the obvious `Jáchym Kubík`, even though +A bank payment from sender `JIŘÍ DVOŘÁK` with the message +`Jáchym Dvořák: 01/2026+03/2026+04/2026` is being inferred as +`[?] Jáchym Novotný (G)` instead of the obvious `Jáchym Dvořák`, even though the message contains his exact full name. -**Root cause** (confirmed with the user): `Jáchym Kubík` is in the **junior** +**Root cause** (confirmed with the user): `Jáchym Dvořák` is in the **junior** attendance sheet only — he does not appear on the main/adults sheet. But [scripts/infer_payments.py:101-102](scripts/infer_payments.py#L101-L102) builds `member_names` by calling `get_members_with_fees()` @@ -15,17 +15,17 @@ builds `member_names` by calling `get_members_with_fees()` `EXPORT_URL` (the adults sheet). Junior-only members are therefore invisible to the matcher. -With Kubík absent from `member_names`, the matcher in +With Dvořák absent from `member_names`, the matcher in [scripts/match_payments.py:65](scripts/match_payments.py#L65) processes the -combined text `jiri kubik jachym kubik: 01/2026+03/2026+04/2026` against an +combined text `jiri dvorak jachym dvorak: 01/2026+03/2026+04/2026` against an adults-only roster: - The exact-full-name short-circuit (`match_payments.py:75-84`) finds nothing — no adult's full name is in the text. -- Hrušák `(G)` is the only adult with first name `Jáchym`. He fails the +- Novotný `(G)` is the only adult with first name `Jáchym`. He fails the auto-rules (his surname isn't in the text) but hits the partial-first-name - review rule (`match_payments.py:123-125`) → returned as `("Jáchym Hrušák (G)", - "review")`, rendered as `[?] Jáchym Hrušák (G)`. + review rule (`match_payments.py:123-125`) → returned as `("Jáchym Novotný (G)", + "review")`, rendered as `[?] Jáchym Novotný (G)`. The user's original framing — "exact match in message should win over everything" — is already implemented for any candidate that **is** in the @@ -33,7 +33,7 @@ roster (the May-04 short-circuit). The bug is upstream: the right candidate was never even considered. **Goal:** make `infer_payments` consider junior members as candidates, so -junior-only names like `Jáchym Kubík` get matched correctly. +junior-only names like `Jáchym Dvořák` get matched correctly. ## Approach @@ -98,23 +98,23 @@ changes are needed. the adults version, so `m[0]` works for both. - [scripts/match_payments.py:65-137](scripts/match_payments.py#L65-L137) — `match_members` already handles the precedence the user wants (exact full-name - short-circuit), so once Kubík is in `member_names`, the case will be auto-matched + short-circuit), so once Dvořák is in `member_names`, the case will be auto-matched with no `[?]`. ## Verification 1. **Manual sanity** — re-run inference on the offending row: - - Clear `Person`/`Purpose` for the Kubík row in the payments sheet. + - Clear `Person`/`Purpose` for the Dvořák row in the payments sheet. - `make infer`. - - Expect `Person = Jáchym Kubík`, `Purpose = 2026-01, 2026-03, 2026-04`, + - Expect `Person = Jáchym Dvořák`, `Purpose = 2026-01, 2026-03, 2026-04`, no `[?]`. 2. **Unit test** — extend [tests/test_match_members.py](tests/test_match_members.py) (or add a small `tests/test_infer_payments.py`) to assert that, given a roster that - includes `Jáchym Hrušák (G)` and `Jáchym Kubík`, the message - `Jáchym Kubík: 01/2026+03/2026+04/2026` resolves to - `[("Jáchym Kubík", "auto")]` only. This is really a regression test for + includes `Jáchym Novotný (G)` and `Jáchym Dvořák`, the message + `Jáchym Dvořák: 01/2026+03/2026+04/2026` resolves to + `[("Jáchym Dvořák", "auto")]` only. This is really a regression test for the May-04 short-circuit — the new behavior under test is just that `infer_payments` now feeds in juniors. @@ -122,7 +122,7 @@ changes are needed. 4. **Dashboard smoke** — `make web`, open `/payments`, confirm the row now shows the correct member; open `/juniors`, confirm the payment is - credited to Kubík for the three months listed. + credited to Dvořák for the three months listed. 5. **Changelog** — once the user confirms the fix, append an entry to [CHANGELOG.md](CHANGELOG.md) per [CLAUDE.md](CLAUDE.md): diff --git a/docs/plans/2026-10-08-1026-pre-feature-cleanup-tasks.md b/docs/plans/2026-10-08-1026-pre-feature-cleanup-tasks.md index 29604cf..6f04bca 100644 --- a/docs/plans/2026-10-08-1026-pre-feature-cleanup-tasks.md +++ b/docs/plans/2026-10-08-1026-pre-feature-cleanup-tasks.md @@ -58,7 +58,8 @@ before new feature work starts. Companion to the Go rewrite tracker, current year. - Same applies to the `sync-2026` Makefile target. - Moot if Python is retired before then. -- [ ] **T4** Remove member names from tracked files (PII rule in `CLAUDE.md`). +- [x] **T4** Remove member names from tracked files (PII rule in `CLAUDE.md`). + Done 2026-10-09 in a narrower form, see Notes & decisions. Original task: - `CHANGELOG.md` line 29 names a member in a fix description. - `tests/test_match_payments.py`, `tests/test_reconcile_exceptions.py` and `go/internal/domain/reconcile/reconcile_test.go` use names that may be @@ -172,3 +173,16 @@ Plan: [2026-10-08-1103-api-parity-vs-months-to-show.md](2026-10-08-1103-api-pari ## Notes & decisions (Add entries as you go. Format: `YYYY-MM-DD — short note`.) + +- 2026-10-09 — T4 scope decided by the owner: a real name as a bare test + input is acceptable; a real name next to payment details, raw bank + messages, or any junior or parent name is not. Replaced the junior + regression-test names (test, changelog, plan) with fictional ones and + scrubbed three parity fixtures that still held raw bank messages. The + other test names stay. Git history was not rewritten. Rule recorded in + `CLAUDE.md`. +- 2026-10-09 — The `real_*` parse_month_references seeds pick rows from the + live payments cache by index (`_REAL_PMR_INDICES`). The sheet has changed + since capture, so `make capture-fixtures` now rewrites those fixtures with + different messages. Commit only the fixtures a change needs, or replace + the index-based seeds with fixed synthetic ones. diff --git a/go/tests/fixtures/pure/parse_month_references/real_filip_prosinec_leden_unor.json b/go/tests/fixtures/pure/parse_month_references/real_filip_prosinec_leden_unor.json index 03c9160..a26c425 100644 --- a/go/tests/fixtures/pure/parse_month_references/real_filip_prosinec_leden_unor.json +++ b/go/tests/fixtures/pure/parse_month_references/real_filip_prosinec_leden_unor.json @@ -3,7 +3,7 @@ "func": "scripts.czech_utils.parse_month_references", "captured_at": "2026-05-06", "input": { - "text": "Filip Halamka - prosinec, leden, unor", + "text": "Member_607b748d - prosinec, leden, unor", "default_year": 2026 }, "output": { diff --git a/go/tests/fixtures/pure/parse_month_references/real_jachym_numeric_multi.json b/go/tests/fixtures/pure/parse_month_references/real_jachym_numeric_multi.json index 049604b..088fddd 100644 --- a/go/tests/fixtures/pure/parse_month_references/real_jachym_numeric_multi.json +++ b/go/tests/fixtures/pure/parse_month_references/real_jachym_numeric_multi.json @@ -3,7 +3,7 @@ "func": "scripts.czech_utils.parse_month_references", "captured_at": "2026-05-06", "input": { - "text": "Jáchym Kubík: 01/2026+03/2026+04/2026", + "text": "Member_71343342: 01/2026+03/2026+04/2026", "default_year": 2026 }, "output": { diff --git a/go/tests/fixtures/pure/parse_month_references/real_martin_prosinec_leden.json b/go/tests/fixtures/pure/parse_month_references/real_martin_prosinec_leden.json index 37f6208..3999be3 100644 --- a/go/tests/fixtures/pure/parse_month_references/real_martin_prosinec_leden.json +++ b/go/tests/fixtures/pure/parse_month_references/real_martin_prosinec_leden.json @@ -3,7 +3,7 @@ "func": "scripts.czech_utils.parse_month_references", "captured_at": "2026-05-06", "input": { - "text": "Martin Bolvansky Pausal Prosinec Leden", + "text": "Member_51a8ec2d Pausal Prosinec Leden", "default_year": 2026 }, "output": { diff --git a/legacy/tests/test_match_members.py b/legacy/tests/test_match_members.py index 6071d1a..9cfe5b3 100644 --- a/legacy/tests/test_match_members.py +++ b/legacy/tests/test_match_members.py @@ -51,21 +51,21 @@ class TestMatchMembersExact(unittest.TestCase): def test_shared_first_name_junior_in_roster_wins_exact(self): # Regression: two members share first name "Jáchym"; message has full name # of the junior-only member → exact match must win, no [?] on the adult. - roster = ["Jáchym Hrušák (G)", "Jáchym Kubík"] + roster = ["Jáchym Novotný (G)", "Jáchym Dvořák"] result = match_members( - "JIŘÍ KUBÍK Jáchym Kubík: 01/2026+03/2026+04/2026", roster + "JIŘÍ DVOŘÁK Jáchym Dvořák: 01/2026+03/2026+04/2026", roster ) - self.assertEqual(result, [("Jáchym Kubík", "auto")]) + self.assertEqual(result, [("Jáchym Dvořák", "auto")]) def test_shared_first_name_without_junior_in_roster_falls_back(self): - # Without Kubík in the roster (old behaviour), Hrušák wins via first-name + # Without Dvořák in the roster (old behaviour), Novotný wins via first-name # partial match — confirms the roster-expansion fix is the real solution. - roster = ["Jáchym Hrušák (G)"] + roster = ["Jáchym Novotný (G)"] result = match_members( - "JIŘÍ KUBÍK Jáchym Kubík: 01/2026+03/2026+04/2026", roster + "JIŘÍ DVOŘÁK Jáchym Dvořák: 01/2026+03/2026+04/2026", roster ) names = [r[0] for r in result] - self.assertIn("Jáchym Hrušák (G)", names) + self.assertIn("Jáchym Novotný (G)", names) if __name__ == "__main__":