Compare commits

...
2 Commits
Author SHA1 Message Date
kacerr ac9f974007 Merge pull request 'fix(pii): remove real member names tied to payments' (#51) from fix/scrub-member-names into main
Reviewed-on: #51
2026-10-09 00:23:31 +02:00
kacerrandClaude Opus 5.5 e53b57f976 fix(pii): remove real member names tied to payments
- Scrub three parity fixtures that still held raw bank messages
- Use fictional names in the junior regression test, its changelog
  entry and its plan file
- State the rule in CLAUDE.md: a real name as a bare test input is
  fine; a real name next to payment details, or any junior or parent
  name, is not
- Tick T4 in the cleanup task list

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 00:22:15 +02:00
8 changed files with 53 additions and 29 deletions
+8 -1
View File
@@ -1,5 +1,12 @@
# Changelog # Changelog
## 2026-10-09 00:21 CEST — fix(pii): remove real member names tied to payments
- Three parity fixtures held raw bank messages with real names; the names are now `Member_<hash>` pseudonyms, as the scrubber produces.
- The junior regression test, its changelog entry and its plan file used a real junior's and parent's name with paid months; replaced with fictional names.
- `CLAUDE.md` now states the rule: a real name as a bare test input is fine, a real name next to payment details or any junior/parent name is not.
- Git history still contains the old text.
## 2026-10-08 21:27 CEST — feat(fees): junior rate for 2026-08 is 0 ## 2026-10-08 21:27 CEST — feat(fees): junior rate for 2026-08 is 0
- Junior monthly rate for 2026-08: 500 → 0 CZK, same as adults. - Junior monthly rate for 2026-08: 500 → 0 CZK, same as adults.
@@ -294,7 +301,7 @@
## 2026-05-06 16:38 CEST — fix: include juniors in payment-inference roster ## 2026-05-06 16:38 CEST — fix: include juniors in payment-inference roster
- `scripts/infer_payments.py`: union adults + junior rosters so junior-only members are visible to the matcher. - `scripts/infer_payments.py`: union adults + junior rosters so junior-only members are visible to the matcher.
- Root cause: `get_members_with_fees()` reads only the adults sheet; junior-only kids like Jáchym Kubík were absent from `member_names`, causing the exact-match short-circuit to never fire and a different adult sharing the first name to win via fuzzy review. - Root cause: `get_members_with_fees()` reads only the adults sheet; junior-only kids like Jáchym Dvořák were absent from `member_names`, causing the exact-match short-circuit to never fire and a different adult sharing the first name to win via fuzzy review.
- Two regression tests added to `tests/test_match_members.py`. - Two regression tests added to `tests/test_match_members.py`.
## 2026-05-06 16:05 CEST — feat(go/M2.10): port domain/reconcile.Reconcile ## 2026-05-06 16:05 CEST — feat(go/M2.10): port domain/reconcile.Reconcile
+3
View File
@@ -29,6 +29,9 @@ These exist only in the Go app on purpose. Do not port them to the legacy Python
## Key Constraints ## Key Constraints
- **PII separation**: Member data (names, emails, payment info) must never be committed to git. Enforce config/data separation from day one. - **PII separation**: Member data (names, emails, payment info) must never be committed to git. Enforce config/data separation from day one.
- Tests may use a real member's name as a bare matching input (diacritics, nicknames, word order).
- Never commit a real name together with payment details or raw bank messages, and never commit names of juniors or their parents. Use fictional names there.
- Parity fixtures under `go/tests/fixtures/` must be scrubbed (`Member_<hash>`); check new or regenerated fixtures before committing.
- **Configuration**: External service IDs, credentials, and tunable parameters are centralized in `go/internal/config/config.go` (mirrored in `legacy/scripts/config.py`). Domain-specific constants (fees, merged months) stay in their respective modules. - **Configuration**: External service IDs, credentials, and tunable parameters are centralized in `go/internal/config/config.go` (mirrored in `legacy/scripts/config.py`). Domain-specific constants (fees, merged months) stay in their respective modules.
## Development Setup ## Development Setup
@@ -2,12 +2,12 @@
## Context ## Context
A bank payment from sender `JIŘÍ KUBÍK` with the message A bank payment from sender `JIŘÍ DVOŘÁK` with the message
`Jáchym Kubík: 01/2026+03/2026+04/2026` is being inferred as `Jáchym Dvořák: 01/2026+03/2026+04/2026` is being inferred as
`[?] Jáchym Hrušák (G)` instead of the obvious `Jáchym Kubík`, even though `[?] Jáchym Novotný (G)` instead of the obvious `Jáchym Dvořák`, even though
the message contains his exact full name. the message contains his exact full name.
**Root cause** (confirmed with the user): `Jáchym Kubík` is in the **junior** **Root cause** (confirmed with the user): `Jáchym Dvořák` is in the **junior**
attendance sheet only — he does not appear on the main/adults sheet. But attendance sheet only — he does not appear on the main/adults sheet. But
[scripts/infer_payments.py:101-102](scripts/infer_payments.py#L101-L102) [scripts/infer_payments.py:101-102](scripts/infer_payments.py#L101-L102)
builds `member_names` by calling `get_members_with_fees()` builds `member_names` by calling `get_members_with_fees()`
@@ -15,17 +15,17 @@ builds `member_names` by calling `get_members_with_fees()`
`EXPORT_URL` (the adults sheet). Junior-only members are therefore invisible `EXPORT_URL` (the adults sheet). Junior-only members are therefore invisible
to the matcher. to the matcher.
With Kubík absent from `member_names`, the matcher in With Dvořák absent from `member_names`, the matcher in
[scripts/match_payments.py:65](scripts/match_payments.py#L65) processes the [scripts/match_payments.py:65](scripts/match_payments.py#L65) processes the
combined text `jiri kubik jachym kubik: 01/2026+03/2026+04/2026` against an combined text `jiri dvorak jachym dvorak: 01/2026+03/2026+04/2026` against an
adults-only roster: adults-only roster:
- The exact-full-name short-circuit (`match_payments.py:75-84`) finds nothing — - The exact-full-name short-circuit (`match_payments.py:75-84`) finds nothing —
no adult's full name is in the text. no adult's full name is in the text.
- Hrušák `(G)` is the only adult with first name `Jáchym`. He fails the - Novotný `(G)` is the only adult with first name `Jáchym`. He fails the
auto-rules (his surname isn't in the text) but hits the partial-first-name auto-rules (his surname isn't in the text) but hits the partial-first-name
review rule (`match_payments.py:123-125`) → returned as `("Jáchym Hrušák (G)", review rule (`match_payments.py:123-125`) → returned as `("Jáchym Novotný (G)",
"review")`, rendered as `[?] Jáchym Hrušák (G)`. "review")`, rendered as `[?] Jáchym Novotný (G)`.
The user's original framing — "exact match in message should win over The user's original framing — "exact match in message should win over
everything" — is already implemented for any candidate that **is** in the everything" — is already implemented for any candidate that **is** in the
@@ -33,7 +33,7 @@ roster (the May-04 short-circuit). The bug is upstream: the right candidate
was never even considered. was never even considered.
**Goal:** make `infer_payments` consider junior members as candidates, so **Goal:** make `infer_payments` consider junior members as candidates, so
junior-only names like `Jáchym Kubík` get matched correctly. junior-only names like `Jáchym Dvořák` get matched correctly.
## Approach ## Approach
@@ -98,23 +98,23 @@ changes are needed.
the adults version, so `m[0]` works for both. the adults version, so `m[0]` works for both.
- [scripts/match_payments.py:65-137](scripts/match_payments.py#L65-L137) — - [scripts/match_payments.py:65-137](scripts/match_payments.py#L65-L137) —
`match_members` already handles the precedence the user wants (exact full-name `match_members` already handles the precedence the user wants (exact full-name
short-circuit), so once Kubík is in `member_names`, the case will be auto-matched short-circuit), so once Dvořák is in `member_names`, the case will be auto-matched
with no `[?]`. with no `[?]`.
## Verification ## Verification
1. **Manual sanity** — re-run inference on the offending row: 1. **Manual sanity** — re-run inference on the offending row:
- Clear `Person`/`Purpose` for the Kubík row in the payments sheet. - Clear `Person`/`Purpose` for the Dvořák row in the payments sheet.
- `make infer`. - `make infer`.
- Expect `Person = Jáchym Kubík`, `Purpose = 2026-01, 2026-03, 2026-04`, - Expect `Person = Jáchym Dvořák`, `Purpose = 2026-01, 2026-03, 2026-04`,
no `[?]`. no `[?]`.
2. **Unit test** — extend 2. **Unit test** — extend
[tests/test_match_members.py](tests/test_match_members.py) (or add a small [tests/test_match_members.py](tests/test_match_members.py) (or add a small
`tests/test_infer_payments.py`) to assert that, given a roster that `tests/test_infer_payments.py`) to assert that, given a roster that
includes `Jáchym Hrušák (G)` and `Jáchym Kubík`, the message includes `Jáchym Novotný (G)` and `Jáchym Dvořák`, the message
`Jáchym Kubík: 01/2026+03/2026+04/2026` resolves to `Jáchym Dvořák: 01/2026+03/2026+04/2026` resolves to
`[("Jáchym Kubík", "auto")]` only. This is really a regression test for `[("Jáchym Dvořák", "auto")]` only. This is really a regression test for
the May-04 short-circuit — the new behavior under test is just that the May-04 short-circuit — the new behavior under test is just that
`infer_payments` now feeds in juniors. `infer_payments` now feeds in juniors.
@@ -122,7 +122,7 @@ changes are needed.
4. **Dashboard smoke** — `make web`, open `/payments`, confirm the row now 4. **Dashboard smoke** — `make web`, open `/payments`, confirm the row now
shows the correct member; open `/juniors`, confirm the payment is shows the correct member; open `/juniors`, confirm the payment is
credited to Kubík for the three months listed. credited to Dvořák for the three months listed.
5. **Changelog** — once the user confirms the fix, append an entry to 5. **Changelog** — once the user confirms the fix, append an entry to
[CHANGELOG.md](CHANGELOG.md) per [CLAUDE.md](CLAUDE.md): [CHANGELOG.md](CHANGELOG.md) per [CLAUDE.md](CLAUDE.md):
@@ -58,7 +58,8 @@ before new feature work starts. Companion to the Go rewrite tracker,
current year. current year.
- Same applies to the `sync-2026` Makefile target. - Same applies to the `sync-2026` Makefile target.
- Moot if Python is retired before then. - Moot if Python is retired before then.
- [ ] **T4** Remove member names from tracked files (PII rule in `CLAUDE.md`). - [x] **T4** Remove member names from tracked files (PII rule in `CLAUDE.md`).
Done 2026-10-09 in a narrower form, see Notes & decisions. Original task:
- `CHANGELOG.md` line 29 names a member in a fix description. - `CHANGELOG.md` line 29 names a member in a fix description.
- `tests/test_match_payments.py`, `tests/test_reconcile_exceptions.py` and - `tests/test_match_payments.py`, `tests/test_reconcile_exceptions.py` and
`go/internal/domain/reconcile/reconcile_test.go` use names that may be `go/internal/domain/reconcile/reconcile_test.go` use names that may be
@@ -172,3 +173,16 @@ Plan: [2026-10-08-1103-api-parity-vs-months-to-show.md](2026-10-08-1103-api-pari
## Notes & decisions ## Notes & decisions
(Add entries as you go. Format: `YYYY-MM-DD — short note`.) (Add entries as you go. Format: `YYYY-MM-DD — short note`.)
- 2026-10-09 — T4 scope decided by the owner: a real name as a bare test
input is acceptable; a real name next to payment details, raw bank
messages, or any junior or parent name is not. Replaced the junior
regression-test names (test, changelog, plan) with fictional ones and
scrubbed three parity fixtures that still held raw bank messages. The
other test names stay. Git history was not rewritten. Rule recorded in
`CLAUDE.md`.
- 2026-10-09 — The `real_*` parse_month_references seeds pick rows from the
live payments cache by index (`_REAL_PMR_INDICES`). The sheet has changed
since capture, so `make capture-fixtures` now rewrites those fixtures with
different messages. Commit only the fixtures a change needs, or replace
the index-based seeds with fixed synthetic ones.
@@ -3,7 +3,7 @@
"func": "scripts.czech_utils.parse_month_references", "func": "scripts.czech_utils.parse_month_references",
"captured_at": "2026-05-06", "captured_at": "2026-05-06",
"input": { "input": {
"text": "Filip Halamka - prosinec, leden, unor", "text": "Member_607b748d - prosinec, leden, unor",
"default_year": 2026 "default_year": 2026
}, },
"output": { "output": {
@@ -3,7 +3,7 @@
"func": "scripts.czech_utils.parse_month_references", "func": "scripts.czech_utils.parse_month_references",
"captured_at": "2026-05-06", "captured_at": "2026-05-06",
"input": { "input": {
"text": "Jáchym Kubík: 01/2026+03/2026+04/2026", "text": "Member_71343342: 01/2026+03/2026+04/2026",
"default_year": 2026 "default_year": 2026
}, },
"output": { "output": {
@@ -3,7 +3,7 @@
"func": "scripts.czech_utils.parse_month_references", "func": "scripts.czech_utils.parse_month_references",
"captured_at": "2026-05-06", "captured_at": "2026-05-06",
"input": { "input": {
"text": "Martin Bolvansky Pausal Prosinec Leden", "text": "Member_51a8ec2d Pausal Prosinec Leden",
"default_year": 2026 "default_year": 2026
}, },
"output": { "output": {
+7 -7
View File
@@ -51,21 +51,21 @@ class TestMatchMembersExact(unittest.TestCase):
def test_shared_first_name_junior_in_roster_wins_exact(self): def test_shared_first_name_junior_in_roster_wins_exact(self):
# Regression: two members share first name "Jáchym"; message has full name # Regression: two members share first name "Jáchym"; message has full name
# of the junior-only member → exact match must win, no [?] on the adult. # of the junior-only member → exact match must win, no [?] on the adult.
roster = ["Jáchym Hrušák (G)", "Jáchym Kubík"] roster = ["Jáchym Novotný (G)", "Jáchym Dvořák"]
result = match_members( result = match_members(
"JIŘÍ KUBÍK Jáchym Kubík: 01/2026+03/2026+04/2026", roster "JIŘÍ DVOŘÁK Jáchym Dvořák: 01/2026+03/2026+04/2026", roster
) )
self.assertEqual(result, [("Jáchym Kubík", "auto")]) self.assertEqual(result, [("Jáchym Dvořák", "auto")])
def test_shared_first_name_without_junior_in_roster_falls_back(self): def test_shared_first_name_without_junior_in_roster_falls_back(self):
# Without Kubík in the roster (old behaviour), Hrušák wins via first-name # Without Dvořák in the roster (old behaviour), Novotný wins via first-name
# partial match — confirms the roster-expansion fix is the real solution. # partial match — confirms the roster-expansion fix is the real solution.
roster = ["Jáchym Hrušák (G)"] roster = ["Jáchym Novotný (G)"]
result = match_members( result = match_members(
"JIŘÍ KUBÍK Jáchym Kubík: 01/2026+03/2026+04/2026", roster "JIŘÍ DVOŘÁK Jáchym Dvořák: 01/2026+03/2026+04/2026", roster
) )
names = [r[0] for r in result] names = [r[0] for r in result]
self.assertIn("Jáchym Hrušák (G)", names) self.assertIn("Jáchym Novotný (G)", names)
if __name__ == "__main__": if __name__ == "__main__":