Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
e53b57f976 |
+8
-1
@@ -1,5 +1,12 @@
|
|||||||
# Changelog
|
# Changelog
|
||||||
|
|
||||||
|
## 2026-10-09 00:21 CEST — fix(pii): remove real member names tied to payments
|
||||||
|
|
||||||
|
- Three parity fixtures held raw bank messages with real names; the names are now `Member_<hash>` pseudonyms, as the scrubber produces.
|
||||||
|
- The junior regression test, its changelog entry and its plan file used a real junior's and parent's name with paid months; replaced with fictional names.
|
||||||
|
- `CLAUDE.md` now states the rule: a real name as a bare test input is fine, a real name next to payment details or any junior/parent name is not.
|
||||||
|
- Git history still contains the old text.
|
||||||
|
|
||||||
## 2026-10-08 21:27 CEST — feat(fees): junior rate for 2026-08 is 0
|
## 2026-10-08 21:27 CEST — feat(fees): junior rate for 2026-08 is 0
|
||||||
|
|
||||||
- Junior monthly rate for 2026-08: 500 → 0 CZK, same as adults.
|
- Junior monthly rate for 2026-08: 500 → 0 CZK, same as adults.
|
||||||
@@ -294,7 +301,7 @@
|
|||||||
## 2026-05-06 16:38 CEST — fix: include juniors in payment-inference roster
|
## 2026-05-06 16:38 CEST — fix: include juniors in payment-inference roster
|
||||||
|
|
||||||
- `scripts/infer_payments.py`: union adults + junior rosters so junior-only members are visible to the matcher.
|
- `scripts/infer_payments.py`: union adults + junior rosters so junior-only members are visible to the matcher.
|
||||||
- Root cause: `get_members_with_fees()` reads only the adults sheet; junior-only kids like Jáchym Kubík were absent from `member_names`, causing the exact-match short-circuit to never fire and a different adult sharing the first name to win via fuzzy review.
|
- Root cause: `get_members_with_fees()` reads only the adults sheet; junior-only kids like Jáchym Dvořák were absent from `member_names`, causing the exact-match short-circuit to never fire and a different adult sharing the first name to win via fuzzy review.
|
||||||
- Two regression tests added to `tests/test_match_members.py`.
|
- Two regression tests added to `tests/test_match_members.py`.
|
||||||
|
|
||||||
## 2026-05-06 16:05 CEST — feat(go/M2.10): port domain/reconcile.Reconcile
|
## 2026-05-06 16:05 CEST — feat(go/M2.10): port domain/reconcile.Reconcile
|
||||||
|
|||||||
@@ -29,6 +29,9 @@ These exist only in the Go app on purpose. Do not port them to the legacy Python
|
|||||||
## Key Constraints
|
## Key Constraints
|
||||||
|
|
||||||
- **PII separation**: Member data (names, emails, payment info) must never be committed to git. Enforce config/data separation from day one.
|
- **PII separation**: Member data (names, emails, payment info) must never be committed to git. Enforce config/data separation from day one.
|
||||||
|
- Tests may use a real member's name as a bare matching input (diacritics, nicknames, word order).
|
||||||
|
- Never commit a real name together with payment details or raw bank messages, and never commit names of juniors or their parents. Use fictional names there.
|
||||||
|
- Parity fixtures under `go/tests/fixtures/` must be scrubbed (`Member_<hash>`); check new or regenerated fixtures before committing.
|
||||||
- **Configuration**: External service IDs, credentials, and tunable parameters are centralized in `go/internal/config/config.go` (mirrored in `legacy/scripts/config.py`). Domain-specific constants (fees, merged months) stay in their respective modules.
|
- **Configuration**: External service IDs, credentials, and tunable parameters are centralized in `go/internal/config/config.go` (mirrored in `legacy/scripts/config.py`). Domain-specific constants (fees, merged months) stay in their respective modules.
|
||||||
|
|
||||||
## Development Setup
|
## Development Setup
|
||||||
|
|||||||
@@ -2,12 +2,12 @@
|
|||||||
|
|
||||||
## Context
|
## Context
|
||||||
|
|
||||||
A bank payment from sender `JIŘÍ KUBÍK` with the message
|
A bank payment from sender `JIŘÍ DVOŘÁK` with the message
|
||||||
`Jáchym Kubík: 01/2026+03/2026+04/2026` is being inferred as
|
`Jáchym Dvořák: 01/2026+03/2026+04/2026` is being inferred as
|
||||||
`[?] Jáchym Hrušák (G)` instead of the obvious `Jáchym Kubík`, even though
|
`[?] Jáchym Novotný (G)` instead of the obvious `Jáchym Dvořák`, even though
|
||||||
the message contains his exact full name.
|
the message contains his exact full name.
|
||||||
|
|
||||||
**Root cause** (confirmed with the user): `Jáchym Kubík` is in the **junior**
|
**Root cause** (confirmed with the user): `Jáchym Dvořák` is in the **junior**
|
||||||
attendance sheet only — he does not appear on the main/adults sheet. But
|
attendance sheet only — he does not appear on the main/adults sheet. But
|
||||||
[scripts/infer_payments.py:101-102](scripts/infer_payments.py#L101-L102)
|
[scripts/infer_payments.py:101-102](scripts/infer_payments.py#L101-L102)
|
||||||
builds `member_names` by calling `get_members_with_fees()`
|
builds `member_names` by calling `get_members_with_fees()`
|
||||||
@@ -15,17 +15,17 @@ builds `member_names` by calling `get_members_with_fees()`
|
|||||||
`EXPORT_URL` (the adults sheet). Junior-only members are therefore invisible
|
`EXPORT_URL` (the adults sheet). Junior-only members are therefore invisible
|
||||||
to the matcher.
|
to the matcher.
|
||||||
|
|
||||||
With Kubík absent from `member_names`, the matcher in
|
With Dvořák absent from `member_names`, the matcher in
|
||||||
[scripts/match_payments.py:65](scripts/match_payments.py#L65) processes the
|
[scripts/match_payments.py:65](scripts/match_payments.py#L65) processes the
|
||||||
combined text `jiri kubik jachym kubik: 01/2026+03/2026+04/2026` against an
|
combined text `jiri dvorak jachym dvorak: 01/2026+03/2026+04/2026` against an
|
||||||
adults-only roster:
|
adults-only roster:
|
||||||
|
|
||||||
- The exact-full-name short-circuit (`match_payments.py:75-84`) finds nothing —
|
- The exact-full-name short-circuit (`match_payments.py:75-84`) finds nothing —
|
||||||
no adult's full name is in the text.
|
no adult's full name is in the text.
|
||||||
- Hrušák `(G)` is the only adult with first name `Jáchym`. He fails the
|
- Novotný `(G)` is the only adult with first name `Jáchym`. He fails the
|
||||||
auto-rules (his surname isn't in the text) but hits the partial-first-name
|
auto-rules (his surname isn't in the text) but hits the partial-first-name
|
||||||
review rule (`match_payments.py:123-125`) → returned as `("Jáchym Hrušák (G)",
|
review rule (`match_payments.py:123-125`) → returned as `("Jáchym Novotný (G)",
|
||||||
"review")`, rendered as `[?] Jáchym Hrušák (G)`.
|
"review")`, rendered as `[?] Jáchym Novotný (G)`.
|
||||||
|
|
||||||
The user's original framing — "exact match in message should win over
|
The user's original framing — "exact match in message should win over
|
||||||
everything" — is already implemented for any candidate that **is** in the
|
everything" — is already implemented for any candidate that **is** in the
|
||||||
@@ -33,7 +33,7 @@ roster (the May-04 short-circuit). The bug is upstream: the right candidate
|
|||||||
was never even considered.
|
was never even considered.
|
||||||
|
|
||||||
**Goal:** make `infer_payments` consider junior members as candidates, so
|
**Goal:** make `infer_payments` consider junior members as candidates, so
|
||||||
junior-only names like `Jáchym Kubík` get matched correctly.
|
junior-only names like `Jáchym Dvořák` get matched correctly.
|
||||||
|
|
||||||
## Approach
|
## Approach
|
||||||
|
|
||||||
@@ -98,23 +98,23 @@ changes are needed.
|
|||||||
the adults version, so `m[0]` works for both.
|
the adults version, so `m[0]` works for both.
|
||||||
- [scripts/match_payments.py:65-137](scripts/match_payments.py#L65-L137) —
|
- [scripts/match_payments.py:65-137](scripts/match_payments.py#L65-L137) —
|
||||||
`match_members` already handles the precedence the user wants (exact full-name
|
`match_members` already handles the precedence the user wants (exact full-name
|
||||||
short-circuit), so once Kubík is in `member_names`, the case will be auto-matched
|
short-circuit), so once Dvořák is in `member_names`, the case will be auto-matched
|
||||||
with no `[?]`.
|
with no `[?]`.
|
||||||
|
|
||||||
## Verification
|
## Verification
|
||||||
|
|
||||||
1. **Manual sanity** — re-run inference on the offending row:
|
1. **Manual sanity** — re-run inference on the offending row:
|
||||||
- Clear `Person`/`Purpose` for the Kubík row in the payments sheet.
|
- Clear `Person`/`Purpose` for the Dvořák row in the payments sheet.
|
||||||
- `make infer`.
|
- `make infer`.
|
||||||
- Expect `Person = Jáchym Kubík`, `Purpose = 2026-01, 2026-03, 2026-04`,
|
- Expect `Person = Jáchym Dvořák`, `Purpose = 2026-01, 2026-03, 2026-04`,
|
||||||
no `[?]`.
|
no `[?]`.
|
||||||
|
|
||||||
2. **Unit test** — extend
|
2. **Unit test** — extend
|
||||||
[tests/test_match_members.py](tests/test_match_members.py) (or add a small
|
[tests/test_match_members.py](tests/test_match_members.py) (or add a small
|
||||||
`tests/test_infer_payments.py`) to assert that, given a roster that
|
`tests/test_infer_payments.py`) to assert that, given a roster that
|
||||||
includes `Jáchym Hrušák (G)` and `Jáchym Kubík`, the message
|
includes `Jáchym Novotný (G)` and `Jáchym Dvořák`, the message
|
||||||
`Jáchym Kubík: 01/2026+03/2026+04/2026` resolves to
|
`Jáchym Dvořák: 01/2026+03/2026+04/2026` resolves to
|
||||||
`[("Jáchym Kubík", "auto")]` only. This is really a regression test for
|
`[("Jáchym Dvořák", "auto")]` only. This is really a regression test for
|
||||||
the May-04 short-circuit — the new behavior under test is just that
|
the May-04 short-circuit — the new behavior under test is just that
|
||||||
`infer_payments` now feeds in juniors.
|
`infer_payments` now feeds in juniors.
|
||||||
|
|
||||||
@@ -122,7 +122,7 @@ changes are needed.
|
|||||||
|
|
||||||
4. **Dashboard smoke** — `make web`, open `/payments`, confirm the row now
|
4. **Dashboard smoke** — `make web`, open `/payments`, confirm the row now
|
||||||
shows the correct member; open `/juniors`, confirm the payment is
|
shows the correct member; open `/juniors`, confirm the payment is
|
||||||
credited to Kubík for the three months listed.
|
credited to Dvořák for the three months listed.
|
||||||
|
|
||||||
5. **Changelog** — once the user confirms the fix, append an entry to
|
5. **Changelog** — once the user confirms the fix, append an entry to
|
||||||
[CHANGELOG.md](CHANGELOG.md) per [CLAUDE.md](CLAUDE.md):
|
[CHANGELOG.md](CHANGELOG.md) per [CLAUDE.md](CLAUDE.md):
|
||||||
|
|||||||
@@ -58,7 +58,8 @@ before new feature work starts. Companion to the Go rewrite tracker,
|
|||||||
current year.
|
current year.
|
||||||
- Same applies to the `sync-2026` Makefile target.
|
- Same applies to the `sync-2026` Makefile target.
|
||||||
- Moot if Python is retired before then.
|
- Moot if Python is retired before then.
|
||||||
- [ ] **T4** Remove member names from tracked files (PII rule in `CLAUDE.md`).
|
- [x] **T4** Remove member names from tracked files (PII rule in `CLAUDE.md`).
|
||||||
|
Done 2026-10-09 in a narrower form, see Notes & decisions. Original task:
|
||||||
- `CHANGELOG.md` line 29 names a member in a fix description.
|
- `CHANGELOG.md` line 29 names a member in a fix description.
|
||||||
- `tests/test_match_payments.py`, `tests/test_reconcile_exceptions.py` and
|
- `tests/test_match_payments.py`, `tests/test_reconcile_exceptions.py` and
|
||||||
`go/internal/domain/reconcile/reconcile_test.go` use names that may be
|
`go/internal/domain/reconcile/reconcile_test.go` use names that may be
|
||||||
@@ -172,3 +173,16 @@ Plan: [2026-10-08-1103-api-parity-vs-months-to-show.md](2026-10-08-1103-api-pari
|
|||||||
## Notes & decisions
|
## Notes & decisions
|
||||||
|
|
||||||
(Add entries as you go. Format: `YYYY-MM-DD — short note`.)
|
(Add entries as you go. Format: `YYYY-MM-DD — short note`.)
|
||||||
|
|
||||||
|
- 2026-10-09 — T4 scope decided by the owner: a real name as a bare test
|
||||||
|
input is acceptable; a real name next to payment details, raw bank
|
||||||
|
messages, or any junior or parent name is not. Replaced the junior
|
||||||
|
regression-test names (test, changelog, plan) with fictional ones and
|
||||||
|
scrubbed three parity fixtures that still held raw bank messages. The
|
||||||
|
other test names stay. Git history was not rewritten. Rule recorded in
|
||||||
|
`CLAUDE.md`.
|
||||||
|
- 2026-10-09 — The `real_*` parse_month_references seeds pick rows from the
|
||||||
|
live payments cache by index (`_REAL_PMR_INDICES`). The sheet has changed
|
||||||
|
since capture, so `make capture-fixtures` now rewrites those fixtures with
|
||||||
|
different messages. Commit only the fixtures a change needs, or replace
|
||||||
|
the index-based seeds with fixed synthetic ones.
|
||||||
|
|||||||
+1
-1
@@ -3,7 +3,7 @@
|
|||||||
"func": "scripts.czech_utils.parse_month_references",
|
"func": "scripts.czech_utils.parse_month_references",
|
||||||
"captured_at": "2026-05-06",
|
"captured_at": "2026-05-06",
|
||||||
"input": {
|
"input": {
|
||||||
"text": "Filip Halamka - prosinec, leden, unor",
|
"text": "Member_607b748d - prosinec, leden, unor",
|
||||||
"default_year": 2026
|
"default_year": 2026
|
||||||
},
|
},
|
||||||
"output": {
|
"output": {
|
||||||
|
|||||||
+1
-1
@@ -3,7 +3,7 @@
|
|||||||
"func": "scripts.czech_utils.parse_month_references",
|
"func": "scripts.czech_utils.parse_month_references",
|
||||||
"captured_at": "2026-05-06",
|
"captured_at": "2026-05-06",
|
||||||
"input": {
|
"input": {
|
||||||
"text": "Jáchym Kubík: 01/2026+03/2026+04/2026",
|
"text": "Member_71343342: 01/2026+03/2026+04/2026",
|
||||||
"default_year": 2026
|
"default_year": 2026
|
||||||
},
|
},
|
||||||
"output": {
|
"output": {
|
||||||
|
|||||||
+1
-1
@@ -3,7 +3,7 @@
|
|||||||
"func": "scripts.czech_utils.parse_month_references",
|
"func": "scripts.czech_utils.parse_month_references",
|
||||||
"captured_at": "2026-05-06",
|
"captured_at": "2026-05-06",
|
||||||
"input": {
|
"input": {
|
||||||
"text": "Martin Bolvansky Pausal Prosinec Leden",
|
"text": "Member_51a8ec2d Pausal Prosinec Leden",
|
||||||
"default_year": 2026
|
"default_year": 2026
|
||||||
},
|
},
|
||||||
"output": {
|
"output": {
|
||||||
|
|||||||
@@ -51,21 +51,21 @@ class TestMatchMembersExact(unittest.TestCase):
|
|||||||
def test_shared_first_name_junior_in_roster_wins_exact(self):
|
def test_shared_first_name_junior_in_roster_wins_exact(self):
|
||||||
# Regression: two members share first name "Jáchym"; message has full name
|
# Regression: two members share first name "Jáchym"; message has full name
|
||||||
# of the junior-only member → exact match must win, no [?] on the adult.
|
# of the junior-only member → exact match must win, no [?] on the adult.
|
||||||
roster = ["Jáchym Hrušák (G)", "Jáchym Kubík"]
|
roster = ["Jáchym Novotný (G)", "Jáchym Dvořák"]
|
||||||
result = match_members(
|
result = match_members(
|
||||||
"JIŘÍ KUBÍK Jáchym Kubík: 01/2026+03/2026+04/2026", roster
|
"JIŘÍ DVOŘÁK Jáchym Dvořák: 01/2026+03/2026+04/2026", roster
|
||||||
)
|
)
|
||||||
self.assertEqual(result, [("Jáchym Kubík", "auto")])
|
self.assertEqual(result, [("Jáchym Dvořák", "auto")])
|
||||||
|
|
||||||
def test_shared_first_name_without_junior_in_roster_falls_back(self):
|
def test_shared_first_name_without_junior_in_roster_falls_back(self):
|
||||||
# Without Kubík in the roster (old behaviour), Hrušák wins via first-name
|
# Without Dvořák in the roster (old behaviour), Novotný wins via first-name
|
||||||
# partial match — confirms the roster-expansion fix is the real solution.
|
# partial match — confirms the roster-expansion fix is the real solution.
|
||||||
roster = ["Jáchym Hrušák (G)"]
|
roster = ["Jáchym Novotný (G)"]
|
||||||
result = match_members(
|
result = match_members(
|
||||||
"JIŘÍ KUBÍK Jáchym Kubík: 01/2026+03/2026+04/2026", roster
|
"JIŘÍ DVOŘÁK Jáchym Dvořák: 01/2026+03/2026+04/2026", roster
|
||||||
)
|
)
|
||||||
names = [r[0] for r in result]
|
names = [r[0] for r in result]
|
||||||
self.assertIn("Jáchym Hrušák (G)", names)
|
self.assertIn("Jáchym Novotný (G)", names)
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
|
|||||||
Reference in New Issue
Block a user