Merge pull request 'feat(infer): messages starting with "other:" are not monthly fees' (#49) from feat/infer-other-purpose into main
Reviewed-on: #49
This commit was merged in pull request #49.
This commit is contained in:
@@ -1,5 +1,14 @@
|
|||||||
# Changelog
|
# Changelog
|
||||||
|
|
||||||
|
## 2026-10-08 12:37 CEST — feat(infer): messages starting with "other:" are not monthly fees
|
||||||
|
|
||||||
|
- When the bank message starts with `other:` (case-insensitive), inference no longer looks for a month. It copies the trimmed message into Purpose, so the payment is linked to the person as an "other" transaction and not counted as a monthly fee. Messages that contain `other:` elsewhere are inferred as before.
|
||||||
|
- If no person is found, Purpose is still written and the row shows under unmatched for a manual fix.
|
||||||
|
- Reconcile applies the same rule to rows not yet processed by inference.
|
||||||
|
- Existing rows are not re-inferred; clear Person and Purpose on a row to reprocess it.
|
||||||
|
- Go and Python: `matching.IsOtherPurpose` / `is_other_purpose`, `InferTransactionDetails` / `infer_transaction_details`, reconcile fallback; four new parity fixtures.
|
||||||
|
- Also fixed (Go): junior cell tooltip showed `Expected: 0` instead of `Expected: ?` when a month with unknown expected fee had a payment.
|
||||||
|
|
||||||
## 2026-10-08 12:16 CEST — feat(go): scheduled bank sync (Go only)
|
## 2026-10-08 12:16 CEST — feat(go): scheduled bank sync (Go only)
|
||||||
|
|
||||||
- The Go server now runs the bank sync (sync → infer → flush cache) by itself at the times in `SYNC_SCHEDULE` (comma-separated `HH:MM`, default `04:00`, `off` disables), read in `SYNC_TZ` (default `Europe/Prague`). Nothing runs at startup; an invalid value stops the server at startup with an error.
|
- The Go server now runs the bank sync (sync → infer → flush cache) by itself at the times in `SYNC_SCHEDULE` (comma-separated `HH:MM`, default `04:00`, `off` disables), read in `SYNC_TZ` (default `Europe/Prague`). Nothing runs at startup; an invalid value stops the server at startup with an error.
|
||||||
|
|||||||
@@ -164,6 +164,8 @@ Both implementations check the Drive API `modifiedTime` and serve a cached JSON
|
|||||||
|
|
||||||
`Person` and `Purpose` are written by the inference step (`fuj infer` / `infer_payments.py`) and can be manually corrected. `manual fix` column presence disables re-inference for that row. Multiple people or months are comma-separated in Person/Purpose.
|
`Person` and `Purpose` are written by the inference step (`fuj infer` / `infer_payments.py`) and can be manually corrected. `manual fix` column presence disables re-inference for that row. Multiple people or months are comma-separated in Person/Purpose.
|
||||||
|
|
||||||
|
**Non-fee payments ("other")**: a Purpose that starts with `other:` (case-insensitive) is attached to the person's "other transactions" and is not counted against any month. Inference produces this by itself when the bank message starts with `other:`: it copies the trimmed message into Purpose and does not look for a month (neither in the text nor from the date). A message that only contains `other:` later in the text is inferred the normal way. The shared check is `matching.IsOtherPurpose` / `is_other_purpose`.
|
||||||
|
|
||||||
### QR codes
|
### QR codes
|
||||||
|
|
||||||
`/qr?account=…&amount=…&message=…` generates a Czech QR Platba PNG (SPD format).
|
`/qr?account=…&amount=…&message=…` generates a Czech QR Platba PNG (SPD format).
|
||||||
|
|||||||
@@ -0,0 +1,109 @@
|
|||||||
|
# Inference: messages starting with "other:" are not monthly fees
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
Some payments are not monthly fees (tournament fees, jerseys, …). Reconcile
|
||||||
|
already supports them: a row whose **Purpose** starts with `other:` is attached
|
||||||
|
to the person's "other transactions" and is not counted against any month.
|
||||||
|
Today that only happens when someone types the purpose by hand, because
|
||||||
|
inference always tries to find a month — from the message, or failing that
|
||||||
|
from the payment date.
|
||||||
|
|
||||||
|
New rule: when the bank message for the recipient **starts with `other:`**,
|
||||||
|
inference must not look for a month. It copies the message into Purpose, so
|
||||||
|
the payment is linked to the person but not counted as a monthly fee. Messages
|
||||||
|
that merely contain `other:` somewhere else are handled the old way.
|
||||||
|
|
||||||
|
Implemented in both Go and legacy Python (this is inference behaviour, so the
|
||||||
|
dual implementation rule applies).
|
||||||
|
|
||||||
|
## Behaviour
|
||||||
|
|
||||||
|
- Trigger: message, after trimming whitespace, starts with `other:`
|
||||||
|
(case-insensitive, same test reconcile already uses for Purpose).
|
||||||
|
- Purpose written to the sheet: the trimmed message, unchanged
|
||||||
|
(e.g. `other: turnaj Praha`). No month parsing, no date fallback.
|
||||||
|
- Person: inferred exactly as today (sender + message + VS, then sender
|
||||||
|
alone), including the `[?]` review marker.
|
||||||
|
- Inferred Amount: the full amount, as today.
|
||||||
|
- No person found: Purpose is still written, Person stays empty. The row then
|
||||||
|
shows up under "unmatched" for a manual fix instead of being guessed again
|
||||||
|
on every run.
|
||||||
|
- Rows not yet processed by inference (Person and Purpose both empty) are
|
||||||
|
inferred on the fly by reconcile; that path follows the same rule, so such a
|
||||||
|
payment is never briefly counted as a monthly fee.
|
||||||
|
- Existing rows are untouched: inference only fills rows where manual fix,
|
||||||
|
Person and Purpose are all empty. To reprocess an old row, clear its Person
|
||||||
|
and Purpose cells.
|
||||||
|
|
||||||
|
## Implementation
|
||||||
|
|
||||||
|
Branch `feat/infer-other-purpose` off `main`; MR via `tea`.
|
||||||
|
|
||||||
|
The output shape of the inference function does not change: for an "other"
|
||||||
|
message the months list holds the single purpose string. That is the same
|
||||||
|
convention reconcile already uses internally (`matchedMonths = [purpose]`),
|
||||||
|
and it means the sheet-writing code needs no change in either language.
|
||||||
|
|
||||||
|
1. **Shared predicate**
|
||||||
|
- Go: `matching.IsOtherPurpose(s string) bool` in
|
||||||
|
`go/internal/domain/matching/infer.go`
|
||||||
|
(`strings.HasPrefix(strings.ToLower(strings.TrimSpace(s)), "other:")`).
|
||||||
|
- Python: `is_other_purpose(s)` in `legacy/scripts/match_payments.py`.
|
||||||
|
- Replace the two existing inline checks in reconcile with it
|
||||||
|
(`go/internal/domain/reconcile/reconcile.go`, `reconcile()` in
|
||||||
|
`match_payments.py`).
|
||||||
|
|
||||||
|
2. **Inference**
|
||||||
|
- Go `matching.InferTransactionDetails` and Python
|
||||||
|
`infer_transaction_details`: after member matching, if the message is an
|
||||||
|
other-purpose, return `months = [trimmed message]` and skip
|
||||||
|
`ParseMonthReferences` and the date fallback. Update the doc comments
|
||||||
|
(months may hold one `other:…` purpose).
|
||||||
|
- `banksync.InferPayments` (`go/internal/services/banksync/infer.go`) and
|
||||||
|
`infer_payments.py`: no logic change — they already join months into
|
||||||
|
Purpose and write the row when either members or months are non-empty.
|
||||||
|
|
||||||
|
3. **Reconcile fallback** (rows with empty Person/Purpose)
|
||||||
|
- In the inference-fallback branch of both reconcile implementations: if
|
||||||
|
the inferred months are a single other-purpose, set the row's purpose to
|
||||||
|
it and mark it as other, so it goes to the member's other-transactions
|
||||||
|
ledger instead of the month allocation.
|
||||||
|
|
||||||
|
4. **Tests and fixtures**
|
||||||
|
- Seeds in `legacy/scripts/_fixture_seeds.py`, captured with the existing
|
||||||
|
capture/scrub scripts for these two functions only:
|
||||||
|
- `infer_transaction_details`: `other_prefix` (message
|
||||||
|
`other: tournament`, member in sender → that purpose, no month),
|
||||||
|
`other_prefix_uppercase` (`Other: …`), `other_not_at_start`
|
||||||
|
(`leden other: x` → old behaviour, month parsed).
|
||||||
|
- `reconcile`: `11_other_message_fallback` (empty Person/Purpose, message
|
||||||
|
starts with `other:` → lands in `other_transactions`, month stays
|
||||||
|
unpaid).
|
||||||
|
- Go unit tests: `IsOtherPurpose` table; cases in
|
||||||
|
`matching/infer_test.go`; a case in `banksync/infer_test.go` asserting
|
||||||
|
the written row (`Person`, `Purpose = "other: …"`, amount) and one with
|
||||||
|
no matching member; a fallback case in `reconcile/reconcile_test.go`.
|
||||||
|
- Python: cases in `legacy/tests/test_match_payments.py` for
|
||||||
|
`infer_transaction_details` and the reconcile fallback.
|
||||||
|
- Existing fixtures must stay byte-identical apart from the new files.
|
||||||
|
|
||||||
|
5. **Docs**
|
||||||
|
- `CLAUDE.md`: add the rule to the payments-sheet / inference description.
|
||||||
|
- `CHANGELOG.md` entry; copy this plan to
|
||||||
|
`docs/plans/2026-10-08-HHMM-infer-other-purpose.md`.
|
||||||
|
|
||||||
|
No change to the JSON API, schemas, templates or the sheet layout.
|
||||||
|
|
||||||
|
## Verification
|
||||||
|
|
||||||
|
1. `make go-test-all` (unit + parity, including the new fixtures) and
|
||||||
|
`make test`.
|
||||||
|
2. `git status go/tests/fixtures` shows only the four new fixture files.
|
||||||
|
3. Dry run against the live sheet in both implementations
|
||||||
|
(`fuj infer --dry-run`, Python `infer_payments.py --dry-run`): confirm it
|
||||||
|
runs cleanly and proposes nothing unexpected. The live sheet probably has
|
||||||
|
no un-inferred row starting with `other:`, so the new rule itself is
|
||||||
|
proven by the tests rather than by live data.
|
||||||
|
4. `make parity` between the two servers (or the two-container run used
|
||||||
|
earlier) still reports 3/3 routes matching.
|
||||||
@@ -3,6 +3,7 @@ package matching
|
|||||||
import (
|
import (
|
||||||
"fmt"
|
"fmt"
|
||||||
"fuj-management/go/internal/domain/czech"
|
"fuj-management/go/internal/domain/czech"
|
||||||
|
"strings"
|
||||||
"time"
|
"time"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -17,12 +18,22 @@ type Transaction struct {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// InferredDetails is the result of InferTransactionDetails.
|
// InferredDetails is the result of InferTransactionDetails.
|
||||||
|
// Months holds "YYYY-MM" keys, or exactly one "other:…" purpose when the
|
||||||
|
// payment is not a monthly fee (see IsOtherPurpose).
|
||||||
type InferredDetails struct {
|
type InferredDetails struct {
|
||||||
Members []Match
|
Members []Match
|
||||||
Months []string
|
Months []string
|
||||||
SearchText string
|
SearchText string
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// IsOtherPurpose reports whether s marks a payment that is not a monthly fee:
|
||||||
|
// it starts with "other:" (case-insensitive, leading whitespace ignored).
|
||||||
|
// Used for both the Purpose column and the bank message.
|
||||||
|
// Ports legacy/scripts/match_payments.py is_other_purpose.
|
||||||
|
func IsOtherPurpose(s string) bool {
|
||||||
|
return strings.HasPrefix(strings.ToLower(strings.TrimSpace(s)), "other:")
|
||||||
|
}
|
||||||
|
|
||||||
// InferTransactionDetails infers which member(s) and month(s) a transaction belongs to.
|
// InferTransactionDetails infers which member(s) and month(s) a transaction belongs to.
|
||||||
//
|
//
|
||||||
// Search text for member matching: sender + message + user_id.
|
// Search text for member matching: sender + message + user_id.
|
||||||
@@ -30,6 +41,10 @@ type InferredDetails struct {
|
|||||||
// Fallback 1: if no members found, retry match on sender alone.
|
// Fallback 1: if no members found, retry match on sender alone.
|
||||||
// Fallback 2: if no months found, derive from tx.Date (Sheets serial or YYYY-MM-DD).
|
// Fallback 2: if no months found, derive from tx.Date (Sheets serial or YYYY-MM-DD).
|
||||||
//
|
//
|
||||||
|
// A message that starts with "other:" is not a monthly fee: no month is
|
||||||
|
// inferred (neither from the text nor from the date) and Months holds the
|
||||||
|
// trimmed message as the single purpose.
|
||||||
|
//
|
||||||
// defaultYear seeds czech.ParseMonthReferences (Python defaulted to the current year;
|
// defaultYear seeds czech.ParseMonthReferences (Python defaulted to the current year;
|
||||||
// callers should pass time.Now().Year() or a fixed year for deterministic tests).
|
// callers should pass time.Now().Year() or a fixed year for deterministic tests).
|
||||||
//
|
//
|
||||||
@@ -38,12 +53,20 @@ func InferTransactionDetails(tx Transaction, memberNames []string, defaultYear i
|
|||||||
searchText := fmt.Sprintf("%s %s %s", tx.Sender, tx.Message, tx.UserID)
|
searchText := fmt.Sprintf("%s %s %s", tx.Sender, tx.Message, tx.UserID)
|
||||||
|
|
||||||
members := MatchMembers(searchText, memberNames)
|
members := MatchMembers(searchText, memberNames)
|
||||||
months := czech.ParseMonthReferences(tx.Message+" "+tx.UserID, defaultYear)
|
|
||||||
|
|
||||||
if len(members) == 0 {
|
if len(members) == 0 {
|
||||||
members = MatchMembers(tx.Sender, memberNames)
|
members = MatchMembers(tx.Sender, memberNames)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if IsOtherPurpose(tx.Message) {
|
||||||
|
return InferredDetails{
|
||||||
|
Members: members,
|
||||||
|
Months: []string{strings.TrimSpace(tx.Message)},
|
||||||
|
SearchText: searchText,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
months := czech.ParseMonthReferences(tx.Message+" "+tx.UserID, defaultYear)
|
||||||
|
|
||||||
if len(months) == 0 && tx.Date != nil && tx.Date != "" {
|
if len(months) == 0 && tx.Date != nil && tx.Date != "" {
|
||||||
if ym := inferMonthFromDate(tx.Date); ym != "" {
|
if ym := inferMonthFromDate(tx.Date); ym != "" {
|
||||||
months = []string{ym}
|
months = []string{ym}
|
||||||
|
|||||||
@@ -32,6 +32,74 @@ import (
|
|||||||
|
|
||||||
var inferMembers = []string{"Tomáš Němeček (Tov)", "Jana Nováková"}
|
var inferMembers = []string{"Tomáš Němeček (Tov)", "Jana Nováková"}
|
||||||
|
|
||||||
|
func TestIsOtherPurpose(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
cases := map[string]bool{
|
||||||
|
"other: tournament": true,
|
||||||
|
"other:shirt": true,
|
||||||
|
" Other: Tournament": true,
|
||||||
|
"OTHER:x": true,
|
||||||
|
"leden other: x": false,
|
||||||
|
"other tournament": false,
|
||||||
|
"another: x": false,
|
||||||
|
"2026-01": false,
|
||||||
|
"": false,
|
||||||
|
}
|
||||||
|
for in, want := range cases {
|
||||||
|
if got := IsOtherPurpose(in); got != want {
|
||||||
|
t.Errorf("IsOtherPurpose(%q) = %v, want %v", in, got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestInferTransactionDetails_OtherMessage(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
jana := []Match{{Name: "Jana Nováková", Confidence: ConfidenceAuto}}
|
||||||
|
cases := []struct {
|
||||||
|
name string
|
||||||
|
tx Transaction
|
||||||
|
wantMembers []Match
|
||||||
|
wantMonths []string
|
||||||
|
}{
|
||||||
|
{
|
||||||
|
// Month words and the date must both be ignored.
|
||||||
|
name: "other prefix: message becomes the purpose, no month",
|
||||||
|
tx: Transaction{Sender: "Jana Novakova", Message: " other: turnaj leden 2026 ", Date: "2026-04-15"},
|
||||||
|
wantMembers: jana,
|
||||||
|
wantMonths: []string{"other: turnaj leden 2026"},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "other prefix is case-insensitive and keeps the original text",
|
||||||
|
tx: Transaction{Sender: "Jana Novakova", Message: "Other: Dres", Date: "2026-04-15"},
|
||||||
|
wantMembers: jana,
|
||||||
|
wantMonths: []string{"Other: Dres"},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "other prefix with no known member still yields the purpose",
|
||||||
|
tx: Transaction{Sender: "Unknown Payer", Message: "other: dres", Date: "2026-04-15"},
|
||||||
|
wantMembers: nil,
|
||||||
|
wantMonths: []string{"other: dres"},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "other: not at the start is handled the old way",
|
||||||
|
tx: Transaction{Sender: "Jana Novakova", Message: "leden other: dres", Date: "2026-04-15"},
|
||||||
|
wantMembers: jana,
|
||||||
|
wantMonths: []string{"2026-01"},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
for _, tc := range cases {
|
||||||
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
|
got := InferTransactionDetails(tc.tx, inferMembers, 2026)
|
||||||
|
if len(got.Members) != len(tc.wantMembers) || (len(got.Members) > 0 && !reflect.DeepEqual(got.Members, tc.wantMembers)) {
|
||||||
|
t.Errorf("Members = %v, want %v", got.Members, tc.wantMembers)
|
||||||
|
}
|
||||||
|
if !reflect.DeepEqual(got.Months, tc.wantMonths) {
|
||||||
|
t.Errorf("Months = %v, want %v", got.Months, tc.wantMonths)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestInferTransactionDetails(t *testing.T) {
|
func TestInferTransactionDetails(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
|
|||||||
@@ -203,7 +203,7 @@ func Reconcile(
|
|||||||
personStr := strings.TrimSpace(tx.Person)
|
personStr := strings.TrimSpace(tx.Person)
|
||||||
purposeStr := strings.TrimSpace(tx.Purpose)
|
purposeStr := strings.TrimSpace(tx.Purpose)
|
||||||
personStr = questionMarkRe.ReplaceAllString(personStr, "")
|
personStr = questionMarkRe.ReplaceAllString(personStr, "")
|
||||||
isOther := strings.HasPrefix(strings.ToLower(purposeStr), "other:")
|
isOther := matching.IsOtherPurpose(purposeStr)
|
||||||
|
|
||||||
var matchedMembers []matching.Match
|
var matchedMembers []matching.Match
|
||||||
var matchedMonths []string
|
var matchedMonths []string
|
||||||
@@ -249,6 +249,11 @@ func Reconcile(
|
|||||||
matchedMembers = inferred.Members
|
matchedMembers = inferred.Members
|
||||||
matchedMonths = inferred.Months
|
matchedMonths = inferred.Months
|
||||||
amount = tx.Amount
|
amount = tx.Amount
|
||||||
|
// A message starting with "other:" is inferred as a non-fee payment.
|
||||||
|
if len(matchedMonths) == 1 && matching.IsOtherPurpose(matchedMonths[0]) {
|
||||||
|
isOther = true
|
||||||
|
purposeStr = matchedMonths[0]
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
if len(matchedMembers) == 0 || len(matchedMonths) == 0 {
|
if len(matchedMembers) == 0 || len(matchedMonths) == 0 {
|
||||||
|
|||||||
@@ -257,6 +257,33 @@ func TestReconcileQuestionMarkMarkerStripped(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// [Go] Purpose "other:shirt" puts payment in OtherTransactions, not in month ledger.
|
// [Go] Purpose "other:shirt" puts payment in OtherTransactions, not in month ledger.
|
||||||
|
// A row not yet processed by inference (empty Person/Purpose) whose message
|
||||||
|
// starts with "other:" must land in the other ledger, not in a month.
|
||||||
|
func TestReconcileOtherMessageFallback(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
members := []Member{{Name: "Alice", Tier: "A", Fees: map[string]FeeData{"2026-01": {Expected: 750, Attendance: 3}}}}
|
||||||
|
txs := []Transaction{{
|
||||||
|
Date: "2026-01-10", Amount: 300,
|
||||||
|
Sender: "Alice", Message: "other: shirt",
|
||||||
|
}}
|
||||||
|
|
||||||
|
result := Reconcile(members, []string{"2026-01"}, txs, nil, 2026)
|
||||||
|
|
||||||
|
alice := result.Members["Alice"]
|
||||||
|
if paid := alice.Months["2026-01"].Paid; paid != 0 {
|
||||||
|
t.Errorf("2026-01 paid = %v, want 0", paid)
|
||||||
|
}
|
||||||
|
if len(alice.OtherTransactions) != 1 {
|
||||||
|
t.Fatalf("other transactions = %d, want 1", len(alice.OtherTransactions))
|
||||||
|
}
|
||||||
|
if got := alice.OtherTransactions[0]; got.Amount != 300 || got.Purpose != "other: shirt" {
|
||||||
|
t.Errorf("other entry = %+v", got)
|
||||||
|
}
|
||||||
|
if len(result.Unmatched) != 0 {
|
||||||
|
t.Errorf("unmatched = %d, want 0", len(result.Unmatched))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestReconcileOtherPurpose(t *testing.T) {
|
func TestReconcileOtherPurpose(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
members := []Member{{Name: "Alice", Tier: "A", Fees: map[string]FeeData{"2026-01": {Expected: 750, Attendance: 3}}}}
|
members := []Member{{Name: "Alice", Tier: "A", Fees: map[string]FeeData{"2026-01": {Expected: 750, Attendance: 3}}}}
|
||||||
|
|||||||
@@ -53,6 +53,42 @@ func TestInferPayments_BasicMatch(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestInferPayments_OtherMessage(t *testing.T) {
|
||||||
|
sh := &sheets.Fake{Values: map[string][][]any{
|
||||||
|
"SHEETID/A1:Z": {
|
||||||
|
paymentsHeader,
|
||||||
|
// Known member: Person filled, message copied to Purpose, no month.
|
||||||
|
{"2026-04-10", 300.0, "", "", "", "", "Jana Novakova", "", "other: dres", "", ""},
|
||||||
|
// Unknown sender: Purpose is still written, Person stays empty.
|
||||||
|
{"2026-04-11", 500.0, "", "", "", "", "Unknown Payer", "", "Other: turnaj", "", ""},
|
||||||
|
},
|
||||||
|
}}
|
||||||
|
att := &fakeAttendance{
|
||||||
|
adults: []reconcile.Member{{Name: "Jana Novakova", Tier: "A"}},
|
||||||
|
}
|
||||||
|
|
||||||
|
n, err := InferPayments(context.Background(), "SHEETID", sh, att, InferOpts{})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if n != 2 {
|
||||||
|
t.Fatalf("want 2 rows updated, got %d", n)
|
||||||
|
}
|
||||||
|
updates := sh.BatchUpdated[0].Updates
|
||||||
|
want := [][]any{
|
||||||
|
{"Jana Novakova", "other: dres", "300"},
|
||||||
|
{"", "Other: turnaj", "500"},
|
||||||
|
}
|
||||||
|
for i, w := range want {
|
||||||
|
got := updates[i].Values[0]
|
||||||
|
for j := range w {
|
||||||
|
if got[j] != w[j] {
|
||||||
|
t.Errorf("row %d col %d: want %q, got %q", i, j, w[j], got[j])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestInferPayments_SkipRule_ManualFix(t *testing.T) {
|
func TestInferPayments_SkipRule_ManualFix(t *testing.T) {
|
||||||
sh := &sheets.Fake{Values: map[string][][]any{
|
sh := &sheets.Fake{Values: map[string][][]any{
|
||||||
"SHEETID/A1:Z": {
|
"SHEETID/A1:Z": {
|
||||||
|
|||||||
@@ -242,7 +242,12 @@ func buildJuniorMemberRow(
|
|||||||
|
|
||||||
tooltip := ""
|
tooltip := ""
|
||||||
if (!md.IsUnknown && md.Expected > 0) || paid > 0 {
|
if (!md.IsUnknown && md.Expected > 0) || paid > 0 {
|
||||||
tooltip = fmt.Sprintf("Received: %d, Expected: %d", paid, md.Expected)
|
// Unknown expected (one session, manual review) shows as "?", as in Python.
|
||||||
|
expectedText := strconv.Itoa(md.Expected)
|
||||||
|
if md.IsUnknown {
|
||||||
|
expectedText = "?"
|
||||||
|
}
|
||||||
|
tooltip = fmt.Sprintf("Received: %d, Expected: %s", paid, expectedText)
|
||||||
}
|
}
|
||||||
|
|
||||||
row.Months = append(row.Months, MonthCell{
|
row.Months = append(row.Months, MonthCell{
|
||||||
|
|||||||
@@ -0,0 +1,36 @@
|
|||||||
|
{
|
||||||
|
"case": "other_not_at_start",
|
||||||
|
"func": "scripts.match_payments.infer_transaction_details",
|
||||||
|
"captured_at": "2026-10-08",
|
||||||
|
"input": {
|
||||||
|
"tx": {
|
||||||
|
"sender": "Jana Novakova",
|
||||||
|
"message": "leden other: dres",
|
||||||
|
"user_id": "",
|
||||||
|
"date": {
|
||||||
|
"type": "string",
|
||||||
|
"value": "2026-04-15"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"member_names": [
|
||||||
|
"Alžběta Testovická",
|
||||||
|
"Tomáš Fiktivný (Tov)",
|
||||||
|
"Pavel Smutný (Štrúdl)",
|
||||||
|
"Jana Nováková",
|
||||||
|
"Adam Novák"
|
||||||
|
],
|
||||||
|
"default_year": 2026
|
||||||
|
},
|
||||||
|
"output": {
|
||||||
|
"matches": [
|
||||||
|
{
|
||||||
|
"name": "Jana Nováková",
|
||||||
|
"confidence": "auto"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"months": [
|
||||||
|
"2026-01"
|
||||||
|
],
|
||||||
|
"search_text": "Jana Novakova leden other: dres "
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,36 @@
|
|||||||
|
{
|
||||||
|
"case": "other_prefix",
|
||||||
|
"func": "scripts.match_payments.infer_transaction_details",
|
||||||
|
"captured_at": "2026-10-08",
|
||||||
|
"input": {
|
||||||
|
"tx": {
|
||||||
|
"sender": "Jana Novakova",
|
||||||
|
"message": "other: tournament leden 2026",
|
||||||
|
"user_id": "",
|
||||||
|
"date": {
|
||||||
|
"type": "string",
|
||||||
|
"value": "2026-04-15"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"member_names": [
|
||||||
|
"Alžběta Testovická",
|
||||||
|
"Tomáš Fiktivný (Tov)",
|
||||||
|
"Pavel Smutný (Štrúdl)",
|
||||||
|
"Jana Nováková",
|
||||||
|
"Adam Novák"
|
||||||
|
],
|
||||||
|
"default_year": 2026
|
||||||
|
},
|
||||||
|
"output": {
|
||||||
|
"matches": [
|
||||||
|
{
|
||||||
|
"name": "Jana Nováková",
|
||||||
|
"confidence": "auto"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"months": [
|
||||||
|
"other: tournament leden 2026"
|
||||||
|
],
|
||||||
|
"search_text": "Jana Novakova other: tournament leden 2026 "
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,31 @@
|
|||||||
|
{
|
||||||
|
"case": "other_prefix_uppercase",
|
||||||
|
"func": "scripts.match_payments.infer_transaction_details",
|
||||||
|
"captured_at": "2026-10-08",
|
||||||
|
"input": {
|
||||||
|
"tx": {
|
||||||
|
"sender": "Test Payer",
|
||||||
|
"message": " Other: Dres ",
|
||||||
|
"user_id": "",
|
||||||
|
"date": {
|
||||||
|
"type": "string",
|
||||||
|
"value": "2026-04-15"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"member_names": [
|
||||||
|
"Alžběta Testovická",
|
||||||
|
"Tomáš Fiktivný (Tov)",
|
||||||
|
"Pavel Smutný (Štrúdl)",
|
||||||
|
"Jana Nováková",
|
||||||
|
"Adam Novák"
|
||||||
|
],
|
||||||
|
"default_year": 2026
|
||||||
|
},
|
||||||
|
"output": {
|
||||||
|
"matches": [],
|
||||||
|
"months": [
|
||||||
|
"Other: Dres"
|
||||||
|
],
|
||||||
|
"search_text": "Test Payer Other: Dres "
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,69 @@
|
|||||||
|
{
|
||||||
|
"case": "11_other_message_fallback",
|
||||||
|
"func": "scripts.match_payments.reconcile",
|
||||||
|
"captured_at": "2026-10-08",
|
||||||
|
"input": {
|
||||||
|
"members": [
|
||||||
|
{
|
||||||
|
"name": "Member_d035d9f9",
|
||||||
|
"tier": "A",
|
||||||
|
"fees": {
|
||||||
|
"2026-01": [
|
||||||
|
750,
|
||||||
|
3
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"sorted_months": [
|
||||||
|
"2026-01"
|
||||||
|
],
|
||||||
|
"transactions": [
|
||||||
|
{
|
||||||
|
"date": "2026-01-10",
|
||||||
|
"amount": 300,
|
||||||
|
"manual_fix": "",
|
||||||
|
"person": "",
|
||||||
|
"purpose": "",
|
||||||
|
"inferred_amount": 300,
|
||||||
|
"sender": "Member_d035d9f9",
|
||||||
|
"message": "other: tournament",
|
||||||
|
"bank_id": ""
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"exceptions": [],
|
||||||
|
"default_year": 2026
|
||||||
|
},
|
||||||
|
"output": {
|
||||||
|
"members": {
|
||||||
|
"Member_d035d9f9": {
|
||||||
|
"tier": "A",
|
||||||
|
"months": {
|
||||||
|
"2026-01": {
|
||||||
|
"expected": 750,
|
||||||
|
"original_expected": 750,
|
||||||
|
"attendance_count": 3,
|
||||||
|
"exception": null,
|
||||||
|
"paid": 0.0,
|
||||||
|
"transactions": []
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"other_transactions": [
|
||||||
|
{
|
||||||
|
"amount": 300.0,
|
||||||
|
"date": "2026-01-10",
|
||||||
|
"sender": "Member_d035d9f9",
|
||||||
|
"message": "other: tournament",
|
||||||
|
"purpose": "other: tournament",
|
||||||
|
"confidence": "auto"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"total_balance": -750
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"unmatched": [],
|
||||||
|
"credits": {
|
||||||
|
"Member_d035d9f9": -750
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -358,6 +358,36 @@ SEEDS[("infer_transaction_details", "serial_date")] = {
|
|||||||
"member_names": _ROSTER,
|
"member_names": _ROSTER,
|
||||||
"default_year": 2026,
|
"default_year": 2026,
|
||||||
}
|
}
|
||||||
|
SEEDS[("infer_transaction_details", "other_prefix")] = {
|
||||||
|
"tx": {
|
||||||
|
"sender": "Jana Novakova",
|
||||||
|
"message": "other: tournament leden 2026",
|
||||||
|
"user_id": "",
|
||||||
|
"date": {"type": "string", "value": "2026-04-15"},
|
||||||
|
},
|
||||||
|
"member_names": _ROSTER,
|
||||||
|
"default_year": 2026,
|
||||||
|
}
|
||||||
|
SEEDS[("infer_transaction_details", "other_prefix_uppercase")] = {
|
||||||
|
"tx": {
|
||||||
|
"sender": "Test Payer",
|
||||||
|
"message": " Other: Dres ",
|
||||||
|
"user_id": "",
|
||||||
|
"date": {"type": "string", "value": "2026-04-15"},
|
||||||
|
},
|
||||||
|
"member_names": _ROSTER,
|
||||||
|
"default_year": 2026,
|
||||||
|
}
|
||||||
|
SEEDS[("infer_transaction_details", "other_not_at_start")] = {
|
||||||
|
"tx": {
|
||||||
|
"sender": "Jana Novakova",
|
||||||
|
"message": "leden other: dres",
|
||||||
|
"user_id": "",
|
||||||
|
"date": {"type": "string", "value": "2026-04-15"},
|
||||||
|
},
|
||||||
|
"member_names": _ROSTER,
|
||||||
|
"default_year": 2026,
|
||||||
|
}
|
||||||
SEEDS[("infer_transaction_details", "no_member_no_month")] = {
|
SEEDS[("infer_transaction_details", "no_member_no_month")] = {
|
||||||
"tx": {
|
"tx": {
|
||||||
"sender": "Unknown Person",
|
"sender": "Unknown Person",
|
||||||
@@ -476,6 +506,14 @@ SEEDS[("reconcile", "07_other_purpose_split")] = _reconcile_seed(
|
|||||||
transactions=[_tx("2026-01-10", 800, "Alice Dvořák, Bob Kratochvíl", "other:tournament", sender="Alice Dvořák")],
|
transactions=[_tx("2026-01-10", 800, "Alice Dvořák, Bob Kratochvíl", "other:tournament", sender="Alice Dvořák")],
|
||||||
)
|
)
|
||||||
|
|
||||||
|
# 11 — message starts with "other:" on a row not yet inferred (empty Person/Purpose):
|
||||||
|
# goes to the other ledger, the month stays unpaid
|
||||||
|
SEEDS[("reconcile", "11_other_message_fallback")] = _reconcile_seed(
|
||||||
|
members=[_member("Alice Dvořák", "A", {"2026-01": (750, 3)})],
|
||||||
|
sorted_months=["2026-01"],
|
||||||
|
transactions=[_tx("2026-01-10", 300, "", "", sender="Alice Dvořák", message="other: tournament")],
|
||||||
|
)
|
||||||
|
|
||||||
# 08 — junior with attendance=1 (expected=0 in reconcile, unknown in UI)
|
# 08 — junior with attendance=1 (expected=0 in reconcile, unknown in UI)
|
||||||
SEEDS[("reconcile", "08_junior_question_mark")] = _reconcile_seed(
|
SEEDS[("reconcile", "08_junior_question_mark")] = _reconcile_seed(
|
||||||
members=[_member("Karel Junior", "A", {"2026-01": (0, 1)})],
|
members=[_member("Karel Junior", "A", {"2026-01": (0, 1)})],
|
||||||
|
|||||||
@@ -150,27 +150,48 @@ def match_members(
|
|||||||
# Reconciliation
|
# Reconciliation
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def is_other_purpose(text) -> bool:
|
||||||
|
"""True when text marks a payment that is not a monthly fee.
|
||||||
|
|
||||||
|
It starts with "other:" (case-insensitive, leading whitespace ignored).
|
||||||
|
Used for both the Purpose column and the bank message.
|
||||||
|
"""
|
||||||
|
return str(text or "").strip().lower().startswith("other:")
|
||||||
|
|
||||||
|
|
||||||
def infer_transaction_details(tx: dict, member_names: list[str]) -> dict:
|
def infer_transaction_details(tx: dict, member_names: list[str]) -> dict:
|
||||||
"""Infer member(s) and month(s) for a single transaction.
|
"""Infer member(s) and month(s) for a single transaction.
|
||||||
|
|
||||||
|
A message that starts with "other:" is not a monthly fee: no month is
|
||||||
|
inferred (neither from the text nor from the date) and 'months' holds the
|
||||||
|
trimmed message as the single purpose.
|
||||||
|
|
||||||
Returns:
|
Returns:
|
||||||
{
|
{
|
||||||
'members': [(name, confidence)],
|
'members': [(name, confidence)],
|
||||||
'months': [YYYY-MM],
|
'months': [YYYY-MM] or ['other:…'],
|
||||||
'matched_text': str
|
'matched_text': str
|
||||||
}
|
}
|
||||||
"""
|
"""
|
||||||
# Combine sender + message for searching
|
# Combine sender + message for searching
|
||||||
search_text = f"{tx.get('sender', '')} {tx.get('message', '')} {tx.get('user_id', '')}"
|
search_text = f"{tx.get('sender', '')} {tx.get('message', '')} {tx.get('user_id', '')}"
|
||||||
matched_members = match_members(search_text, member_names)
|
matched_members = match_members(search_text, member_names)
|
||||||
matched_months = parse_month_references(
|
|
||||||
tx.get("message", "") + " " + tx.get("user_id", "")
|
|
||||||
)
|
|
||||||
|
|
||||||
if not matched_members:
|
if not matched_members:
|
||||||
# Try matching sender name alone with more lenient matching
|
# Try matching sender name alone with more lenient matching
|
||||||
matched_members = match_members(tx.get("sender", ""), member_names)
|
matched_members = match_members(tx.get("sender", ""), member_names)
|
||||||
|
|
||||||
|
if is_other_purpose(tx.get("message", "")):
|
||||||
|
return {
|
||||||
|
"members": matched_members,
|
||||||
|
"months": [str(tx.get("message", "")).strip()],
|
||||||
|
"search_text": search_text
|
||||||
|
}
|
||||||
|
|
||||||
|
matched_months = parse_month_references(
|
||||||
|
tx.get("message", "") + " " + tx.get("user_id", "")
|
||||||
|
)
|
||||||
|
|
||||||
if not matched_months:
|
if not matched_months:
|
||||||
# If no month specified, try to infer from payment date
|
# If no month specified, try to infer from payment date
|
||||||
tx_date = tx.get("date")
|
tx_date = tx.get("date")
|
||||||
@@ -394,7 +415,7 @@ def reconcile(
|
|||||||
|
|
||||||
# Strip markers like [?]
|
# Strip markers like [?]
|
||||||
person_str = re.sub(r"\[\?\]\s*", "", person_str)
|
person_str = re.sub(r"\[\?\]\s*", "", person_str)
|
||||||
is_other = purpose_str.lower().startswith("other:")
|
is_other = is_other_purpose(purpose_str)
|
||||||
|
|
||||||
if person_str and purpose_str:
|
if person_str and purpose_str:
|
||||||
# We have pre-matched data (either from script or manual)
|
# We have pre-matched data (either from script or manual)
|
||||||
@@ -415,6 +436,10 @@ def reconcile(
|
|||||||
inference = infer_transaction_details(tx, member_names)
|
inference = infer_transaction_details(tx, member_names)
|
||||||
matched_members = inference["members"]
|
matched_members = inference["members"]
|
||||||
matched_months = inference["months"]
|
matched_months = inference["months"]
|
||||||
|
# A message starting with "other:" is inferred as a non-fee payment.
|
||||||
|
if len(matched_months) == 1 and is_other_purpose(matched_months[0]):
|
||||||
|
is_other = True
|
||||||
|
purpose_str = matched_months[0]
|
||||||
amount = tx.get("amount", 0)
|
amount = tx.get("amount", 0)
|
||||||
try:
|
try:
|
||||||
amount = float(amount)
|
amount = float(amount)
|
||||||
|
|||||||
@@ -1,7 +1,9 @@
|
|||||||
import unittest
|
import unittest
|
||||||
from unittest.mock import MagicMock, patch
|
from unittest.mock import MagicMock, patch
|
||||||
|
|
||||||
from scripts.match_payments import canonical_member_key, fetch_sheet_data, reconcile
|
from scripts.match_payments import (
|
||||||
|
canonical_member_key, fetch_sheet_data, infer_transaction_details, is_other_purpose, reconcile,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
class TestCanonicalMemberKey(unittest.TestCase):
|
class TestCanonicalMemberKey(unittest.TestCase):
|
||||||
@@ -88,3 +90,44 @@ class TestFetchSheetDataVS(unittest.TestCase):
|
|||||||
|
|
||||||
def test_string_vs_is_kept(self):
|
def test_string_vs_is_kept(self):
|
||||||
self.assertEqual(self._fetch("0308")[0]["vs"], "0308")
|
self.assertEqual(self._fetch("0308")[0]["vs"], "0308")
|
||||||
|
|
||||||
|
|
||||||
|
class TestOtherPurposeInference(unittest.TestCase):
|
||||||
|
"""A message starting with "other:" is not a monthly fee."""
|
||||||
|
|
||||||
|
MEMBERS = ["Jana Nováková", "Adam Novák"]
|
||||||
|
|
||||||
|
def test_is_other_purpose(self):
|
||||||
|
for text in ("other: tournament", "other:shirt", " Other: Tournament", "OTHER:x"):
|
||||||
|
self.assertTrue(is_other_purpose(text), text)
|
||||||
|
for text in ("leden other: x", "other tournament", "another: x", "2026-01", "", None):
|
||||||
|
self.assertFalse(is_other_purpose(text), text)
|
||||||
|
|
||||||
|
def test_message_becomes_purpose_and_no_month_is_inferred(self):
|
||||||
|
tx = {"sender": "Jana Novakova", "message": " other: turnaj leden 2026 ", "date": "2026-04-15"}
|
||||||
|
result = infer_transaction_details(tx, self.MEMBERS)
|
||||||
|
self.assertEqual(result["members"], [("Jana Nováková", "auto")])
|
||||||
|
self.assertEqual(result["months"], ["other: turnaj leden 2026"])
|
||||||
|
|
||||||
|
def test_unknown_member_still_yields_purpose(self):
|
||||||
|
tx = {"sender": "Unknown Payer", "message": "Other: Dres", "date": "2026-04-15"}
|
||||||
|
result = infer_transaction_details(tx, self.MEMBERS)
|
||||||
|
self.assertEqual(result["members"], [])
|
||||||
|
self.assertEqual(result["months"], ["Other: Dres"])
|
||||||
|
|
||||||
|
def test_other_not_at_start_is_handled_the_old_way(self):
|
||||||
|
tx = {"sender": "Jana Novakova", "message": "leden other: dres", "date": "2026-04-15"}
|
||||||
|
result = infer_transaction_details(tx, self.MEMBERS)
|
||||||
|
self.assertEqual(result["months"], ["2026-01"])
|
||||||
|
|
||||||
|
def test_reconcile_fallback_puts_it_in_other_ledger(self):
|
||||||
|
members = [("Jana Nováková", "A", {"2026-01": (750, 3)})]
|
||||||
|
tx = {"date": "2026-01-10", "amount": 300, "person": "", "purpose": "",
|
||||||
|
"sender": "Jana Novakova", "message": "other: dres", "inferred_amount": ""}
|
||||||
|
result = reconcile(members, ["2026-01"], [tx], {})
|
||||||
|
member = result["members"]["Jana Nováková"]
|
||||||
|
self.assertEqual(member["months"]["2026-01"]["paid"], 0)
|
||||||
|
self.assertEqual(len(member["other_transactions"]), 1)
|
||||||
|
self.assertEqual(member["other_transactions"][0]["amount"], 300)
|
||||||
|
self.assertEqual(member["other_transactions"][0]["purpose"], "other: dres")
|
||||||
|
self.assertEqual(result["unmatched"], [])
|
||||||
|
|||||||
Reference in New Issue
Block a user