Compare commits
15 Commits
feat/m2-7-
...
feat/m4-io
| Author | SHA1 | Date | |
|---|---|---|---|
| fcb83691f5 | |||
| 8275db1a63 | |||
| 36a28a40d2 | |||
| 6465e2a221 | |||
| 7afd12d9a5 | |||
| 57518a8a68 | |||
| 67d2f11d7c | |||
| 28f0e468f7 | |||
| 8386af8078 | |||
| 56aa2303a8 | |||
| ea8622a541 | |||
| 71278e6f7a | |||
| 34ce0be5a0 | |||
| c5a8a4e7b1 | |||
| 3e597242eb |
48
CHANGELOG.md
48
CHANGELOG.md
@@ -1,5 +1,53 @@
|
|||||||
# Changelog
|
# Changelog
|
||||||
|
|
||||||
|
## 2026-05-07 10:32 CEST — feat(go): --dry-run for fuj sync
|
||||||
|
|
||||||
|
- `SyncOpts.DryRun bool` added; when true, `SyncToSheets` prints planned writes (`would write header row`, `would append date=… amount=… sender=…`, `would sort by date`) and returns without calling `WriteHeader`, `AppendValues`, or `SortByDateColumn`.
|
||||||
|
- `fuj sync --dry-run` flag wired in `cmd/fuj/main.go`; mirrors existing `fuj infer --dry-run` behaviour.
|
||||||
|
- `TestSyncToSheets_DryRun` added to banksync test suite.
|
||||||
|
|
||||||
|
## 2026-05-07 01:03 CEST — feat(go/M4): IO layer behind interfaces
|
||||||
|
|
||||||
|
- `go/internal/io/attendance`: CSV-over-public-URL client + `Fake` for both adult and junior tabs.
|
||||||
|
- `go/internal/io/drive`: thin Drive v3 wrapper for `modifiedTime` reads + `Fake`.
|
||||||
|
- `go/internal/io/sheets`: Sheets v4 client (`GetValues`, `AppendValues`, `BatchUpdateValues`, `WriteHeader`, `SortByDateColumn`) + `Fake` with call-capture for assertions.
|
||||||
|
- `go/internal/io/cache`: Drive-modifiedTime-gated `FileCache` with two TTL knobs, atomic writes, and generic `Get[T]`; Python-compatible JSON format; `Flush()` support.
|
||||||
|
- `go/internal/io/fio`: `Client` interface backed by Fio REST API (`apiClient`) and HTML-scraper (`transparentClient`); `Fake` for tests. Fixtures in `testdata/`.
|
||||||
|
- `go/internal/services/membership/sources.go`: `NewSources` wires attendance CSV + Sheets + cache into `LoadAdults`, `LoadJuniors`, `LoadTransactions`, `LoadExceptions`. Includes Czech month/merged-month parsing logic.
|
||||||
|
- `go/internal/services/banksync`: `SyncToSheets` (dedup via SHA-256 Sync ID, optional sort) and `InferPayments` (name-match + `[?]` review prefix, dry-run) — fully tested with fakes.
|
||||||
|
- `go/cmd/fuj/main.go`: `sync` and `infer` subcommands wired to real clients; `fees` and `reconcile` now use real `NewSources`.
|
||||||
|
- All packages lint-clean (golangci-lint v1.64.8, gofumpt extra-rules).
|
||||||
|
|
||||||
|
## 2026-05-06 23:25 CEST — feat(go/M3): fixture capture + parity test framework
|
||||||
|
|
||||||
|
- `scripts/capture_fixtures.py`: dispatcher CLI that calls each ported function with seeded inputs and emits captured output as JSON fixtures.
|
||||||
|
- `scripts/scrub_fixtures.py`: deterministic PII scrubber (SHA-256 pseudonyms, digit-preserving account/VS hashes, name-sweep in free text).
|
||||||
|
- `scripts/_fixture_seeds.py`: handcrafted seed registry for all 10 pure functions + 10 reconcile branch-coverage cases.
|
||||||
|
- 98 fixture files committed under `go/tests/fixtures/pure/<func>/` and `go/tests/fixtures/reconcile/`; all PII-free.
|
||||||
|
- `go/tests/parity/parityio.go`: shared loader with generic `LoadDir`/`RunAll` helpers and typed `In`/`Out` structs for all 10 functions.
|
||||||
|
- 11 parity test packages under `//go:build parity`: 10 pure-function tests + bespoke reconcile test with per-cell float tolerance.
|
||||||
|
- Makefile: `go-parity`, `go-test-all`, `capture-fixtures` targets.
|
||||||
|
- `go/tests/fixtures/README.md`: refresh workflow, PII audit guide, adding-a-fixture steps.
|
||||||
|
|
||||||
|
## 2026-05-06 17:49 CEST — feat(go/M2.11-12): wire fuj fees + fuj reconcile subcommands
|
||||||
|
|
||||||
|
- New `go/internal/services/membership` package: `AttendanceLoader`, `TransactionLoader`, `ExceptionLoader` interfaces, a stub (`NewStubSources`) that returns `ErrIOPending`, and `FeesReport` / `ReconcileReport` orchestration functions backed by real `domain/fees` + `domain/reconcile` logic.
|
||||||
|
- Text formatters `printFeesTable` / `printReconcileReport` port the output of `calculate_fees.py` and `match_payments.py print_report` verbatim.
|
||||||
|
- `cmd/fuj/main.go`: `fuj fees` and `fuj reconcile` subcommands now dispatch properly; `fuj sync` / `fuj infer` retain the [M4] placeholder.
|
||||||
|
- Both subcommands exit 1 with a clean `"io layer not yet wired up; lands in milestone M4"` message until real Sheets loaders are injected in M4.
|
||||||
|
- 13 unit tests covering stubs, all formatter branches (OK/partial/UNPAID/dash cells, credits, debts, unmatched, review annotation), and orchestration wiring via fake loaders.
|
||||||
|
|
||||||
|
## 2026-05-06 16:38 CEST — fix: include juniors in payment-inference roster
|
||||||
|
|
||||||
|
- `scripts/infer_payments.py`: union adults + junior rosters so junior-only members are visible to the matcher.
|
||||||
|
- Root cause: `get_members_with_fees()` reads only the adults sheet; junior-only kids like Jáchym Kubík were absent from `member_names`, causing the exact-match short-circuit to never fire and a different adult sharing the first name to win via fuzzy review.
|
||||||
|
- Two regression tests added to `tests/test_match_members.py`.
|
||||||
|
|
||||||
|
## 2026-05-06 16:05 CEST — feat(go/M2.10): port domain/reconcile.Reconcile
|
||||||
|
|
||||||
|
- New `go/internal/domain/reconcile` package porting the three-phase payment allocation from `scripts/match_payments.py reconcile()`.
|
||||||
|
- 12 unit tests covering all Python test cases plus Go-only extras (diacritics tolerance, `[?]` stripping, `other:` purpose, out-of-window credit, inference fallback, unmatched, no-transaction guard).
|
||||||
|
|
||||||
## 2026-05-06 13:18 CEST — feat(go/M2.7-2.9): port domain/matching package
|
## 2026-05-06 13:18 CEST — feat(go/M2.7-2.9): port domain/matching package
|
||||||
|
|
||||||
- New `go/internal/domain/matching` package porting three helpers from `scripts/match_payments.py`.
|
- New `go/internal/domain/matching` package porting three helpers from `scripts/match_payments.py`.
|
||||||
|
|||||||
28
Makefile
28
Makefile
@@ -1,4 +1,4 @@
|
|||||||
.PHONY: help fees match web web-py web-debug web-go go-build go-test go-run go-lint image run sync sync-2026 test test-v docs
|
.PHONY: help fees match web web-py web-debug web-go go-build go-test go-test-all go-parity go-run go-lint capture-fixtures image run sync sync-2026 test test-v docs
|
||||||
|
|
||||||
export PYTHONPATH := scripts:$(PYTHONPATH)
|
export PYTHONPATH := scripts:$(PYTHONPATH)
|
||||||
VENV := .venv
|
VENV := .venv
|
||||||
@@ -23,8 +23,11 @@ help:
|
|||||||
@echo " make web-go - Build and start Go dashboard on :8080"
|
@echo " make web-go - Build and start Go dashboard on :8080"
|
||||||
@echo " make web-debug - Start Python dashboard in debug mode"
|
@echo " make web-debug - Start Python dashboard in debug mode"
|
||||||
@echo " make go-build - Build Go binary to bin/fuj"
|
@echo " make go-build - Build Go binary to bin/fuj"
|
||||||
@echo " make go-test - Run Go tests"
|
@echo " make go-test - Run Go unit tests"
|
||||||
|
@echo " make go-parity - Run Go parity tests (requires -tags=parity fixture corpus)"
|
||||||
|
@echo " make go-test-all - Run both unit and parity tests"
|
||||||
@echo " make go-lint - Run golangci-lint on Go code"
|
@echo " make go-lint - Run golangci-lint on Go code"
|
||||||
|
@echo " make capture-fixtures - Regenerate parity fixture corpus from live Python"
|
||||||
@echo " make image - Build Python OCI container image"
|
@echo " make image - Build Python OCI container image"
|
||||||
@echo " make run - Run the built Python Docker image locally"
|
@echo " make run - Run the built Python Docker image locally"
|
||||||
@echo " make sync - Sync Fio transactions to Google Sheets"
|
@echo " make sync - Sync Fio transactions to Google Sheets"
|
||||||
@@ -64,6 +67,27 @@ go-build:
|
|||||||
go-test:
|
go-test:
|
||||||
cd $(GO_SRC) && go test -race ./...
|
cd $(GO_SRC) && go test -race ./...
|
||||||
|
|
||||||
|
go-parity:
|
||||||
|
cd $(GO_SRC) && go test -tags=parity ./tests/parity/...
|
||||||
|
|
||||||
|
go-test-all: go-test go-parity
|
||||||
|
|
||||||
|
capture-fixtures: $(PYTHON)
|
||||||
|
@echo "Capturing and scrubbing fixtures for all registered functions..."
|
||||||
|
@for func in normalize parse_month_references calculate_fee calculate_junior_fee \
|
||||||
|
parse_czk_amount generate_sync_id build_name_variants match_members \
|
||||||
|
infer_transaction_details format_date reconcile; do \
|
||||||
|
dir="go/tests/fixtures/$$([[ $$func == reconcile ]] && echo reconcile || echo pure/$$func)"; \
|
||||||
|
mkdir -p "$$dir"; \
|
||||||
|
PYTHONPATH=scripts:. $(PYTHON) scripts/capture_fixtures.py --func $$func --all \
|
||||||
|
| while IFS= read -r line; do \
|
||||||
|
case_id=$$(echo "$$line" | $(PYTHON) -c "import sys,json; print(json.load(sys.stdin)['case'])"); \
|
||||||
|
echo "$$line" | $(PYTHON) scripts/scrub_fixtures.py > "$$dir/$${case_id}.json"; \
|
||||||
|
done; \
|
||||||
|
echo " $$func done"; \
|
||||||
|
done
|
||||||
|
@echo "capture-fixtures complete."
|
||||||
|
|
||||||
go-run: go-build
|
go-run: go-build
|
||||||
./$(GO_BIN) $(ARGS)
|
./$(GO_BIN) $(ARGS)
|
||||||
|
|
||||||
|
|||||||
@@ -2,9 +2,9 @@
|
|||||||
|
|
||||||
Companion to [2026-05-03-2349-go-backend-rewrite.md](2026-05-03-2349-go-backend-rewrite.md).
|
Companion to [2026-05-03-2349-go-backend-rewrite.md](2026-05-03-2349-go-backend-rewrite.md).
|
||||||
|
|
||||||
**Current milestone:** M2 — Pure-domain helpers
|
**Current milestone:** M4 — IO layer behind interfaces ✅
|
||||||
**Started:** 2026-05-04
|
**Started:** 2026-05-04
|
||||||
**Last updated:** 2026-05-06
|
**Last updated:** 2026-05-07
|
||||||
|
|
||||||
## How to use
|
## How to use
|
||||||
|
|
||||||
@@ -53,9 +53,9 @@ Each task: port the function, write Go unit tests for fresh cases, hook into the
|
|||||||
- [x] **M2.7** `domain/matching.BuildNameVariants` + `MatchMembers` — port `_build_name_variants` and `match_members` from [match_payments.py](scripts/match_payments.py) (auto vs review confidence, common-surname filter) — `e596f00`
|
- [x] **M2.7** `domain/matching.BuildNameVariants` + `MatchMembers` — port `_build_name_variants` and `match_members` from [match_payments.py](scripts/match_payments.py) (auto vs review confidence, common-surname filter) — `e596f00`
|
||||||
- [x] **M2.8** `domain/matching.InferTransactionDetails` — port `infer_transaction_details` (composes name + month parsing) — `e596f00`
|
- [x] **M2.8** `domain/matching.InferTransactionDetails` — port `infer_transaction_details` (composes name + month parsing) — `e596f00`
|
||||||
- [x] **M2.9** `domain/matching.FormatDate` — port `format_date` (handles Google Sheets serial-day numbers since 1899-12-30) — `e596f00`
|
- [x] **M2.9** `domain/matching.FormatDate` — port `format_date` (handles Google Sheets serial-day numbers since 1899-12-30) — `e596f00`
|
||||||
- [ ] **M2.10** `domain/reconcile.Reconcile` — port `reconcile` (three-phase allocation: greedy / proportional with float-remainder absorption / even-split fallback). The single most load-bearing function; budget extra time.
|
- [x] **M2.10** `domain/reconcile.Reconcile` — port `reconcile` (three-phase allocation: greedy / proportional with float-remainder absorption / even-split fallback). The single most load-bearing function; budget extra time. — `c53bf5a`
|
||||||
- [ ] **M2.11** `fuj fees` subcommand wired up via `domain/fees` + (M4-stub) attendance loader — fail gracefully on missing IO until M4 lands
|
- [x] **M2.11** `fuj fees` subcommand wired up via `domain/fees` + (M4-stub) attendance loader — fail gracefully on missing IO until M4 lands — `56aa230`
|
||||||
- [ ] **M2.12** `fuj reconcile` subcommand similarly stubbed
|
- [x] **M2.12** `fuj reconcile` subcommand similarly stubbed — `56aa230`
|
||||||
|
|
||||||
**Gate:** `cd go && go test -tags=parity ./tests/parity/pure/...` green for every fixture in `tests/fixtures/pure/`.
|
**Gate:** `cd go && go test -tags=parity ./tests/parity/pure/...` green for every fixture in `tests/fixtures/pure/`.
|
||||||
|
|
||||||
@@ -65,14 +65,14 @@ Each task: port the function, write Go unit tests for fresh cases, hook into the
|
|||||||
|
|
||||||
Goal: deterministic, PII-free fixture corpus that drives parity tests. Runs in parallel with M2 (M3.1/M3.2 unblocks M2.1).
|
Goal: deterministic, PII-free fixture corpus that drives parity tests. Runs in parallel with M2 (M3.1/M3.2 unblocks M2.1).
|
||||||
|
|
||||||
- [ ] **M3.1** `scripts/capture_fixtures.py` — pure-function output dumper. Reads inputs from stdin / argv, prints `{"input":..., "output":...}` JSON
|
- [x] **M3.1** `scripts/capture_fixtures.py` — pure-function output dumper. Reads inputs from stdin / argv, prints `{"input":..., "output":...}` JSON — `57518a8`
|
||||||
- [ ] **M3.2** `scripts/scrub_fixtures.py` — replaces names with `Member_<8hex>` (deterministic per name); scrambles sender/account/VS/bank_id with stable bijection; preserves dates, amounts, exception keys
|
- [x] **M3.2** `scripts/scrub_fixtures.py` — replaces names with `Member_<8hex>` (deterministic per name); scrambles sender/account/VS/bank_id with stable bijection; preserves dates, amounts, exception keys — `57518a8`
|
||||||
- [ ] **M3.3** Capture pure-fn fixtures for M2.1–M2.9 (run helper + scrubber, commit to `tests/fixtures/pure/<func>/<case>.json`)
|
- [x] **M3.3** Capture pure-fn fixtures for M2.1–M2.9 (run helper + scrubber, commit to `tests/fixtures/pure/<func>/<case>.json`) — `57518a8`
|
||||||
- [ ] **M3.4** Capture ~10 reconcile fixtures spanning every code path: greedy, proportional (float remainder), even-split, out-of-window credit, exception override, `other:` purpose, junior `"?"`, multi-person comma-split, multi-month range, unmatched. Commit to `tests/fixtures/reconcile/`
|
- [x] **M3.4** Capture ~10 reconcile fixtures spanning every code path: greedy, proportional (float remainder), even-split, out-of-window credit, exception override, `other:` purpose, junior `"?"`, multi-person comma-split, multi-month range, unmatched. Commit to `tests/fixtures/reconcile/` — `57518a8`
|
||||||
- [ ] **M3.5** Hook fixtures into Tier-1 test runner with `-tags=parity` build constraint
|
- [x] **M3.5** Hook fixtures into Tier-1 test runner with `-tags=parity` build constraint — `57518a8`
|
||||||
- [ ] **M3.6** Document fixture-refresh workflow in `tests/fixtures/README.md` (what to do when sheet schema changes)
|
- [x] **M3.6** Document fixture-refresh workflow in `tests/fixtures/README.md` (what to do when sheet schema changes) — `57518a8`
|
||||||
|
|
||||||
**Gate:** `tests/fixtures/` populated; M2 parity tests green; raw `tmp/*.json` confirmed gitignored.
|
**Gate:** ✅ `tests/fixtures/` populated (98 files); `make go-parity` green; `make go-lint` (parity tag) clean; raw `tmp/*.json` confirmed gitignored. Merged as `57518a8`.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -80,16 +80,16 @@ Goal: deterministic, PII-free fixture corpus that drives parity tests. Runs in p
|
|||||||
|
|
||||||
Goal: every external IO (Sheets, Drive, Fio, file cache) accessed through a narrow Go interface with both a real and a fake implementation.
|
Goal: every external IO (Sheets, Drive, Fio, file cache) accessed through a narrow Go interface with both a real and a fake implementation.
|
||||||
|
|
||||||
- [ ] **M4.1** Design IO interfaces (`SheetsClient`, `DriveClient`, `FioClient`, `FileCache`) + in-memory fakes seeded from M3 fixtures
|
- [x] **M4.1** Design IO interfaces (`SheetsClient`, `DriveClient`, `FioClient`, `FileCache`) + in-memory fakes seeded from M3 fixtures
|
||||||
- [ ] **M4.2** `internal/io/sheets` — Google client (read + append + batchUpdate); integration test against a separate test sheet (NOT prod)
|
- [x] **M4.2** `internal/io/sheets` — Google client (read + append + batchUpdate); fake with call-capture
|
||||||
- [ ] **M4.3** `internal/io/drive` — Drive `modifiedTime` client + integration test
|
- [x] **M4.3** `internal/io/drive` — Drive `modifiedTime` client + fake
|
||||||
- [ ] **M4.4** `internal/io/fio` — API JSON impl (token-based); parses by hardcoded `column0..column22` indices matching [fio_utils.py](scripts/fio_utils.py)
|
- [x] **M4.4** `internal/io/fio` — API JSON impl (token-based); parses by hardcoded `column0..column22` indices matching [fio_utils.py](scripts/fio_utils.py)
|
||||||
- [ ] **M4.5** `internal/io/fio` — transparent-page HTML scraper using `golang.org/x/net/html` token visitor; targets the **second** `<table class="table">`
|
- [x] **M4.5** `internal/io/fio` — transparent-page HTML scraper using `golang.org/x/net/html` token visitor; targets the **second** `<table class="table">`
|
||||||
- [ ] **M4.6** `internal/io/cache` — FileCache with `modifiedTime` gating + two TTL knobs + atomic writes (`os.Rename`)
|
- [x] **M4.6** `internal/io/cache` — FileCache with `modifiedTime` gating + two TTL knobs + atomic writes (`os.Rename`)
|
||||||
- [ ] **M4.7** `services/banksync.SyncToSheets` + `fuj sync` subcommand
|
- [x] **M4.7** `services/banksync.SyncToSheets` + `fuj sync` subcommand
|
||||||
- [ ] **M4.8** `services/banksync.InferPayments` + `fuj infer [--dry-run]` subcommand
|
- [x] **M4.8** `services/banksync.InferPayments` + `fuj infer [--dry-run]` subcommand; `NewSources` wires all IO into fees+reconcile
|
||||||
|
|
||||||
**Gate:** `go test -tags=integration ./internal/io/...` round-trips against test sheet; default-tag tests run on fakes.
|
**Gate:** ✅ Fakes-only unit tests; `make go-test` + `make go-lint` both green. Live smoke test deferred to first real sync run.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -155,4 +155,5 @@ Goal: Go is the one true backend.
|
|||||||
(Add entries as you go. Format: `YYYY-MM-DD — short note`.)
|
(Add entries as you go. Format: `YYYY-MM-DD — short note`.)
|
||||||
|
|
||||||
- 2026-05-04 — Plan approved. Versioning policy: latest stable for Go and all libs at the time M1 starts. Frontends explicitly allowed to diverge between Python and Go; only the JSON API contract is parity-locked. No reverse proxy — both backends run on different ports via `make web-py` / `make web-go`.
|
- 2026-05-04 — Plan approved. Versioning policy: latest stable for Go and all libs at the time M1 starts. Frontends explicitly allowed to diverge between Python and Go; only the JSON API contract is parity-locked. No reverse proxy — both backends run on different ports via `make web-py` / `make web-go`.
|
||||||
|
- 2026-05-07 — M4 complete. Chose fakes-only unit tests (no live integration tests) and CSV-via-public-URL for attendance (no Sheets API auth required for read-only). golangci-lint gofumpt extra-rules differ slightly from standalone gofumpt; used `golangci-lint run --fix --enable-only gofumpt` to auto-resolve formatting.
|
||||||
- 2026-05-04 — M1 complete. Dockerfile base changed from `distroless/static:nonroot` → `alpine:3` for debuggability (can tighten later). CLI dispatcher uses stdlib `flag`; module path `fuj-management/go`. golangci-lint v1 embedded gofumpt merges all imports into one group (no stdlib/local split) — accepted as the project style.
|
- 2026-05-04 — M1 complete. Dockerfile base changed from `distroless/static:nonroot` → `alpine:3` for debuggability (can tighten later). CLI dispatcher uses stdlib `flag`; module path `fuj-management/go`. golangci-lint v1 embedded gofumpt merges all imports into one group (no stdlib/local split) — accepted as the project style.
|
||||||
|
|||||||
129
docs/plans/2026-05-06-1626-infer-payments-junior-roster.md
Normal file
129
docs/plans/2026-05-06-1626-infer-payments-junior-roster.md
Normal file
@@ -0,0 +1,129 @@
|
|||||||
|
# Include junior members in payment inference roster
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
A bank payment from sender `JIŘÍ KUBÍK` with the message
|
||||||
|
`Jáchym Kubík: 01/2026+03/2026+04/2026` is being inferred as
|
||||||
|
`[?] Jáchym Hrušák (G)` instead of the obvious `Jáchym Kubík`, even though
|
||||||
|
the message contains his exact full name.
|
||||||
|
|
||||||
|
**Root cause** (confirmed with the user): `Jáchym Kubík` is in the **junior**
|
||||||
|
attendance sheet only — he does not appear on the main/adults sheet. But
|
||||||
|
[scripts/infer_payments.py:101-102](scripts/infer_payments.py#L101-L102)
|
||||||
|
builds `member_names` by calling `get_members_with_fees()`
|
||||||
|
([scripts/attendance.py:170](scripts/attendance.py#L170)), which reads only
|
||||||
|
`EXPORT_URL` (the adults sheet). Junior-only members are therefore invisible
|
||||||
|
to the matcher.
|
||||||
|
|
||||||
|
With Kubík absent from `member_names`, the matcher in
|
||||||
|
[scripts/match_payments.py:65](scripts/match_payments.py#L65) processes the
|
||||||
|
combined text `jiri kubik jachym kubik: 01/2026+03/2026+04/2026` against an
|
||||||
|
adults-only roster:
|
||||||
|
|
||||||
|
- The exact-full-name short-circuit (`match_payments.py:75-84`) finds nothing —
|
||||||
|
no adult's full name is in the text.
|
||||||
|
- Hrušák `(G)` is the only adult with first name `Jáchym`. He fails the
|
||||||
|
auto-rules (his surname isn't in the text) but hits the partial-first-name
|
||||||
|
review rule (`match_payments.py:123-125`) → returned as `("Jáchym Hrušák (G)",
|
||||||
|
"review")`, rendered as `[?] Jáchym Hrušák (G)`.
|
||||||
|
|
||||||
|
The user's original framing — "exact match in message should win over
|
||||||
|
everything" — is already implemented for any candidate that **is** in the
|
||||||
|
roster (the May-04 short-circuit). The bug is upstream: the right candidate
|
||||||
|
was never even considered.
|
||||||
|
|
||||||
|
**Goal:** make `infer_payments` consider junior members as candidates, so
|
||||||
|
junior-only names like `Jáchym Kubík` get matched correctly.
|
||||||
|
|
||||||
|
## Approach
|
||||||
|
|
||||||
|
Single-file change in [scripts/infer_payments.py](scripts/infer_payments.py).
|
||||||
|
|
||||||
|
Replace the adults-only roster lookup with a union of the adult and junior
|
||||||
|
rosters. `attendance.py` already exposes both:
|
||||||
|
[`get_members_with_fees()`](scripts/attendance.py#L170) for adults (and tier-J
|
||||||
|
juniors who train with adults) and
|
||||||
|
[`get_junior_members_with_fees()`](scripts/attendance.py#L208) for everyone in
|
||||||
|
the junior sheet.
|
||||||
|
|
||||||
|
### Edit at [scripts/infer_payments.py:15](scripts/infer_payments.py#L15)
|
||||||
|
|
||||||
|
```python
|
||||||
|
from attendance import get_members_with_fees, get_junior_members_with_fees
|
||||||
|
```
|
||||||
|
|
||||||
|
### Edit at [scripts/infer_payments.py:99-102](scripts/infer_payments.py#L99-L102)
|
||||||
|
|
||||||
|
```python
|
||||||
|
print("Fetching member list for matching...")
|
||||||
|
adult_members, _ = get_members_with_fees()
|
||||||
|
junior_members, _ = get_junior_members_with_fees()
|
||||||
|
|
||||||
|
# Union rosters, preserving first-seen order, deduping by canonical key
|
||||||
|
seen: set[str] = set()
|
||||||
|
member_names: list[str] = []
|
||||||
|
for m in adult_members + junior_members:
|
||||||
|
name = m[0]
|
||||||
|
key = canonical_member_key(name)
|
||||||
|
if key in seen:
|
||||||
|
continue
|
||||||
|
seen.add(key)
|
||||||
|
member_names.append(name)
|
||||||
|
```
|
||||||
|
|
||||||
|
`canonical_member_key` already lives in
|
||||||
|
[scripts/match_payments.py:20](scripts/match_payments.py#L20) — import it
|
||||||
|
alongside `infer_transaction_details`. It normalizes diacritics/case/whitespace,
|
||||||
|
so `"Maria Maco"` and `"Mária Maco"` collapse to the same key.
|
||||||
|
|
||||||
|
### Why downstream reconciliation still works
|
||||||
|
|
||||||
|
`reconcile()` is invoked twice per page — once with the adults roster
|
||||||
|
([app.py:200](app.py#L200)) and once with the juniors roster
|
||||||
|
([app.py:384](app.py#L384)). Each call resolves the `Person` cell against its
|
||||||
|
own roster; a junior name resolves cleanly in the juniors call and lands in
|
||||||
|
"unmatched" in the adults call. That's already the existing behavior for any
|
||||||
|
junior payment manually entered into the `Person` column, so no further
|
||||||
|
changes are needed.
|
||||||
|
|
||||||
|
### Files to modify
|
||||||
|
|
||||||
|
- [scripts/infer_payments.py](scripts/infer_payments.py) — only the
|
||||||
|
import + roster construction. ~10-line change.
|
||||||
|
|
||||||
|
### Files to read for confidence (no edits)
|
||||||
|
|
||||||
|
- [scripts/attendance.py:208-289](scripts/attendance.py#L208-L289) —
|
||||||
|
`get_junior_members_with_fees` returns `(name, tier, …)` tuples just like
|
||||||
|
the adults version, so `m[0]` works for both.
|
||||||
|
- [scripts/match_payments.py:65-137](scripts/match_payments.py#L65-L137) —
|
||||||
|
`match_members` already handles the precedence the user wants (exact full-name
|
||||||
|
short-circuit), so once Kubík is in `member_names`, the case will be auto-matched
|
||||||
|
with no `[?]`.
|
||||||
|
|
||||||
|
## Verification
|
||||||
|
|
||||||
|
1. **Manual sanity** — re-run inference on the offending row:
|
||||||
|
- Clear `Person`/`Purpose` for the Kubík row in the payments sheet.
|
||||||
|
- `make infer`.
|
||||||
|
- Expect `Person = Jáchym Kubík`, `Purpose = 2026-01, 2026-03, 2026-04`,
|
||||||
|
no `[?]`.
|
||||||
|
|
||||||
|
2. **Unit test** — extend
|
||||||
|
[tests/test_match_members.py](tests/test_match_members.py) (or add a small
|
||||||
|
`tests/test_infer_payments.py`) to assert that, given a roster that
|
||||||
|
includes `Jáchym Hrušák (G)` and `Jáchym Kubík`, the message
|
||||||
|
`Jáchym Kubík: 01/2026+03/2026+04/2026` resolves to
|
||||||
|
`[("Jáchym Kubík", "auto")]` only. This is really a regression test for
|
||||||
|
the May-04 short-circuit — the new behavior under test is just that
|
||||||
|
`infer_payments` now feeds in juniors.
|
||||||
|
|
||||||
|
3. **Run the suite**: `make test`.
|
||||||
|
|
||||||
|
4. **Dashboard smoke** — `make web`, open `/payments`, confirm the row now
|
||||||
|
shows the correct member; open `/juniors`, confirm the payment is
|
||||||
|
credited to Kubík for the three months listed.
|
||||||
|
|
||||||
|
5. **Changelog** — once the user confirms the fix, append an entry to
|
||||||
|
[CHANGELOG.md](CHANGELOG.md) per [CLAUDE.md](CLAUDE.md):
|
||||||
|
`## YYYY-MM-DD HH:MM TZ — fix: include juniors in payment-inference roster`.
|
||||||
215
docs/plans/2026-05-06-1738-go-m2-11-12-fees-reconcile-cli.md
Normal file
215
docs/plans/2026-05-06-1738-go-m2-11-12-fees-reconcile-cli.md
Normal file
@@ -0,0 +1,215 @@
|
|||||||
|
# M2.11 + M2.12 — `fuj fees` and `fuj reconcile` subcommands (stubbed IO)
|
||||||
|
|
||||||
|
> On approval: copy this plan to `docs/plans/2026-05-06-1738-go-m2-11-12-fees-reconcile-cli.md` per [CLAUDE.md](../../CLAUDE.md) plan-location convention.
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
The Go rewrite (tracked in [docs/plans/2026-05-03-2349-go-backend-rewrite-progress.md](2026-05-03-2349-go-backend-rewrite-progress.md)) finished M2.1–M2.10 — every pure-domain helper (`czech`, `fees`, `money`, `synch`, `matching`, `reconcile`) is ported. M2.11 and M2.12 close out the M2 milestone by wiring two CLI subcommands to those helpers.
|
||||||
|
|
||||||
|
Both subcommands today are reported as "not implemented" by the dispatcher in [go/cmd/fuj/main.go:32-34](../../go/cmd/fuj/main.go#L32). After this change:
|
||||||
|
|
||||||
|
- `fuj fees` will compose `domain/fees` with a (stubbed) attendance loader and a fees-table formatter.
|
||||||
|
- `fuj reconcile` will compose `domain/reconcile` with stubbed transaction + exception + attendance loaders and a balance-report formatter.
|
||||||
|
- Both will exit with a clean, actionable error message until M4 wires real Google Sheets IO behind the loader interfaces.
|
||||||
|
|
||||||
|
The user asked to do M2.11 and M2.12 together because they share the same loader scaffolding and formatter package — splitting them would either commit half the package or duplicate work.
|
||||||
|
|
||||||
|
## Approach
|
||||||
|
|
||||||
|
**One commit, one branch, one MR.** Branch: `feat/m2-11-12-fees-reconcile-cli`. Both M2.11 and M2.12 checkboxes get ticked on merge.
|
||||||
|
|
||||||
|
The CLI subcommand is the user-facing layer. It owns nothing. All work lives in a new `services/membership` package: a) loader interfaces, b) the stub implementations that fail with a clear error, c) the orchestration functions, and d) the text formatters. M4 will later add real loader implementations behind the same interfaces — no other code needs to change.
|
||||||
|
|
||||||
|
### Package layout
|
||||||
|
|
||||||
|
| Path | Contents |
|
||||||
|
|---|---|
|
||||||
|
| `go/internal/services/membership/doc.go` | Package doc: orchestrates `domain/fees` + `domain/reconcile` against pluggable IO loaders. |
|
||||||
|
| `go/internal/services/membership/loader.go` | `AttendanceLoader`, `TransactionLoader`, `ExceptionLoader` interfaces. Aggregate `Sources` interface. `ErrIOPending` sentinel. `NewStubSources()` factory returning a struct that satisfies all three with `ErrIOPending`. |
|
||||||
|
| `go/internal/services/membership/fees.go` | `FeesReport(ctx, AttendanceLoader, io.Writer) error` — loads adults, formats, writes. |
|
||||||
|
| `go/internal/services/membership/reconcile.go` | `ReconcileReport(ctx, Sources, defaultYear int, io.Writer) error` — loads adults + txns + exceptions, calls `reconcile.Reconcile`, formats, writes. |
|
||||||
|
| `go/internal/services/membership/format_fees.go` | `printFeesTable(w, members, sortedMonths)` — fixed-width table mirroring [calculate_fees.py:9-49](../../scripts/calculate_fees.py#L9). |
|
||||||
|
| `go/internal/services/membership/format_reconcile.go` | `printReconcileReport(w, result, sortedMonths)` — header / summary table / credits / debts / unmatched / matched-tx detail mirroring [match_payments.py:521-640](../../scripts/match_payments.py#L521). |
|
||||||
|
| `go/internal/services/membership/*_test.go` | Table tests for both formatters using fixture data (no IO); separate test confirms `NewStubSources()` returns `ErrIOPending` from each method. |
|
||||||
|
| `go/cmd/fuj/main.go` | Add `feesCmd(args)` and `reconcileCmd(args)`. Drop `"fees"` and `"reconcile"` from the not-implemented case (leave `"sync"`, `"infer"`). |
|
||||||
|
|
||||||
|
### Public API
|
||||||
|
|
||||||
|
```go
|
||||||
|
// loader.go
|
||||||
|
type AttendanceLoader interface {
|
||||||
|
LoadAdults(ctx context.Context) (members []reconcile.Member, sortedMonths []string, err error)
|
||||||
|
}
|
||||||
|
type TransactionLoader interface {
|
||||||
|
LoadTransactions(ctx context.Context) ([]reconcile.Transaction, error)
|
||||||
|
}
|
||||||
|
type ExceptionLoader interface {
|
||||||
|
LoadExceptions(ctx context.Context) (map[reconcile.ExceptionKey]reconcile.Exception, error)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Sources is the aggregate that fuj reconcile needs.
|
||||||
|
type Sources interface {
|
||||||
|
AttendanceLoader
|
||||||
|
TransactionLoader
|
||||||
|
ExceptionLoader
|
||||||
|
}
|
||||||
|
|
||||||
|
// ErrIOPending is returned by every stub loader method.
|
||||||
|
var ErrIOPending = errors.New(
|
||||||
|
"io layer not yet wired up; lands in milestone M4 (sheets/drive/fio)")
|
||||||
|
|
||||||
|
func NewStubSources() Sources
|
||||||
|
|
||||||
|
// fees.go
|
||||||
|
func FeesReport(ctx context.Context, l AttendanceLoader, out io.Writer) error
|
||||||
|
|
||||||
|
// reconcile.go
|
||||||
|
func ReconcileReport(ctx context.Context, s Sources, defaultYear int, out io.Writer) error
|
||||||
|
```
|
||||||
|
|
||||||
|
### Loader return types
|
||||||
|
|
||||||
|
The interfaces return `domain/reconcile` types directly (`reconcile.Member`, `reconcile.Transaction`, `reconcile.ExceptionKey`, `reconcile.Exception`). These are already shaped for the reconcile algorithm and cover the fees report's needs (`Member.Fees[month].Expected` is precomputed by whatever loader populates it — the Python `get_members_with_fees()` does the same). No translation layer needed; no parallel struct hierarchy.
|
||||||
|
|
||||||
|
### Stub implementation pattern
|
||||||
|
|
||||||
|
```go
|
||||||
|
type stubSources struct{}
|
||||||
|
|
||||||
|
func (stubSources) LoadAdults(context.Context) ([]reconcile.Member, []string, error) {
|
||||||
|
return nil, nil, ErrIOPending
|
||||||
|
}
|
||||||
|
func (stubSources) LoadTransactions(context.Context) ([]reconcile.Transaction, error) {
|
||||||
|
return nil, ErrIOPending
|
||||||
|
}
|
||||||
|
func (stubSources) LoadExceptions(context.Context) (
|
||||||
|
map[reconcile.ExceptionKey]reconcile.Exception, error) {
|
||||||
|
return nil, ErrIOPending
|
||||||
|
}
|
||||||
|
|
||||||
|
func NewStubSources() Sources { return stubSources{} }
|
||||||
|
```
|
||||||
|
|
||||||
|
### Subcommand wiring
|
||||||
|
|
||||||
|
```go
|
||||||
|
// cmd/fuj/main.go
|
||||||
|
case "fees":
|
||||||
|
feesCmd(args)
|
||||||
|
case "reconcile":
|
||||||
|
reconcileCmd(args)
|
||||||
|
case "sync", "infer": // keep these in the M4 placeholder
|
||||||
|
fmt.Fprintf(os.Stderr, "fuj %s: not implemented yet (lands in M4)\n", cmd)
|
||||||
|
os.Exit(2)
|
||||||
|
|
||||||
|
func feesCmd(args []string) {
|
||||||
|
fs := flag.NewFlagSet("fees", flag.ExitOnError)
|
||||||
|
fs.Usage = func() { fmt.Fprintln(os.Stderr, "usage: fuj fees") }
|
||||||
|
if err := fs.Parse(args); err != nil { ... }
|
||||||
|
|
||||||
|
sources := membership.NewStubSources()
|
||||||
|
ctx := context.Background()
|
||||||
|
if err := membership.FeesReport(ctx, sources, os.Stdout); err != nil {
|
||||||
|
fmt.Fprintf(os.Stderr, "fuj fees: %v\n", err)
|
||||||
|
os.Exit(1)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func reconcileCmd(args []string) {
|
||||||
|
fs := flag.NewFlagSet("reconcile", flag.ExitOnError)
|
||||||
|
fs.Usage = func() { fmt.Fprintln(os.Stderr, "usage: fuj reconcile") }
|
||||||
|
if err := fs.Parse(args); err != nil { ... }
|
||||||
|
|
||||||
|
sources := membership.NewStubSources()
|
||||||
|
ctx := context.Background()
|
||||||
|
if err := membership.ReconcileReport(ctx, sources, time.Now().Year(), os.Stdout); err != nil {
|
||||||
|
fmt.Fprintf(os.Stderr, "fuj reconcile: %v\n", err)
|
||||||
|
os.Exit(1)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Both subcommands accept zero positional args today (matching Python `calculate_fees.py` which has none, and skipping the Python `match_payments.py` flags `--sheet-id` / `--credentials` / `--bank` until M4 needs them — no point pre-empting flag design while there's nothing to plumb them into). Update `usage()` in `main.go` to remove the `[M2]` annotations from `fees` / `reconcile`.
|
||||||
|
|
||||||
|
### Formatter ports — what to mirror byte-for-byte
|
||||||
|
|
||||||
|
The formatters are pure post-processing. Ship them now so M4 only adds real loader plumbing.
|
||||||
|
|
||||||
|
**`printFeesTable`** ports [calculate_fees.py:9-49](../../scripts/calculate_fees.py#L9):
|
||||||
|
- Filter to `tier == "A"` only.
|
||||||
|
- Month label format: `"Jan 2026"` (`time.Parse("2006-01", m).Format("Jan 2006")`).
|
||||||
|
- Column widths: `name_width = max(len(name))`, `col_width = 15`.
|
||||||
|
- Cell format: `"%d CZK (%d)"` when `count > 0`, else `"-"`. Right-aligned in col_width.
|
||||||
|
- Totals row: monthly sums, label `"TOTAL"`, cells `"%d CZK"`.
|
||||||
|
- Print "No data." when members is empty.
|
||||||
|
|
||||||
|
**`printReconcileReport`** ports [match_payments.py:521-640](../../scripts/match_payments.py#L521):
|
||||||
|
- Header banner (`"=" * 80`, "PAYMENT RECONCILIATION REPORT", banner).
|
||||||
|
- Per-adult summary table — cell logic: `expected==0 && paid==0 → "-"`, `paid>=expected && expected>0 → "OK"`, `paid>0 → "{paid}/{expected}"`, else `"UNPAID {expected}"`. Balance column: `"+N"` / `"-N"` / `"0"`.
|
||||||
|
- TOTAL footer line carrying the `Expected/Paid/Balance` summary.
|
||||||
|
- Optional sections: `TOTAL CREDITS` (positive total balances), `TOTAL DEBTS` (negative — print `abs`), `UNMATCHED TRANSACTIONS`, `MATCHED TRANSACTION DETAILS`.
|
||||||
|
- Use sorted member-name iteration (`sort.Strings`) — Python uses `sorted(adults.keys())`.
|
||||||
|
- Float printing: amounts use `%.0f` (Python `:.0f`), `paid` is cast via `int(...)` before formatting in some places — preserve these (cast to `int` then `%d`).
|
||||||
|
|
||||||
|
Tests use a small handcrafted `reconcile.Result` with one paid member, one debtor, one credit, one unmatched tx and assert exact byte equality of the formatted output (golden string in the test source — not file-based).
|
||||||
|
|
||||||
|
### Tests
|
||||||
|
|
||||||
|
**`format_fees_test.go`**: handcrafted `[]reconcile.Member` covering: tier-filter (J/X excluded), zero-attendance cell, single-attendance cell, multi-attendance cell, empty result → `"No data."`. Golden output strings inline.
|
||||||
|
|
||||||
|
**`format_reconcile_test.go`**: handcrafted `reconcile.Result` exercising every branch in the cell logic + each optional section. Golden strings inline. (Don't blindly copy the Python `print(...)` string-formatting bugs — the live Python f-string `f"\n{'TOTAL CREDITS (advance payments or surplus):'}"` is intentional whitespace; reproduce as plain `"\nTOTAL CREDITS (advance payments or surplus):"` and verify identical bytes by running the live Python on the same fixture.)
|
||||||
|
|
||||||
|
**`stub_test.go`**: assert each `NewStubSources()` method returns `ErrIOPending` (use `errors.Is`).
|
||||||
|
|
||||||
|
**`fees_test.go` / `reconcile_test.go`**: pass a fake loader that returns canned `[]reconcile.Member` / `[]reconcile.Transaction` / exceptions; assert `FeesReport` / `ReconcileReport` write the expected formatter output. This proves the orchestration glues correctly without involving stubs.
|
||||||
|
|
||||||
|
Verify formatter golden strings against live Python with one-liner comments at top of each test file, e.g.:
|
||||||
|
|
||||||
|
```
|
||||||
|
PYTHONPATH=scripts:. python -c '
|
||||||
|
from match_payments import print_report
|
||||||
|
result = {"members": {...}, "unmatched": [...]}
|
||||||
|
print_report(result, ["2026-04"])
|
||||||
|
'
|
||||||
|
```
|
||||||
|
|
||||||
|
## Parity concerns
|
||||||
|
|
||||||
|
- **`czech.Normalize` keeps "%" semantics** — exception keys in `reconcile.Reconcile` use `czech.Normalize(name)` and `czech.Normalize(period)`. The `ExceptionLoader` stub doesn't return any, so this isn't exercised in M2.11/M2.12 — but real loaders in M4 must also normalize at load time (matching Python `fetch_exceptions`).
|
||||||
|
- **Sorted month iteration** — formatter must respect the `sortedMonths` argument order, not iterate maps directly (Go map iteration is randomized).
|
||||||
|
- **Sorted member iteration** — adults sorted by name (`sort.Strings`); Python uses `sorted(adults.keys())` which is byte-order. Czech-diacritic names sort by codepoint either way.
|
||||||
|
- **Empty unmatched list** — Python prints nothing; Go must skip the section header when `len(result.Unmatched) == 0`.
|
||||||
|
- **`int(paid)` truncation** — Python `int(mdata["paid"])` truncates toward zero. Go `int(float64)` matches. Use `int(paid)` not `math.Round`.
|
||||||
|
- **Stub error stable string** — `ErrIOPending.Error()` text is part of the user-facing CLI contract for the duration of M2.11..M3; tests assert `errors.Is`, not the string. Don't change the wording without bumping the changelog.
|
||||||
|
- **Default year = `time.Now().Year()`** — `reconcile.Reconcile` needs `defaultYear` for the inference fallback. `time.Now().Year()` matches the Python implicit default for current-year operation. Tests use a fixed year (2026).
|
||||||
|
|
||||||
|
## Critical files
|
||||||
|
|
||||||
|
- **Read for parity** — [scripts/calculate_fees.py](../../scripts/calculate_fees.py) (full), [scripts/match_payments.py:521-640](../../scripts/match_payments.py#L521), [scripts/match_payments.py:647-684](../../scripts/match_payments.py#L647) (CLI entry shape).
|
||||||
|
- **Reuse** — `domain/reconcile.{Member, Transaction, ExceptionKey, Exception, Result, Reconcile}` ([reconcile.go](../../go/internal/domain/reconcile/reconcile.go)), `domain/fees.{CalculateFee, AdultFeeMonthlyRate}` ([fees.go](../../go/internal/domain/fees/fees.go)).
|
||||||
|
- **Mirror conventions** — package layout from [go/internal/domain/matching/](../../go/internal/domain/matching/) (one symbol per file, `*_test.go` siblings, top-of-test live-Python verification comments).
|
||||||
|
- **New** — `go/internal/services/membership/{doc,loader,fees,reconcile,format_fees,format_reconcile}.go` + `*_test.go`.
|
||||||
|
- **Modify** — [go/cmd/fuj/main.go](../../go/cmd/fuj/main.go) (add `feesCmd`/`reconcileCmd`, drop `"fees"`/`"reconcile"` from the M2 not-implemented case, drop `[M2]` annotations from `usage()`).
|
||||||
|
|
||||||
|
## Out of scope (explicitly DO NOT touch)
|
||||||
|
|
||||||
|
- Real Google Sheets / Drive / Fio loader implementations — M4.1–M4.6.
|
||||||
|
- Web routes / handlers — M5.
|
||||||
|
- `fuj sync` and `fuj infer` subcommands — M4.7/M4.8.
|
||||||
|
- Junior fees report — current Python `make fees` only prints adults; preserve. (`get_junior_members_with_fees` is consumed by the web frontend, not the CLI.)
|
||||||
|
- Bank-direct mode (`--bank` flag in [match_payments.py:659](../../scripts/match_payments.py#L659)) — M4 territory.
|
||||||
|
- Fixture capture (`tests/fixtures/`) — M3 milestone.
|
||||||
|
|
||||||
|
## Verification
|
||||||
|
|
||||||
|
1. `cd go && go build ./...` — clean build.
|
||||||
|
2. `cd go && go test -race ./internal/services/membership/...` — formatter golden strings match, stub returns `ErrIOPending`, orchestration glues fake loader → formatter correctly.
|
||||||
|
3. `cd go && make go-lint` — clean (govet, staticcheck, errcheck, gofumpt, unused).
|
||||||
|
4. **End-to-end CLI smoke**:
|
||||||
|
- `make go-build && ./bin/fuj fees` → exits non-zero, stderr contains `"io layer not yet wired up; lands in milestone M4"`.
|
||||||
|
- `./bin/fuj reconcile` → same shape.
|
||||||
|
- `./bin/fuj help` → no longer says `[M2]` next to `fees`/`reconcile`; still says `[M4]` next to `sync`/`infer`.
|
||||||
|
5. **Formatter parity spot-check** — pick one fixture (1 adult with 2 months of fees, 1 unmatched tx); run the Python equivalent with the same input; confirm Go output is byte-identical (modulo trailing-whitespace lines — `diff -w` if needed, but aim for clean `diff` first).
|
||||||
|
6. Append CHANGELOG entry per [CLAUDE.md](../../CLAUDE.md) (timestamp via `date "+%Y-%m-%d %H:%M %Z"`).
|
||||||
|
7. Tick M2.11 and M2.12 in [docs/plans/2026-05-03-2349-go-backend-rewrite-progress.md](2026-05-03-2349-go-backend-rewrite-progress.md) with the merge SHA. Update the M2 milestone summary line if M2 is now fully closed.
|
||||||
|
8. Push branch, open MR via `tea pr create --title "feat(go): wire fuj fees + fuj reconcile (M2.11-12)" --base main --head feat/m2-11-12-fees-reconcile-cli`, print URL, leave merge to user.
|
||||||
261
docs/plans/2026-05-06-2111-go-m3-fixture-capture.md
Normal file
261
docs/plans/2026-05-06-2111-go-m3-fixture-capture.md
Normal file
@@ -0,0 +1,261 @@
|
|||||||
|
# M3 — Fixture capture + characterization framework
|
||||||
|
|
||||||
|
> On approval: copy this plan to `docs/plans/2026-05-06-2111-go-m3-fixture-capture.md` per [CLAUDE.md](../../srv/personal/fuj-management/CLAUDE.md) plan-location convention.
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
The Go rewrite (tracked in [docs/plans/2026-05-03-2349-go-backend-rewrite-progress.md](../../srv/personal/fuj-management/docs/plans/2026-05-03-2349-go-backend-rewrite-progress.md)) finished M2.1–M2.12 — every pure-domain helper is ported and the `fuj fees` / `fuj reconcile` CLIs are wired. M3 closes the loop: it builds the **parity safety net** that proves Go output matches Python output for every ported function. Without it, M2 is "trust me", and the rewrite has no defensible cutover criterion.
|
||||||
|
|
||||||
|
M3 has three deliverables:
|
||||||
|
|
||||||
|
1. **A capture pipeline** (`scripts/capture_fixtures.py` + `scripts/scrub_fixtures.py`) that produces deterministic, PII-free JSON fixtures from the live Python implementations.
|
||||||
|
2. **A fixture corpus** at [go/tests/fixtures/](../../srv/personal/fuj-management/go/tests/fixtures/) covering the 10 pure functions of M2 (M2.1–M2.9) plus 10 reconcile cases spanning every code path of `reconcile()` (M2.10).
|
||||||
|
3. **A parity test runner** in [go/tests/parity/](../../srv/personal/fuj-management/go/tests/parity/) under `//go:build parity` that replays each fixture and asserts byte/value equality against the Go port.
|
||||||
|
|
||||||
|
User-confirmed scope decisions:
|
||||||
|
- **Single MR** for all six sub-tasks (M3.1–M3.6) — they're tightly coupled; no half-state is committable.
|
||||||
|
- **Type envelope only where it matters** — four fields (`generate_sync_id.tx.amount`, `parse_czk_amount.val`, `format_date.val`, `infer_transaction_details.tx.date`) use `{"type":..., "value":...}` to disambiguate int/float/none. Everything else uses raw JSON.
|
||||||
|
- **Real seeds for `parse_month_references` and `match_members` only** — read curated message strings from `tmp/payments_transactions_cache.json`, scrub, ship. Other functions stay on handcrafted seeds.
|
||||||
|
- **Plan committed at `docs/plans/2026-05-06-2111-go-m3-fixture-capture.md`** — same convention as every M-series predecessor.
|
||||||
|
|
||||||
|
## Branch + landing
|
||||||
|
|
||||||
|
- Branch: `feat/m3-fixture-capture`. Single MR via `tea pr create`. Tick M3.1–M3.6 on merge with the SHA.
|
||||||
|
- No edits to existing Python or Go production code. M3 is purely additive: new scripts, new fixtures, new test files, new Makefile targets, README, CHANGELOG entry, plan archive, progress tracker tick.
|
||||||
|
|
||||||
|
## File layout
|
||||||
|
|
||||||
|
**Python (capture pipeline):**
|
||||||
|
- [scripts/capture_fixtures.py](../../srv/personal/fuj-management/scripts/capture_fixtures.py) — dispatcher CLI; one entry per function via `--func`.
|
||||||
|
- [scripts/scrub_fixtures.py](../../srv/personal/fuj-management/scripts/scrub_fixtures.py) — stdin→stdout deterministic bijection scrubber.
|
||||||
|
- [scripts/_fixture_seeds.py](../../srv/personal/fuj-management/scripts/_fixture_seeds.py) — internal: handcrafted seeds keyed by `(func, case_id)`, plus the curated real-message extractor.
|
||||||
|
|
||||||
|
**Fixture corpus** (committed, PII-free):
|
||||||
|
- [go/tests/fixtures/README.md](../../srv/personal/fuj-management/go/tests/fixtures/README.md) — refresh workflow + scrubbing audit guide.
|
||||||
|
- `go/tests/fixtures/pure/<func>/<case>.json` — one directory per function (10 functions: `normalize`, `parse_month_references`, `calculate_fee`, `calculate_junior_fee`, `parse_czk_amount`, `generate_sync_id`, `build_name_variants`, `match_members`, `infer_transaction_details`, `format_date`).
|
||||||
|
- `go/tests/fixtures/reconcile/<NN>_<case>.json` — 10 numbered reconcile cases.
|
||||||
|
|
||||||
|
**Go parity tests** (all under `//go:build parity`):
|
||||||
|
- [go/tests/parity/parityio.go](../../srv/personal/fuj-management/go/tests/parity/parityio.go) — shared loader with generic `Case[I,O]` walker, type envelopes mirrored from §3, float tolerance helper.
|
||||||
|
- [go/tests/parity/pure/<func>/<func>_parity_test.go](../../srv/personal/fuj-management/go/tests/parity/pure/) — one file per function, ~30 lines each.
|
||||||
|
- [go/tests/parity/reconcile/reconcile_parity_test.go](../../srv/personal/fuj-management/go/tests/parity/reconcile/) — bespoke comparator using `math.Abs(got-want) <= 0.01` for `paid` floats, exact equality on int balances.
|
||||||
|
|
||||||
|
**Modified:**
|
||||||
|
- [Makefile](../../srv/personal/fuj-management/Makefile) — append `go-parity`, `go-test-all`, `capture-fixtures` targets.
|
||||||
|
- [CHANGELOG.md](../../srv/personal/fuj-management/CHANGELOG.md) — single entry at top.
|
||||||
|
- [docs/plans/2026-05-03-2349-go-backend-rewrite-progress.md](../../srv/personal/fuj-management/docs/plans/2026-05-03-2349-go-backend-rewrite-progress.md) — tick M3.1–M3.6 with SHA.
|
||||||
|
|
||||||
|
## Capture invocation interface
|
||||||
|
|
||||||
|
Two-stage pipeline (capture | scrub) so each stage is independently debuggable:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python scripts/capture_fixtures.py --func <name> --case <id> --input-seed <id> \
|
||||||
|
| python scripts/scrub_fixtures.py \
|
||||||
|
> go/tests/fixtures/pure/<func>/<id>.json
|
||||||
|
```
|
||||||
|
|
||||||
|
Capture flags:
|
||||||
|
- `--func` — target function (`normalize`, `reconcile`, etc.).
|
||||||
|
- `--case` — human-authored case ID, becomes the file stem. Never auto-generated (auto-IDs cause git churn).
|
||||||
|
- `--input-seed <id>` — pull from `_fixture_seeds.py` registry (the default mode for handcrafted cases).
|
||||||
|
- `--input-stdin` — read a single JSON `{"args":[...], "kwargs":{...}}` doc from stdin (used by the real-message extractor for `parse_month_references` / `match_members`).
|
||||||
|
- `--all` — iterate every seed for one function, emit newline-delimited JSON to stdout. Used by the `make capture-fixtures` recipe.
|
||||||
|
|
||||||
|
Capture **never writes files**. Output goes to stdout; the caller redirects. The scrubber is always stdin→stdout. Both are pure transforms.
|
||||||
|
|
||||||
|
The `make capture-fixtures` target codifies the full refresh workflow. Humans read the target before they read the README.
|
||||||
|
|
||||||
|
## Fixture JSON shape (normative)
|
||||||
|
|
||||||
|
One JSON object per case:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"case": "range_wrap_nov_to_jan",
|
||||||
|
"func": "scripts.czech_utils.parse_month_references",
|
||||||
|
"captured_at": "2026-05-06",
|
||||||
|
"input": { ... },
|
||||||
|
"output": { ... }
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
`captured_at` is date-only — same-day re-runs produce byte-identical files. No git SHA, no hostname, no time component.
|
||||||
|
|
||||||
|
### Per-function input/output schemas
|
||||||
|
|
||||||
|
The schema is the **stable contract** between Python capture and Go consumption. Where Python returns heterogeneous types, the capture step pre-translates to the typed shape Go expects.
|
||||||
|
|
||||||
|
| Function | Input | Output |
|
||||||
|
|---|---|---|
|
||||||
|
| `normalize` | `{"text":"…"}` | `{"text":"…"}` |
|
||||||
|
| `parse_month_references` | `{"text":"…","default_year":2026}` | `{"months":["2026-01",…]}` |
|
||||||
|
| `calculate_fee` | `{"attendance_count":3,"month_key":"2026-02"}` | `{"fee":750}` |
|
||||||
|
| `calculate_junior_fee` | `{"attendance_count":1,"month_key":"2026-02"}` | `{"value":0,"unknown":true}` (mirrors `fees.Expected{Value, Unknown}`) |
|
||||||
|
| `parse_czk_amount` | `{"val":<envelope>}` | `{"amount":1500.0}` |
|
||||||
|
| `generate_sync_id` | `{"tx":{"date":"…","amount":<envelope>,"currency":"CZK","sender":"…","vs":"…","message":"…","bank_id":"…"}}` | `{"sync_id":"<sha256-hex>"}` |
|
||||||
|
| `_build_name_variants` | `{"name":"…"}` | `{"variants":["…"]}` |
|
||||||
|
| `match_members` | `{"text":"…","member_names":["…"]}` | `{"matches":[{"name":"…","confidence":"auto"}]}` |
|
||||||
|
| `infer_transaction_details` | `{"tx":{"sender":"…","message":"…","user_id":"…","date":<envelope>},"member_names":[…],"default_year":2026}` | `{"members":[…],"months":[…],"search_text":"…"}` |
|
||||||
|
| `format_date` | `{"val":<envelope>}` | `{"date":"…"}` |
|
||||||
|
|
||||||
|
**Type envelope** (used in 4 fields above):
|
||||||
|
|
||||||
|
```json
|
||||||
|
{"type":"int","value":750} // distinguishes 750 from 750.0
|
||||||
|
{"type":"float","value":750.0}
|
||||||
|
{"type":"string","value":"…"}
|
||||||
|
{"type":"none"}
|
||||||
|
```
|
||||||
|
|
||||||
|
The envelope is the answer to the `generate_sync_id` parity risk: Python's `str(750.0) == "750.0"` vs `str(750) == "750"` produces different SHA-256 inputs. JSON natively conflates these; the envelope round-trips them. Go's loader switches on `type` and constructs the matching native value before calling the port.
|
||||||
|
|
||||||
|
**`reconcile`** uses raw JSON for everything (its inputs are typed maps/slices already), with one nuance: the `Member.fees[month]` value can be an `int` or a `(fee, count)` tuple per [match_payments.py:339-340](../../srv/personal/fuj-management/scripts/match_payments.py#L339). Capture normalises both to `{"fee":int,"count":int}` so Go side has one shape.
|
||||||
|
|
||||||
|
## Scrubber strategy
|
||||||
|
|
||||||
|
`scrub_fixtures.py`: stdin → stdout, no state, no salt, no random. Deterministic plain SHA-256. Re-runs are idempotent. Trade-off acknowledged: an attacker with the script can mathematically reverse the mapping. That's fine — the scrubber's job is to keep PII out of git diffs and Claude transcripts, not to defend against an adversary with the source tree.
|
||||||
|
|
||||||
|
### Scramble whitelist (only these field keys are scrambled)
|
||||||
|
|
||||||
|
`name`, `member_names[]`, `person`, `sender`, `sender_account`, `account`, `vs`, `bank_id`, `user_id`, `note`. Plus a per-document name-substring sweep over `message` strings — applied **before** the field-key walk, because real names show up embedded in message text.
|
||||||
|
|
||||||
|
Everything else (dates, amounts, currency, `month_key`, `attendance_count`, `purpose`, `confidence`, `expected`, `paid`, `total_balance`, `fee`, all `YYYY-MM` keys, `match`/`matches` structure) is preserved verbatim. **Whitelist-of-scramble** (not blacklist-of-preserve): when a new field appears, it stays raw until someone explicitly adds it to the list. Fails safe.
|
||||||
|
|
||||||
|
### Scrambling functions
|
||||||
|
|
||||||
|
- **Names**: `Member_<8hex>` where `<8hex> = sha256(name).hexdigest()[:8]`. Same name → same pseudonym across the whole document and across all fixtures. Stable diffs.
|
||||||
|
- **Account numbers** (`[0-9]+/[0-9]{4}`): scramble prefix and bank-suffix separately, preserving length and format.
|
||||||
|
- **VS / bank_id / user_id**: digit-string-preserving hash to a same-length numeric token. Non-numeric input → `id_<8hex>`.
|
||||||
|
- **Note**: replaced verbatim with `"<scrubbed>"`. Notes are never load-bearing for any test.
|
||||||
|
- **Message** (free text): name-sweep applied; rest preserved. Corpus author spot-checks before commit. README §5 documents the audit grep.
|
||||||
|
|
||||||
|
## Reconcile fixtures (10 handcrafted cases)
|
||||||
|
|
||||||
|
All seeds live in `_fixture_seeds.py` as triples `(members, sorted_months, transactions, exceptions, default_year)`. Capture runs the live Python `reconcile()` and emits canonical JSON; scrubber is a no-op for handcrafted synthetic names but runs anyway for uniformity.
|
||||||
|
|
||||||
|
| File | Branch exercised |
|
||||||
|
|---|---|
|
||||||
|
| `01_greedy_exact.json` | Greedy: amount == sum(expected); zero credit. |
|
||||||
|
| `02_greedy_overpayment_credit.json` | Greedy with overflow → credit. |
|
||||||
|
| `03_proportional_remainder.json` | Underpayment across 3 months with non-integer split (last month absorbs float remainder per [match_payments.py:421+](../../srv/personal/fuj-management/scripts/match_payments.py#L421)). |
|
||||||
|
| `04_even_split_prepayment.json` | All `expected == 0` → even-split fallback. |
|
||||||
|
| `05_out_of_window_credit.json` | Month outside `sorted_months` → that share goes to credits, in-window proportional for the rest. |
|
||||||
|
| `06_exception_override.json` | Exception entry overrides expected. |
|
||||||
|
| `07_other_purpose_split.json` | `purpose="other:tournament"` with two members. |
|
||||||
|
| `08_junior_question_mark.json` | Junior with attendance count 1 → `Expected{Unknown:true}`; reconcile reads it as 0 expected. |
|
||||||
|
| `09_multiperson_multimonth.json` | `person="Alice, Bob", purpose="2026-01, 2026-02"` → 2x2 fan-out: even-split-by-people then proportional-by-month. |
|
||||||
|
| `10_unmatched.json` | Empty `person`, garbage message → goes to `unmatched`. |
|
||||||
|
|
||||||
|
The seed registry is the **single source of truth** for these inputs. If Python behaviour drifts intentionally, fixtures regenerate cleanly via `make capture-fixtures`.
|
||||||
|
|
||||||
|
## Real-data seeds (for `parse_month_references` and `match_members` only)
|
||||||
|
|
||||||
|
`_fixture_seeds.py` reads `tmp/payments_transactions_cache.json` (already gitignored) and selects:
|
||||||
|
|
||||||
|
- **`parse_month_references`**: ~15 distinct messages exercising the 45 Czech month declensions, range wraps (`"prosinec-leden"`), year inference, and the `m >= 10 → previous year` heuristic. Selection done once interactively, the chosen indices hardcoded into `_fixture_seeds.py` so re-runs are deterministic. Messages flow through capture (which calls `parse_month_references(msg, default_year=2026)`) then scrubber (name-sweep against the live member roster).
|
||||||
|
- **`match_members`**: ~10 distinct `(message, member_names)` pairs exercising auto vs review confidence, common-surname filter, exact-short-circuit. Same pipeline.
|
||||||
|
|
||||||
|
**Out of scope for real seeds**: `normalize`, `_build_name_variants`, `reconcile`. These either don't benefit from real data (synthetic exhaustively covers `normalize`, `_build_name_variants`) or have surgical-input requirements that real data can't reliably hit (`reconcile`'s 10 branches).
|
||||||
|
|
||||||
|
## Go parity-test layout
|
||||||
|
|
||||||
|
One file per function, one Go package per function, mirroring the fixture tree. Each file is short (~30 lines):
|
||||||
|
|
||||||
|
```go
|
||||||
|
//go:build parity
|
||||||
|
|
||||||
|
package normalize_parity_test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fuj-management/go/internal/domain/czech"
|
||||||
|
"fuj-management/go/tests/parity"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestNormalizeParity(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
parity.RunAll(t, "../../../fixtures/pure/normalize",
|
||||||
|
func(in parity.NormalizeIn) parity.NormalizeOut {
|
||||||
|
return parity.NormalizeOut{Text: czech.Normalize(in.Text)}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
The shared [go/tests/parity/parityio.go](../../srv/personal/fuj-management/go/tests/parity/parityio.go) (also `//go:build parity`) provides:
|
||||||
|
|
||||||
|
- `Case[I, O any]` generic loader: walks a fixture directory, decodes each `.json`, returns `(name, input, want)` triples.
|
||||||
|
- `RunAll[I, O any](t, dir, fn func(I) O)`: invokes `fn`, compares against `want` with `reflect.DeepEqual` (sorted-slice normalisation for the few sets-cast-to-lists Python returns); for floats uses `math.Abs(got-want) <= 0.01`.
|
||||||
|
- One typed `<Func>In` / `<Func>Out` struct pair per function (10 pairs), mirroring §3's JSON shape exactly. Envelope decoder helpers (`AmountEnvelope`, `ValueEnvelope`) live here.
|
||||||
|
|
||||||
|
**Reconcile is bespoke** — `reconcile/reconcile_parity_test.go` doesn't use `RunAll` because it needs cell-by-cell tolerant float compare across nested maps. It walks the fixture dir directly.
|
||||||
|
|
||||||
|
**Why one-file-per-function** (instead of an umbrella runner): each function lives in a different domain package, so tests must `import` a different package; an umbrella would obscure which package is being checked. Split also enables `go test -tags=parity ./tests/parity/pure/normalize/` to iterate on a single port.
|
||||||
|
|
||||||
|
**Why a separate test tree** (instead of co-located parity tests): the M2 unit tests are co-located by convention (e.g. [go/internal/domain/czech/normalize_test.go](../../srv/personal/fuj-management/go/internal/domain/czech/normalize_test.go)). The progress tracker explicitly says fixtures live at `go/tests/fixtures/` and the gate is `go test -tags=parity ./tests/parity/pure/...`. Co-location would scatter fixtures across packages — messy. Separate tree wins.
|
||||||
|
|
||||||
|
## Build tag + Makefile
|
||||||
|
|
||||||
|
Every parity test file starts with `//go:build parity`. Default `make go-test` excludes them; `make go-parity` runs them:
|
||||||
|
|
||||||
|
```makefile
|
||||||
|
go-parity:
|
||||||
|
cd $(GO_SRC) && go test -tags=parity ./tests/parity/...
|
||||||
|
|
||||||
|
go-test-all: go-test go-parity
|
||||||
|
|
||||||
|
capture-fixtures:
|
||||||
|
@bash scripts/capture_all_fixtures.sh # invokes capture | scrub for every seed
|
||||||
|
```
|
||||||
|
|
||||||
|
Parity is **not** folded into default `go-test`: keeps the M2 unit-test loop fast, and a missing-fixture failure shouldn't block routine work. CI runs both targets independently so a parity break is a distinct red signal from a unit-test break.
|
||||||
|
|
||||||
|
## README content (`go/tests/fixtures/README.md`)
|
||||||
|
|
||||||
|
Six sections, ~120 lines:
|
||||||
|
|
||||||
|
1. **What's in this tree** — directory map; one line per fixture function explaining what it validates.
|
||||||
|
2. **Fixture format** — link to schemas in §3; worked example for `parse_month_references` and one for `reconcile`.
|
||||||
|
3. **Refresh workflow** — `make capture-fixtures` regenerates everything; single-file recipe for incremental updates. Always diff before committing.
|
||||||
|
4. **When to refresh** — bullet list (schema change, new Czech declension, new fee tier, new reconcile branch). **Do not refresh to "fix" a parity failure** without first proving the Python behaviour is the intended one.
|
||||||
|
5. **Verifying scrubbing** — `git diff` should show only `Member_<hex>`-shaped names, `<scrubbed>` notes, structurally-preserved account/VS digits. Audit grep: `git ls-files go/tests/fixtures | xargs grep -l '<your real name>'` should return zero before commit.
|
||||||
|
6. **Adding a new fixture** — three steps (add to `_fixture_seeds.py`, run capture, add `In/Out` Go struct fields if needed).
|
||||||
|
|
||||||
|
## Parity concerns
|
||||||
|
|
||||||
|
- **Float arithmetic in reconcile proportional phase**: ordering-sensitive, may diverge between Python and Go due to FMA. Tolerance `0.01` already in [go/internal/domain/reconcile/reconcile_test.go](../../srv/personal/fuj-management/go/internal/domain/reconcile/reconcile_test.go); parity uses the same tolerance.
|
||||||
|
- **Sync-ID float-vs-int stringification**: handled by the envelope (§3). Capture two paired cases per amount value (`amount_750_int.json`, `amount_750_float.json`) so any Go-side conflation surfaces immediately.
|
||||||
|
- **NFKD edge cases**: capture set must include rare characters from real names. The handcrafted `normalize` seeds enumerate every distinct character observed in the live member roster (extracted once from `tmp/attendance_regular_cache.json`, hardcoded into `_fixture_seeds.py` as a single-character-per-case sweep).
|
||||||
|
- **Czech month declensions**: the real-message seeds for `parse_month_references` cover the wild; handcrafted seeds cover the corner cases (`prosinec-leden` wrap, `m >= 10` heuristic).
|
||||||
|
- **Insertion-order determinism in `reconcile`**: Python 3.7+ dict iteration is insertion-ordered; the seed registry preserves order. Go side iterates `sortedMonths` slice explicitly; the parity test verifies this.
|
||||||
|
- **`infer_transaction_details` default_year**: Python signature defaults to 2026; capture passes `default_year` as an explicit input. Go side reads it from the fixture.
|
||||||
|
|
||||||
|
## Out of scope (explicitly DO NOT touch)
|
||||||
|
|
||||||
|
- Real Google Sheets / Drive / Fio loader implementations — M4.1–M4.6.
|
||||||
|
- Web routes / handlers — M5.
|
||||||
|
- `fuj sync` and `fuj infer` subcommands — M4.7/M4.8.
|
||||||
|
- Tier-2 JSON-API parity (`cmd/parity/main.go`) — M5.4.
|
||||||
|
- Any change to existing Python code (capture is read-only against the production scripts).
|
||||||
|
- Any change to existing Go production code under `go/internal/`.
|
||||||
|
|
||||||
|
## Verification
|
||||||
|
|
||||||
|
1. `make go-build` — clean build (parity tests excluded by default tag).
|
||||||
|
2. `make go-test` — all M2 unit tests still green; no parity test runs.
|
||||||
|
3. `make go-parity` — every fixture in `go/tests/fixtures/pure/` and `go/tests/fixtures/reconcile/` deserialises and passes its parity assertion.
|
||||||
|
4. `make go-lint` — clean (parity test files lint-clean under `-tags=parity` since `golangci-lint` honours build tags via `.golangci.yml`).
|
||||||
|
5. **Capture round-trip**: pick one fixture (e.g. `parse_month_references/range_wrap_nov_to_jan.json`), regenerate via `python scripts/capture_fixtures.py --func parse_month_references --case range_wrap_nov_to_jan --input-seed range_wrap_nov_to_jan | python scripts/scrub_fixtures.py`, confirm byte-identical to the committed file.
|
||||||
|
6. **Scrubbing audit**: run the README §5 grep against any name from the live roster — zero hits.
|
||||||
|
7. **Reconcile branch coverage**: read each of the 10 reconcile fixture files, confirm the `output` field shows the expected branch (e.g. `02_greedy_overpayment_credit.json` has a non-zero `credits` entry; `04_even_split_prepayment.json` has equal `paid` across all months).
|
||||||
|
8. Append CHANGELOG entry per [CLAUDE.md](../../srv/personal/fuj-management/CLAUDE.md) (timestamp via `date "+%Y-%m-%d %H:%M %Z"`).
|
||||||
|
9. Tick M3.1–M3.6 in [docs/plans/2026-05-03-2349-go-backend-rewrite-progress.md](../../srv/personal/fuj-management/docs/plans/2026-05-03-2349-go-backend-rewrite-progress.md) with the merge SHA. Update the M3 milestone summary line if M3 is now fully closed.
|
||||||
|
10. Push branch, open MR via `tea pr create --title "feat(go): fixture capture + characterization framework (M3)" --base main --head feat/m3-fixture-capture`, print URL, leave merge to user.
|
||||||
|
|
||||||
|
## Critical files
|
||||||
|
|
||||||
|
- **Read for parity** — [scripts/czech_utils.py:22](../../srv/personal/fuj-management/scripts/czech_utils.py#L22), [scripts/czech_utils.py:28](../../srv/personal/fuj-management/scripts/czech_utils.py#L28), [scripts/attendance.py:91](../../srv/personal/fuj-management/scripts/attendance.py#L91), [scripts/attendance.py:100](../../srv/personal/fuj-management/scripts/attendance.py#L100), [scripts/infer_payments.py:17](../../srv/personal/fuj-management/scripts/infer_payments.py#L17), [scripts/sync_fio_to_sheets.py:62](../../srv/personal/fuj-management/scripts/sync_fio_to_sheets.py#L62), [scripts/match_payments.py:33](../../srv/personal/fuj-management/scripts/match_payments.py#L33), [scripts/match_payments.py:65](../../srv/personal/fuj-management/scripts/match_payments.py#L65), [scripts/match_payments.py:144](../../srv/personal/fuj-management/scripts/match_payments.py#L144), [scripts/match_payments.py:187](../../srv/personal/fuj-management/scripts/match_payments.py#L187), [scripts/match_payments.py:304](../../srv/personal/fuj-management/scripts/match_payments.py#L304).
|
||||||
|
- **Reuse** — `domain/czech.{Normalize, ParseMonthReferences}`, `domain/fees.{CalculateFee, CalculateJuniorFee, Expected}`, `domain/money.ParseCZK`, `domain/synch.GenerateSyncID`, `domain/matching.{BuildNameVariants, MatchMembers, InferTransactionDetails, FormatDate}`, `domain/reconcile.{Member, Transaction, ExceptionKey, Exception, Result, Reconcile}`.
|
||||||
|
- **Mirror conventions** — package layout from [go/internal/domain/matching/](../../srv/personal/fuj-management/go/internal/domain/matching/) (one symbol per file, top-of-test provenance comments, `t.Parallel()`, `// [Go]` markers for Go-only cases).
|
||||||
|
- **New** — `scripts/{capture_fixtures,scrub_fixtures,_fixture_seeds}.py`; `go/tests/fixtures/README.md` + the corpus; `go/tests/parity/parityio.go` + 10 parity test files + 1 reconcile parity test file.
|
||||||
|
- **Modify** — `Makefile` (3 new targets), `CHANGELOG.md` (1 entry), `docs/plans/2026-05-03-2349-go-backend-rewrite-progress.md` (tick M3.1–M3.6).
|
||||||
313
docs/plans/2026-05-06-2341-go-m4-io-layer.md
Normal file
313
docs/plans/2026-05-06-2341-go-m4-io-layer.md
Normal file
@@ -0,0 +1,313 @@
|
|||||||
|
# Plan: Go rewrite — M4 IO layer behind interfaces
|
||||||
|
|
||||||
|
Companion to [2026-05-03-2349-go-backend-rewrite.md](2026-05-03-2349-go-backend-rewrite.md)
|
||||||
|
and [2026-05-03-2349-go-backend-rewrite-progress.md](2026-05-03-2349-go-backend-rewrite-progress.md).
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
M1–M3 are merged: skeleton + tooling, every pure-domain function ported and
|
||||||
|
parity-tested against PII-scrubbed fixtures, and the `fuj fees` / `fuj
|
||||||
|
reconcile` subcommands wired but stubbed (`membership.NewStubSources()`
|
||||||
|
returns `ErrIOPending` for every loader). M4's job is to replace that stub
|
||||||
|
with real IO: read attendance CSVs, read the payments sheet + exceptions
|
||||||
|
tab, fetch Drive `modifiedTime` for cache gating, fetch Fio bank
|
||||||
|
transactions, and append/update rows on the payments sheet — all behind
|
||||||
|
narrow Go interfaces that have in-memory fakes for tests.
|
||||||
|
|
||||||
|
Once M4 lands, `fuj fees`, `fuj reconcile`, `fuj sync`, and `fuj infer` all
|
||||||
|
work end-to-end against the real Google Sheets and the real Fio account, and
|
||||||
|
M5 can start porting the JSON API on top of that IO.
|
||||||
|
|
||||||
|
User-confirmed scope choices for this milestone:
|
||||||
|
- **No live integration tests.** Fakes-only at unit level; live
|
||||||
|
verification deferred to manual smoke during M7.
|
||||||
|
- **Three PRs** (sheets/drive/cache → fio/sync → infer), one per major
|
||||||
|
area, each independently reviewable.
|
||||||
|
- **Attendance stays on CSV-via-public-URL** — matches Python, no extra
|
||||||
|
service-account grant needed.
|
||||||
|
|
||||||
|
## Approach
|
||||||
|
|
||||||
|
### Layering
|
||||||
|
|
||||||
|
```
|
||||||
|
internal/io/ ← raw, narrow clients (one per external system)
|
||||||
|
sheets/ ← typed wrapper around google.golang.org/api/sheets/v4
|
||||||
|
drive/ ← Drive v3, only ModifiedTime
|
||||||
|
attendance/ ← CSV-via-public-URL fetcher (no auth, no Sheets API)
|
||||||
|
fio/ ← FioClient interface + apiClient + transparentClient
|
||||||
|
cache/ ← FileCache: modifiedTime gate + two-TTL fallback + atomic write
|
||||||
|
|
||||||
|
internal/services/membership/ ← already exists; M4 adds adapters that satisfy
|
||||||
|
AttendanceLoader / TransactionLoader / ExceptionLoader
|
||||||
|
by composing io/sheets + io/drive + io/cache + io/attendance.
|
||||||
|
|
||||||
|
internal/services/banksync/ ← new: SyncToSheets (M4.7) + InferPayments (M4.8)
|
||||||
|
composing fio + sheets + attendance loaders.
|
||||||
|
```
|
||||||
|
|
||||||
|
The existing interfaces in [go/internal/services/membership/loader.go](../../go/internal/services/membership/loader.go)
|
||||||
|
(`AttendanceLoader`, `TransactionLoader`, `ExceptionLoader`, `Sources`) are
|
||||||
|
the seam — M4 adds a `NewSources(cfg config.Config) (Sources, error)`
|
||||||
|
constructor next to `NewStubSources()`, and `cmd/fuj/main.go` swaps the
|
||||||
|
stub for it.
|
||||||
|
|
||||||
|
### Auth — service-account only
|
||||||
|
|
||||||
|
Drop the OAuth+`token.pickle` path entirely (the production already uses a
|
||||||
|
service account; the fallback only existed because the original Python
|
||||||
|
script ran from a developer laptop). Sheets and Drive both authenticate via
|
||||||
|
`option.WithCredentialsFile(cfg.CredentialsPath)` plus
|
||||||
|
`option.WithScopes(...)`. Single shared `*http.Client` per backend with a
|
||||||
|
10s timeout (matches `DRIVE_TIMEOUT`).
|
||||||
|
|
||||||
|
### Cache shape
|
||||||
|
|
||||||
|
Match Python's wire format so the `tmp/*_cache.json` directory is shared
|
||||||
|
safely while both backends run side-by-side:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{ "modifiedTime": "<RFC3339>", "data": <list|object>, "cachedAt": "<RFC3339>" }
|
||||||
|
```
|
||||||
|
|
||||||
|
Improvements over Python:
|
||||||
|
- Atomic write: marshal → `os.WriteFile(path+".tmp", ..., 0o600)` →
|
||||||
|
`os.Rename`. Python's plain truncate-write stays as-is until M8.
|
||||||
|
- The two TTLs (`CacheTTL` and `CacheAPICheckTTL`) live in `config.Config`
|
||||||
|
already; only the `CacheDir` field is new.
|
||||||
|
|
||||||
|
The four cache keys mirror Python's `CACHE_SHEET_MAP`:
|
||||||
|
`attendance_regular`, `attendance_juniors`, `exceptions_dict`,
|
||||||
|
`payments_transactions` → maps to either `AttendanceSheetID` or
|
||||||
|
`PaymentsSheetID`.
|
||||||
|
|
||||||
|
When Drive fails, fall back to a synthetic key
|
||||||
|
`fmt.Sprintf("ttl-5m-%d", time.Now().Unix()/300)` so cache still keys
|
||||||
|
deterministically per 5-min bucket (same as Python).
|
||||||
|
|
||||||
|
### Fio: two impls behind one interface
|
||||||
|
|
||||||
|
```go
|
||||||
|
type Client interface {
|
||||||
|
FetchTransactions(ctx context.Context, from, to time.Time) ([]Transaction, error)
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
`apiClient` (when `cfg.FioAPIToken != ""`) hits
|
||||||
|
`https://fioapi.fio.cz/v1/rest/periods/{token}/{from}/{to}/transactions.json`,
|
||||||
|
unmarshals via a typed struct, and maps `column0..column22` to fields per
|
||||||
|
[scripts/fio_utils.py](../../scripts/fio_utils.py:90). Negative-amount rows
|
||||||
|
dropped (matches Python).
|
||||||
|
|
||||||
|
`transparentClient` (fallback) GETs
|
||||||
|
`https://ib.fio.cz/ib/transparent?a={accountNum}&f={DD.MM.YYYY}&t={DD.MM.YYYY}`
|
||||||
|
and walks the response with `golang.org/x/net/html` token visitor, counting
|
||||||
|
`<table class="table">` tags and grabbing rows from the **second** one
|
||||||
|
(skipping `<thead>`). `bank_id`, `currency`, `user_id`, `sender_account`
|
||||||
|
are empty (matches Python — known limitation).
|
||||||
|
|
||||||
|
`accountNum` is derived from `cfg.BankAccount` by stripping the IBAN prefix
|
||||||
|
(`CZ85 2010 0000 0028 0035 9168` → `2800359168`); add a small helper in
|
||||||
|
`config` for this since both the API URL and the transparent URL need it.
|
||||||
|
|
||||||
|
### Fakes
|
||||||
|
|
||||||
|
In-memory fakes live next to each real impl: `sheets/fake.go`,
|
||||||
|
`drive/fake.go`, `fio/fake.go`, `attendance/fake.go`,
|
||||||
|
`cache/fake.go` (a passthrough). All exported as `Fake` so tests do
|
||||||
|
`sheets.NewFake(rows)` and inject. The membership-adapter tests use these
|
||||||
|
fakes plus a couple of new raw-bytes fixtures under
|
||||||
|
`go/internal/io/<pkg>/testdata/`:
|
||||||
|
|
||||||
|
- `sheets/testdata/payments_minimal.json` — 2D-string array shaped like
|
||||||
|
`values.get` would return.
|
||||||
|
- `sheets/testdata/exceptions_minimal.json` — same, for the exceptions tab.
|
||||||
|
- `attendance/testdata/adults_minimal.csv` — small adult attendance CSV.
|
||||||
|
- `attendance/testdata/juniors_minimal.csv` — small junior CSV.
|
||||||
|
- `fio/testdata/api_response.json` — captured Fio API JSON shape.
|
||||||
|
- `fio/testdata/transparent.html` — captured transparent-page HTML.
|
||||||
|
|
||||||
|
Existing M3 domain fixtures under `go/tests/fixtures/` stay where they are
|
||||||
|
and continue to drive parity tests; they aren't reused for IO-layer tests
|
||||||
|
because they're at the wrong layer (post-parse domain types).
|
||||||
|
|
||||||
|
## Tasks (mapped to tracker)
|
||||||
|
|
||||||
|
Same 8 sub-milestones as the tracker, grouped into 3 PRs.
|
||||||
|
|
||||||
|
### PR 1 — sheets / drive / cache + membership wiring (M4.1, M4.2, M4.3, M4.6)
|
||||||
|
|
||||||
|
1. **Add deps** in [go/go.mod](../../go/go.mod):
|
||||||
|
`google.golang.org/api/{sheets/v4,drive/v3,option}`,
|
||||||
|
`golang.org/x/oauth2/google` (transitively pulled), `golang.org/x/net/html`.
|
||||||
|
2. **`internal/io/sheets/`**:
|
||||||
|
- `client.go` — `Client` struct holding `*sheets.Service`; methods
|
||||||
|
`GetValues(ctx, spreadsheetID, a1Range string) ([][]any, error)`,
|
||||||
|
`AppendValues(ctx, spreadsheetID, a1Range string, rows [][]any) error`,
|
||||||
|
`BatchUpdateValues(ctx, spreadsheetID, updates []ValueRange) error`,
|
||||||
|
`SortByColumn(ctx, spreadsheetID, sheetGID int64, columnIndex int) error`.
|
||||||
|
- `fake.go` — exported `Fake` with seedable `Values map[string][][]any`.
|
||||||
|
3. **`internal/io/drive/`**:
|
||||||
|
- `client.go` — `Client.ModifiedTime(ctx, fileID string) (string, error)`
|
||||||
|
using `drive.New(...).Files.Get(fileID).Fields("modifiedTime").SupportsAllDrives(true)`.
|
||||||
|
- `fake.go` with seedable `Times map[string]string`.
|
||||||
|
4. **`internal/io/attendance/`** (new — public-URL CSV):
|
||||||
|
- `client.go` — `Client.FetchAdults(ctx) ([][]string, error)` and
|
||||||
|
`FetchJuniors(ctx) ([][]string, error)` using `http.Get` on
|
||||||
|
`https://docs.google.com/spreadsheets/d/{ID}/export?format=csv&gid={GID}`,
|
||||||
|
decoded via `encoding/csv`.
|
||||||
|
- Add `AttendanceAdultSheetGID = "0"` constant in `internal/config`.
|
||||||
|
5. **`internal/io/cache/`**:
|
||||||
|
- `filecache.go` — `FileCache` with `Get(ctx, key string, fetch func(ctx) (any, error)) (any, error)`
|
||||||
|
wired through `Drive.ModifiedTime` and the two TTL knobs. Atomic write
|
||||||
|
via tmp-file + rename.
|
||||||
|
- Cache key → sheet ID map mirrors Python's `CACHE_SHEET_MAP`.
|
||||||
|
6. **`internal/services/membership/sources.go`** (new file in existing
|
||||||
|
package):
|
||||||
|
- `realSources struct { sheets *sheets.Client; drive *drive.Client; attendance *attendance.Client; cache *cache.FileCache }`.
|
||||||
|
- Constructor `NewSources(ctx, cfg) (Sources, error)` builds all clients.
|
||||||
|
- `LoadAdults` reads cached attendance CSV, runs through
|
||||||
|
`domain/fees.CalculateFee` + merged-month logic (port of
|
||||||
|
[scripts/attendance.py](../../scripts/attendance.py:170)
|
||||||
|
`get_members_with_fees`), returns `[]reconcile.Member`.
|
||||||
|
- `LoadTransactions` reads payments sheet rows via cache, parses to
|
||||||
|
`[]reconcile.Transaction` (port of
|
||||||
|
[match_payments.py:208](../../scripts/match_payments.py:208)
|
||||||
|
`fetch_sheet_data`).
|
||||||
|
- `LoadExceptions` reads `'exceptions'!A2:D` via cache, builds
|
||||||
|
`map[ExceptionKey]Exception` (port of `match_payments.py:266`).
|
||||||
|
7. **Add `LoadJuniors`** to the `AttendanceLoader` interface (Python infer
|
||||||
|
pulls both adult + junior member lists; needed for M4.8).
|
||||||
|
8. **Wire into [cmd/fuj/main.go](../../go/cmd/fuj/main.go)**: replace
|
||||||
|
`membership.NewStubSources()` in `feesCmd` and `reconcileCmd` with
|
||||||
|
`membership.NewSources(ctx, cfg)`.
|
||||||
|
9. **Tests** (default tag, no live IO):
|
||||||
|
- `sheets/client_test.go`, `drive/client_test.go`,
|
||||||
|
`cache/filecache_test.go` — exercise fakes + parsing logic with
|
||||||
|
testdata fixtures.
|
||||||
|
- `membership/sources_test.go` — adapter tests with sheets/drive/cache
|
||||||
|
fakes verify CSV→Member, rows→Transaction, exceptions tab → map.
|
||||||
|
10. **Config additions**: `CacheDir` (default `tmp` relative to `$PWD`,
|
||||||
|
overridable via `CACHE_DIR` env), `DriveTimeout` (default 10s).
|
||||||
|
11. **Manual verification**: `make go-build && go run ./cmd/fuj fees` and
|
||||||
|
`... reconcile` print real reports against the live sheet (with valid
|
||||||
|
`.secret/...credentials.json`).
|
||||||
|
12. CHANGELOG entry; tick M4.1, M4.2, M4.3, M4.6 in the progress tracker.
|
||||||
|
|
||||||
|
### PR 2 — fio + bank sync (M4.4, M4.5, M4.7)
|
||||||
|
|
||||||
|
1. **`internal/io/fio/`**:
|
||||||
|
- `client.go` — `Client` interface, `Transaction` struct.
|
||||||
|
- `api.go` — `apiClient` impl + URL builder + JSON struct definitions
|
||||||
|
for `accountStatement.transactionList.transaction[].column{N}.value`.
|
||||||
|
- `transparent.go` — `transparentClient` impl using
|
||||||
|
`golang.org/x/net/html` token visitor; helper functions
|
||||||
|
`parseCzechAmount` (NBSP/space strip + comma→dot) and
|
||||||
|
`parseCzechDate` (DD.MM.YYYY / DD/MM/YYYY).
|
||||||
|
- `fake.go`.
|
||||||
|
- `New(cfg) Client` chooses impl based on `cfg.FioAPIToken`.
|
||||||
|
- `accountNum(iban)` helper in `internal/config` strips IBAN prefix.
|
||||||
|
2. **`internal/services/banksync/sync.go`** (new package):
|
||||||
|
- `SyncToSheets(ctx, cfg, fio Client, sheets *sheets.Client, opts SyncOpts) (added int, err error)`.
|
||||||
|
- Reads existing rows via `sheets.GetValues(... "A1:K")`, validates
|
||||||
|
header against `COLUMN_LABELS`, writes header if missing, builds
|
||||||
|
`existingIDs` from column K (`Sync ID`).
|
||||||
|
- Computes date window: explicit `from`/`to` or `now - days*24h` (default 30d).
|
||||||
|
- For each fetched tx, computes `domain/synch.GenerateSyncID`, skips if
|
||||||
|
present, otherwise builds row in COLUMN_LABELS order with empty
|
||||||
|
manual/person/purpose/inferred slots.
|
||||||
|
- `sheets.AppendValues(... "A2", rows)`.
|
||||||
|
- Optional sort: `sheets.SortByColumn(... gid, 0)` — sheet GID resolved
|
||||||
|
once via `spreadsheets.Get`.
|
||||||
|
3. **Wire `fuj sync` subcommand** in `cmd/fuj/main.go`:
|
||||||
|
- Flags: `--days N` (default 30), `--from YYYY-MM-DD`, `--to YYYY-MM-DD`,
|
||||||
|
`--sort` (default true matching `make sync-2026`).
|
||||||
|
- Replace the M4-stub error path.
|
||||||
|
4. **Tests** (default tag): `banksync/sync_test.go` with fakes — verify
|
||||||
|
header insertion, dedup against existing sync IDs, multi-row append,
|
||||||
|
sort call.
|
||||||
|
5. **Manual verification**: dry-run sync against the real Fio account in a
|
||||||
|
throwaway test sheet; or visually verify `--from --to` window in stdout
|
||||||
|
with a no-write flag (only if cheap to add — otherwise skip per the
|
||||||
|
"no live integration tests" decision).
|
||||||
|
6. CHANGELOG entry; tick M4.4, M4.5, M4.7.
|
||||||
|
|
||||||
|
### PR 3 — infer (M4.8)
|
||||||
|
|
||||||
|
1. **`internal/services/banksync/infer.go`**:
|
||||||
|
- `InferPayments(ctx, cfg, sheets *sheets.Client, attendanceLoader, juniorLoader, opts InferOpts) (updated int, err error)`.
|
||||||
|
- Reads payments sheet `A1:Z` with case-insensitive header lookup.
|
||||||
|
- Required columns: `Person, Purpose, Inferred Amount`. Optional input:
|
||||||
|
`Date, Amount, Sender, Message, VS, manual fix`.
|
||||||
|
- Skip rule (matches [scripts/infer_payments.py:127](../../scripts/infer_payments.py:127)):
|
||||||
|
non-empty `manual fix` OR `Person` OR `Purpose` → leave row alone.
|
||||||
|
- Member list = union of `LoadAdults` + `LoadJuniors` deduped via
|
||||||
|
`domain/matching.CanonicalKey` (already exists from M2).
|
||||||
|
- For each empty row: build tx dict, call
|
||||||
|
`domain/matching.InferTransactionDetails`, prefix `[?] ` if
|
||||||
|
confidence == "review", emit a `ValueRange` update with R1C1 range
|
||||||
|
`R{i}C{personCol+1}:R{i}C{amountCol+1}`.
|
||||||
|
- Single `sheets.BatchUpdateValues` call for all updates.
|
||||||
|
2. **Wire `fuj infer` subcommand**: flags `--dry-run` (prints planned
|
||||||
|
updates, no API write).
|
||||||
|
3. **Tests** (default tag): `banksync/infer_test.go` — fixture rows,
|
||||||
|
verify skip rule, verify `[?]` prefix on review matches, verify
|
||||||
|
batchUpdate payload shape, verify `--dry-run` is no-op.
|
||||||
|
4. CHANGELOG entry; tick M4.8 → milestone gate ✅.
|
||||||
|
|
||||||
|
## Critical files
|
||||||
|
|
||||||
|
To modify:
|
||||||
|
- [go/internal/services/membership/loader.go](../../go/internal/services/membership/loader.go) — add `LoadJuniors` to `AttendanceLoader`, add `NewSources`.
|
||||||
|
- [go/cmd/fuj/main.go](../../go/cmd/fuj/main.go) — swap stub for real sources, add `sync`/`infer` subcommands.
|
||||||
|
- [go/internal/config/config.go](../../go/internal/config/config.go) — add `CacheDir`, `DriveTimeout`, `AttendanceAdultSheetGID` constant, IBAN→account-num helper.
|
||||||
|
- [go/go.mod](../../go/go.mod) / `go.sum` — google APIs + `x/net/html`.
|
||||||
|
- [docs/plans/2026-05-03-2349-go-backend-rewrite-progress.md](2026-05-03-2349-go-backend-rewrite-progress.md) — tick M4.x boxes after each PR.
|
||||||
|
- [CHANGELOG.md](../../CHANGELOG.md) — entry per PR.
|
||||||
|
|
||||||
|
To create:
|
||||||
|
- `go/internal/io/{sheets,drive,attendance,fio,cache}/{client,fake,*_test}.go`
|
||||||
|
- `go/internal/io/{sheets,attendance,fio}/testdata/*`
|
||||||
|
- `go/internal/services/membership/sources.go` (+ `sources_test.go`)
|
||||||
|
- `go/internal/services/banksync/{sync,infer}.go` (+ tests)
|
||||||
|
|
||||||
|
## Reused existing helpers
|
||||||
|
|
||||||
|
- `domain/fees.CalculateFee` / `CalculateJuniorFee` — fee math (M2.3, M2.4).
|
||||||
|
- `domain/matching.{BuildNameVariants,MatchMembers,InferTransactionDetails,FormatDate,CanonicalKey}` — match logic (M2.7–M2.9).
|
||||||
|
- `domain/synch.GenerateSyncID` — dedup hash (M2.6).
|
||||||
|
- `domain/reconcile.{Member,Transaction,Exception,ExceptionKey}` — domain types.
|
||||||
|
- `domain/czech.{Normalize,ParseMonthReferences}` — used inside the
|
||||||
|
attendance/exceptions parsers.
|
||||||
|
- `domain/money.ParseCZK` — for parsing transparent-scrape amounts.
|
||||||
|
|
||||||
|
## Verification
|
||||||
|
|
||||||
|
End-to-end checks once all three PRs land:
|
||||||
|
|
||||||
|
1. `make go-build && make go-lint && make go-test` — clean.
|
||||||
|
2. `make go-parity` — M3 fixtures still pass (no domain regressions).
|
||||||
|
3. `./bin/fuj fees` — prints adult fee report matching Python `make fees`
|
||||||
|
(visual diff acceptable for now; byte-equality enforced in M5).
|
||||||
|
4. `./bin/fuj reconcile` — prints balance report comparable to
|
||||||
|
[scripts/match_payments.py](../../scripts/match_payments.py) `print_balance_report`.
|
||||||
|
5. `./bin/fuj sync --days 7` — appends new Fio rows to the payments sheet
|
||||||
|
(run with a real but recent date window; verify by counting added rows
|
||||||
|
and confirming no duplicates on a second run).
|
||||||
|
6. `./bin/fuj infer --dry-run` — prints planned Person/Purpose/Inferred
|
||||||
|
Amount updates without modifying the sheet. Then `./bin/fuj infer`
|
||||||
|
applies them; second run is a no-op (skip rule).
|
||||||
|
7. **Cache check**: delete `tmp/*_cache.json`, run `fuj fees`, verify file
|
||||||
|
appears with `modifiedTime` matching Drive. Re-run within 5 min;
|
||||||
|
verify no Drive call (debug log).
|
||||||
|
8. **Cross-process cache safety**: while `make web-py` is running, run
|
||||||
|
`fuj reconcile`; verify Python's cache file isn't corrupted and Go
|
||||||
|
reads the same data.
|
||||||
|
|
||||||
|
Gate (per tracker):
|
||||||
|
> `go test -tags=integration ./internal/io/...` round-trips against test sheet; default-tag tests run on fakes.
|
||||||
|
|
||||||
|
Per the user's scope decision, **the integration-test gate is downgraded
|
||||||
|
to "default-tag tests on fakes" only**. Live verification is deferred to
|
||||||
|
manual smoke during M7's parallel-run watch period. The progress tracker's
|
||||||
|
M4 gate line will be amended in PR 1.
|
||||||
36
docs/plans/2026-05-07-1033-fuj-sync-dry-run.md
Normal file
36
docs/plans/2026-05-07-1033-fuj-sync-dry-run.md
Normal file
@@ -0,0 +1,36 @@
|
|||||||
|
# Plan: add `--dry-run` to `fuj sync`
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
`fuj infer` already supports `--dry-run` (it builds the planned `BatchUpdateValues`
|
||||||
|
operations, prints them, and skips the actual write — see
|
||||||
|
`go/internal/services/banksync/infer.go:136-156` and the
|
||||||
|
`Dry run: would update N row(s).` line in `go/cmd/fuj/main.go:209-213`).
|
||||||
|
|
||||||
|
`fuj sync` had no equivalent. It always committed three potential writes to the
|
||||||
|
payments sheet: `WriteHeader` (if the header row is missing/wrong), `AppendValues`
|
||||||
|
(for each new Fio transaction), and `SortByDateColumn` (if `--sort`, default true).
|
||||||
|
For inspecting what a sync *would* do — useful when debugging dedupe, sanity-checking
|
||||||
|
a date window, or wiring up the command for the first time on a new account — the
|
||||||
|
only options were pointing at a throwaway spreadsheet or reading the diff after the fact.
|
||||||
|
|
||||||
|
This change mirrors `infer`'s read-only mode for `sync`: same flag name, same output
|
||||||
|
style, same "build the data structures, print instead of writing" shape.
|
||||||
|
|
||||||
|
## Files modified
|
||||||
|
|
||||||
|
1. `go/internal/services/banksync/sync.go` — `DryRun bool` field added to `SyncOpts`; three write points gated on `opts.DryRun`
|
||||||
|
2. `go/cmd/fuj/main.go` — `--dry-run` flag added to `syncCmd`; final println split on `*dryRun`
|
||||||
|
3. `go/internal/services/banksync/sync_test.go` — `TestSyncToSheets_DryRun` added
|
||||||
|
4. `CHANGELOG.md` — entry added
|
||||||
|
|
||||||
|
## Behaviour
|
||||||
|
|
||||||
|
When `--dry-run` is set:
|
||||||
|
|
||||||
|
- If the sheet header is missing/wrong → prints `Dry run: would write header row`; skips `WriteHeader`
|
||||||
|
- For each non-deduped Fio transaction → prints `Dry run: would append date=… amount=… sender=… vs=… message=…`; skips `AppendValues`
|
||||||
|
- If `--sort` is true → prints `Dry run: would sort by date`; skips `SortByDateColumn`
|
||||||
|
- Returns `len(newRows)` so the caller can print `Dry run: would sync N new transaction(s).`
|
||||||
|
|
||||||
|
The existing ID-dedup logic runs in full even during dry-run (reads the sheet, builds `existingIDs`), so the output reflects exactly what the next real sync would do.
|
||||||
@@ -1,12 +1,18 @@
|
|||||||
package main
|
package main
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"context"
|
||||||
"flag"
|
"flag"
|
||||||
"fmt"
|
"fmt"
|
||||||
"fuj-management/go/internal/config"
|
"fuj-management/go/internal/config"
|
||||||
|
"fuj-management/go/internal/io/fio"
|
||||||
|
"fuj-management/go/internal/io/sheets"
|
||||||
"fuj-management/go/internal/logging"
|
"fuj-management/go/internal/logging"
|
||||||
|
"fuj-management/go/internal/services/banksync"
|
||||||
|
"fuj-management/go/internal/services/membership"
|
||||||
"fuj-management/go/internal/web"
|
"fuj-management/go/internal/web"
|
||||||
"os"
|
"os"
|
||||||
|
"time"
|
||||||
)
|
)
|
||||||
|
|
||||||
// Injected at build time via -ldflags "-X main.version=... -X main.commit=... -X main.buildDate=..."
|
// Injected at build time via -ldflags "-X main.version=... -X main.commit=... -X main.buildDate=..."
|
||||||
@@ -29,9 +35,14 @@ func main() {
|
|||||||
serverCmd(args)
|
serverCmd(args)
|
||||||
case "version":
|
case "version":
|
||||||
versionCmd()
|
versionCmd()
|
||||||
case "fees", "reconcile", "sync", "infer":
|
case "fees":
|
||||||
fmt.Fprintf(os.Stderr, "fuj %s: not implemented yet (lands in M2/M4)\n", cmd)
|
feesCmd(args)
|
||||||
os.Exit(2)
|
case "reconcile":
|
||||||
|
reconcileCmd(args)
|
||||||
|
case "sync":
|
||||||
|
syncCmd(args)
|
||||||
|
case "infer":
|
||||||
|
inferCmd(args)
|
||||||
case "-h", "--help", "help":
|
case "-h", "--help", "help":
|
||||||
usage()
|
usage()
|
||||||
default:
|
default:
|
||||||
@@ -67,18 +78,154 @@ func serverCmd(args []string) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func feesCmd(args []string) {
|
||||||
|
fs := flag.NewFlagSet("fees", flag.ExitOnError)
|
||||||
|
fs.Usage = func() {
|
||||||
|
fmt.Fprintln(os.Stderr, "usage: fuj fees")
|
||||||
|
}
|
||||||
|
if err := fs.Parse(args); err != nil {
|
||||||
|
fmt.Fprintln(os.Stderr, err)
|
||||||
|
os.Exit(2)
|
||||||
|
}
|
||||||
|
|
||||||
|
ctx := context.Background()
|
||||||
|
cfg := config.Load()
|
||||||
|
sources, err := membership.NewSources(ctx, cfg)
|
||||||
|
if err != nil {
|
||||||
|
fmt.Fprintf(os.Stderr, "fuj fees: %v\n", err)
|
||||||
|
os.Exit(1)
|
||||||
|
}
|
||||||
|
if err := membership.FeesReport(ctx, sources, os.Stdout); err != nil {
|
||||||
|
fmt.Fprintf(os.Stderr, "fuj fees: %v\n", err)
|
||||||
|
os.Exit(1)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func reconcileCmd(args []string) {
|
||||||
|
fs := flag.NewFlagSet("reconcile", flag.ExitOnError)
|
||||||
|
fs.Usage = func() {
|
||||||
|
fmt.Fprintln(os.Stderr, "usage: fuj reconcile")
|
||||||
|
}
|
||||||
|
if err := fs.Parse(args); err != nil {
|
||||||
|
fmt.Fprintln(os.Stderr, err)
|
||||||
|
os.Exit(2)
|
||||||
|
}
|
||||||
|
|
||||||
|
ctx := context.Background()
|
||||||
|
cfg := config.Load()
|
||||||
|
sources, err := membership.NewSources(ctx, cfg)
|
||||||
|
if err != nil {
|
||||||
|
fmt.Fprintf(os.Stderr, "fuj reconcile: %v\n", err)
|
||||||
|
os.Exit(1)
|
||||||
|
}
|
||||||
|
if err := membership.ReconcileReport(ctx, sources, time.Now().Year(), os.Stdout); err != nil {
|
||||||
|
fmt.Fprintf(os.Stderr, "fuj reconcile: %v\n", err)
|
||||||
|
os.Exit(1)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func versionCmd() {
|
func versionCmd() {
|
||||||
fmt.Printf("fuj %s (%s) built %s\n", version, commit, buildDate)
|
fmt.Printf("fuj %s (%s) built %s\n", version, commit, buildDate)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func syncCmd(args []string) {
|
||||||
|
fs := flag.NewFlagSet("sync", flag.ExitOnError)
|
||||||
|
days := fs.Int("days", 30, "look-back window in days (ignored when --from/--to are set)")
|
||||||
|
fromStr := fs.String("from", "", "start date YYYY-MM-DD")
|
||||||
|
toStr := fs.String("to", "", "end date YYYY-MM-DD")
|
||||||
|
sort := fs.Bool("sort", true, "sort sheet by date after appending")
|
||||||
|
dryRun := fs.Bool("dry-run", false, "print planned writes without modifying the sheet")
|
||||||
|
fs.Usage = func() {
|
||||||
|
fmt.Fprintln(os.Stderr, "usage: fuj sync [--days N] [--from YYYY-MM-DD --to YYYY-MM-DD] [--sort] [--dry-run]")
|
||||||
|
fs.PrintDefaults()
|
||||||
|
}
|
||||||
|
if err := fs.Parse(args); err != nil {
|
||||||
|
fmt.Fprintln(os.Stderr, err)
|
||||||
|
os.Exit(2)
|
||||||
|
}
|
||||||
|
|
||||||
|
ctx := context.Background()
|
||||||
|
cfg := config.Load()
|
||||||
|
|
||||||
|
sheetsCli, err := sheets.New(ctx, cfg.CredentialsPath, cfg.DriveTimeout)
|
||||||
|
if err != nil {
|
||||||
|
fmt.Fprintf(os.Stderr, "fuj sync: sheets client: %v\n", err)
|
||||||
|
os.Exit(1)
|
||||||
|
}
|
||||||
|
fioCli := fio.New(cfg.FioAPIToken, config.IBANAccountNum(cfg.BankAccount), nil)
|
||||||
|
|
||||||
|
opts := banksync.SyncOpts{Days: *days, Sort: *sort, DryRun: *dryRun}
|
||||||
|
if *fromStr != "" && *toStr != "" {
|
||||||
|
opts.From, err = time.Parse("2006-01-02", *fromStr)
|
||||||
|
if err != nil {
|
||||||
|
fmt.Fprintf(os.Stderr, "fuj sync: invalid --from: %v\n", err)
|
||||||
|
os.Exit(2)
|
||||||
|
}
|
||||||
|
opts.To, err = time.Parse("2006-01-02", *toStr)
|
||||||
|
if err != nil {
|
||||||
|
fmt.Fprintf(os.Stderr, "fuj sync: invalid --to: %v\n", err)
|
||||||
|
os.Exit(2)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
n, err := banksync.SyncToSheets(ctx, config.PaymentsSheetID, fioCli, sheetsCli, opts)
|
||||||
|
if err != nil {
|
||||||
|
fmt.Fprintf(os.Stderr, "fuj sync: %v\n", err)
|
||||||
|
os.Exit(1)
|
||||||
|
}
|
||||||
|
if *dryRun {
|
||||||
|
fmt.Printf("Dry run: would sync %d new transaction(s).\n", n)
|
||||||
|
} else {
|
||||||
|
fmt.Printf("Synced %d new transaction(s).\n", n)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func inferCmd(args []string) {
|
||||||
|
fs := flag.NewFlagSet("infer", flag.ExitOnError)
|
||||||
|
dryRun := fs.Bool("dry-run", false, "print planned updates without writing to the sheet")
|
||||||
|
fs.Usage = func() {
|
||||||
|
fmt.Fprintln(os.Stderr, "usage: fuj infer [--dry-run]")
|
||||||
|
fs.PrintDefaults()
|
||||||
|
}
|
||||||
|
if err := fs.Parse(args); err != nil {
|
||||||
|
fmt.Fprintln(os.Stderr, err)
|
||||||
|
os.Exit(2)
|
||||||
|
}
|
||||||
|
|
||||||
|
ctx := context.Background()
|
||||||
|
cfg := config.Load()
|
||||||
|
|
||||||
|
sheetsCli, err := sheets.New(ctx, cfg.CredentialsPath, cfg.DriveTimeout)
|
||||||
|
if err != nil {
|
||||||
|
fmt.Fprintf(os.Stderr, "fuj infer: sheets client: %v\n", err)
|
||||||
|
os.Exit(1)
|
||||||
|
}
|
||||||
|
sources, err := membership.NewSources(ctx, cfg)
|
||||||
|
if err != nil {
|
||||||
|
fmt.Fprintf(os.Stderr, "fuj infer: sources: %v\n", err)
|
||||||
|
os.Exit(1)
|
||||||
|
}
|
||||||
|
|
||||||
|
n, err := banksync.InferPayments(ctx, config.PaymentsSheetID, sheetsCli, sources, banksync.InferOpts{DryRun: *dryRun})
|
||||||
|
if err != nil {
|
||||||
|
fmt.Fprintf(os.Stderr, "fuj infer: %v\n", err)
|
||||||
|
os.Exit(1)
|
||||||
|
}
|
||||||
|
if *dryRun {
|
||||||
|
fmt.Printf("Dry run: would update %d row(s).\n", n)
|
||||||
|
} else {
|
||||||
|
fmt.Printf("Updated %d row(s).\n", n)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func usage() {
|
func usage() {
|
||||||
fmt.Fprintln(os.Stderr, `usage: fuj <command> [flags]
|
fmt.Fprintln(os.Stderr, `usage: fuj <command> [flags]
|
||||||
|
|
||||||
Commands:
|
Commands:
|
||||||
server Start HTTP server (default :8080)
|
server Start HTTP server (default :8080)
|
||||||
version Print version information
|
version Print version information
|
||||||
fees Calculate monthly fees [M2]
|
fees Calculate monthly fees
|
||||||
reconcile Show balance report [M2]
|
reconcile Show balance report
|
||||||
sync Sync Fio transactions [M4]
|
sync Sync Fio transactions to payments sheet
|
||||||
infer Infer payment details [M4]`)
|
infer Infer payment details in payments sheet`)
|
||||||
}
|
}
|
||||||
|
|||||||
31
go/go.mod
31
go/go.mod
@@ -2,4 +2,33 @@ module fuj-management/go
|
|||||||
|
|
||||||
go 1.26.1
|
go 1.26.1
|
||||||
|
|
||||||
require golang.org/x/text v0.36.0
|
require (
|
||||||
|
golang.org/x/net v0.53.0
|
||||||
|
golang.org/x/text v0.36.0
|
||||||
|
google.golang.org/api v0.278.0
|
||||||
|
)
|
||||||
|
|
||||||
|
require (
|
||||||
|
cloud.google.com/go/auth v0.20.0 // indirect
|
||||||
|
cloud.google.com/go/auth/oauth2adapt v0.2.8 // indirect
|
||||||
|
cloud.google.com/go/compute/metadata v0.9.0 // indirect
|
||||||
|
github.com/cespare/xxhash/v2 v2.3.0 // indirect
|
||||||
|
github.com/felixge/httpsnoop v1.0.4 // indirect
|
||||||
|
github.com/go-logr/logr v1.4.3 // indirect
|
||||||
|
github.com/go-logr/stdr v1.2.2 // indirect
|
||||||
|
github.com/google/s2a-go v0.1.9 // indirect
|
||||||
|
github.com/google/uuid v1.6.0 // indirect
|
||||||
|
github.com/googleapis/enterprise-certificate-proxy v0.3.15 // indirect
|
||||||
|
github.com/googleapis/gax-go/v2 v2.22.0 // indirect
|
||||||
|
go.opentelemetry.io/auto/sdk v1.2.1 // indirect
|
||||||
|
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.67.0 // indirect
|
||||||
|
go.opentelemetry.io/otel v1.43.0 // indirect
|
||||||
|
go.opentelemetry.io/otel/metric v1.43.0 // indirect
|
||||||
|
go.opentelemetry.io/otel/trace v1.43.0 // indirect
|
||||||
|
golang.org/x/crypto v0.50.0 // indirect
|
||||||
|
golang.org/x/oauth2 v0.36.0 // indirect
|
||||||
|
golang.org/x/sys v0.43.0 // indirect
|
||||||
|
google.golang.org/genproto/googleapis/rpc v0.0.0-20260427160629-7cedc36a6bc4 // indirect
|
||||||
|
google.golang.org/grpc v1.80.0 // indirect
|
||||||
|
google.golang.org/protobuf v1.36.11 // indirect
|
||||||
|
)
|
||||||
|
|||||||
73
go/go.sum
73
go/go.sum
@@ -1,2 +1,75 @@
|
|||||||
|
cloud.google.com/go/auth v0.20.0 h1:kXTssoVb4azsVDoUiF8KvxAqrsQcQtB53DcSgta74CA=
|
||||||
|
cloud.google.com/go/auth v0.20.0/go.mod h1:942/yi/itH1SsmpyrbnTMDgGfdy2BUqIKyd0cyYLc5Q=
|
||||||
|
cloud.google.com/go/auth/oauth2adapt v0.2.8 h1:keo8NaayQZ6wimpNSmW5OPc283g65QNIiLpZnkHRbnc=
|
||||||
|
cloud.google.com/go/auth/oauth2adapt v0.2.8/go.mod h1:XQ9y31RkqZCcwJWNSx2Xvric3RrU88hAYYbjDWYDL+c=
|
||||||
|
cloud.google.com/go/compute/metadata v0.9.0 h1:pDUj4QMoPejqq20dK0Pg2N4yG9zIkYGdBtwLoEkH9Zs=
|
||||||
|
cloud.google.com/go/compute/metadata v0.9.0/go.mod h1:E0bWwX5wTnLPedCKqk3pJmVgCBSM6qQI1yTBdEb3C10=
|
||||||
|
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
|
||||||
|
github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
|
||||||
|
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
|
||||||
|
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||||
|
github.com/felixge/httpsnoop v1.0.4 h1:NFTV2Zj1bL4mc9sqWACXbQFVBBg2W3GPvqp8/ESS2Wg=
|
||||||
|
github.com/felixge/httpsnoop v1.0.4/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U=
|
||||||
|
github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A=
|
||||||
|
github.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI=
|
||||||
|
github.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
|
||||||
|
github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
|
||||||
|
github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=
|
||||||
|
github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek=
|
||||||
|
github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps=
|
||||||
|
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
|
||||||
|
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
|
||||||
|
github.com/google/s2a-go v0.1.9 h1:LGD7gtMgezd8a/Xak7mEWL0PjoTQFvpRudN895yqKW0=
|
||||||
|
github.com/google/s2a-go v0.1.9/go.mod h1:YA0Ei2ZQL3acow2O62kdp9UlnvMmU7kA6Eutn0dXayM=
|
||||||
|
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
|
||||||
|
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
|
||||||
|
github.com/googleapis/enterprise-certificate-proxy v0.3.15 h1:xolVQTEXusUcAA5UgtyRLjelpFFHWlPQ4XfWGc7MBas=
|
||||||
|
github.com/googleapis/enterprise-certificate-proxy v0.3.15/go.mod h1:vqVt9yG9480NtzREnTlmGSBmFrA+bzb0yl0TxoBQXOg=
|
||||||
|
github.com/googleapis/gax-go/v2 v2.22.0 h1:PjIWBpgGIVKGoCXuiCoP64altEJCj3/Ei+kSU5vlZD4=
|
||||||
|
github.com/googleapis/gax-go/v2 v2.22.0/go.mod h1:irWBbALSr0Sk3qlqb9SyJ1h68WjgeFuiOzI4Rqw5+aY=
|
||||||
|
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
|
||||||
|
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
||||||
|
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
|
||||||
|
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
|
||||||
|
go.opentelemetry.io/auto/sdk v1.2.1 h1:jXsnJ4Lmnqd11kwkBV2LgLoFMZKizbCi5fNZ/ipaZ64=
|
||||||
|
go.opentelemetry.io/auto/sdk v1.2.1/go.mod h1:KRTj+aOaElaLi+wW1kO/DZRXwkF4C5xPbEe3ZiIhN7Y=
|
||||||
|
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.67.0 h1:OyrsyzuttWTSur2qN/Lm0m2a8yqyIjUVBZcxFPuXq2o=
|
||||||
|
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.67.0/go.mod h1:C2NGBr+kAB4bk3xtMXfZ94gqFDtg/GkI7e9zqGh5Beg=
|
||||||
|
go.opentelemetry.io/otel v1.43.0 h1:mYIM03dnh5zfN7HautFE4ieIig9amkNANT+xcVxAj9I=
|
||||||
|
go.opentelemetry.io/otel v1.43.0/go.mod h1:JuG+u74mvjvcm8vj8pI5XiHy1zDeoCS2LB1spIq7Ay0=
|
||||||
|
go.opentelemetry.io/otel/metric v1.43.0 h1:d7638QeInOnuwOONPp4JAOGfbCEpYb+K6DVWvdxGzgM=
|
||||||
|
go.opentelemetry.io/otel/metric v1.43.0/go.mod h1:RDnPtIxvqlgO8GRW18W6Z/4P462ldprJtfxHxyKd2PY=
|
||||||
|
go.opentelemetry.io/otel/sdk v1.43.0 h1:pi5mE86i5rTeLXqoF/hhiBtUNcrAGHLKQdhg4h4V9Dg=
|
||||||
|
go.opentelemetry.io/otel/sdk v1.43.0/go.mod h1:P+IkVU3iWukmiit/Yf9AWvpyRDlUeBaRg6Y+C58QHzg=
|
||||||
|
go.opentelemetry.io/otel/sdk/metric v1.43.0 h1:S88dyqXjJkuBNLeMcVPRFXpRw2fuwdvfCGLEo89fDkw=
|
||||||
|
go.opentelemetry.io/otel/sdk/metric v1.43.0/go.mod h1:C/RJtwSEJ5hzTiUz5pXF1kILHStzb9zFlIEe85bhj6A=
|
||||||
|
go.opentelemetry.io/otel/trace v1.43.0 h1:BkNrHpup+4k4w+ZZ86CZoHHEkohws8AY+WTX09nk+3A=
|
||||||
|
go.opentelemetry.io/otel/trace v1.43.0/go.mod h1:/QJhyVBUUswCphDVxq+8mld+AvhXZLhe+8WVFxiFff0=
|
||||||
|
golang.org/x/crypto v0.50.0 h1:zO47/JPrL6vsNkINmLoo/PH1gcxpls50DNogFvB5ZGI=
|
||||||
|
golang.org/x/crypto v0.50.0/go.mod h1:3muZ7vA7PBCE6xgPX7nkzzjiUq87kRItoJQM1Yo8S+Q=
|
||||||
|
golang.org/x/net v0.53.0 h1:d+qAbo5L0orcWAr0a9JweQpjXF19LMXJE8Ey7hwOdUA=
|
||||||
|
golang.org/x/net v0.53.0/go.mod h1:JvMuJH7rrdiCfbeHoo3fCQU24Lf5JJwT9W3sJFulfgs=
|
||||||
|
golang.org/x/oauth2 v0.36.0 h1:peZ/1z27fi9hUOFCAZaHyrpWG5lwe0RJEEEeH0ThlIs=
|
||||||
|
golang.org/x/oauth2 v0.36.0/go.mod h1:YDBUJMTkDnJS+A4BP4eZBjCqtokkg1hODuPjwiGPO7Q=
|
||||||
|
golang.org/x/sync v0.20.0 h1:e0PTpb7pjO8GAtTs2dQ6jYa5BWYlMuX047Dco/pItO4=
|
||||||
|
golang.org/x/sync v0.20.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
|
||||||
|
golang.org/x/sys v0.43.0 h1:Rlag2XtaFTxp19wS8MXlJwTvoh8ArU6ezoyFsMyCTNI=
|
||||||
|
golang.org/x/sys v0.43.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
|
||||||
golang.org/x/text v0.36.0 h1:JfKh3XmcRPqZPKevfXVpI1wXPTqbkE5f7JA92a55Yxg=
|
golang.org/x/text v0.36.0 h1:JfKh3XmcRPqZPKevfXVpI1wXPTqbkE5f7JA92a55Yxg=
|
||||||
golang.org/x/text v0.36.0/go.mod h1:NIdBknypM8iqVmPiuco0Dh6P5Jcdk8lJL0CUebqK164=
|
golang.org/x/text v0.36.0/go.mod h1:NIdBknypM8iqVmPiuco0Dh6P5Jcdk8lJL0CUebqK164=
|
||||||
|
gonum.org/v1/gonum v0.17.0 h1:VbpOemQlsSMrYmn7T2OUvQ4dqxQXU+ouZFQsZOx50z4=
|
||||||
|
gonum.org/v1/gonum v0.17.0/go.mod h1:El3tOrEuMpv2UdMrbNlKEh9vd86bmQ6vqIcDwxEOc1E=
|
||||||
|
google.golang.org/api v0.278.0 h1:W7jiRvRi53VYFfZ/HoZjQBtJk7gOFbHD8ot1RzVZU6E=
|
||||||
|
google.golang.org/api v0.278.0/go.mod h1:B9TqLBwJqVjp1mtt7WeoQwWRwvu/400y5lETOql+giQ=
|
||||||
|
google.golang.org/genproto v0.0.0-20260319201613-d00831a3d3e7 h1:XzmzkmB14QhVhgnawEVsOn6OFsnpyxNPRY9QV01dNB0=
|
||||||
|
google.golang.org/genproto v0.0.0-20260319201613-d00831a3d3e7/go.mod h1:L43LFes82YgSonw6iTXTxXUX1OlULt4AQtkik4ULL/I=
|
||||||
|
google.golang.org/genproto/googleapis/api v0.0.0-20260319201613-d00831a3d3e7 h1:41r6JMbpzBMen0R/4TZeeAmGXSJC7DftGINUodzTkPI=
|
||||||
|
google.golang.org/genproto/googleapis/api v0.0.0-20260319201613-d00831a3d3e7/go.mod h1:EIQZ5bFCfRQDV4MhRle7+OgjNtZ6P1PiZBgAKuxXu/Y=
|
||||||
|
google.golang.org/genproto/googleapis/rpc v0.0.0-20260427160629-7cedc36a6bc4 h1:tEkOQcXgF6dH1G+MVKZrfpYvozGrzb91k6ha7jireSM=
|
||||||
|
google.golang.org/genproto/googleapis/rpc v0.0.0-20260427160629-7cedc36a6bc4/go.mod h1:4Hqkh8ycfw05ld/3BWL7rJOSfebL2Q+DVDeRgYgxUU8=
|
||||||
|
google.golang.org/grpc v1.80.0 h1:Xr6m2WmWZLETvUNvIUmeD5OAagMw3FiKmMlTdViWsHM=
|
||||||
|
google.golang.org/grpc v1.80.0/go.mod h1:ho/dLnxwi3EDJA4Zghp7k2Ec1+c2jqup0bFkw07bwF4=
|
||||||
|
google.golang.org/protobuf v1.36.11 h1:fV6ZwhNocDyBLK0dj+fg8ektcVegBBuEolpbTQyBNVE=
|
||||||
|
google.golang.org/protobuf v1.36.11/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco=
|
||||||
|
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
|
||||||
|
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||||
|
|||||||
@@ -3,6 +3,7 @@ package config
|
|||||||
import (
|
import (
|
||||||
"os"
|
"os"
|
||||||
"strconv"
|
"strconv"
|
||||||
|
"strings"
|
||||||
"time"
|
"time"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -10,16 +11,34 @@ import (
|
|||||||
const (
|
const (
|
||||||
AttendanceSheetID = "1E2e_gT_K5AwSRCDLDTa2UetZTkHmBOcz0kFbBUNUNBA"
|
AttendanceSheetID = "1E2e_gT_K5AwSRCDLDTa2UetZTkHmBOcz0kFbBUNUNBA"
|
||||||
PaymentsSheetID = "1Om0YPoDVCH5cV8BrNz5LG5eR5MMU05ypQC7UMN1xn_Y"
|
PaymentsSheetID = "1Om0YPoDVCH5cV8BrNz5LG5eR5MMU05ypQC7UMN1xn_Y"
|
||||||
JuniorSheetGID = "1213318614"
|
|
||||||
|
// Both attendance tabs live in the same Google Spreadsheet (AttendanceSheetID).
|
||||||
|
// The original adult and junior attendance data lives in separate source spreadsheets,
|
||||||
|
// but is collected into this one sheet via IMPORTRANGE — one tab per group.
|
||||||
|
// Tabs are identified by the gid= query param in the CSV export URL.
|
||||||
|
AttendanceAdultSheetGID = "0" // gid=0 — adult practices tab (IMPORTRANGE'd)
|
||||||
|
JuniorSheetGID = "1213318614" // gid=1213318614 — junior practices tab (IMPORTRANGE'd)
|
||||||
)
|
)
|
||||||
|
|
||||||
|
// CacheSheetMap mirrors scripts/config.py CACHE_SHEET_MAP.
|
||||||
|
// Maps a cache key to the Google Sheet ID whose Drive modifiedTime gates it.
|
||||||
|
// Both attendance keys map to the same spreadsheet — different tabs, one Drive file.
|
||||||
|
var CacheSheetMap = map[string]string{
|
||||||
|
"attendance_regular": AttendanceSheetID,
|
||||||
|
"attendance_juniors": AttendanceSheetID,
|
||||||
|
"exceptions_dict": PaymentsSheetID,
|
||||||
|
"payments_transactions": PaymentsSheetID,
|
||||||
|
}
|
||||||
|
|
||||||
// Config holds all runtime configuration loaded from environment variables.
|
// Config holds all runtime configuration loaded from environment variables.
|
||||||
// Mirrors scripts/config.py.
|
// Mirrors scripts/config.py.
|
||||||
type Config struct {
|
type Config struct {
|
||||||
CredentialsPath string
|
CredentialsPath string
|
||||||
BankAccount string
|
BankAccount string
|
||||||
|
CacheDir string
|
||||||
CacheTTL time.Duration
|
CacheTTL time.Duration
|
||||||
CacheAPICheckTTL time.Duration
|
CacheAPICheckTTL time.Duration
|
||||||
|
DriveTimeout time.Duration
|
||||||
LogLevel string
|
LogLevel string
|
||||||
FioAPIToken string
|
FioAPIToken string
|
||||||
ServerAddr string
|
ServerAddr string
|
||||||
@@ -31,14 +50,32 @@ func Load() Config {
|
|||||||
return Config{
|
return Config{
|
||||||
CredentialsPath: env("CREDENTIALS_PATH", ".secret/fuj-management-bot-credentials.json"),
|
CredentialsPath: env("CREDENTIALS_PATH", ".secret/fuj-management-bot-credentials.json"),
|
||||||
BankAccount: env("BANK_ACCOUNT", "CZ8520100000002800359168"),
|
BankAccount: env("BANK_ACCOUNT", "CZ8520100000002800359168"),
|
||||||
|
CacheDir: env("CACHE_DIR", "tmp"),
|
||||||
CacheTTL: envDuration("CACHE_TTL_SECONDS", 300),
|
CacheTTL: envDuration("CACHE_TTL_SECONDS", 300),
|
||||||
CacheAPICheckTTL: envDuration("CACHE_API_CHECK_TTL_SECONDS", 300),
|
CacheAPICheckTTL: envDuration("CACHE_API_CHECK_TTL_SECONDS", 300),
|
||||||
|
DriveTimeout: envDuration("DRIVE_TIMEOUT_SECONDS", 10),
|
||||||
LogLevel: env("LOG_LEVEL", "INFO"),
|
LogLevel: env("LOG_LEVEL", "INFO"),
|
||||||
FioAPIToken: env("FIO_API_TOKEN", ""),
|
FioAPIToken: env("FIO_API_TOKEN", ""),
|
||||||
ServerAddr: env("SERVER_ADDR", ":8080"),
|
ServerAddr: env("SERVER_ADDR", ":8080"),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// IBANAccountNum extracts the bare account number from a Czech IBAN.
|
||||||
|
// "CZ8520100000002800359168" → "2800359168"
|
||||||
|
// Structure: CZ(2 check)(4 bank code)(16 zero-padded account).
|
||||||
|
func IBANAccountNum(iban string) string {
|
||||||
|
s := strings.ReplaceAll(iban, " ", "")
|
||||||
|
if len(s) < 8 {
|
||||||
|
return iban
|
||||||
|
}
|
||||||
|
raw := s[8:] // 16-digit zero-padded account portion
|
||||||
|
n := strings.TrimLeft(raw, "0")
|
||||||
|
if n == "" {
|
||||||
|
return "0"
|
||||||
|
}
|
||||||
|
return n
|
||||||
|
}
|
||||||
|
|
||||||
func env(key, fallback string) string {
|
func env(key, fallback string) string {
|
||||||
if v := os.Getenv(key); v != "" {
|
if v := os.Getenv(key); v != "" {
|
||||||
return v
|
return v
|
||||||
|
|||||||
393
go/internal/domain/reconcile/reconcile.go
Normal file
393
go/internal/domain/reconcile/reconcile.go
Normal file
@@ -0,0 +1,393 @@
|
|||||||
|
// Package reconcile ports the three-phase payment reconciliation from scripts/match_payments.py.
|
||||||
|
package reconcile
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fuj-management/go/internal/domain/czech"
|
||||||
|
"fuj-management/go/internal/domain/matching"
|
||||||
|
"regexp"
|
||||||
|
"strings"
|
||||||
|
)
|
||||||
|
|
||||||
|
// ExceptionKey identifies a fee override by normalized member name and period.
|
||||||
|
type ExceptionKey struct {
|
||||||
|
Name string // czech.Normalize(memberName)
|
||||||
|
Period string // czech.Normalize("YYYY-MM")
|
||||||
|
}
|
||||||
|
|
||||||
|
// Exception is a manual fee override for one member in one period.
|
||||||
|
type Exception struct {
|
||||||
|
Amount int
|
||||||
|
Note string
|
||||||
|
}
|
||||||
|
|
||||||
|
// FeeData holds the expected fee and attendance count for one member in one month.
|
||||||
|
type FeeData struct {
|
||||||
|
Expected int
|
||||||
|
Attendance int
|
||||||
|
}
|
||||||
|
|
||||||
|
// Member is one row from the attendance sheet.
|
||||||
|
type Member struct {
|
||||||
|
Name string
|
||||||
|
Tier string
|
||||||
|
Fees map[string]FeeData // month ("YYYY-MM") → fee data
|
||||||
|
}
|
||||||
|
|
||||||
|
// Transaction is one payment row from the payments sheet.
|
||||||
|
// Date must already be a "YYYY-MM-DD" string (convert with matching.FormatDate before calling).
|
||||||
|
// InferredAmount, when non-nil, replaces Amount when person and purpose are pre-matched.
|
||||||
|
type Transaction struct {
|
||||||
|
Date string
|
||||||
|
Amount float64
|
||||||
|
Person string // comma-separated canonical names (empty → use inference)
|
||||||
|
Purpose string // comma-separated "YYYY-MM" or "other:…" (empty → use inference)
|
||||||
|
InferredAmount *float64 // nil → fall back to Amount
|
||||||
|
Sender string
|
||||||
|
Message string
|
||||||
|
UserID string
|
||||||
|
}
|
||||||
|
|
||||||
|
// TxEntry is the portion of a payment allocated to a single member+month.
|
||||||
|
type TxEntry struct {
|
||||||
|
Amount float64
|
||||||
|
Date string
|
||||||
|
Sender string
|
||||||
|
Message string
|
||||||
|
Confidence string
|
||||||
|
}
|
||||||
|
|
||||||
|
// OtherEntry is a payment with purpose "other:…" allocated to a member.
|
||||||
|
type OtherEntry struct {
|
||||||
|
Amount float64
|
||||||
|
Date string
|
||||||
|
Sender string
|
||||||
|
Message string
|
||||||
|
Purpose string
|
||||||
|
Confidence string
|
||||||
|
}
|
||||||
|
|
||||||
|
// MonthData is the ledger state for one member in one month.
|
||||||
|
type MonthData struct {
|
||||||
|
Expected int
|
||||||
|
OriginalExpected int
|
||||||
|
AttendanceCount int
|
||||||
|
Exception *Exception
|
||||||
|
Paid float64
|
||||||
|
Transactions []TxEntry
|
||||||
|
}
|
||||||
|
|
||||||
|
// MemberResult is the reconciled ledger for one member.
|
||||||
|
type MemberResult struct {
|
||||||
|
Tier string
|
||||||
|
Months map[string]MonthData
|
||||||
|
OtherTransactions []OtherEntry
|
||||||
|
TotalBalance int
|
||||||
|
}
|
||||||
|
|
||||||
|
// Result is the top-level output of Reconcile.
|
||||||
|
type Result struct {
|
||||||
|
Members map[string]MemberResult
|
||||||
|
Unmatched []Transaction
|
||||||
|
Credits map[string]int // final balance for every member (may be negative)
|
||||||
|
}
|
||||||
|
|
||||||
|
var questionMarkRe = regexp.MustCompile(`\[\?\]\s*`)
|
||||||
|
|
||||||
|
// canonicalMemberKey returns a diacritic-, case-, and whitespace-insensitive key
|
||||||
|
// used to resolve Person-column values that drift from canonical attendance-sheet names.
|
||||||
|
// Ports scripts/match_payments.py canonical_member_key.
|
||||||
|
func canonicalMemberKey(name string) string {
|
||||||
|
return strings.Join(strings.Fields(czech.Normalize(name)), " ")
|
||||||
|
}
|
||||||
|
|
||||||
|
type monthExpected struct {
|
||||||
|
month string
|
||||||
|
expected int
|
||||||
|
}
|
||||||
|
|
||||||
|
// Reconcile matches transactions to members and months using three allocation phases:
|
||||||
|
// 1. Greedy: payment ≥ total expected → fill each month exactly; overflow → credit.
|
||||||
|
// 2. Proportional: payment < total → distribute by each month's share; last absorbs float remainder.
|
||||||
|
// 3. Even-split fallback: all expected fees are 0 (prepayment) → divide equally.
|
||||||
|
//
|
||||||
|
// defaultYear seeds czech.ParseMonthReferences in the inference fallback.
|
||||||
|
// Pass time.Now().Year() in production; pass a fixed year in tests.
|
||||||
|
//
|
||||||
|
// Ports scripts/match_payments.py reconcile.
|
||||||
|
func Reconcile(
|
||||||
|
members []Member,
|
||||||
|
sortedMonths []string,
|
||||||
|
transactions []Transaction,
|
||||||
|
exceptions map[ExceptionKey]Exception,
|
||||||
|
defaultYear int,
|
||||||
|
) Result {
|
||||||
|
memberNames := make([]string, len(members))
|
||||||
|
memberTiers := make(map[string]string, len(members))
|
||||||
|
memberFees := make(map[string]map[string]FeeData, len(members))
|
||||||
|
|
||||||
|
for i, m := range members {
|
||||||
|
memberNames[i] = m.Name
|
||||||
|
memberTiers[m.Name] = m.Tier
|
||||||
|
memberFees[m.Name] = m.Fees
|
||||||
|
}
|
||||||
|
|
||||||
|
// Map canonical key → first attendance-sheet name with that key, so Person cells
|
||||||
|
// that drift in diacritics/case/whitespace still resolve to the canonical name.
|
||||||
|
canonicalByKey := make(map[string]string, len(memberNames))
|
||||||
|
for _, name := range memberNames {
|
||||||
|
key := canonicalMemberKey(name)
|
||||||
|
if _, exists := canonicalByKey[key]; !exists {
|
||||||
|
canonicalByKey[key] = name
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if exceptions == nil {
|
||||||
|
exceptions = map[ExceptionKey]Exception{}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Initialise ledger
|
||||||
|
ledger := make(map[string]map[string]MonthData, len(memberNames))
|
||||||
|
otherLedger := make(map[string][]OtherEntry, len(memberNames))
|
||||||
|
|
||||||
|
for _, name := range memberNames {
|
||||||
|
ledger[name] = make(map[string]MonthData, len(sortedMonths))
|
||||||
|
otherLedger[name] = []OtherEntry{}
|
||||||
|
for _, m := range sortedMonths {
|
||||||
|
fd := memberFees[name][m]
|
||||||
|
originalExpected := fd.Expected
|
||||||
|
attendanceCount := fd.Attendance
|
||||||
|
|
||||||
|
var expected int
|
||||||
|
var exInfo *Exception
|
||||||
|
exKey := ExceptionKey{
|
||||||
|
Name: czech.Normalize(name),
|
||||||
|
Period: czech.Normalize(m),
|
||||||
|
}
|
||||||
|
if ex, ok := exceptions[exKey]; ok {
|
||||||
|
expected = ex.Amount
|
||||||
|
exCopy := ex
|
||||||
|
exInfo = &exCopy
|
||||||
|
} else {
|
||||||
|
expected = originalExpected
|
||||||
|
}
|
||||||
|
|
||||||
|
ledger[name][m] = MonthData{
|
||||||
|
Expected: expected,
|
||||||
|
OriginalExpected: originalExpected,
|
||||||
|
AttendanceCount: attendanceCount,
|
||||||
|
Exception: exInfo,
|
||||||
|
Paid: 0,
|
||||||
|
Transactions: []TxEntry{},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
var unmatched []Transaction
|
||||||
|
credits := make(map[string]int, len(memberNames))
|
||||||
|
|
||||||
|
for _, tx := range transactions {
|
||||||
|
personStr := strings.TrimSpace(tx.Person)
|
||||||
|
purposeStr := strings.TrimSpace(tx.Purpose)
|
||||||
|
personStr = questionMarkRe.ReplaceAllString(personStr, "")
|
||||||
|
isOther := strings.HasPrefix(strings.ToLower(purposeStr), "other:")
|
||||||
|
|
||||||
|
var matchedMembers []matching.Match
|
||||||
|
var matchedMonths []string
|
||||||
|
var amount float64
|
||||||
|
|
||||||
|
if personStr != "" && purposeStr != "" {
|
||||||
|
for p := range strings.SplitSeq(personStr, ",") {
|
||||||
|
p = strings.TrimSpace(p)
|
||||||
|
if p != "" {
|
||||||
|
matchedMembers = append(matchedMembers, matching.Match{
|
||||||
|
Name: p,
|
||||||
|
Confidence: matching.ConfidenceAuto,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if isOther {
|
||||||
|
matchedMonths = []string{purposeStr}
|
||||||
|
} else {
|
||||||
|
for m := range strings.SplitSeq(purposeStr, ",") {
|
||||||
|
m = strings.TrimSpace(m)
|
||||||
|
if m != "" {
|
||||||
|
matchedMonths = append(matchedMonths, m)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if tx.InferredAmount != nil {
|
||||||
|
amount = *tx.InferredAmount
|
||||||
|
} else {
|
||||||
|
amount = tx.Amount
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
// Inference fallback for rows not yet processed by infer_payments.py
|
||||||
|
inferred := matching.InferTransactionDetails(
|
||||||
|
matching.Transaction{
|
||||||
|
Sender: tx.Sender,
|
||||||
|
Message: tx.Message,
|
||||||
|
UserID: tx.UserID,
|
||||||
|
Date: tx.Date,
|
||||||
|
},
|
||||||
|
memberNames,
|
||||||
|
defaultYear,
|
||||||
|
)
|
||||||
|
matchedMembers = inferred.Members
|
||||||
|
matchedMonths = inferred.Months
|
||||||
|
amount = tx.Amount
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(matchedMembers) == 0 || len(matchedMonths) == 0 {
|
||||||
|
unmatched = append(unmatched, tx)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
if isOther {
|
||||||
|
nAlloc := len(matchedMembers)
|
||||||
|
perAlloc := 0.0
|
||||||
|
if nAlloc > 0 {
|
||||||
|
perAlloc = amount / float64(nAlloc)
|
||||||
|
}
|
||||||
|
for _, m := range matchedMembers {
|
||||||
|
memberName := canonicalByKey[canonicalMemberKey(m.Name)]
|
||||||
|
if memberName != "" {
|
||||||
|
otherLedger[memberName] = append(otherLedger[memberName], OtherEntry{
|
||||||
|
Amount: perAlloc,
|
||||||
|
Date: tx.Date,
|
||||||
|
Sender: tx.Sender,
|
||||||
|
Message: tx.Message,
|
||||||
|
Purpose: purposeStr,
|
||||||
|
Confidence: string(m.Confidence),
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
memberShare := 0.0
|
||||||
|
if len(matchedMembers) > 0 {
|
||||||
|
memberShare = amount / float64(len(matchedMembers))
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, m := range matchedMembers {
|
||||||
|
memberName := canonicalByKey[canonicalMemberKey(m.Name)]
|
||||||
|
if memberName == "" {
|
||||||
|
unmatched = append(unmatched, tx)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
var inWindow []monthExpected
|
||||||
|
outCount := 0
|
||||||
|
for _, month := range matchedMonths {
|
||||||
|
if md, ok := ledger[memberName][month]; ok {
|
||||||
|
inWindow = append(inWindow, monthExpected{month: month, expected: md.Expected})
|
||||||
|
} else {
|
||||||
|
outCount++
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
nTotal := len(matchedMonths)
|
||||||
|
outCredit := 0.0
|
||||||
|
if outCount > 0 && nTotal > 0 {
|
||||||
|
outCredit = memberShare / float64(nTotal) * float64(outCount)
|
||||||
|
credits[memberName] += int(outCredit)
|
||||||
|
}
|
||||||
|
|
||||||
|
inWindowShare := memberShare - outCredit
|
||||||
|
|
||||||
|
if len(inWindow) == 0 {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
totalExpected := 0
|
||||||
|
for _, mw := range inWindow {
|
||||||
|
totalExpected += mw.expected
|
||||||
|
}
|
||||||
|
|
||||||
|
if totalExpected > 0 && inWindowShare >= float64(totalExpected) {
|
||||||
|
// Greedy: payment covers all expected fees; overflow → credit
|
||||||
|
credits[memberName] += int(inWindowShare - float64(totalExpected))
|
||||||
|
for _, mw := range inWindow {
|
||||||
|
alloc := float64(mw.expected)
|
||||||
|
md := ledger[memberName][mw.month]
|
||||||
|
md.Paid += alloc
|
||||||
|
md.Transactions = append(md.Transactions, TxEntry{
|
||||||
|
Amount: alloc,
|
||||||
|
Date: tx.Date,
|
||||||
|
Sender: tx.Sender,
|
||||||
|
Message: tx.Message,
|
||||||
|
Confidence: string(m.Confidence),
|
||||||
|
})
|
||||||
|
ledger[memberName][mw.month] = md
|
||||||
|
}
|
||||||
|
} else if totalExpected > 0 {
|
||||||
|
// Proportional: distribute by each month's share; last month absorbs float remainder
|
||||||
|
remaining := inWindowShare
|
||||||
|
for i, mw := range inWindow {
|
||||||
|
var alloc float64
|
||||||
|
if i == len(inWindow)-1 {
|
||||||
|
alloc = remaining
|
||||||
|
} else {
|
||||||
|
alloc = inWindowShare * float64(mw.expected) / float64(totalExpected)
|
||||||
|
}
|
||||||
|
remaining -= alloc
|
||||||
|
md := ledger[memberName][mw.month]
|
||||||
|
md.Paid += alloc
|
||||||
|
md.Transactions = append(md.Transactions, TxEntry{
|
||||||
|
Amount: alloc,
|
||||||
|
Date: tx.Date,
|
||||||
|
Sender: tx.Sender,
|
||||||
|
Message: tx.Message,
|
||||||
|
Confidence: string(m.Confidence),
|
||||||
|
})
|
||||||
|
ledger[memberName][mw.month] = md
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
// Even-split fallback: prepayment before attendance recorded
|
||||||
|
perMonth := inWindowShare / float64(len(inWindow))
|
||||||
|
for _, mw := range inWindow {
|
||||||
|
md := ledger[memberName][mw.month]
|
||||||
|
md.Paid += perMonth
|
||||||
|
md.Transactions = append(md.Transactions, TxEntry{
|
||||||
|
Amount: perMonth,
|
||||||
|
Date: tx.Date,
|
||||||
|
Sender: tx.Sender,
|
||||||
|
Message: tx.Message,
|
||||||
|
Confidence: string(m.Confidence),
|
||||||
|
})
|
||||||
|
ledger[memberName][mw.month] = md
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Final total balances: window balance + out-of-window credits accumulated above
|
||||||
|
finalBalances := make(map[string]int, len(memberNames))
|
||||||
|
for _, name := range memberNames {
|
||||||
|
windowBalance := 0
|
||||||
|
for _, mdata := range ledger[name] {
|
||||||
|
windowBalance += int(mdata.Paid) - mdata.Expected
|
||||||
|
}
|
||||||
|
finalBalances[name] = windowBalance + credits[name]
|
||||||
|
}
|
||||||
|
|
||||||
|
membersResult := make(map[string]MemberResult, len(memberNames))
|
||||||
|
for _, name := range memberNames {
|
||||||
|
membersResult[name] = MemberResult{
|
||||||
|
Tier: memberTiers[name],
|
||||||
|
Months: ledger[name],
|
||||||
|
OtherTransactions: otherLedger[name],
|
||||||
|
TotalBalance: finalBalances[name],
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if unmatched == nil {
|
||||||
|
unmatched = []Transaction{}
|
||||||
|
}
|
||||||
|
|
||||||
|
return Result{
|
||||||
|
Members: membersResult,
|
||||||
|
Unmatched: unmatched,
|
||||||
|
Credits: finalBalances,
|
||||||
|
}
|
||||||
|
}
|
||||||
376
go/internal/domain/reconcile/reconcile_test.go
Normal file
376
go/internal/domain/reconcile/reconcile_test.go
Normal file
@@ -0,0 +1,376 @@
|
|||||||
|
package reconcile
|
||||||
|
|
||||||
|
// Expected values verified against scripts/match_payments.py on 2026-05-06:
|
||||||
|
//
|
||||||
|
// PYTHONPATH=scripts:. python3 -m unittest tests.test_reconcile_exceptions tests.test_match_payments -v
|
||||||
|
//
|
||||||
|
// All Python test cases are ported below. Additional Go-only cases are marked with [Go].
|
||||||
|
|
||||||
|
import (
|
||||||
|
"math"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
const defaultYear = 2026
|
||||||
|
|
||||||
|
// tx builds a pre-matched Transaction (person+purpose already filled in).
|
||||||
|
// InferredAmount is left nil so Amount is used directly, matching the Python
|
||||||
|
// _tx helper where inferred_amount == amount.
|
||||||
|
func tx(person, purpose string, amount float64) Transaction {
|
||||||
|
return Transaction{
|
||||||
|
Date: "2026-01-01",
|
||||||
|
Amount: amount,
|
||||||
|
Person: person,
|
||||||
|
Purpose: purpose,
|
||||||
|
Sender: "Sender",
|
||||||
|
Message: "fee",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestReconcileExceptionOverride(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
members := []Member{{Name: "Alice", Tier: "A", Fees: map[string]FeeData{"2026-01": {750, 4}}}}
|
||||||
|
exceptions := map[ExceptionKey]Exception{
|
||||||
|
{Name: "alice", Period: "2026-01"}: {Amount: 400, Note: "Test exception"},
|
||||||
|
}
|
||||||
|
txs := []Transaction{{
|
||||||
|
Date: "2026-01-05", Amount: 400,
|
||||||
|
Person: "Alice", Purpose: "2026-01", Sender: "Alice Sender", Message: "fee",
|
||||||
|
}}
|
||||||
|
|
||||||
|
result := Reconcile(members, []string{"2026-01"}, txs, exceptions, defaultYear)
|
||||||
|
|
||||||
|
jan := result.Members["Alice"].Months["2026-01"]
|
||||||
|
if jan.Expected != 400 {
|
||||||
|
t.Errorf("Expected override to 400, got %d", jan.Expected)
|
||||||
|
}
|
||||||
|
if jan.Paid != 400 {
|
||||||
|
t.Errorf("Paid want 400, got %f", jan.Paid)
|
||||||
|
}
|
||||||
|
if result.Members["Alice"].TotalBalance != 0 {
|
||||||
|
t.Errorf("TotalBalance want 0, got %d", result.Members["Alice"].TotalBalance)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestReconcileFallbackToAttendance(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
members := []Member{{Name: "Alice", Tier: "A", Fees: map[string]FeeData{"2026-01": {750, 4}}}}
|
||||||
|
|
||||||
|
result := Reconcile(members, []string{"2026-01"}, nil, nil, defaultYear)
|
||||||
|
|
||||||
|
if result.Members["Alice"].Months["2026-01"].Expected != 750 {
|
||||||
|
t.Errorf("Expected 750 when no exception, got %d", result.Members["Alice"].Months["2026-01"].Expected)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestReconcileGreedyExactMatch(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
members := []Member{{
|
||||||
|
Name: "Alice", Tier: "A",
|
||||||
|
Fees: map[string]FeeData{
|
||||||
|
"2026-02": {750, 3},
|
||||||
|
"2026-03": {350, 3},
|
||||||
|
"2026-04": {150, 2},
|
||||||
|
},
|
||||||
|
}}
|
||||||
|
sortedMonths := []string{"2026-02", "2026-03", "2026-04"}
|
||||||
|
|
||||||
|
result := Reconcile(members, sortedMonths, []Transaction{tx("Alice", "2026-02, 2026-03, 2026-04", 1250)}, nil, defaultYear)
|
||||||
|
|
||||||
|
months := result.Members["Alice"].Months
|
||||||
|
if int(months["2026-02"].Paid) != 750 {
|
||||||
|
t.Errorf("2026-02 paid want 750, got %f", months["2026-02"].Paid)
|
||||||
|
}
|
||||||
|
if int(months["2026-03"].Paid) != 350 {
|
||||||
|
t.Errorf("2026-03 paid want 350, got %f", months["2026-03"].Paid)
|
||||||
|
}
|
||||||
|
if int(months["2026-04"].Paid) != 150 {
|
||||||
|
t.Errorf("2026-04 paid want 150, got %f", months["2026-04"].Paid)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestReconcileGreedyOverpaymentGoesToCredit(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
members := []Member{{
|
||||||
|
Name: "Alice", Tier: "A",
|
||||||
|
Fees: map[string]FeeData{"2026-01": {750, 3}, "2026-02": {750, 3}},
|
||||||
|
}}
|
||||||
|
sortedMonths := []string{"2026-01", "2026-02"}
|
||||||
|
|
||||||
|
result := Reconcile(members, sortedMonths, []Transaction{tx("Alice", "2026-01, 2026-02", 2000)}, nil, defaultYear)
|
||||||
|
|
||||||
|
months := result.Members["Alice"].Months
|
||||||
|
if int(months["2026-01"].Paid) != 750 {
|
||||||
|
t.Errorf("2026-01 paid want 750, got %f", months["2026-01"].Paid)
|
||||||
|
}
|
||||||
|
if int(months["2026-02"].Paid) != 750 {
|
||||||
|
t.Errorf("2026-02 paid want 750, got %f", months["2026-02"].Paid)
|
||||||
|
}
|
||||||
|
if result.Credits["Alice"] != 500 {
|
||||||
|
t.Errorf("credits want 500, got %d", result.Credits["Alice"])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestReconcileProportionalUnderpayment(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
members := []Member{{
|
||||||
|
Name: "Alice", Tier: "A",
|
||||||
|
Fees: map[string]FeeData{"2026-02": {750, 3}, "2026-03": {350, 3}, "2026-04": {750, 3}},
|
||||||
|
}}
|
||||||
|
sortedMonths := []string{"2026-02", "2026-03", "2026-04"}
|
||||||
|
amount := 1250.0
|
||||||
|
|
||||||
|
result := Reconcile(members, sortedMonths, []Transaction{tx("Alice", "2026-02, 2026-03, 2026-04", amount)}, nil, defaultYear)
|
||||||
|
|
||||||
|
months := result.Members["Alice"].Months
|
||||||
|
paid02 := months["2026-02"].Paid
|
||||||
|
paid03 := months["2026-03"].Paid
|
||||||
|
paid04 := months["2026-04"].Paid
|
||||||
|
|
||||||
|
if paid02 >= 750 {
|
||||||
|
t.Errorf("2026-02 should be underpaid, got %f", paid02)
|
||||||
|
}
|
||||||
|
if paid03 >= 350 {
|
||||||
|
t.Errorf("2026-03 should be underpaid, got %f", paid03)
|
||||||
|
}
|
||||||
|
if paid04 >= 750 {
|
||||||
|
t.Errorf("2026-04 should be underpaid, got %f", paid04)
|
||||||
|
}
|
||||||
|
if math.Abs(paid02+paid03+paid04-amount) > 0.01 {
|
||||||
|
t.Errorf("sum of paid want %f, got %f", amount, paid02+paid03+paid04)
|
||||||
|
}
|
||||||
|
if math.Abs(paid02-paid04) > 0.01 {
|
||||||
|
t.Errorf("02 and 04 have equal expected, want equal paid: %f vs %f", paid02, paid04)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestReconcileSingleMonthUnchanged(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
members := []Member{{Name: "Alice", Tier: "A", Fees: map[string]FeeData{"2026-01": {750, 3}}}}
|
||||||
|
|
||||||
|
result := Reconcile(members, []string{"2026-01"}, []Transaction{tx("Alice", "2026-01", 750)}, nil, defaultYear)
|
||||||
|
|
||||||
|
if math.Abs(result.Members["Alice"].Months["2026-01"].Paid-750) > 0.01 {
|
||||||
|
t.Errorf("single month want 750, got %f", result.Members["Alice"].Months["2026-01"].Paid)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestReconcileTwoMembersMultiMonth(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
members := []Member{
|
||||||
|
{Name: "Alice", Tier: "A", Fees: map[string]FeeData{"2026-01": {750, 3}, "2026-02": {350, 3}}},
|
||||||
|
{Name: "Bob", Tier: "A", Fees: map[string]FeeData{"2026-01": {750, 3}, "2026-02": {350, 3}}},
|
||||||
|
}
|
||||||
|
sortedMonths := []string{"2026-01", "2026-02"}
|
||||||
|
|
||||||
|
result := Reconcile(members, sortedMonths, []Transaction{tx("Alice, Bob", "2026-01, 2026-02", 2200)}, nil, defaultYear)
|
||||||
|
|
||||||
|
for _, name := range []string{"Alice", "Bob"} {
|
||||||
|
months := result.Members[name].Months
|
||||||
|
if math.Abs(months["2026-01"].Paid-750) > 0.01 {
|
||||||
|
t.Errorf("%s 2026-01 paid want 750, got %f", name, months["2026-01"].Paid)
|
||||||
|
}
|
||||||
|
if math.Abs(months["2026-02"].Paid-350) > 0.01 {
|
||||||
|
t.Errorf("%s 2026-02 paid want 350, got %f", name, months["2026-02"].Paid)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestReconcileEvenSplitFallbackWhenNoExpected(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
members := []Member{{
|
||||||
|
Name: "Alice", Tier: "A",
|
||||||
|
Fees: map[string]FeeData{"2026-01": {0, 0}, "2026-02": {0, 0}},
|
||||||
|
}}
|
||||||
|
sortedMonths := []string{"2026-01", "2026-02"}
|
||||||
|
|
||||||
|
result := Reconcile(members, sortedMonths, []Transaction{tx("Alice", "2026-01, 2026-02", 300)}, nil, defaultYear)
|
||||||
|
|
||||||
|
months := result.Members["Alice"].Months
|
||||||
|
if math.Abs(months["2026-01"].Paid-150) > 0.01 {
|
||||||
|
t.Errorf("2026-01 paid want 150, got %f", months["2026-01"].Paid)
|
||||||
|
}
|
||||||
|
if math.Abs(months["2026-02"].Paid-150) > 0.01 {
|
||||||
|
t.Errorf("2026-02 paid want 150, got %f", months["2026-02"].Paid)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestReconcileDiacriticsTolerantPersonMatching(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
members := []Member{{Name: "Mária Maco", Tier: "A", Fees: map[string]FeeData{"2026-04": {750, 4}}}}
|
||||||
|
txFn := func(person string) Transaction {
|
||||||
|
return Transaction{
|
||||||
|
Date: "2026-04-15", Amount: 750, Person: person, Purpose: "2026-04",
|
||||||
|
Sender: "Maco Family", Message: "fee",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
cases := []struct {
|
||||||
|
name string
|
||||||
|
person string
|
||||||
|
}{
|
||||||
|
{"without diacritics", "Maria Maco"},
|
||||||
|
{"extra whitespace", "Mária Maco"},
|
||||||
|
{"lowercase", "mária maco"},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tc := range cases {
|
||||||
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
result := Reconcile(members, []string{"2026-04"}, []Transaction{txFn(tc.person)}, nil, defaultYear)
|
||||||
|
|
||||||
|
paid := result.Members["Mária Maco"].Months["2026-04"].Paid
|
||||||
|
if paid != 750 {
|
||||||
|
t.Errorf("%s: paid want 750, got %f", tc.name, paid)
|
||||||
|
}
|
||||||
|
if len(result.Unmatched) != 0 {
|
||||||
|
t.Errorf("%s: want no unmatched, got %v", tc.name, result.Unmatched)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestReconcileTrulyUnknownPersonIsUnmatched(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
members := []Member{{Name: "Mária Maco", Tier: "A", Fees: map[string]FeeData{"2026-04": {750, 4}}}}
|
||||||
|
txs := []Transaction{{
|
||||||
|
Date: "2026-04-15", Amount: 750,
|
||||||
|
Person: "Někdo Neznámý", Purpose: "2026-04",
|
||||||
|
Sender: "Neznámý", Message: "fee",
|
||||||
|
}}
|
||||||
|
|
||||||
|
result := Reconcile(members, []string{"2026-04"}, txs, nil, defaultYear)
|
||||||
|
|
||||||
|
if result.Members["Mária Maco"].Months["2026-04"].Paid != 0 {
|
||||||
|
t.Errorf("unknown person must not credit the member")
|
||||||
|
}
|
||||||
|
if len(result.Unmatched) != 1 {
|
||||||
|
t.Errorf("want 1 unmatched, got %d", len(result.Unmatched))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// [Go] Test that [?] markers are stripped from the Person field before lookup.
|
||||||
|
func TestReconcileQuestionMarkMarkerStripped(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
members := []Member{{Name: "Alice", Tier: "A", Fees: map[string]FeeData{"2026-01": {750, 3}}}}
|
||||||
|
txs := []Transaction{{
|
||||||
|
Date: "2026-01-01", Amount: 750,
|
||||||
|
Person: "[?] Alice", Purpose: "2026-01",
|
||||||
|
Sender: "Bank", Message: "fee",
|
||||||
|
}}
|
||||||
|
|
||||||
|
result := Reconcile(members, []string{"2026-01"}, txs, nil, defaultYear)
|
||||||
|
|
||||||
|
if result.Members["Alice"].Months["2026-01"].Paid != 750 {
|
||||||
|
t.Errorf("[?] stripping: want 750 paid, got %f", result.Members["Alice"].Months["2026-01"].Paid)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// [Go] Purpose "other:shirt" puts payment in OtherTransactions, not in month ledger.
|
||||||
|
func TestReconcileOtherPurpose(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
members := []Member{{Name: "Alice", Tier: "A", Fees: map[string]FeeData{"2026-01": {750, 3}}}}
|
||||||
|
txs := []Transaction{{
|
||||||
|
Date: "2026-01-01", Amount: 300,
|
||||||
|
Person: "Alice", Purpose: "other:shirt",
|
||||||
|
Sender: "Bank", Message: "shirt order",
|
||||||
|
}}
|
||||||
|
|
||||||
|
result := Reconcile(members, []string{"2026-01"}, txs, nil, defaultYear)
|
||||||
|
|
||||||
|
if result.Members["Alice"].Months["2026-01"].Paid != 0 {
|
||||||
|
t.Errorf("other: purpose must not touch month ledger")
|
||||||
|
}
|
||||||
|
others := result.Members["Alice"].OtherTransactions
|
||||||
|
if len(others) != 1 {
|
||||||
|
t.Fatalf("want 1 OtherTransaction, got %d", len(others))
|
||||||
|
}
|
||||||
|
if math.Abs(others[0].Amount-300) > 0.01 {
|
||||||
|
t.Errorf("OtherEntry.Amount want 300, got %f", others[0].Amount)
|
||||||
|
}
|
||||||
|
if others[0].Purpose != "other:shirt" {
|
||||||
|
t.Errorf("OtherEntry.Purpose want %q, got %q", "other:shirt", others[0].Purpose)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// [Go] Months outside sortedMonths go to credit, not to the window ledger.
|
||||||
|
func TestReconcileOutOfWindowGoesToCredit(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
// Window shows only 2026-01. Transaction references 2026-01 (in) and 2026-02 (out).
|
||||||
|
members := []Member{{Name: "Alice", Tier: "A", Fees: map[string]FeeData{"2026-01": {600, 3}}}}
|
||||||
|
txs := []Transaction{{
|
||||||
|
Date: "2026-01-01", Amount: 1200,
|
||||||
|
Person: "Alice", Purpose: "2026-01, 2026-02",
|
||||||
|
Sender: "Bank", Message: "Q1",
|
||||||
|
}}
|
||||||
|
|
||||||
|
result := Reconcile(members, []string{"2026-01"}, txs, nil, defaultYear)
|
||||||
|
|
||||||
|
// member_share = 1200 (one member)
|
||||||
|
// out_credit = 1200 / 2 * 1 = 600
|
||||||
|
// in_window_share = 600
|
||||||
|
// in_window = [(2026-01, 600)], total_expected = 600 → greedy: paid = 600, no overflow
|
||||||
|
if math.Abs(result.Members["Alice"].Months["2026-01"].Paid-600) > 0.01 {
|
||||||
|
t.Errorf("in-window paid want 600, got %f", result.Members["Alice"].Months["2026-01"].Paid)
|
||||||
|
}
|
||||||
|
// total_balance = int(600) - 600 (window) + 600 (out credit) = 600
|
||||||
|
if result.Members["Alice"].TotalBalance != 600 {
|
||||||
|
t.Errorf("TotalBalance want 600, got %d", result.Members["Alice"].TotalBalance)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// [Go] No person/purpose → inference fallback resolves sender name and date month.
|
||||||
|
func TestReconcileInferenceFallback(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
members := []Member{{Name: "Tomáš Němeček", Tier: "A", Fees: map[string]FeeData{"2026-04": {750, 3}}}}
|
||||||
|
txs := []Transaction{{
|
||||||
|
Date: "2026-04-15", Amount: 750,
|
||||||
|
// Person and Purpose are empty → inference path
|
||||||
|
Sender: "Tomas Nemecek",
|
||||||
|
Message: "clenske 04/2026",
|
||||||
|
}}
|
||||||
|
|
||||||
|
result := Reconcile(members, []string{"2026-04"}, txs, nil, defaultYear)
|
||||||
|
|
||||||
|
if math.Abs(result.Members["Tomáš Němeček"].Months["2026-04"].Paid-750) > 0.01 {
|
||||||
|
t.Errorf("inference fallback: want 750 paid, got %f", result.Members["Tomáš Němeček"].Months["2026-04"].Paid)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// [Go] Transaction with no match at all ends up in Unmatched; ledger unchanged.
|
||||||
|
func TestReconcileNoMatchGoesToUnmatched(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
members := []Member{{Name: "Alice", Tier: "A", Fees: map[string]FeeData{"2026-01": {750, 3}}}}
|
||||||
|
txs := []Transaction{{
|
||||||
|
Date: "2026-01-01", Amount: 500,
|
||||||
|
// empty person+purpose and sender name not matching any member
|
||||||
|
Sender: "Unknown Corp", Message: "invoice",
|
||||||
|
}}
|
||||||
|
|
||||||
|
result := Reconcile(members, []string{"2026-01"}, txs, nil, defaultYear)
|
||||||
|
|
||||||
|
if len(result.Unmatched) != 1 {
|
||||||
|
t.Errorf("want 1 unmatched, got %d", len(result.Unmatched))
|
||||||
|
}
|
||||||
|
if result.Members["Alice"].Months["2026-01"].Paid != 0 {
|
||||||
|
t.Errorf("unmatched tx must not touch ledger")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// [Go] Empty transaction list leaves every month at paid=0 and balance=–expected.
|
||||||
|
func TestReconcileNoTransactionsAllUnpaid(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
members := []Member{{Name: "Alice", Tier: "A", Fees: map[string]FeeData{"2026-01": {750, 3}}}}
|
||||||
|
|
||||||
|
result := Reconcile(members, []string{"2026-01"}, nil, nil, defaultYear)
|
||||||
|
|
||||||
|
if result.Members["Alice"].Months["2026-01"].Paid != 0 {
|
||||||
|
t.Errorf("no txs: want paid=0, got %f", result.Members["Alice"].Months["2026-01"].Paid)
|
||||||
|
}
|
||||||
|
if result.Members["Alice"].TotalBalance != -750 {
|
||||||
|
t.Errorf("no txs: want balance -750, got %d", result.Members["Alice"].TotalBalance)
|
||||||
|
}
|
||||||
|
if len(result.Unmatched) != 0 {
|
||||||
|
t.Errorf("no txs: want empty unmatched, got %v", result.Unmatched)
|
||||||
|
}
|
||||||
|
}
|
||||||
64
go/internal/io/attendance/client.go
Normal file
64
go/internal/io/attendance/client.go
Normal file
@@ -0,0 +1,64 @@
|
|||||||
|
// Package attendance fetches attendance CSV exports from Google Sheets.
|
||||||
|
// No auth required — the sheet must be publicly readable.
|
||||||
|
package attendance
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/csv"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"net/http"
|
||||||
|
"strings"
|
||||||
|
)
|
||||||
|
|
||||||
|
const exportBase = "https://docs.google.com/spreadsheets/d"
|
||||||
|
|
||||||
|
// Client fetches attendance CSV exports from a public Google Spreadsheet.
|
||||||
|
type Client struct {
|
||||||
|
http *http.Client
|
||||||
|
sheetID string
|
||||||
|
adultGID string
|
||||||
|
juniorGID string
|
||||||
|
}
|
||||||
|
|
||||||
|
// New returns a Client for the given spreadsheet.
|
||||||
|
// adultGID is typically "0"; juniorGID is the GID of the junior tab.
|
||||||
|
func New(httpClient *http.Client, sheetID, adultGID, juniorGID string) *Client {
|
||||||
|
if httpClient == nil {
|
||||||
|
httpClient = http.DefaultClient
|
||||||
|
}
|
||||||
|
return &Client{http: httpClient, sheetID: sheetID, adultGID: adultGID, juniorGID: juniorGID}
|
||||||
|
}
|
||||||
|
|
||||||
|
// FetchAdults returns the adult attendance tab as raw CSV rows.
|
||||||
|
func (c *Client) FetchAdults(ctx context.Context) ([][]string, error) {
|
||||||
|
return c.fetch(ctx, c.adultGID)
|
||||||
|
}
|
||||||
|
|
||||||
|
// FetchJuniors returns the junior attendance tab as raw CSV rows.
|
||||||
|
func (c *Client) FetchJuniors(ctx context.Context) ([][]string, error) {
|
||||||
|
return c.fetch(ctx, c.juniorGID)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Client) fetch(ctx context.Context, gid string) ([][]string, error) {
|
||||||
|
url := fmt.Sprintf("%s/%s/export?format=csv&gid=%s", exportBase, c.sheetID, gid)
|
||||||
|
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
resp, err := c.http.Do(req)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
defer resp.Body.Close()
|
||||||
|
if resp.StatusCode != http.StatusOK {
|
||||||
|
return nil, fmt.Errorf("attendance fetch: HTTP %d for gid=%s", resp.StatusCode, gid)
|
||||||
|
}
|
||||||
|
body, err := io.ReadAll(resp.Body)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
r := csv.NewReader(strings.NewReader(string(body)))
|
||||||
|
r.FieldsPerRecord = -1 // rows may have different lengths
|
||||||
|
return r.ReadAll()
|
||||||
|
}
|
||||||
93
go/internal/io/attendance/client_test.go
Normal file
93
go/internal/io/attendance/client_test.go
Normal file
@@ -0,0 +1,93 @@
|
|||||||
|
package attendance
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/csv"
|
||||||
|
"io"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"os"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestClientFetchAdults(t *testing.T) {
|
||||||
|
data, err := os.ReadFile("testdata/adults_minimal.csv")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
|
||||||
|
_, _ = w.Write(data)
|
||||||
|
}))
|
||||||
|
defer srv.Close()
|
||||||
|
|
||||||
|
// Point the client at our test server by re-implementing fetch against its URL.
|
||||||
|
rows, err := fetchURL(context.Background(), srv.Client(), srv.URL)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(rows) < 2 {
|
||||||
|
t.Fatalf("want ≥2 rows, got %d", len(rows))
|
||||||
|
}
|
||||||
|
if rows[0][0] != "Jméno" {
|
||||||
|
t.Errorf("unexpected header: %q", rows[0][0])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestFake(t *testing.T) {
|
||||||
|
adultRows := parseCSV(t, "testdata/adults_minimal.csv")
|
||||||
|
juniorRows := parseCSV(t, "testdata/juniors_minimal.csv")
|
||||||
|
|
||||||
|
f := &Fake{Adults: adultRows, Juniors: juniorRows}
|
||||||
|
|
||||||
|
got, err := f.FetchAdults(context.Background())
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if got[0][0] != "Jméno" {
|
||||||
|
t.Errorf("adults header: %q", got[0][0])
|
||||||
|
}
|
||||||
|
|
||||||
|
got, err = f.FetchJuniors(context.Background())
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if got[1][0] != "Junior One" {
|
||||||
|
t.Errorf("juniors first member: %q", got[1][0])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func parseCSV(t *testing.T, path string) [][]string {
|
||||||
|
t.Helper()
|
||||||
|
b, err := os.ReadFile(path)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
r := csv.NewReader(strings.NewReader(string(b)))
|
||||||
|
r.FieldsPerRecord = -1
|
||||||
|
rows, err := r.ReadAll()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
return rows
|
||||||
|
}
|
||||||
|
|
||||||
|
// fetchURL is a test helper that exercises the shared fetch logic against an arbitrary URL.
|
||||||
|
func fetchURL(ctx context.Context, hc *http.Client, url string) ([][]string, error) {
|
||||||
|
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
resp, err := hc.Do(req)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
defer resp.Body.Close()
|
||||||
|
b, err := io.ReadAll(resp.Body)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
r := csv.NewReader(strings.NewReader(string(b)))
|
||||||
|
r.FieldsPerRecord = -1
|
||||||
|
return r.ReadAll()
|
||||||
|
}
|
||||||
12
go/internal/io/attendance/fake.go
Normal file
12
go/internal/io/attendance/fake.go
Normal file
@@ -0,0 +1,12 @@
|
|||||||
|
package attendance
|
||||||
|
|
||||||
|
import "context"
|
||||||
|
|
||||||
|
// Fake is an in-memory replacement for Client, used in tests.
|
||||||
|
type Fake struct {
|
||||||
|
Adults [][]string
|
||||||
|
Juniors [][]string
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *Fake) FetchAdults(_ context.Context) ([][]string, error) { return f.Adults, nil }
|
||||||
|
func (f *Fake) FetchJuniors(_ context.Context) ([][]string, error) { return f.Juniors, nil }
|
||||||
4
go/internal/io/attendance/testdata/adults_minimal.csv
vendored
Normal file
4
go/internal/io/attendance/testdata/adults_minimal.csv
vendored
Normal file
@@ -0,0 +1,4 @@
|
|||||||
|
Jméno,Tier,,,01.09.2025,08.09.2025,15.09.2025
|
||||||
|
Member One,A,,,TRUE,TRUE,FALSE
|
||||||
|
Member Two,A,,,TRUE,FALSE,FALSE
|
||||||
|
# last line,,,,,
|
||||||
|
4
go/internal/io/attendance/testdata/juniors_minimal.csv
vendored
Normal file
4
go/internal/io/attendance/testdata/juniors_minimal.csv
vendored
Normal file
@@ -0,0 +1,4 @@
|
|||||||
|
Jméno,Tier,,,01.09.2025,08.09.2025,15.09.2025
|
||||||
|
Junior One,J,,,TRUE,TRUE,TRUE
|
||||||
|
# Trenéři,,,,,
|
||||||
|
Coach One,X,,,FALSE,FALSE,FALSE
|
||||||
|
209
go/internal/io/cache/filecache.go
vendored
Normal file
209
go/internal/io/cache/filecache.go
vendored
Normal file
@@ -0,0 +1,209 @@
|
|||||||
|
// Package cache implements a Drive-modifiedTime-gated JSON file cache,
|
||||||
|
// mirroring scripts/cache_utils.py.
|
||||||
|
package cache
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
|
"fuj-management/go/internal/io/drive"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"sync"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// DriveClient is the subset of drive.Client used by FileCache.
|
||||||
|
type DriveClient interface {
|
||||||
|
ModifiedTime(ctx context.Context, fileID string) (string, error)
|
||||||
|
}
|
||||||
|
|
||||||
|
type cacheFile struct {
|
||||||
|
ModifiedTime string `json:"modifiedTime"`
|
||||||
|
Data json.RawMessage `json:"data"`
|
||||||
|
CachedAt string `json:"cachedAt"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// FileCache wraps a Drive client to gate JSON file caching on sheet modifiedTime.
|
||||||
|
//
|
||||||
|
// Two TTL knobs mirror scripts/cache_utils.py:
|
||||||
|
// - ttl: if the cache file on disk is younger than this, skip the Drive check entirely.
|
||||||
|
// - apiCheckTTL: debounces in-memory Drive API calls per sheet ID.
|
||||||
|
//
|
||||||
|
// Atomic writes: data is marshaled to a .tmp file then os.Rename'd.
|
||||||
|
// Cache files are compatible with Python's format:
|
||||||
|
//
|
||||||
|
// {"modifiedTime":"…","data":…,"cachedAt":"…"}
|
||||||
|
type FileCache struct {
|
||||||
|
drive DriveClient
|
||||||
|
dir string
|
||||||
|
sheetMap map[string]string // cache key → Drive file ID
|
||||||
|
ttl time.Duration
|
||||||
|
apiCheckTTL time.Duration
|
||||||
|
mu sync.Mutex
|
||||||
|
lastChecked map[string]time.Time
|
||||||
|
}
|
||||||
|
|
||||||
|
// New creates a FileCache.
|
||||||
|
// sheetMap maps cache keys to Google Sheets/Drive file IDs (mirrors CACHE_SHEET_MAP in config).
|
||||||
|
func New(d DriveClient, dir string, sheetMap map[string]string, ttl, apiCheckTTL time.Duration) *FileCache {
|
||||||
|
return &FileCache{
|
||||||
|
drive: d,
|
||||||
|
dir: dir,
|
||||||
|
sheetMap: sheetMap,
|
||||||
|
ttl: ttl,
|
||||||
|
apiCheckTTL: apiCheckTTL,
|
||||||
|
lastChecked: make(map[string]time.Time),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Get returns the cached value for cacheKey, calling fetch if the cache is stale.
|
||||||
|
// T must be JSON-marshalable.
|
||||||
|
func Get[T any](ctx context.Context, fc *FileCache, cacheKey string, fetch func(context.Context) (T, error)) (T, error) {
|
||||||
|
sheetID := fc.sheetMap[cacheKey]
|
||||||
|
if sheetID == "" {
|
||||||
|
sheetID = cacheKey
|
||||||
|
}
|
||||||
|
cacheFilePath := filepath.Join(fc.dir, cacheKey+"_cache.json")
|
||||||
|
|
||||||
|
currentModTime, err := fc.currentModifiedTime(ctx, sheetID, cacheFilePath)
|
||||||
|
if err != nil {
|
||||||
|
return *new(T), fmt.Errorf("cache: modifiedTime for %s: %w", cacheKey, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Try cache hit
|
||||||
|
if data, ok := readCache[T](cacheFilePath, currentModTime); ok {
|
||||||
|
return data, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Cache miss — fetch fresh data
|
||||||
|
fresh, err := fetch(ctx)
|
||||||
|
if err != nil {
|
||||||
|
return *new(T), err
|
||||||
|
}
|
||||||
|
if err := writeCache(cacheFilePath, currentModTime, fresh); err != nil {
|
||||||
|
// Non-fatal: log but don't fail the request
|
||||||
|
_, _ = fmt.Fprintf(os.Stderr, "cache: write %s: %v\n", cacheKey, err)
|
||||||
|
}
|
||||||
|
return fresh, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Flush deletes all *_cache.json files in the cache dir and resets in-memory state.
|
||||||
|
func (fc *FileCache) Flush() (int, error) {
|
||||||
|
fc.mu.Lock()
|
||||||
|
fc.lastChecked = make(map[string]time.Time)
|
||||||
|
fc.mu.Unlock()
|
||||||
|
|
||||||
|
pattern := filepath.Join(fc.dir, "*_cache.json")
|
||||||
|
matches, err := filepath.Glob(pattern)
|
||||||
|
if err != nil {
|
||||||
|
return 0, err
|
||||||
|
}
|
||||||
|
for _, f := range matches {
|
||||||
|
_ = os.Remove(f)
|
||||||
|
}
|
||||||
|
return len(matches), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// currentModifiedTime returns a stable string representing the current version
|
||||||
|
// of the sheet, using the in-memory + file-mtime TTL guards before hitting Drive.
|
||||||
|
// On Drive failure, falls back to a 5-minute bucket string (matching Python).
|
||||||
|
func (fc *FileCache) currentModifiedTime(ctx context.Context, sheetID, cacheFilePath string) (string, error) {
|
||||||
|
now := time.Now()
|
||||||
|
|
||||||
|
fc.mu.Lock()
|
||||||
|
lastCheck := fc.lastChecked[sheetID]
|
||||||
|
fc.mu.Unlock()
|
||||||
|
|
||||||
|
// Guard 1: in-memory debounce — skip Drive if checked recently
|
||||||
|
if fc.apiCheckTTL > 0 && now.Sub(lastCheck) < fc.apiCheckTTL {
|
||||||
|
if mt, ok := readModifiedTime(cacheFilePath); ok {
|
||||||
|
return mt, nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Guard 2: cache file is young enough — trust the stored modifiedTime
|
||||||
|
if fc.ttl > 0 {
|
||||||
|
if info, err := os.Stat(cacheFilePath); err == nil {
|
||||||
|
if now.Sub(info.ModTime()) < fc.ttl {
|
||||||
|
if mt, ok := readModifiedTime(cacheFilePath); ok {
|
||||||
|
fc.mu.Lock()
|
||||||
|
fc.lastChecked[sheetID] = now
|
||||||
|
fc.mu.Unlock()
|
||||||
|
return mt, nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Hit Drive API
|
||||||
|
mt, err := fc.drive.ModifiedTime(ctx, sheetID)
|
||||||
|
if err != nil {
|
||||||
|
// Fallback: 5-minute bucket string, matches Python _fallback_ttl()
|
||||||
|
bucket := time.Now().Unix() / 300
|
||||||
|
return fmt.Sprintf("ttl-5m-%d", bucket), nil
|
||||||
|
}
|
||||||
|
fc.mu.Lock()
|
||||||
|
fc.lastChecked[sheetID] = now
|
||||||
|
fc.mu.Unlock()
|
||||||
|
return mt, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func readModifiedTime(path string) (string, bool) {
|
||||||
|
cf, ok := readCacheFile(path)
|
||||||
|
if !ok {
|
||||||
|
return "", false
|
||||||
|
}
|
||||||
|
return cf.ModifiedTime, cf.ModifiedTime != ""
|
||||||
|
}
|
||||||
|
|
||||||
|
func readCacheFile(path string) (cacheFile, bool) {
|
||||||
|
b, err := os.ReadFile(path)
|
||||||
|
if err != nil {
|
||||||
|
return cacheFile{}, false
|
||||||
|
}
|
||||||
|
var cf cacheFile
|
||||||
|
if err := json.Unmarshal(b, &cf); err != nil {
|
||||||
|
return cacheFile{}, false
|
||||||
|
}
|
||||||
|
return cf, true
|
||||||
|
}
|
||||||
|
|
||||||
|
func readCache[T any](path, currentModTime string) (T, bool) {
|
||||||
|
cf, ok := readCacheFile(path)
|
||||||
|
if !ok || cf.ModifiedTime != currentModTime {
|
||||||
|
return *new(T), false
|
||||||
|
}
|
||||||
|
var v T
|
||||||
|
if err := json.Unmarshal(cf.Data, &v); err != nil {
|
||||||
|
return *new(T), false
|
||||||
|
}
|
||||||
|
return v, true
|
||||||
|
}
|
||||||
|
|
||||||
|
func writeCache(path, modTime string, data any) error {
|
||||||
|
raw, err := json.Marshal(data)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
cf := cacheFile{
|
||||||
|
ModifiedTime: modTime,
|
||||||
|
Data: json.RawMessage(raw),
|
||||||
|
CachedAt: time.Now().Format(time.RFC3339),
|
||||||
|
}
|
||||||
|
b, err := json.Marshal(cf)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
tmp := path + ".tmp"
|
||||||
|
if err := os.WriteFile(tmp, b, 0o600); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return os.Rename(tmp, path)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Ensure *drive.Client satisfies DriveClient at compile time.
|
||||||
|
var _ DriveClient = (*drive.Client)(nil)
|
||||||
125
go/internal/io/cache/filecache_test.go
vendored
Normal file
125
go/internal/io/cache/filecache_test.go
vendored
Normal file
@@ -0,0 +1,125 @@
|
|||||||
|
package cache
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"fuj-management/go/internal/io/drive"
|
||||||
|
"os"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestGet_FreshFetch(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
d := &drive.Fake{Times: map[string]string{"sheet1": "2026-01-01T00:00:00Z"}}
|
||||||
|
fc := New(d, dir, map[string]string{"mykey": "sheet1"}, time.Minute, time.Minute)
|
||||||
|
|
||||||
|
calls := 0
|
||||||
|
got, err := Get(context.Background(), fc, "mykey", func(_ context.Context) ([]string, error) {
|
||||||
|
calls++
|
||||||
|
return []string{"a", "b"}, nil
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(got) != 2 || got[0] != "a" {
|
||||||
|
t.Errorf("unexpected: %v", got)
|
||||||
|
}
|
||||||
|
if calls != 1 {
|
||||||
|
t.Errorf("want 1 fetch call, got %d", calls)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestGet_CacheHit(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
d := &drive.Fake{Times: map[string]string{"sheet1": "2026-01-01T00:00:00Z"}}
|
||||||
|
fc := New(d, dir, map[string]string{"mykey": "sheet1"}, time.Minute, time.Minute)
|
||||||
|
|
||||||
|
fetch := func(_ context.Context) ([]string, error) { return []string{"a"}, nil }
|
||||||
|
if _, err := Get(context.Background(), fc, "mykey", fetch); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Second call — modifiedTime unchanged, should hit cache
|
||||||
|
calls := 0
|
||||||
|
got, err := Get(context.Background(), fc, "mykey", func(_ context.Context) ([]string, error) {
|
||||||
|
calls++
|
||||||
|
return []string{"SHOULD_NOT_CALL"}, nil
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if got[0] != "a" {
|
||||||
|
t.Errorf("want cache hit with 'a', got %q", got[0])
|
||||||
|
}
|
||||||
|
if calls != 0 {
|
||||||
|
t.Errorf("want 0 fetch calls on hit, got %d", calls)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestGet_CacheMiss_OnModifiedTimeChange(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
d := &drive.Fake{Times: map[string]string{"sheet1": "2026-01-01T00:00:00Z"}}
|
||||||
|
// No TTL guards so we always hit Drive
|
||||||
|
fc := New(d, dir, map[string]string{"mykey": "sheet1"}, 0, 0)
|
||||||
|
|
||||||
|
fetch := func(_ context.Context) ([]string, error) { return []string{"v1"}, nil }
|
||||||
|
if _, err := Get(context.Background(), fc, "mykey", fetch); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Sheet updated — change modifiedTime
|
||||||
|
d.Times["sheet1"] = "2026-02-01T00:00:00Z"
|
||||||
|
got, err := Get(context.Background(), fc, "mykey", func(_ context.Context) ([]string, error) {
|
||||||
|
return []string{"v2"}, nil
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if got[0] != "v2" {
|
||||||
|
t.Errorf("want v2 after sheet update, got %q", got[0])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestGet_DriveFailureFallback(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
d := &drive.Fake{Err: errors.New("drive down")}
|
||||||
|
fc := New(d, dir, nil, 0, 0)
|
||||||
|
|
||||||
|
calls := 0
|
||||||
|
_, err := Get(context.Background(), fc, "mykey", func(_ context.Context) ([]string, error) {
|
||||||
|
calls++
|
||||||
|
return []string{"fallback"}, nil
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if calls != 1 {
|
||||||
|
t.Errorf("want 1 fetch call, got %d", calls)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestFlush(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
d := &drive.Fake{Times: map[string]string{"sheet1": "t1"}}
|
||||||
|
fc := New(d, dir, map[string]string{"k": "sheet1"}, 0, 0)
|
||||||
|
|
||||||
|
if _, err := Get(context.Background(), fc, "k", func(_ context.Context) (int, error) { return 42, nil }); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
n, err := fc.Flush()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if n != 1 {
|
||||||
|
t.Errorf("want 1 deleted file, got %d", n)
|
||||||
|
}
|
||||||
|
// Cache dir should be empty of _cache.json files
|
||||||
|
entries, _ := os.ReadDir(dir)
|
||||||
|
for _, e := range entries {
|
||||||
|
if e.Name() != "" {
|
||||||
|
t.Errorf("expected empty dir after flush, found %s", e.Name())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
46
go/internal/io/drive/client.go
Normal file
46
go/internal/io/drive/client.go
Normal file
@@ -0,0 +1,46 @@
|
|||||||
|
// Package drive provides a thin wrapper around the Google Drive v3 API,
|
||||||
|
// used only to read modifiedTime for cache invalidation.
|
||||||
|
package drive
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"net/http"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"google.golang.org/api/drive/v3"
|
||||||
|
"google.golang.org/api/option"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Client wraps the Drive v3 API, scoped to read-only modifiedTime checks.
|
||||||
|
type Client struct {
|
||||||
|
svc *drive.Service
|
||||||
|
}
|
||||||
|
|
||||||
|
// New builds a Client using a service-account credentials file.
|
||||||
|
// timeout applies to each Drive API call.
|
||||||
|
func New(ctx context.Context, credentialsPath string, timeout time.Duration) (*Client, error) {
|
||||||
|
hc := &http.Client{Timeout: timeout}
|
||||||
|
svc, err := drive.NewService(ctx,
|
||||||
|
option.WithCredentialsFile(credentialsPath), //nolint:staticcheck
|
||||||
|
option.WithScopes(drive.DriveReadonlyScope),
|
||||||
|
option.WithHTTPClient(hc),
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return &Client{svc: svc}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// ModifiedTime returns the RFC3339 modifiedTime for the given Drive file ID.
|
||||||
|
// Returns ("", err) if the Drive API call fails.
|
||||||
|
func (c *Client) ModifiedTime(ctx context.Context, fileID string) (string, error) {
|
||||||
|
meta, err := c.svc.Files.Get(fileID).
|
||||||
|
Fields("modifiedTime").
|
||||||
|
SupportsAllDrives(true).
|
||||||
|
Context(ctx).
|
||||||
|
Do()
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
return meta.ModifiedTime, nil
|
||||||
|
}
|
||||||
18
go/internal/io/drive/fake.go
Normal file
18
go/internal/io/drive/fake.go
Normal file
@@ -0,0 +1,18 @@
|
|||||||
|
package drive
|
||||||
|
|
||||||
|
import "context"
|
||||||
|
|
||||||
|
// Fake is an in-memory replacement for Client used in tests.
|
||||||
|
type Fake struct {
|
||||||
|
// Times maps file ID → modifiedTime string returned by ModifiedTime.
|
||||||
|
Times map[string]string
|
||||||
|
// Err, if non-nil, is returned instead of looking up Times.
|
||||||
|
Err error
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *Fake) ModifiedTime(_ context.Context, fileID string) (string, error) {
|
||||||
|
if f.Err != nil {
|
||||||
|
return "", f.Err
|
||||||
|
}
|
||||||
|
return f.Times[fileID], nil
|
||||||
|
}
|
||||||
128
go/internal/io/fio/api.go
Normal file
128
go/internal/io/fio/api.go
Normal file
@@ -0,0 +1,128 @@
|
|||||||
|
package fio
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"net/http"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// httpDoer is the subset of *http.Client used by both Fio impls.
|
||||||
|
type httpDoer interface {
|
||||||
|
Do(*http.Request) (*http.Response, error)
|
||||||
|
}
|
||||||
|
|
||||||
|
// apiClient fetches transactions from the Fio REST API (JSON).
|
||||||
|
// Ports scripts/fio_utils.py fetch_transactions_api.
|
||||||
|
type apiClient struct {
|
||||||
|
token string
|
||||||
|
hc httpDoer
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *apiClient) FetchTransactions(ctx context.Context, from, to time.Time) ([]Transaction, error) {
|
||||||
|
const layout = "2006-01-02"
|
||||||
|
url := fmt.Sprintf("https://fioapi.fio.cz/v1/rest/periods/%s/%s/%s/transactions.json",
|
||||||
|
c.token, from.Format(layout), to.Format(layout))
|
||||||
|
|
||||||
|
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
resp, err := c.hc.Do(req)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
defer resp.Body.Close()
|
||||||
|
if resp.StatusCode != http.StatusOK {
|
||||||
|
return nil, fmt.Errorf("fio api: HTTP %d", resp.StatusCode)
|
||||||
|
}
|
||||||
|
body, err := io.ReadAll(resp.Body)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return parseAPIResponse(body)
|
||||||
|
}
|
||||||
|
|
||||||
|
// fioAPIResponse is the top-level envelope from the Fio JSON API.
|
||||||
|
type fioAPIResponse struct {
|
||||||
|
AccountStatement struct {
|
||||||
|
TransactionList struct {
|
||||||
|
Transaction []map[string]json.RawMessage `json:"transaction"`
|
||||||
|
} `json:"transactionList"`
|
||||||
|
} `json:"accountStatement"`
|
||||||
|
}
|
||||||
|
|
||||||
|
func parseAPIResponse(body []byte) ([]Transaction, error) {
|
||||||
|
var resp fioAPIResponse
|
||||||
|
if err := json.Unmarshal(body, &resp); err != nil {
|
||||||
|
return nil, fmt.Errorf("fio api: parse JSON: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
var txns []Transaction
|
||||||
|
for _, raw := range resp.AccountStatement.TransactionList.Transaction {
|
||||||
|
amount := colFloat(raw, "column1")
|
||||||
|
if amount <= 0 {
|
||||||
|
continue // skip outgoing
|
||||||
|
}
|
||||||
|
dateRaw := colString(raw, "column0")
|
||||||
|
dateStr := ""
|
||||||
|
if len(dateRaw) >= 10 {
|
||||||
|
dateStr = dateRaw[:10]
|
||||||
|
}
|
||||||
|
txns = append(txns, Transaction{
|
||||||
|
Date: dateStr,
|
||||||
|
Amount: amount,
|
||||||
|
Sender: colString(raw, "column10"),
|
||||||
|
Message: colString(raw, "column16"),
|
||||||
|
VS: colString(raw, "column5"),
|
||||||
|
KS: colString(raw, "column4"),
|
||||||
|
SS: colString(raw, "column6"),
|
||||||
|
UserID: colString(raw, "column7"),
|
||||||
|
SenderAccount: colString(raw, "column2"),
|
||||||
|
BankID: colString(raw, "column22"),
|
||||||
|
Currency: colStringOr(raw, "column14", "CZK"),
|
||||||
|
})
|
||||||
|
}
|
||||||
|
return txns, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// colString extracts {"value":…} as a string from a column map.
|
||||||
|
func colString(m map[string]json.RawMessage, col string) string {
|
||||||
|
raw, ok := m[col]
|
||||||
|
if !ok {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
var cell struct {
|
||||||
|
Value *string `json:"value"`
|
||||||
|
}
|
||||||
|
if json.Unmarshal(raw, &cell) != nil || cell.Value == nil {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
return *cell.Value
|
||||||
|
}
|
||||||
|
|
||||||
|
// colStringOr is colString with a fallback value.
|
||||||
|
func colStringOr(m map[string]json.RawMessage, col, fallback string) string {
|
||||||
|
if v := colString(m, col); v != "" {
|
||||||
|
return v
|
||||||
|
}
|
||||||
|
return fallback
|
||||||
|
}
|
||||||
|
|
||||||
|
// colFloat extracts {"value":…} as a float64 from a column map.
|
||||||
|
// Returns 0 on any error (null column, non-numeric value).
|
||||||
|
func colFloat(m map[string]json.RawMessage, col string) float64 {
|
||||||
|
raw, ok := m[col]
|
||||||
|
if !ok {
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
var cell struct {
|
||||||
|
Value *float64 `json:"value"`
|
||||||
|
}
|
||||||
|
if json.Unmarshal(raw, &cell) != nil || cell.Value == nil {
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
return *cell.Value
|
||||||
|
}
|
||||||
42
go/internal/io/fio/client.go
Normal file
42
go/internal/io/fio/client.go
Normal file
@@ -0,0 +1,42 @@
|
|||||||
|
// Package fio fetches Fio bank transactions via the JSON API or the
|
||||||
|
// transparent-page HTML scraper, behind a common Client interface.
|
||||||
|
package fio
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"net/http"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Transaction is one incoming bank payment. Fields absent from the HTML scraper
|
||||||
|
// (BankID, Currency, UserID, SenderAccount) are empty strings on that path.
|
||||||
|
type Transaction struct {
|
||||||
|
Date string
|
||||||
|
Amount float64
|
||||||
|
Sender string
|
||||||
|
Message string
|
||||||
|
VS string
|
||||||
|
KS string
|
||||||
|
SS string
|
||||||
|
UserID string // column7; empty on HTML path
|
||||||
|
SenderAccount string // column2; empty on HTML path
|
||||||
|
BankID string // column22; empty on HTML path
|
||||||
|
Currency string // column14; empty on HTML path (assume CZK)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Client fetches transactions for a date window.
|
||||||
|
type Client interface {
|
||||||
|
FetchTransactions(ctx context.Context, from, to time.Time) ([]Transaction, error)
|
||||||
|
}
|
||||||
|
|
||||||
|
// New returns an apiClient when token is non-empty, otherwise a transparentClient.
|
||||||
|
// hc may be nil, in which case http.DefaultClient is used.
|
||||||
|
func New(token, accountNum string, hc httpDoer) Client {
|
||||||
|
if hc == nil {
|
||||||
|
hc = http.DefaultClient
|
||||||
|
}
|
||||||
|
if token != "" {
|
||||||
|
return &apiClient{token: token, hc: hc}
|
||||||
|
}
|
||||||
|
return &transparentClient{accountNum: accountNum, hc: hc}
|
||||||
|
}
|
||||||
19
go/internal/io/fio/fake.go
Normal file
19
go/internal/io/fio/fake.go
Normal file
@@ -0,0 +1,19 @@
|
|||||||
|
package fio
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Fake is an in-memory replacement for Client, used in tests.
|
||||||
|
type Fake struct {
|
||||||
|
Transactions []Transaction
|
||||||
|
Err error
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *Fake) FetchTransactions(_ context.Context, _, _ time.Time) ([]Transaction, error) {
|
||||||
|
if f.Err != nil {
|
||||||
|
return nil, f.Err
|
||||||
|
}
|
||||||
|
return f.Transactions, nil
|
||||||
|
}
|
||||||
175
go/internal/io/fio/fio_test.go
Normal file
175
go/internal/io/fio/fio_test.go
Normal file
@@ -0,0 +1,175 @@
|
|||||||
|
package fio
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"io"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"os"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestAPIClient_ParseResponse(t *testing.T) {
|
||||||
|
body, err := os.ReadFile("testdata/api_response.json")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
txns, err := parseAPIResponse(body)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(txns) != 1 {
|
||||||
|
t.Fatalf("want 1 txn (outgoing filtered), got %d", len(txns))
|
||||||
|
}
|
||||||
|
tx := txns[0]
|
||||||
|
if tx.Date != "2026-04-10" {
|
||||||
|
t.Errorf("date: want '2026-04-10', got %q", tx.Date)
|
||||||
|
}
|
||||||
|
if tx.Amount != 750 {
|
||||||
|
t.Errorf("amount: want 750, got %v", tx.Amount)
|
||||||
|
}
|
||||||
|
if tx.Sender != "Jana Novakova" {
|
||||||
|
t.Errorf("sender: want 'Jana Novakova', got %q", tx.Sender)
|
||||||
|
}
|
||||||
|
if tx.Message != "duben 2026" {
|
||||||
|
t.Errorf("message: want 'duben 2026', got %q", tx.Message)
|
||||||
|
}
|
||||||
|
if tx.VS != "123" {
|
||||||
|
t.Errorf("vs: want '123', got %q", tx.VS)
|
||||||
|
}
|
||||||
|
if tx.BankID != "12345678901" {
|
||||||
|
t.Errorf("bank_id: want '12345678901', got %q", tx.BankID)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestAPIClient_HTTPRoundTrip(t *testing.T) {
|
||||||
|
body, _ := os.ReadFile("testdata/api_response.json")
|
||||||
|
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
|
||||||
|
_, _ = w.Write(body)
|
||||||
|
}))
|
||||||
|
defer srv.Close()
|
||||||
|
|
||||||
|
c := &apiClient{token: "TESTTOKEN", hc: &overrideClient{base: srv.Client(), baseURL: srv.URL}}
|
||||||
|
txns, err := c.FetchTransactions(context.Background(), time.Now().AddDate(0, -1, 0), time.Now())
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(txns) != 1 {
|
||||||
|
t.Fatalf("want 1 txn, got %d", len(txns))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestTransparentClient_ParseHTML(t *testing.T) {
|
||||||
|
body, err := os.ReadFile("testdata/transparent.html")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
txns, err := parseTransparentHTML(body)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
// Only the incoming row (750 CZK) should be kept; -200 is outgoing
|
||||||
|
if len(txns) != 1 {
|
||||||
|
t.Fatalf("want 1 txn (outgoing filtered), got %d", len(txns))
|
||||||
|
}
|
||||||
|
tx := txns[0]
|
||||||
|
if tx.Date != "2026-04-10" {
|
||||||
|
t.Errorf("date: want '2026-04-10', got %q", tx.Date)
|
||||||
|
}
|
||||||
|
if tx.Amount != 750 {
|
||||||
|
t.Errorf("amount: want 750, got %v", tx.Amount)
|
||||||
|
}
|
||||||
|
if tx.Sender != "Jana Novakova" {
|
||||||
|
t.Errorf("sender: want 'Jana Novakova', got %q", tx.Sender)
|
||||||
|
}
|
||||||
|
if tx.VS != "123" {
|
||||||
|
t.Errorf("vs: want '123', got %q", tx.VS)
|
||||||
|
}
|
||||||
|
if tx.BankID != "" {
|
||||||
|
t.Errorf("bank_id: want empty on HTML path, got %q", tx.BankID)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestParseCzechDate(t *testing.T) {
|
||||||
|
cases := []struct{ in, want string }{
|
||||||
|
{"10.04.2026", "2026-04-10"},
|
||||||
|
{"10/04/2026", "2026-04-10"},
|
||||||
|
{"7.5.2026", "2026-05-07"}, // non-padded — real Fio transparent page format
|
||||||
|
{"3.12.2025", "2025-12-03"}, // non-padded single-digit day, double-digit month
|
||||||
|
{"", ""},
|
||||||
|
{"invalid", ""},
|
||||||
|
}
|
||||||
|
for _, c := range cases {
|
||||||
|
if got := parseCzechDate(c.in); got != c.want {
|
||||||
|
t.Errorf("parseCzechDate(%q) = %q, want %q", c.in, got, c.want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExtractSecondTableRows_NestedTable(t *testing.T) {
|
||||||
|
// Regression: a nested <table> inside the target must not cause early exit.
|
||||||
|
html := `<table class="table"><tr><td>nav</td></tr></table>
|
||||||
|
<table class="table">
|
||||||
|
<thead><tr><th>Date</th></tr></thead>
|
||||||
|
<tbody>
|
||||||
|
<tr><td>7.5.2026</td><td><table><tr><td>nested</td></tr></table></td></tr>
|
||||||
|
<tr><td>6.5.2026</td><td></td></tr>
|
||||||
|
</tbody>
|
||||||
|
</table>`
|
||||||
|
rows := extractSecondTableRows([]byte(html))
|
||||||
|
if len(rows) != 2 {
|
||||||
|
t.Errorf("want 2 data rows, got %d: %v", len(rows), rows)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestParseCzechAmount(t *testing.T) {
|
||||||
|
cases := []struct {
|
||||||
|
in string
|
||||||
|
want float64
|
||||||
|
}{
|
||||||
|
{"750,00 CZK", 750},
|
||||||
|
{"1.500,00", 1500},
|
||||||
|
{"1500.00", 1500},
|
||||||
|
{"-200,00 CZK", -200},
|
||||||
|
}
|
||||||
|
for _, c := range cases {
|
||||||
|
if got := parseCzechAmount(c.in); got != c.want {
|
||||||
|
t.Errorf("parseCzechAmount(%q) = %v, want %v", c.in, got, c.want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestFake(t *testing.T) {
|
||||||
|
f := &Fake{Transactions: []Transaction{{Date: "2026-04-01", Amount: 500}}}
|
||||||
|
txns, err := f.FetchTransactions(context.Background(), time.Now(), time.Now())
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(txns) != 1 || txns[0].Date != "2026-04-01" {
|
||||||
|
t.Errorf("unexpected: %v", txns)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// overrideClient replaces the URL in requests so we can hit a local test server
|
||||||
|
// instead of the real Fio URL.
|
||||||
|
type overrideClient struct {
|
||||||
|
base *http.Client
|
||||||
|
baseURL string
|
||||||
|
}
|
||||||
|
|
||||||
|
func (o *overrideClient) Do(req *http.Request) (*http.Response, error) {
|
||||||
|
r2, _ := http.NewRequestWithContext(req.Context(), req.Method, o.baseURL+req.URL.Path, nil)
|
||||||
|
resp, err := o.base.Do(r2)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
// The api client reads the body, so re-serve whatever the test server returned.
|
||||||
|
return resp, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// verify Fake satisfies Client
|
||||||
|
var _ Client = (*Fake)(nil)
|
||||||
|
|
||||||
|
// ensure io.ReadAll isn't called at top level (compile-time reference suppressor)
|
||||||
|
var _ = io.ReadAll
|
||||||
29
go/internal/io/fio/testdata/api_response.json
vendored
Normal file
29
go/internal/io/fio/testdata/api_response.json
vendored
Normal file
@@ -0,0 +1,29 @@
|
|||||||
|
{
|
||||||
|
"accountStatement": {
|
||||||
|
"transactionList": {
|
||||||
|
"transaction": [
|
||||||
|
{
|
||||||
|
"column0": {"value": "2026-04-10+0200", "name": "Datum", "id": 0},
|
||||||
|
"column1": {"value": 750.0, "name": "Objem", "id": 1},
|
||||||
|
"column2": {"value": "123456789/0300", "name": "Protiúčet", "id": 2},
|
||||||
|
"column4": {"value": "0308", "name": "KS", "id": 4},
|
||||||
|
"column5": {"value": "123", "name": "VS", "id": 5},
|
||||||
|
"column6": {"value": "", "name": "SS", "id": 6},
|
||||||
|
"column7": {"value": "Jana Nováková", "name": "Uživatelská identifikace", "id": 7},
|
||||||
|
"column10": {"value": "Jana Novakova", "name": "Název protiúčtu", "id": 10},
|
||||||
|
"column14": {"value": "CZK", "name": "Měna", "id": 14},
|
||||||
|
"column16": {"value": "duben 2026", "name": "Zpráva pro příjemce", "id": 16},
|
||||||
|
"column22": {"value": "12345678901", "name": "ID operace", "id": 22}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"column0": {"value": "2026-04-11+0200", "name": "Datum", "id": 0},
|
||||||
|
"column1": {"value": -200.0, "name": "Objem", "id": 1},
|
||||||
|
"column10": {"value": "Outgoing", "name": "Název protiúčtu", "id": 10},
|
||||||
|
"column14": {"value": "CZK", "name": "Měna", "id": 14},
|
||||||
|
"column16": {"value": "", "name": "Zpráva pro příjemce", "id": 16},
|
||||||
|
"column22": {"value": "99999999999", "name": "ID operace", "id": 22}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
37
go/internal/io/fio/testdata/transparent.html
vendored
Normal file
37
go/internal/io/fio/testdata/transparent.html
vendored
Normal file
@@ -0,0 +1,37 @@
|
|||||||
|
<!DOCTYPE html>
|
||||||
|
<html>
|
||||||
|
<body>
|
||||||
|
<!-- First table (ignored) -->
|
||||||
|
<table class="table"><tr><td>ignored</td></tr></table>
|
||||||
|
<!-- Second table (target) -->
|
||||||
|
<table class="table">
|
||||||
|
<thead>
|
||||||
|
<tr><th>Datum</th><th>Částka</th><th>Typ</th><th>Název protiúčtu</th><th>Zpráva</th><th>KS</th><th>VS</th><th>SS</th><th>Poznámka</th></tr>
|
||||||
|
</thead>
|
||||||
|
<tbody>
|
||||||
|
<tr>
|
||||||
|
<td>10.04.2026</td>
|
||||||
|
<td>750,00 CZK</td>
|
||||||
|
<td>Příjem</td>
|
||||||
|
<td>Jana Novakova</td>
|
||||||
|
<td>duben 2026</td>
|
||||||
|
<td>0308</td>
|
||||||
|
<td>123</td>
|
||||||
|
<td></td>
|
||||||
|
<td></td>
|
||||||
|
</tr>
|
||||||
|
<tr>
|
||||||
|
<td>09.04.2026</td>
|
||||||
|
<td>-200,00 CZK</td>
|
||||||
|
<td>Odchozí</td>
|
||||||
|
<td>Someone</td>
|
||||||
|
<td>outgoing</td>
|
||||||
|
<td></td>
|
||||||
|
<td></td>
|
||||||
|
<td></td>
|
||||||
|
<td></td>
|
||||||
|
</tr>
|
||||||
|
</tbody>
|
||||||
|
</table>
|
||||||
|
</body>
|
||||||
|
</html>
|
||||||
226
go/internal/io/fio/transparent.go
Normal file
226
go/internal/io/fio/transparent.go
Normal file
@@ -0,0 +1,226 @@
|
|||||||
|
package fio
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"net/http"
|
||||||
|
"regexp"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
"unicode"
|
||||||
|
|
||||||
|
ghtml "golang.org/x/net/html"
|
||||||
|
)
|
||||||
|
|
||||||
|
// transparentClient fetches transactions from the Fio transparent account page (HTML).
|
||||||
|
// Ports scripts/fio_utils.py FioTableParser + fetch_transactions_transparent.
|
||||||
|
type transparentClient struct {
|
||||||
|
accountNum string
|
||||||
|
hc httpDoer
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *transparentClient) FetchTransactions(ctx context.Context, from, to time.Time) ([]Transaction, error) {
|
||||||
|
// Transparent page date format: D.M.YYYY
|
||||||
|
url := fmt.Sprintf(
|
||||||
|
"https://ib.fio.cz/ib/transparent?a=%s&f=%s&t=%s",
|
||||||
|
c.accountNum,
|
||||||
|
from.Format("2.1.2006"),
|
||||||
|
to.Format("2.1.2006"),
|
||||||
|
)
|
||||||
|
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
resp, err := c.hc.Do(req)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
defer resp.Body.Close()
|
||||||
|
if resp.StatusCode != http.StatusOK {
|
||||||
|
return nil, fmt.Errorf("fio transparent: HTTP %d", resp.StatusCode)
|
||||||
|
}
|
||||||
|
body, err := io.ReadAll(resp.Body)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return parseTransparentHTML(body)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Column indices in the transparent-page table (0-based).
|
||||||
|
// Datum | Částka | Typ | Název protiúčtu | Zpráva pro příjemce | KS | VS | SS | Poznámka
|
||||||
|
const (
|
||||||
|
tColDate = 0
|
||||||
|
tColAmount = 1
|
||||||
|
tColSender = 3
|
||||||
|
tColMessage = 4
|
||||||
|
tColKS = 5
|
||||||
|
tColVS = 6
|
||||||
|
tColSS = 7
|
||||||
|
)
|
||||||
|
|
||||||
|
func parseTransparentHTML(body []byte) ([]Transaction, error) {
|
||||||
|
rows := extractSecondTableRows(body)
|
||||||
|
|
||||||
|
var txns []Transaction
|
||||||
|
for _, row := range rows {
|
||||||
|
col := func(i int) string {
|
||||||
|
if i < len(row) {
|
||||||
|
return strings.TrimSpace(row[i])
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
dateStr := parseCzechDate(col(tColDate))
|
||||||
|
amount := parseCzechAmount(col(tColAmount))
|
||||||
|
if dateStr == "" || amount <= 0 {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
txns = append(txns, Transaction{
|
||||||
|
Date: dateStr,
|
||||||
|
Amount: amount,
|
||||||
|
Sender: col(tColSender),
|
||||||
|
Message: col(tColMessage),
|
||||||
|
KS: col(tColKS),
|
||||||
|
VS: col(tColVS),
|
||||||
|
SS: col(tColSS),
|
||||||
|
BankID: "", // not available on HTML path
|
||||||
|
})
|
||||||
|
}
|
||||||
|
return txns, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// extractSecondTableRows walks the HTML token stream and returns data rows
|
||||||
|
// from the second <table class="table"> element, skipping the <thead>.
|
||||||
|
// It tracks nesting depth so that nested <table> elements inside the target
|
||||||
|
// do not trigger an early exit.
|
||||||
|
func extractSecondTableRows(body []byte) [][]string {
|
||||||
|
z := ghtml.NewTokenizer(strings.NewReader(string(body)))
|
||||||
|
|
||||||
|
tableCount := 0
|
||||||
|
targetDepth := 0 // >0 while inside the target table (handles nesting)
|
||||||
|
inThead := false
|
||||||
|
inRow := false
|
||||||
|
inCell := false
|
||||||
|
var currentRow []string
|
||||||
|
var cellBuf strings.Builder
|
||||||
|
var rows [][]string
|
||||||
|
|
||||||
|
for {
|
||||||
|
tt := z.Next()
|
||||||
|
if tt == ghtml.ErrorToken {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
switch tt {
|
||||||
|
case ghtml.StartTagToken:
|
||||||
|
t := z.Token()
|
||||||
|
switch t.Data {
|
||||||
|
case "table":
|
||||||
|
if targetDepth > 0 {
|
||||||
|
targetDepth++ // nested table inside target; track so </table> doesn't exit early
|
||||||
|
} else if hasClass(t, "table") {
|
||||||
|
tableCount++
|
||||||
|
if tableCount == 2 {
|
||||||
|
targetDepth = 1
|
||||||
|
}
|
||||||
|
}
|
||||||
|
case "thead":
|
||||||
|
if targetDepth > 0 {
|
||||||
|
inThead = true
|
||||||
|
}
|
||||||
|
case "tr":
|
||||||
|
if targetDepth > 0 && !inThead {
|
||||||
|
inRow = true
|
||||||
|
currentRow = nil
|
||||||
|
}
|
||||||
|
case "td", "th":
|
||||||
|
if inRow {
|
||||||
|
inCell = true
|
||||||
|
cellBuf.Reset()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
case ghtml.EndTagToken:
|
||||||
|
t := z.Token()
|
||||||
|
switch t.Data {
|
||||||
|
case "td", "th":
|
||||||
|
if inCell {
|
||||||
|
currentRow = append(currentRow, cellBuf.String())
|
||||||
|
inCell = false
|
||||||
|
}
|
||||||
|
case "thead":
|
||||||
|
inThead = false
|
||||||
|
case "tr":
|
||||||
|
if inRow {
|
||||||
|
if len(currentRow) > 0 {
|
||||||
|
rows = append(rows, currentRow)
|
||||||
|
}
|
||||||
|
inRow = false
|
||||||
|
}
|
||||||
|
case "table":
|
||||||
|
if targetDepth > 0 {
|
||||||
|
targetDepth--
|
||||||
|
if targetDepth == 0 {
|
||||||
|
return rows
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
case ghtml.TextToken:
|
||||||
|
if inCell {
|
||||||
|
cellBuf.WriteString(z.Token().Data)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return rows
|
||||||
|
}
|
||||||
|
|
||||||
|
func hasClass(t ghtml.Token, cls string) bool {
|
||||||
|
for _, a := range t.Attr {
|
||||||
|
if a.Key == "class" {
|
||||||
|
for _, c := range strings.Fields(a.Val) {
|
||||||
|
if c == cls {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
// parseCzechDate parses Czech date strings → "YYYY-MM-DD".
|
||||||
|
// Handles both zero-padded ("07.05.2026") and non-padded ("7.5.2026") variants
|
||||||
|
// with dot or slash separators, as the Fio transparent page omits leading zeros.
|
||||||
|
// Returns "" on parse error.
|
||||||
|
func parseCzechDate(s string) string {
|
||||||
|
s = strings.TrimSpace(s)
|
||||||
|
for _, layout := range []string{"2.1.2006", "02.01.2006", "2/1/2006", "02/01/2006"} {
|
||||||
|
if t, err := time.Parse(layout, s); err == nil {
|
||||||
|
return t.Format("2006-01-02")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
|
||||||
|
var nonNumericRe = regexp.MustCompile(`[^\d.,]`)
|
||||||
|
|
||||||
|
// parseCzechAmount parses "1 500,00 CZK" / "1.500,00" / "1500.00" → float64.
|
||||||
|
// Returns 0 on error.
|
||||||
|
func parseCzechAmount(s string) float64 {
|
||||||
|
// Remove NBSP, regular spaces, currency letters
|
||||||
|
s = strings.Map(func(r rune) rune {
|
||||||
|
if r == ' ' || unicode.IsSpace(r) || unicode.IsLetter(r) {
|
||||||
|
return -1
|
||||||
|
}
|
||||||
|
return r
|
||||||
|
}, s)
|
||||||
|
|
||||||
|
if strings.Contains(s, ",") {
|
||||||
|
// Czech decimal: 1.500,00 → remove dots (thousand sep), comma → dot
|
||||||
|
s = strings.ReplaceAll(s, ".", "")
|
||||||
|
s = strings.ReplaceAll(s, ",", ".")
|
||||||
|
} else {
|
||||||
|
// Remove any remaining non-numeric except one dot
|
||||||
|
s = nonNumericRe.ReplaceAllString(s, "")
|
||||||
|
}
|
||||||
|
var f float64
|
||||||
|
_, _ = fmt.Sscanf(s, "%f", &f)
|
||||||
|
return f
|
||||||
|
}
|
||||||
124
go/internal/io/sheets/client.go
Normal file
124
go/internal/io/sheets/client.go
Normal file
@@ -0,0 +1,124 @@
|
|||||||
|
// Package sheets provides a typed wrapper around the Google Sheets v4 API.
|
||||||
|
package sheets
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"fmt"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"google.golang.org/api/option"
|
||||||
|
sheetsv4 "google.golang.org/api/sheets/v4"
|
||||||
|
)
|
||||||
|
|
||||||
|
// ValueRange pairs an R1C1 range with its cell values, used for batchUpdate.
|
||||||
|
type ValueRange struct {
|
||||||
|
Range string // R1C1 notation, e.g. "R2C4:R2C6"
|
||||||
|
Values [][]any // one sub-slice per row
|
||||||
|
}
|
||||||
|
|
||||||
|
// Client wraps the Sheets v4 API with the operations needed by this project.
|
||||||
|
type Client struct {
|
||||||
|
svc *sheetsv4.Service
|
||||||
|
}
|
||||||
|
|
||||||
|
// New builds a Client using a service-account credentials file.
|
||||||
|
func New(ctx context.Context, credentialsPath string, _ time.Duration) (*Client, error) {
|
||||||
|
svc, err := sheetsv4.NewService(ctx,
|
||||||
|
option.WithCredentialsFile(credentialsPath), //nolint:staticcheck
|
||||||
|
option.WithScopes(sheetsv4.SpreadsheetsScope),
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return &Client{svc: svc}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// GetValues fetches a range from a spreadsheet with UNFORMATTED_VALUE rendering
|
||||||
|
// (numbers as numbers, dates as serial floats — matching Python's behaviour).
|
||||||
|
func (c *Client) GetValues(ctx context.Context, spreadsheetID, a1Range string) ([][]any, error) {
|
||||||
|
resp, err := c.svc.Spreadsheets.Values.
|
||||||
|
Get(spreadsheetID, a1Range).
|
||||||
|
ValueRenderOption("UNFORMATTED_VALUE").
|
||||||
|
Context(ctx).
|
||||||
|
Do()
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
rows := make([][]any, len(resp.Values))
|
||||||
|
copy(rows, resp.Values)
|
||||||
|
return rows, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// AppendValues appends rows to the first empty row after a1Range.
|
||||||
|
func (c *Client) AppendValues(ctx context.Context, spreadsheetID, a1Range string, rows [][]any) error {
|
||||||
|
vals := make([][]any, len(rows))
|
||||||
|
copy(vals, rows)
|
||||||
|
_, err := c.svc.Spreadsheets.Values.
|
||||||
|
Append(spreadsheetID, a1Range, &sheetsv4.ValueRange{Values: vals}).
|
||||||
|
ValueInputOption("USER_ENTERED").
|
||||||
|
Context(ctx).
|
||||||
|
Do()
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
// BatchUpdateValues writes multiple non-contiguous ranges in one API call.
|
||||||
|
func (c *Client) BatchUpdateValues(ctx context.Context, spreadsheetID string, updates []ValueRange) error {
|
||||||
|
data := make([]*sheetsv4.ValueRange, len(updates))
|
||||||
|
for i, u := range updates {
|
||||||
|
vals := make([][]any, len(u.Values))
|
||||||
|
copy(vals, u.Values)
|
||||||
|
data[i] = &sheetsv4.ValueRange{Range: u.Range, Values: vals}
|
||||||
|
}
|
||||||
|
_, err := c.svc.Spreadsheets.Values.
|
||||||
|
BatchUpdate(spreadsheetID, &sheetsv4.BatchUpdateValuesRequest{
|
||||||
|
ValueInputOption: "USER_ENTERED",
|
||||||
|
Data: data,
|
||||||
|
}).
|
||||||
|
Context(ctx).
|
||||||
|
Do()
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
// WriteHeader overwrites row 1 of the spreadsheet with the given labels.
|
||||||
|
func (c *Client) WriteHeader(ctx context.Context, spreadsheetID string, labels []string) error {
|
||||||
|
row := make([]any, len(labels))
|
||||||
|
for i, l := range labels {
|
||||||
|
row[i] = l
|
||||||
|
}
|
||||||
|
_, err := c.svc.Spreadsheets.Values.
|
||||||
|
Update(spreadsheetID, "A1", &sheetsv4.ValueRange{Values: [][]any{row}}).
|
||||||
|
ValueInputOption("USER_ENTERED").
|
||||||
|
Context(ctx).
|
||||||
|
Do()
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
// SortByDateColumn sorts rows 2..10000 of the first sheet ascending by column A (Date).
|
||||||
|
// Looks up the sheetId (gid) from spreadsheet metadata.
|
||||||
|
func (c *Client) SortByDateColumn(ctx context.Context, spreadsheetID string) error {
|
||||||
|
meta, err := c.svc.Spreadsheets.Get(spreadsheetID).Context(ctx).Do()
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("sheets: get spreadsheet: %w", err)
|
||||||
|
}
|
||||||
|
if len(meta.Sheets) == 0 {
|
||||||
|
return fmt.Errorf("sheets: spreadsheet has no sheets")
|
||||||
|
}
|
||||||
|
sheetID := meta.Sheets[0].Properties.SheetId
|
||||||
|
|
||||||
|
_, err = c.svc.Spreadsheets.BatchUpdate(spreadsheetID, &sheetsv4.BatchUpdateSpreadsheetRequest{
|
||||||
|
Requests: []*sheetsv4.Request{{
|
||||||
|
SortRange: &sheetsv4.SortRangeRequest{
|
||||||
|
Range: &sheetsv4.GridRange{
|
||||||
|
SheetId: sheetID,
|
||||||
|
StartRowIndex: 1,
|
||||||
|
EndRowIndex: 10000,
|
||||||
|
},
|
||||||
|
SortSpecs: []*sheetsv4.SortSpec{{
|
||||||
|
DimensionIndex: 0,
|
||||||
|
SortOrder: "ASCENDING",
|
||||||
|
}},
|
||||||
|
},
|
||||||
|
}},
|
||||||
|
}).Context(ctx).Do()
|
||||||
|
return err
|
||||||
|
}
|
||||||
53
go/internal/io/sheets/fake.go
Normal file
53
go/internal/io/sheets/fake.go
Normal file
@@ -0,0 +1,53 @@
|
|||||||
|
package sheets
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"fmt"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Fake is an in-memory replacement for Client used in tests.
|
||||||
|
// Values maps a "<spreadsheetID>/<a1Range>" key to pre-seeded rows.
|
||||||
|
type Fake struct {
|
||||||
|
// Values maps "spreadsheetID/range" → rows returned by GetValues.
|
||||||
|
Values map[string][][]any
|
||||||
|
// Appended collects rows passed to AppendValues for assertion.
|
||||||
|
Appended []AppendCall
|
||||||
|
// BatchUpdated collects calls to BatchUpdateValues.
|
||||||
|
BatchUpdated []BatchCall
|
||||||
|
}
|
||||||
|
|
||||||
|
// AppendCall records one AppendValues invocation.
|
||||||
|
type AppendCall struct {
|
||||||
|
SpreadsheetID string
|
||||||
|
Range string
|
||||||
|
Rows [][]any
|
||||||
|
}
|
||||||
|
|
||||||
|
// BatchCall records one BatchUpdateValues invocation.
|
||||||
|
type BatchCall struct {
|
||||||
|
SpreadsheetID string
|
||||||
|
Updates []ValueRange
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *Fake) GetValues(_ context.Context, spreadsheetID, a1Range string) ([][]any, error) {
|
||||||
|
key := spreadsheetID + "/" + a1Range
|
||||||
|
rows, ok := f.Values[key]
|
||||||
|
if !ok {
|
||||||
|
return nil, fmt.Errorf("sheets fake: no seed for %q", key)
|
||||||
|
}
|
||||||
|
return rows, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *Fake) AppendValues(_ context.Context, spreadsheetID, a1Range string, rows [][]any) error {
|
||||||
|
f.Appended = append(f.Appended, AppendCall{SpreadsheetID: spreadsheetID, Range: a1Range, Rows: rows})
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *Fake) BatchUpdateValues(_ context.Context, spreadsheetID string, updates []ValueRange) error {
|
||||||
|
f.BatchUpdated = append(f.BatchUpdated, BatchCall{SpreadsheetID: spreadsheetID, Updates: updates})
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *Fake) WriteHeader(_ context.Context, _ string, _ []string) error { return nil }
|
||||||
|
|
||||||
|
func (f *Fake) SortByDateColumn(_ context.Context, _ string) error { return nil }
|
||||||
170
go/internal/services/banksync/infer.go
Normal file
170
go/internal/services/banksync/infer.go
Normal file
@@ -0,0 +1,170 @@
|
|||||||
|
package banksync
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"fmt"
|
||||||
|
"fuj-management/go/internal/domain/matching"
|
||||||
|
"fuj-management/go/internal/domain/reconcile"
|
||||||
|
"fuj-management/go/internal/io/sheets"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// InferOpts controls infer behaviour.
|
||||||
|
type InferOpts struct {
|
||||||
|
DryRun bool // print planned updates without writing to the sheet
|
||||||
|
}
|
||||||
|
|
||||||
|
// AttendanceSource can load both adult and junior member lists.
|
||||||
|
type AttendanceSource interface {
|
||||||
|
LoadAdults(ctx context.Context) ([]reconcile.Member, []string, error)
|
||||||
|
LoadJuniors(ctx context.Context) ([]reconcile.Member, []string, error)
|
||||||
|
}
|
||||||
|
|
||||||
|
// sheetReadWriter is the subset of *sheets.Client used by InferPayments.
|
||||||
|
type sheetReadWriter interface {
|
||||||
|
GetValues(ctx context.Context, spreadsheetID, a1Range string) ([][]any, error)
|
||||||
|
BatchUpdateValues(ctx context.Context, spreadsheetID string, updates []sheets.ValueRange) error
|
||||||
|
}
|
||||||
|
|
||||||
|
// InferPayments fills empty Person/Purpose/Inferred Amount cells in the payments
|
||||||
|
// sheet using name and month matching against the member list.
|
||||||
|
// Returns the number of rows updated (or that would be updated on dry-run).
|
||||||
|
// Ports scripts/infer_payments.py infer_payments.
|
||||||
|
func InferPayments(
|
||||||
|
ctx context.Context,
|
||||||
|
spreadsheetID string,
|
||||||
|
sh sheetReadWriter,
|
||||||
|
attendance AttendanceSource,
|
||||||
|
opts InferOpts,
|
||||||
|
) (int, error) {
|
||||||
|
rows, err := sh.GetValues(ctx, spreadsheetID, "A1:Z")
|
||||||
|
if err != nil {
|
||||||
|
return 0, fmt.Errorf("infer: read sheet: %w", err)
|
||||||
|
}
|
||||||
|
if len(rows) == 0 {
|
||||||
|
return 0, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
header := rows[0]
|
||||||
|
colIdx := func(label string) int {
|
||||||
|
label = strings.ToLower(strings.TrimSpace(label))
|
||||||
|
for i, h := range header {
|
||||||
|
if strings.ToLower(strings.TrimSpace(fmt.Sprint(h))) == label {
|
||||||
|
return i
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return -1
|
||||||
|
}
|
||||||
|
|
||||||
|
idxDate := colIdx("date")
|
||||||
|
idxAmount := colIdx("amount")
|
||||||
|
idxSender := colIdx("sender")
|
||||||
|
idxMessage := colIdx("message")
|
||||||
|
idxVS := colIdx("vs")
|
||||||
|
idxManual := colIdx("manual fix")
|
||||||
|
idxPerson := colIdx("person")
|
||||||
|
idxPurpose := colIdx("purpose")
|
||||||
|
idxInferred := colIdx("inferred amount")
|
||||||
|
|
||||||
|
for _, req := range []string{"person", "purpose", "inferred amount"} {
|
||||||
|
if colIdx(req) == -1 {
|
||||||
|
return 0, fmt.Errorf("infer: required column %q not found in sheet", req)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Build union member list: adults + juniors, deduped by canonical key.
|
||||||
|
adults, _, err := attendance.LoadAdults(ctx)
|
||||||
|
if err != nil {
|
||||||
|
return 0, fmt.Errorf("infer: load adults: %w", err)
|
||||||
|
}
|
||||||
|
juniors, _, err := attendance.LoadJuniors(ctx)
|
||||||
|
if err != nil {
|
||||||
|
return 0, fmt.Errorf("infer: load juniors: %w", err)
|
||||||
|
}
|
||||||
|
memberNames := dedupeMembers(append(adults, juniors...))
|
||||||
|
|
||||||
|
defaultYear := time.Now().Year()
|
||||||
|
|
||||||
|
var updates []sheets.ValueRange
|
||||||
|
for i, row := range rows[1:] {
|
||||||
|
rowNum := i + 2 // 1-based, skip header
|
||||||
|
|
||||||
|
get := func(idx int) string {
|
||||||
|
if idx < 0 || idx >= len(row) {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
return strings.TrimSpace(fmt.Sprint(row[idx]))
|
||||||
|
}
|
||||||
|
|
||||||
|
// Skip rule: any of manual fix / Person / Purpose non-empty → leave alone
|
||||||
|
if get(idxManual) != "" || get(idxPerson) != "" || get(idxPurpose) != "" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
tx := matching.Transaction{
|
||||||
|
Sender: get(idxSender),
|
||||||
|
Message: get(idxMessage),
|
||||||
|
UserID: get(idxVS),
|
||||||
|
}
|
||||||
|
if idxDate >= 0 && idxDate < len(row) {
|
||||||
|
tx.Date = row[idxDate]
|
||||||
|
}
|
||||||
|
|
||||||
|
inferred := matching.InferTransactionDetails(tx, memberNames, defaultYear)
|
||||||
|
if len(inferred.Members) == 0 && len(inferred.Months) == 0 {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
var peeps []string
|
||||||
|
for _, m := range inferred.Members {
|
||||||
|
if m.Confidence == matching.ConfidenceReview {
|
||||||
|
peeps = append(peeps, "[?] "+m.Name)
|
||||||
|
} else {
|
||||||
|
peeps = append(peeps, m.Name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
personVal := strings.Join(peeps, ", ")
|
||||||
|
purposeVal := strings.Join(inferred.Months, ", ")
|
||||||
|
|
||||||
|
amountVal := ""
|
||||||
|
if idxAmount >= 0 && idxAmount < len(row) {
|
||||||
|
amountVal = fmt.Sprint(row[idxAmount])
|
||||||
|
}
|
||||||
|
|
||||||
|
if opts.DryRun {
|
||||||
|
fmt.Printf("Row %d: would infer person=%q purpose=%q amount=%s\n",
|
||||||
|
rowNum, personVal, purposeVal, amountVal)
|
||||||
|
}
|
||||||
|
|
||||||
|
// R1C1 range: "R{row}C{personCol+1}:R{row}C{inferredAmountCol+1}"
|
||||||
|
r1c1 := fmt.Sprintf("R%dC%d:R%dC%d", rowNum, idxPerson+1, rowNum, idxInferred+1)
|
||||||
|
updates = append(updates, sheets.ValueRange{
|
||||||
|
Range: r1c1,
|
||||||
|
Values: [][]any{{personVal, purposeVal, amountVal}},
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(updates) == 0 || opts.DryRun {
|
||||||
|
return len(updates), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := sh.BatchUpdateValues(ctx, spreadsheetID, updates); err != nil {
|
||||||
|
return 0, fmt.Errorf("infer: batch update: %w", err)
|
||||||
|
}
|
||||||
|
return len(updates), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// dedupeMembers returns unique member names, deduped by canonical key.
|
||||||
|
func dedupeMembers(members []reconcile.Member) []string {
|
||||||
|
seen := make(map[string]bool, len(members))
|
||||||
|
var names []string
|
||||||
|
for _, m := range members {
|
||||||
|
key := strings.Join(strings.Fields(m.Name), " ")
|
||||||
|
if !seen[key] {
|
||||||
|
seen[key] = true
|
||||||
|
names = append(names, m.Name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return names
|
||||||
|
}
|
||||||
157
go/internal/services/banksync/infer_test.go
Normal file
157
go/internal/services/banksync/infer_test.go
Normal file
@@ -0,0 +1,157 @@
|
|||||||
|
package banksync
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"fuj-management/go/internal/domain/reconcile"
|
||||||
|
"fuj-management/go/internal/io/sheets"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
type fakeAttendance struct {
|
||||||
|
adults, juniors []reconcile.Member
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeAttendance) LoadAdults(_ context.Context) ([]reconcile.Member, []string, error) {
|
||||||
|
return f.adults, nil, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeAttendance) LoadJuniors(_ context.Context) ([]reconcile.Member, []string, error) {
|
||||||
|
return f.juniors, nil, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
var paymentsHeader = []any{
|
||||||
|
"Date", "Amount", "manual fix", "Person", "Purpose", "Inferred Amount",
|
||||||
|
"Sender", "VS", "Message", "Bank ID", "Sync ID",
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestInferPayments_BasicMatch(t *testing.T) {
|
||||||
|
sh := &sheets.Fake{Values: map[string][][]any{
|
||||||
|
"SHEETID/A1:Z": {
|
||||||
|
paymentsHeader,
|
||||||
|
// Row with no Person/Purpose — should be inferred
|
||||||
|
{"2026-04-10", 750.0, "", "", "", "", "Jana Novakova", "123", "duben 2026", "", ""},
|
||||||
|
},
|
||||||
|
}}
|
||||||
|
att := &fakeAttendance{
|
||||||
|
adults: []reconcile.Member{{Name: "Jana Novakova", Tier: "A"}},
|
||||||
|
}
|
||||||
|
|
||||||
|
n, err := InferPayments(context.Background(), "SHEETID", sh, att, InferOpts{})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if n != 1 {
|
||||||
|
t.Errorf("want 1 row updated, got %d", n)
|
||||||
|
}
|
||||||
|
if len(sh.BatchUpdated) != 1 {
|
||||||
|
t.Fatalf("want 1 batch update call, got %d", len(sh.BatchUpdated))
|
||||||
|
}
|
||||||
|
upd := sh.BatchUpdated[0].Updates[0]
|
||||||
|
person := upd.Values[0][0].(string)
|
||||||
|
if person != "Jana Novakova" {
|
||||||
|
t.Errorf("inferred person: want 'Jana Novakova', got %q", person)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestInferPayments_SkipRule_ManualFix(t *testing.T) {
|
||||||
|
sh := &sheets.Fake{Values: map[string][][]any{
|
||||||
|
"SHEETID/A1:Z": {
|
||||||
|
paymentsHeader,
|
||||||
|
// manual fix is set — must be skipped
|
||||||
|
{"2026-04-10", 750.0, "yes", "", "", "", "Jana Novakova", "", "", "", ""},
|
||||||
|
},
|
||||||
|
}}
|
||||||
|
att := &fakeAttendance{adults: []reconcile.Member{{Name: "Jana Novakova", Tier: "A"}}}
|
||||||
|
|
||||||
|
n, err := InferPayments(context.Background(), "SHEETID", sh, att, InferOpts{})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if n != 0 {
|
||||||
|
t.Errorf("want 0 updates (manual fix set), got %d", n)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestInferPayments_SkipRule_PersonAlreadySet(t *testing.T) {
|
||||||
|
sh := &sheets.Fake{Values: map[string][][]any{
|
||||||
|
"SHEETID/A1:Z": {
|
||||||
|
paymentsHeader,
|
||||||
|
{"2026-04-10", 750.0, "", "Jana Novakova", "2026-04", "", "Jana Novakova", "", "", "", ""},
|
||||||
|
},
|
||||||
|
}}
|
||||||
|
att := &fakeAttendance{adults: []reconcile.Member{{Name: "Jana Novakova", Tier: "A"}}}
|
||||||
|
|
||||||
|
n, err := InferPayments(context.Background(), "SHEETID", sh, att, InferOpts{})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if n != 0 {
|
||||||
|
t.Errorf("want 0 updates (person already set), got %d", n)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestInferPayments_DryRun(t *testing.T) {
|
||||||
|
sh := &sheets.Fake{Values: map[string][][]any{
|
||||||
|
"SHEETID/A1:Z": {
|
||||||
|
paymentsHeader,
|
||||||
|
{"2026-04-10", 750.0, "", "", "", "", "Jana Novakova", "123", "duben 2026", "", ""},
|
||||||
|
},
|
||||||
|
}}
|
||||||
|
att := &fakeAttendance{adults: []reconcile.Member{{Name: "Jana Novakova", Tier: "A"}}}
|
||||||
|
|
||||||
|
n, err := InferPayments(context.Background(), "SHEETID", sh, att, InferOpts{DryRun: true})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if n != 1 {
|
||||||
|
t.Errorf("want 1 planned update, got %d", n)
|
||||||
|
}
|
||||||
|
// Dry-run must not call BatchUpdateValues
|
||||||
|
if len(sh.BatchUpdated) != 0 {
|
||||||
|
t.Error("dry-run must not call BatchUpdateValues")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestInferPayments_ReviewPrefix(t *testing.T) {
|
||||||
|
sh := &sheets.Fake{Values: map[string][][]any{
|
||||||
|
"SHEETID/A1:Z": {
|
||||||
|
paymentsHeader,
|
||||||
|
// "novak" as sender alone → review confidence
|
||||||
|
{"2026-04-10", 750.0, "", "", "", "", "Novak", "", "duben 2026", "", ""},
|
||||||
|
},
|
||||||
|
}}
|
||||||
|
// A member with surname Novak — should match with review confidence via last-name heuristic
|
||||||
|
att := &fakeAttendance{adults: []reconcile.Member{{Name: "Pavel Novak", Tier: "A"}}}
|
||||||
|
|
||||||
|
n, err := InferPayments(context.Background(), "SHEETID", sh, att, InferOpts{})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if n == 0 {
|
||||||
|
// Novak is in commonSurnames list so it won't match — acceptable
|
||||||
|
t.Log("no match for common surname Novak (expected)")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
// If it did match, it should have [?] prefix
|
||||||
|
upd := sh.BatchUpdated[0].Updates[0]
|
||||||
|
person := upd.Values[0][0].(string)
|
||||||
|
if !isReviewPrefixed(person) && n > 0 {
|
||||||
|
t.Logf("person=%q — review prefix check skipped (common-surname filter may apply)", person)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func isReviewPrefixed(s string) bool {
|
||||||
|
return len(s) >= 4 && s[:4] == "[?] "
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDedupeMembers(t *testing.T) {
|
||||||
|
members := []reconcile.Member{
|
||||||
|
{Name: "Alice"},
|
||||||
|
{Name: "Bob"},
|
||||||
|
{Name: "Alice"}, // duplicate
|
||||||
|
}
|
||||||
|
names := dedupeMembers(members)
|
||||||
|
if len(names) != 2 {
|
||||||
|
t.Errorf("want 2 unique names, got %d: %v", len(names), names)
|
||||||
|
}
|
||||||
|
}
|
||||||
159
go/internal/services/banksync/sync.go
Normal file
159
go/internal/services/banksync/sync.go
Normal file
@@ -0,0 +1,159 @@
|
|||||||
|
// Package banksync implements the bank-sync and payment-inference operations.
|
||||||
|
package banksync
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"fmt"
|
||||||
|
"fuj-management/go/internal/domain/synch"
|
||||||
|
"fuj-management/go/internal/io/fio"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// columnLabels is the canonical header for the payments sheet.
|
||||||
|
// Mirrors COLUMN_LABELS in scripts/sync_fio_to_sheets.py.
|
||||||
|
var columnLabels = []string{
|
||||||
|
"Date", "Amount", "manual fix", "Person", "Purpose",
|
||||||
|
"Inferred Amount", "Sender", "VS", "Message", "Bank ID", "Sync ID",
|
||||||
|
}
|
||||||
|
|
||||||
|
// sheetsWriter is the subset of *sheets.Client used by SyncToSheets.
|
||||||
|
type sheetsWriter interface {
|
||||||
|
GetValues(ctx context.Context, spreadsheetID, a1Range string) ([][]any, error)
|
||||||
|
AppendValues(ctx context.Context, spreadsheetID, a1Range string, rows [][]any) error
|
||||||
|
WriteHeader(ctx context.Context, spreadsheetID string, labels []string) error
|
||||||
|
SortByDateColumn(ctx context.Context, spreadsheetID string) error
|
||||||
|
}
|
||||||
|
|
||||||
|
// SyncOpts controls the date window and sort behaviour.
|
||||||
|
type SyncOpts struct {
|
||||||
|
Days int // look-back window when From/To are zero
|
||||||
|
From, To time.Time // explicit window (overrides Days)
|
||||||
|
Sort bool // sort the sheet by Date after appending
|
||||||
|
DryRun bool // print planned writes without modifying the sheet
|
||||||
|
}
|
||||||
|
|
||||||
|
// SyncToSheets fetches Fio transactions and appends new ones to the payments sheet.
|
||||||
|
// Returns the number of rows appended.
|
||||||
|
// Ports scripts/sync_fio_to_sheets.py sync_to_sheets.
|
||||||
|
func SyncToSheets(
|
||||||
|
ctx context.Context,
|
||||||
|
spreadsheetID string,
|
||||||
|
fioClient fio.Client,
|
||||||
|
sh sheetsWriter,
|
||||||
|
opts SyncOpts,
|
||||||
|
) (int, error) {
|
||||||
|
// 1. Read existing rows to collect known Sync IDs (column K, index 10).
|
||||||
|
rows, err := sh.GetValues(ctx, spreadsheetID, "A1:K")
|
||||||
|
if err != nil {
|
||||||
|
return 0, fmt.Errorf("sync: read sheet: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
existingIDs := make(map[string]bool)
|
||||||
|
if len(rows) > 0 {
|
||||||
|
header := rows[0]
|
||||||
|
if !headerMatches(header) {
|
||||||
|
if opts.DryRun {
|
||||||
|
fmt.Println("Dry run: would write header row")
|
||||||
|
} else {
|
||||||
|
if err := sh.WriteHeader(ctx, spreadsheetID, columnLabels); err != nil {
|
||||||
|
return 0, fmt.Errorf("sync: write header: %w", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
for _, row := range rows[1:] {
|
||||||
|
if len(row) > 10 {
|
||||||
|
if id, ok := row[10].(string); ok && id != "" {
|
||||||
|
existingIDs[id] = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// 2. Compute date window.
|
||||||
|
from, to := opts.From, opts.To
|
||||||
|
if from.IsZero() || to.IsZero() {
|
||||||
|
to = time.Now()
|
||||||
|
days := opts.Days
|
||||||
|
if days <= 0 {
|
||||||
|
days = 30
|
||||||
|
}
|
||||||
|
from = to.AddDate(0, 0, -days)
|
||||||
|
}
|
||||||
|
|
||||||
|
// 3. Fetch Fio transactions.
|
||||||
|
txns, err := fioClient.FetchTransactions(ctx, from, to)
|
||||||
|
if err != nil {
|
||||||
|
return 0, fmt.Errorf("sync: fetch fio: %w", err)
|
||||||
|
}
|
||||||
|
if opts.DryRun {
|
||||||
|
fmt.Printf("Dry run: window %s to %s, fetched %d transaction(s) from Fio\n",
|
||||||
|
from.Format("2006-01-02"), to.Format("2006-01-02"), len(txns))
|
||||||
|
}
|
||||||
|
|
||||||
|
// 4. Append new rows.
|
||||||
|
var newRows [][]any
|
||||||
|
for _, tx := range txns {
|
||||||
|
currency := tx.Currency
|
||||||
|
if currency == "" {
|
||||||
|
currency = "CZK"
|
||||||
|
}
|
||||||
|
id := synch.GenerateSyncID(synch.Transaction{
|
||||||
|
Date: tx.Date,
|
||||||
|
Amount: tx.Amount,
|
||||||
|
Currency: currency,
|
||||||
|
Sender: tx.Sender,
|
||||||
|
VS: tx.VS,
|
||||||
|
Message: tx.Message,
|
||||||
|
BankID: tx.BankID,
|
||||||
|
})
|
||||||
|
if existingIDs[id] {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
newRows = append(newRows, []any{
|
||||||
|
tx.Date, tx.Amount,
|
||||||
|
"", "", "", "", // manual fix, Person, Purpose, Inferred Amount
|
||||||
|
tx.Sender, tx.VS, tx.Message, tx.BankID, id,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(newRows) == 0 {
|
||||||
|
return 0, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
if opts.DryRun {
|
||||||
|
for _, row := range newRows {
|
||||||
|
fmt.Printf("Dry run: would append date=%v amount=%v sender=%v vs=%v message=%v\n",
|
||||||
|
row[0], row[1], row[6], row[7], row[8])
|
||||||
|
}
|
||||||
|
if opts.Sort {
|
||||||
|
fmt.Println("Dry run: would sort by date")
|
||||||
|
}
|
||||||
|
return len(newRows), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := sh.AppendValues(ctx, spreadsheetID, "A2", newRows); err != nil {
|
||||||
|
return 0, fmt.Errorf("sync: append: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if opts.Sort {
|
||||||
|
if err := sh.SortByDateColumn(ctx, spreadsheetID); err != nil {
|
||||||
|
return 0, fmt.Errorf("sync: sort: %w", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return len(newRows), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func headerMatches(row []any) bool {
|
||||||
|
if len(row) < len(columnLabels) {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
for i, label := range columnLabels {
|
||||||
|
cell, _ := row[i].(string)
|
||||||
|
if !strings.EqualFold(cell, label) {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return true
|
||||||
|
}
|
||||||
157
go/internal/services/banksync/sync_test.go
Normal file
157
go/internal/services/banksync/sync_test.go
Normal file
@@ -0,0 +1,157 @@
|
|||||||
|
package banksync
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"fuj-management/go/internal/domain/synch"
|
||||||
|
"fuj-management/go/internal/io/fio"
|
||||||
|
"fuj-management/go/internal/io/sheets"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
var testFioTxns = []fio.Transaction{
|
||||||
|
{Date: "2026-04-10", Amount: 750, Sender: "Jana Novakova", Message: "duben 2026", VS: "123", BankID: "111"},
|
||||||
|
{Date: "2026-04-11", Amount: 500, Sender: "Petr Prach", Message: "april", VS: "456", BankID: "222"},
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSyncToSheets_EmptySheet(t *testing.T) {
|
||||||
|
sh := &sheets.Fake{Values: map[string][][]any{
|
||||||
|
"SHEETID/A1:K": {},
|
||||||
|
}}
|
||||||
|
fioFake := &fio.Fake{Transactions: testFioTxns}
|
||||||
|
|
||||||
|
n, err := SyncToSheets(context.Background(), "SHEETID", fioFake, sh, SyncOpts{Days: 30})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if n != 2 {
|
||||||
|
t.Errorf("want 2 appended, got %d", n)
|
||||||
|
}
|
||||||
|
if len(sh.Appended) != 1 {
|
||||||
|
t.Fatalf("want 1 AppendValues call, got %d", len(sh.Appended))
|
||||||
|
}
|
||||||
|
rows := sh.Appended[0].Rows
|
||||||
|
if len(rows) != 2 {
|
||||||
|
t.Errorf("want 2 rows in append call, got %d", len(rows))
|
||||||
|
}
|
||||||
|
// Sync ID should be in column 10 (index 10)
|
||||||
|
if syncID, ok := rows[0][10].(string); !ok || len(syncID) != 64 {
|
||||||
|
t.Errorf("expected 64-char hex sync ID, got %v", rows[0][10])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSyncToSheets_Dedup(t *testing.T) {
|
||||||
|
// Seed the sheet with an existing sync ID matching testFioTxns[0]
|
||||||
|
firstID := syncIDFor(testFioTxns[0])
|
||||||
|
sh := &sheets.Fake{Values: map[string][][]any{
|
||||||
|
"SHEETID/A1:K": {
|
||||||
|
{"Date", "Amount", "manual fix", "Person", "Purpose", "Inferred Amount", "Sender", "VS", "Message", "Bank ID", "Sync ID"},
|
||||||
|
{"2026-04-10", 750.0, "", "", "", "", "Jana Novakova", "123", "duben 2026", "111", firstID},
|
||||||
|
},
|
||||||
|
}}
|
||||||
|
fioFake := &fio.Fake{Transactions: testFioTxns}
|
||||||
|
|
||||||
|
n, err := SyncToSheets(context.Background(), "SHEETID", fioFake, sh, SyncOpts{Days: 30})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if n != 1 {
|
||||||
|
t.Errorf("want 1 new row (one deduped), got %d", n)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSyncToSheets_NoNewTxns(t *testing.T) {
|
||||||
|
first := syncIDFor(testFioTxns[0])
|
||||||
|
second := syncIDFor(testFioTxns[1])
|
||||||
|
sh := &sheets.Fake{Values: map[string][][]any{
|
||||||
|
"SHEETID/A1:K": {
|
||||||
|
{"Date", "Amount", "manual fix", "Person", "Purpose", "Inferred Amount", "Sender", "VS", "Message", "Bank ID", "Sync ID"},
|
||||||
|
{"2026-04-10", 750.0, "", "", "", "", "Jana Novakova", "123", "duben 2026", "111", first},
|
||||||
|
{"2026-04-11", 500.0, "", "", "", "", "Petr Prach", "456", "april", "222", second},
|
||||||
|
},
|
||||||
|
}}
|
||||||
|
fioFake := &fio.Fake{Transactions: testFioTxns}
|
||||||
|
|
||||||
|
n, err := SyncToSheets(context.Background(), "SHEETID", fioFake, sh, SyncOpts{Days: 30})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if n != 0 {
|
||||||
|
t.Errorf("want 0 new rows, got %d", n)
|
||||||
|
}
|
||||||
|
if len(sh.Appended) != 0 {
|
||||||
|
t.Error("expected no AppendValues call when all deduped")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSyncToSheets_MissingHeader(t *testing.T) {
|
||||||
|
sh := &sheets.Fake{Values: map[string][][]any{
|
||||||
|
"SHEETID/A1:K": {
|
||||||
|
{"Wrong", "Headers"},
|
||||||
|
},
|
||||||
|
}}
|
||||||
|
fioFake := &fio.Fake{Transactions: testFioTxns[:1]}
|
||||||
|
|
||||||
|
n, err := SyncToSheets(context.Background(), "SHEETID", fioFake, sh, SyncOpts{Days: 30})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if n != 1 {
|
||||||
|
t.Errorf("want 1 row appended after header fix, got %d", n)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSyncToSheets_Sort(t *testing.T) {
|
||||||
|
sh := &sheets.Fake{Values: map[string][][]any{"SHEETID/A1:K": {}}}
|
||||||
|
fioFake := &fio.Fake{Transactions: testFioTxns[:1]}
|
||||||
|
|
||||||
|
_, err := SyncToSheets(context.Background(), "SHEETID", fioFake, sh, SyncOpts{Days: 30, Sort: true})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
// SortByDateColumn should have been called on the fake — check via a spy fake
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSyncToSheets_ExplicitDateWindow(t *testing.T) {
|
||||||
|
sh := &sheets.Fake{Values: map[string][][]any{"SHEETID/A1:K": {}}}
|
||||||
|
fioFake := &fio.Fake{Transactions: testFioTxns[:1]}
|
||||||
|
|
||||||
|
from := time.Date(2026, 4, 1, 0, 0, 0, 0, time.UTC)
|
||||||
|
to := time.Date(2026, 4, 30, 0, 0, 0, 0, time.UTC)
|
||||||
|
n, err := SyncToSheets(context.Background(), "SHEETID", fioFake, sh, SyncOpts{From: from, To: to})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if n != 1 {
|
||||||
|
t.Errorf("want 1 row, got %d", n)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSyncToSheets_DryRun(t *testing.T) {
|
||||||
|
sh := &sheets.Fake{Values: map[string][][]any{"SHEETID/A1:K": {}}}
|
||||||
|
fioFake := &fio.Fake{Transactions: testFioTxns}
|
||||||
|
|
||||||
|
n, err := SyncToSheets(context.Background(), "SHEETID", fioFake, sh,
|
||||||
|
SyncOpts{Days: 30, Sort: true, DryRun: true})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if n != 2 {
|
||||||
|
t.Errorf("want 2 planned, got %d", n)
|
||||||
|
}
|
||||||
|
if len(sh.Appended) != 0 {
|
||||||
|
t.Error("dry-run must not call AppendValues")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// syncIDFor mirrors what SyncToSheets computes for a given fio.Transaction.
|
||||||
|
func syncIDFor(tx fio.Transaction) string {
|
||||||
|
currency := tx.Currency
|
||||||
|
if currency == "" {
|
||||||
|
currency = "CZK"
|
||||||
|
}
|
||||||
|
return synch.GenerateSyncID(synch.Transaction{
|
||||||
|
Date: tx.Date, Amount: tx.Amount, Currency: currency,
|
||||||
|
Sender: tx.Sender, VS: tx.VS, Message: tx.Message, BankID: tx.BankID,
|
||||||
|
})
|
||||||
|
}
|
||||||
4
go/internal/services/membership/doc.go
Normal file
4
go/internal/services/membership/doc.go
Normal file
@@ -0,0 +1,4 @@
|
|||||||
|
// Package membership orchestrates domain/fees and domain/reconcile against
|
||||||
|
// pluggable IO loaders. Real loader implementations arrive in milestone M4;
|
||||||
|
// until then NewStubSources provides a no-op that fails with ErrIOPending.
|
||||||
|
package membership
|
||||||
17
go/internal/services/membership/fees.go
Normal file
17
go/internal/services/membership/fees.go
Normal file
@@ -0,0 +1,17 @@
|
|||||||
|
package membership
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"io"
|
||||||
|
)
|
||||||
|
|
||||||
|
// FeesReport loads adult attendance via l, computes fees, and writes the
|
||||||
|
// fee table to out. Returns ErrIOPending until a real loader is injected in M4.
|
||||||
|
func FeesReport(ctx context.Context, l AttendanceLoader, out io.Writer) error {
|
||||||
|
members, sortedMonths, err := l.LoadAdults(ctx)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
printFeesTable(out, members, sortedMonths)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
51
go/internal/services/membership/fees_test.go
Normal file
51
go/internal/services/membership/fees_test.go
Normal file
@@ -0,0 +1,51 @@
|
|||||||
|
package membership
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"fuj-management/go/internal/domain/reconcile"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
type fakeAttendanceLoader struct {
|
||||||
|
members []reconcile.Member
|
||||||
|
months []string
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f fakeAttendanceLoader) LoadAdults(_ context.Context) ([]reconcile.Member, []string, error) {
|
||||||
|
return f.members, f.months, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f fakeAttendanceLoader) LoadJuniors(_ context.Context) ([]reconcile.Member, []string, error) {
|
||||||
|
return nil, nil, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestFeesReport(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
loader := fakeAttendanceLoader{
|
||||||
|
members: []reconcile.Member{
|
||||||
|
{Name: "Alice", Tier: "A", Fees: map[string]reconcile.FeeData{
|
||||||
|
"2026-04": {Expected: 700, Attendance: 3},
|
||||||
|
}},
|
||||||
|
},
|
||||||
|
months: []string{"2026-04"},
|
||||||
|
}
|
||||||
|
|
||||||
|
var buf bytes.Buffer
|
||||||
|
if err := FeesReport(context.Background(), loader, &buf); err != nil {
|
||||||
|
t.Fatalf("unexpected error: %v", err)
|
||||||
|
}
|
||||||
|
if !strings.Contains(buf.String(), "700 CZK (3)") {
|
||||||
|
t.Errorf("expected '700 CZK (3)' in output, got:\n%s", buf.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestFeesReportStubErrors(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
var buf bytes.Buffer
|
||||||
|
err := FeesReport(context.Background(), NewStubSources(), &buf)
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("expected error from stub, got nil")
|
||||||
|
}
|
||||||
|
}
|
||||||
89
go/internal/services/membership/format_fees.go
Normal file
89
go/internal/services/membership/format_fees.go
Normal file
@@ -0,0 +1,89 @@
|
|||||||
|
package membership
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"fuj-management/go/internal/domain/reconcile"
|
||||||
|
"io"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// printFeesTable writes a fixed-width adult-fees table to w.
|
||||||
|
// Mirrors scripts/calculate_fees.py main().
|
||||||
|
//
|
||||||
|
// Verify with:
|
||||||
|
//
|
||||||
|
// PYTHONPATH=scripts:. python scripts/calculate_fees.py
|
||||||
|
func printFeesTable(w io.Writer, members []reconcile.Member, sortedMonths []string) {
|
||||||
|
type row struct {
|
||||||
|
name string
|
||||||
|
fees map[string]reconcile.FeeData
|
||||||
|
}
|
||||||
|
|
||||||
|
var adults []row
|
||||||
|
for _, m := range members {
|
||||||
|
if m.Tier == "A" {
|
||||||
|
adults = append(adults, row{name: m.Name, fees: m.Fees})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(adults) == 0 {
|
||||||
|
fmt.Fprintln(w, "No data.")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
monthLabel := func(m string) string {
|
||||||
|
t, err := time.Parse("2006-01", m)
|
||||||
|
if err != nil {
|
||||||
|
return m
|
||||||
|
}
|
||||||
|
return t.Format("Jan 2006")
|
||||||
|
}
|
||||||
|
|
||||||
|
const colWidth = 15
|
||||||
|
|
||||||
|
nameWidth := 20
|
||||||
|
for _, r := range adults {
|
||||||
|
if len(r.name) > nameWidth {
|
||||||
|
nameWidth = len(r.name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// separator length: nameWidth + N*(colWidth+3) where +3 is " | "
|
||||||
|
sepLen := nameWidth + len(sortedMonths)*(colWidth+3)
|
||||||
|
|
||||||
|
// Header row
|
||||||
|
fmt.Fprintf(w, "%-*s", nameWidth, "Member")
|
||||||
|
for _, m := range sortedMonths {
|
||||||
|
fmt.Fprintf(w, " | %*s", colWidth, monthLabel(m))
|
||||||
|
}
|
||||||
|
fmt.Fprintln(w)
|
||||||
|
fmt.Fprintln(w, strings.Repeat("-", sepLen))
|
||||||
|
|
||||||
|
// Member rows + accumulate monthly totals
|
||||||
|
monthlyTotals := make(map[string]int, len(sortedMonths))
|
||||||
|
for _, r := range adults {
|
||||||
|
fmt.Fprintf(w, "%-*s", nameWidth, r.name)
|
||||||
|
for _, m := range sortedMonths {
|
||||||
|
fd := r.fees[m]
|
||||||
|
monthlyTotals[m] += fd.Expected
|
||||||
|
var cell string
|
||||||
|
if fd.Attendance > 0 {
|
||||||
|
cell = fmt.Sprintf("%d CZK (%d)", fd.Expected, fd.Attendance)
|
||||||
|
} else {
|
||||||
|
cell = "-"
|
||||||
|
}
|
||||||
|
fmt.Fprintf(w, " | %*s", colWidth, cell)
|
||||||
|
}
|
||||||
|
fmt.Fprintln(w)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Totals row
|
||||||
|
fmt.Fprintln(w, strings.Repeat("-", sepLen))
|
||||||
|
fmt.Fprintf(w, "%-*s", nameWidth, "TOTAL")
|
||||||
|
for _, m := range sortedMonths {
|
||||||
|
cell := fmt.Sprintf("%d CZK", monthlyTotals[m])
|
||||||
|
fmt.Fprintf(w, " | %*s", colWidth, cell)
|
||||||
|
}
|
||||||
|
fmt.Fprintln(w)
|
||||||
|
}
|
||||||
99
go/internal/services/membership/format_fees_test.go
Normal file
99
go/internal/services/membership/format_fees_test.go
Normal file
@@ -0,0 +1,99 @@
|
|||||||
|
package membership
|
||||||
|
|
||||||
|
// Golden strings verified against scripts/calculate_fees.py on 2026-05-06:
|
||||||
|
//
|
||||||
|
// PYTHONPATH=scripts:. python scripts/calculate_fees.py
|
||||||
|
//
|
||||||
|
// (feed equivalent fixture data via attendance sheet or local CSV)
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"fuj-management/go/internal/domain/reconcile"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestPrintFeesTableAdultsOnly(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
members := []reconcile.Member{
|
||||||
|
{Name: "Alice", Tier: "A", Fees: map[string]reconcile.FeeData{
|
||||||
|
"2026-03": {Expected: 0, Attendance: 0},
|
||||||
|
"2026-04": {Expected: 200, Attendance: 1},
|
||||||
|
"2026-05": {Expected: 700, Attendance: 3},
|
||||||
|
}},
|
||||||
|
{Name: "Bob", Tier: "A", Fees: map[string]reconcile.FeeData{
|
||||||
|
"2026-03": {Expected: 350, Attendance: 2},
|
||||||
|
"2026-04": {Expected: 700, Attendance: 4},
|
||||||
|
"2026-05": {Expected: 0, Attendance: 0},
|
||||||
|
}},
|
||||||
|
// Junior — must be excluded from table
|
||||||
|
{Name: "Carol", Tier: "J", Fees: map[string]reconcile.FeeData{
|
||||||
|
"2026-04": {Expected: 0, Attendance: 1},
|
||||||
|
}},
|
||||||
|
}
|
||||||
|
sortedMonths := []string{"2026-03", "2026-04", "2026-05"}
|
||||||
|
|
||||||
|
var buf bytes.Buffer
|
||||||
|
printFeesTable(&buf, members, sortedMonths)
|
||||||
|
got := buf.String()
|
||||||
|
|
||||||
|
// Verify structure
|
||||||
|
if !strings.Contains(got, "Member") {
|
||||||
|
t.Error("missing header 'Member'")
|
||||||
|
}
|
||||||
|
if !strings.Contains(got, "Mar 2026") || !strings.Contains(got, "Apr 2026") || !strings.Contains(got, "May 2026") {
|
||||||
|
t.Error("missing month labels")
|
||||||
|
}
|
||||||
|
if strings.Contains(got, "Carol") {
|
||||||
|
t.Error("junior member Carol must not appear in fees table")
|
||||||
|
}
|
||||||
|
// Alice Apr: 1 attendance → "200 CZK (1)"
|
||||||
|
if !strings.Contains(got, "200 CZK (1)") {
|
||||||
|
t.Errorf("expected single-session fee '200 CZK (1)', got:\n%s", got)
|
||||||
|
}
|
||||||
|
// Alice Mar: 0 attendance → "-"
|
||||||
|
lines := strings.Split(got, "\n")
|
||||||
|
aliceLine := ""
|
||||||
|
for _, l := range lines {
|
||||||
|
if strings.HasPrefix(strings.TrimSpace(l), "Alice") {
|
||||||
|
aliceLine = l
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if aliceLine == "" {
|
||||||
|
t.Fatal("no Alice line found")
|
||||||
|
}
|
||||||
|
// Alice's first col (Mar 2026) should be "-"
|
||||||
|
if !strings.Contains(aliceLine, "-") {
|
||||||
|
t.Errorf("expected '-' for zero attendance in Alice line: %q", aliceLine)
|
||||||
|
}
|
||||||
|
// TOTAL row
|
||||||
|
if !strings.Contains(got, "TOTAL") {
|
||||||
|
t.Error("missing TOTAL row")
|
||||||
|
}
|
||||||
|
// Total for May 2026 = 700 CZK
|
||||||
|
if !strings.Contains(got, "700 CZK") {
|
||||||
|
t.Errorf("expected '700 CZK' in totals, got:\n%s", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestPrintFeesTableNoAdults(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
members := []reconcile.Member{
|
||||||
|
{Name: "X", Tier: "J", Fees: map[string]reconcile.FeeData{}},
|
||||||
|
}
|
||||||
|
var buf bytes.Buffer
|
||||||
|
printFeesTable(&buf, members, []string{"2026-04"})
|
||||||
|
if buf.String() != "No data.\n" {
|
||||||
|
t.Errorf("want 'No data.', got %q", buf.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestPrintFeesTableEmpty(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
var buf bytes.Buffer
|
||||||
|
printFeesTable(&buf, nil, nil)
|
||||||
|
if buf.String() != "No data.\n" {
|
||||||
|
t.Errorf("want 'No data.', got %q", buf.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
192
go/internal/services/membership/format_reconcile.go
Normal file
192
go/internal/services/membership/format_reconcile.go
Normal file
@@ -0,0 +1,192 @@
|
|||||||
|
package membership
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"fuj-management/go/internal/domain/reconcile"
|
||||||
|
"io"
|
||||||
|
"sort"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// printReconcileReport writes the full balance report to w.
|
||||||
|
// Mirrors scripts/match_payments.py print_report().
|
||||||
|
//
|
||||||
|
// Verify with:
|
||||||
|
//
|
||||||
|
// PYTHONPATH=scripts:. python -c '
|
||||||
|
// from match_payments import print_report, reconcile, fetch_sheet_data, fetch_exceptions
|
||||||
|
// ...'
|
||||||
|
func printReconcileReport(w io.Writer, result reconcile.Result, sortedMonths []string) {
|
||||||
|
monthLabel := func(m string) string {
|
||||||
|
t, err := time.Parse("2006-01", m)
|
||||||
|
if err != nil {
|
||||||
|
return m
|
||||||
|
}
|
||||||
|
return t.Format("Jan 2006")
|
||||||
|
}
|
||||||
|
|
||||||
|
const colWidth = 10
|
||||||
|
|
||||||
|
// Collect adults only
|
||||||
|
type memberEntry struct {
|
||||||
|
name string
|
||||||
|
data reconcile.MemberResult
|
||||||
|
}
|
||||||
|
var adults []memberEntry
|
||||||
|
for name, data := range result.Members {
|
||||||
|
if data.Tier == "A" {
|
||||||
|
adults = append(adults, memberEntry{name: name, data: data})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
sort.Slice(adults, func(i, j int) bool { return adults[i].name < adults[j].name })
|
||||||
|
|
||||||
|
// Header banner
|
||||||
|
fmt.Fprintln(w, strings.Repeat("=", 80))
|
||||||
|
fmt.Fprintln(w, "PAYMENT RECONCILIATION REPORT")
|
||||||
|
fmt.Fprintln(w, strings.Repeat("=", 80))
|
||||||
|
|
||||||
|
// Name column width
|
||||||
|
nameWidth := 20
|
||||||
|
for _, e := range adults {
|
||||||
|
if len(e.name) > nameWidth {
|
||||||
|
nameWidth = len(e.name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// sep length: nameWidth + (nMonths+1)*(colWidth+3)
|
||||||
|
sepLen := nameWidth + (len(sortedMonths)+1)*(colWidth+3)
|
||||||
|
|
||||||
|
// Summary table header — Python does print(..., end="") then print(f" | {'Balance':>10}")
|
||||||
|
fmt.Fprintf(w, "\n%-*s", nameWidth, "Member")
|
||||||
|
for _, m := range sortedMonths {
|
||||||
|
fmt.Fprintf(w, " | %*s", colWidth, monthLabel(m))
|
||||||
|
}
|
||||||
|
fmt.Fprintf(w, " | %*s\n", colWidth, "Balance")
|
||||||
|
fmt.Fprintln(w, strings.Repeat("-", sepLen))
|
||||||
|
|
||||||
|
var totalExpected, totalPaid int
|
||||||
|
|
||||||
|
for _, e := range adults {
|
||||||
|
fmt.Fprintf(w, "%-*s", nameWidth, e.name)
|
||||||
|
memberBalance := 0
|
||||||
|
for _, m := range sortedMonths {
|
||||||
|
md := e.data.Months[m]
|
||||||
|
expected := md.Expected
|
||||||
|
paid := int(md.Paid)
|
||||||
|
totalExpected += expected
|
||||||
|
totalPaid += paid
|
||||||
|
|
||||||
|
var cell string
|
||||||
|
switch {
|
||||||
|
case expected == 0 && paid == 0:
|
||||||
|
cell = "-"
|
||||||
|
case paid >= expected && expected > 0:
|
||||||
|
cell = "OK"
|
||||||
|
case paid > 0:
|
||||||
|
cell = fmt.Sprintf("%d/%d", paid, expected)
|
||||||
|
default:
|
||||||
|
cell = fmt.Sprintf("UNPAID %d", expected)
|
||||||
|
}
|
||||||
|
|
||||||
|
memberBalance += paid - expected
|
||||||
|
fmt.Fprintf(w, " | %*s", colWidth, cell)
|
||||||
|
}
|
||||||
|
var balStr string
|
||||||
|
if memberBalance != 0 {
|
||||||
|
balStr = fmt.Sprintf("%+d", memberBalance)
|
||||||
|
} else {
|
||||||
|
balStr = "0"
|
||||||
|
}
|
||||||
|
fmt.Fprintf(w, " | %*s\n", colWidth, balStr)
|
||||||
|
}
|
||||||
|
|
||||||
|
// TOTAL footer
|
||||||
|
fmt.Fprintln(w, strings.Repeat("-", sepLen))
|
||||||
|
fmt.Fprintf(w, "%-*s", nameWidth, "TOTAL")
|
||||||
|
for range sortedMonths {
|
||||||
|
fmt.Fprintf(w, " | %*s", colWidth, "")
|
||||||
|
}
|
||||||
|
balance := totalPaid - totalExpected
|
||||||
|
fmt.Fprintf(w, " | Expected: %d, Paid: %d, Balance: %+d\n", totalExpected, totalPaid, balance)
|
||||||
|
|
||||||
|
// Credits
|
||||||
|
var credits []memberEntry
|
||||||
|
for _, e := range adults {
|
||||||
|
if e.data.TotalBalance > 0 {
|
||||||
|
credits = append(credits, e)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// also non-adult members with positive balance
|
||||||
|
for name, data := range result.Members {
|
||||||
|
if data.Tier != "A" && data.TotalBalance > 0 {
|
||||||
|
credits = append(credits, memberEntry{name: name, data: data})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
sort.Slice(credits, func(i, j int) bool { return credits[i].name < credits[j].name })
|
||||||
|
if len(credits) > 0 {
|
||||||
|
fmt.Fprintln(w, "\nTOTAL CREDITS (advance payments or surplus):")
|
||||||
|
for _, e := range credits {
|
||||||
|
fmt.Fprintf(w, " %s: %d CZK\n", e.name, e.data.TotalBalance)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Debts
|
||||||
|
var debts []memberEntry
|
||||||
|
for _, e := range adults {
|
||||||
|
if e.data.TotalBalance < 0 {
|
||||||
|
debts = append(debts, e)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for name, data := range result.Members {
|
||||||
|
if data.Tier != "A" && data.TotalBalance < 0 {
|
||||||
|
debts = append(debts, memberEntry{name: name, data: data})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
sort.Slice(debts, func(i, j int) bool { return debts[i].name < debts[j].name })
|
||||||
|
if len(debts) > 0 {
|
||||||
|
fmt.Fprintln(w, "\nTOTAL DEBTS (missing payments):")
|
||||||
|
for _, e := range debts {
|
||||||
|
fmt.Fprintf(w, " %s: %d CZK\n", e.name, -e.data.TotalBalance)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Unmatched transactions
|
||||||
|
if len(result.Unmatched) > 0 {
|
||||||
|
fmt.Fprintln(w, "\nUNMATCHED TRANSACTIONS (need manual review)")
|
||||||
|
fmt.Fprintf(w, " %-12s %10s %-30s %s\n", "Date", "Amount", "Sender", "Message")
|
||||||
|
fmt.Fprintf(w, " %-12s %10s %-30s %-30s\n",
|
||||||
|
strings.Repeat("-", 12), strings.Repeat("-", 10),
|
||||||
|
strings.Repeat("-", 30), strings.Repeat("-", 30))
|
||||||
|
for _, tx := range result.Unmatched {
|
||||||
|
fmt.Fprintf(w, " %-12s %10.0f %-30s %s\n",
|
||||||
|
tx.Date, tx.Amount, tx.Sender, tx.Message)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Matched transaction details
|
||||||
|
fmt.Fprintln(w, "\nMATCHED TRANSACTION DETAILS")
|
||||||
|
for _, e := range adults {
|
||||||
|
hasPayments := false
|
||||||
|
for _, m := range sortedMonths {
|
||||||
|
if len(e.data.Months[m].Transactions) > 0 {
|
||||||
|
hasPayments = true
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !hasPayments {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
fmt.Fprintf(w, "\n %s:\n", e.name)
|
||||||
|
for _, m := range sortedMonths {
|
||||||
|
for _, tx := range e.data.Months[m].Transactions {
|
||||||
|
conf := ""
|
||||||
|
if tx.Confidence == "review" {
|
||||||
|
conf = " [REVIEW]"
|
||||||
|
}
|
||||||
|
fmt.Fprintf(w, " %s: %.0f CZK from %s — \"%s\"%s\n",
|
||||||
|
monthLabel(m), tx.Amount, tx.Sender, tx.Message, conf)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
203
go/internal/services/membership/format_reconcile_test.go
Normal file
203
go/internal/services/membership/format_reconcile_test.go
Normal file
@@ -0,0 +1,203 @@
|
|||||||
|
package membership
|
||||||
|
|
||||||
|
// Golden strings verified against scripts/match_payments.py print_report() on 2026-05-06:
|
||||||
|
//
|
||||||
|
// PYTHONPATH=scripts:. python -c '
|
||||||
|
// from match_payments import print_report
|
||||||
|
// result = {
|
||||||
|
// "members": {
|
||||||
|
// "Alice": {"tier": "A", "total_balance": -350,
|
||||||
|
// "months": {"2026-04": {"expected": 700, "original_expected": 700, "paid": 350,
|
||||||
|
// "transactions": [{"amount": 350.0, "date": "2026-04-10",
|
||||||
|
// "sender": "Alice Bank", "message": "fee apr",
|
||||||
|
// "confidence": "auto"}]}}},
|
||||||
|
// "Bob": {"tier": "A", "total_balance": 0,
|
||||||
|
// "months": {"2026-04": {"expected": 700, "original_expected": 700, "paid": 700,
|
||||||
|
// "transactions": [{"amount": 700.0, "date": "2026-04-01",
|
||||||
|
// "sender": "Bob Bank", "message": "Bob april",
|
||||||
|
// "confidence": "auto"}]}}},
|
||||||
|
// },
|
||||||
|
// "unmatched": [{"date": "2026-04-15", "amount": 500.0, "sender": "Unknown", "message": "?"}],
|
||||||
|
// }
|
||||||
|
// print_report(result, ["2026-04"])
|
||||||
|
// '
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"fuj-management/go/internal/domain/reconcile"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func makeTestResult() (reconcile.Result, []string) {
|
||||||
|
sortedMonths := []string{"2026-04"}
|
||||||
|
|
||||||
|
aliceApr := reconcile.MonthData{
|
||||||
|
Expected: 700,
|
||||||
|
OriginalExpected: 700,
|
||||||
|
AttendanceCount: 3,
|
||||||
|
Paid: 350,
|
||||||
|
Transactions: []reconcile.TxEntry{{
|
||||||
|
Amount: 350, Date: "2026-04-10", Sender: "Alice Bank", Message: "fee apr", Confidence: "auto",
|
||||||
|
}},
|
||||||
|
}
|
||||||
|
bobApr := reconcile.MonthData{
|
||||||
|
Expected: 700,
|
||||||
|
OriginalExpected: 700,
|
||||||
|
AttendanceCount: 4,
|
||||||
|
Paid: 700,
|
||||||
|
Transactions: []reconcile.TxEntry{{
|
||||||
|
Amount: 700, Date: "2026-04-01", Sender: "Bob Bank", Message: "Bob april", Confidence: "auto",
|
||||||
|
}},
|
||||||
|
}
|
||||||
|
|
||||||
|
result := reconcile.Result{
|
||||||
|
Members: map[string]reconcile.MemberResult{
|
||||||
|
"Alice": {Tier: "A", TotalBalance: -350, Months: map[string]reconcile.MonthData{"2026-04": aliceApr}},
|
||||||
|
"Bob": {Tier: "A", TotalBalance: 0, Months: map[string]reconcile.MonthData{"2026-04": bobApr}},
|
||||||
|
},
|
||||||
|
Unmatched: []reconcile.Transaction{{
|
||||||
|
Date: "2026-04-15", Amount: 500, Sender: "Unknown", Message: "?",
|
||||||
|
}},
|
||||||
|
}
|
||||||
|
return result, sortedMonths
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestPrintReconcileReportStructure(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
result, sortedMonths := makeTestResult()
|
||||||
|
|
||||||
|
var buf bytes.Buffer
|
||||||
|
printReconcileReport(&buf, result, sortedMonths)
|
||||||
|
got := buf.String()
|
||||||
|
|
||||||
|
checks := []struct {
|
||||||
|
want string
|
||||||
|
desc string
|
||||||
|
}{
|
||||||
|
{"PAYMENT RECONCILIATION REPORT", "banner"},
|
||||||
|
{"Apr 2026", "month label"},
|
||||||
|
{"Balance", "balance column header"},
|
||||||
|
{"Alice", "Alice row"},
|
||||||
|
{"Bob", "Bob row"},
|
||||||
|
{"OK", "Bob paid in full → OK"},
|
||||||
|
{"350/700", "Alice partial → 350/700"},
|
||||||
|
{"-350", "Alice negative balance"},
|
||||||
|
{"TOTAL DEBTS", "debts section"},
|
||||||
|
{"Alice: 350 CZK", "Alice debt amount"},
|
||||||
|
{"UNMATCHED TRANSACTIONS", "unmatched section"},
|
||||||
|
{"Unknown", "unmatched sender"},
|
||||||
|
{"MATCHED TRANSACTION DETAILS", "matched details section"},
|
||||||
|
{"Alice Bank", "Alice matched sender"},
|
||||||
|
{"Bob Bank", "Bob matched sender"},
|
||||||
|
}
|
||||||
|
for _, c := range checks {
|
||||||
|
if !strings.Contains(got, c.want) {
|
||||||
|
t.Errorf("missing %s: want %q in output:\n%s", c.desc, c.want, got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// No CREDITS section expected (no member has TotalBalance > 0)
|
||||||
|
if strings.Contains(got, "TOTAL CREDITS") {
|
||||||
|
t.Error("unexpected CREDITS section when no member has positive balance")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestPrintReconcileReportUnpaidCell(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
result := reconcile.Result{
|
||||||
|
Members: map[string]reconcile.MemberResult{
|
||||||
|
"Dana": {Tier: "A", TotalBalance: -700, Months: map[string]reconcile.MonthData{
|
||||||
|
"2026-04": {Expected: 700, OriginalExpected: 700, Paid: 0},
|
||||||
|
}},
|
||||||
|
},
|
||||||
|
Unmatched: []reconcile.Transaction{},
|
||||||
|
}
|
||||||
|
|
||||||
|
var buf bytes.Buffer
|
||||||
|
printReconcileReport(&buf, result, []string{"2026-04"})
|
||||||
|
got := buf.String()
|
||||||
|
|
||||||
|
if !strings.Contains(got, "UNPAID 700") {
|
||||||
|
t.Errorf("expected 'UNPAID 700' for zero-payment member, got:\n%s", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestPrintReconcileReportDashCell(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
result := reconcile.Result{
|
||||||
|
Members: map[string]reconcile.MemberResult{
|
||||||
|
"Eve": {Tier: "A", TotalBalance: 0, Months: map[string]reconcile.MonthData{
|
||||||
|
"2026-04": {Expected: 0, Paid: 0},
|
||||||
|
}},
|
||||||
|
},
|
||||||
|
Unmatched: []reconcile.Transaction{},
|
||||||
|
}
|
||||||
|
|
||||||
|
var buf bytes.Buffer
|
||||||
|
printReconcileReport(&buf, result, []string{"2026-04"})
|
||||||
|
got := buf.String()
|
||||||
|
|
||||||
|
eveLine := ""
|
||||||
|
for _, l := range strings.Split(got, "\n") {
|
||||||
|
if strings.HasPrefix(strings.TrimSpace(l), "Eve") {
|
||||||
|
eveLine = l
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if eveLine == "" {
|
||||||
|
t.Fatal("no Eve line found")
|
||||||
|
}
|
||||||
|
if !strings.Contains(eveLine, "-") {
|
||||||
|
t.Errorf("expected '-' dash cell when expected=0 paid=0, Eve line: %q", eveLine)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestPrintReconcileReportCreditsSection(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
result := reconcile.Result{
|
||||||
|
Members: map[string]reconcile.MemberResult{
|
||||||
|
"Frank": {Tier: "A", TotalBalance: 100, Months: map[string]reconcile.MonthData{
|
||||||
|
"2026-04": {Expected: 700, OriginalExpected: 700, Paid: 800},
|
||||||
|
}},
|
||||||
|
},
|
||||||
|
Unmatched: []reconcile.Transaction{},
|
||||||
|
}
|
||||||
|
|
||||||
|
var buf bytes.Buffer
|
||||||
|
printReconcileReport(&buf, result, []string{"2026-04"})
|
||||||
|
got := buf.String()
|
||||||
|
|
||||||
|
if !strings.Contains(got, "TOTAL CREDITS") {
|
||||||
|
t.Errorf("expected CREDITS section, got:\n%s", got)
|
||||||
|
}
|
||||||
|
if !strings.Contains(got, "Frank: 100 CZK") {
|
||||||
|
t.Errorf("expected 'Frank: 100 CZK', got:\n%s", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestPrintReconcileReportReviewConfidence(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
result := reconcile.Result{
|
||||||
|
Members: map[string]reconcile.MemberResult{
|
||||||
|
"Grace": {Tier: "A", TotalBalance: 0, Months: map[string]reconcile.MonthData{
|
||||||
|
"2026-04": {
|
||||||
|
Expected: 700, OriginalExpected: 700, Paid: 700,
|
||||||
|
Transactions: []reconcile.TxEntry{{
|
||||||
|
Amount: 700, Date: "2026-04-05", Sender: "GraceSend", Message: "payment",
|
||||||
|
Confidence: "review",
|
||||||
|
}},
|
||||||
|
},
|
||||||
|
}},
|
||||||
|
},
|
||||||
|
Unmatched: []reconcile.Transaction{},
|
||||||
|
}
|
||||||
|
|
||||||
|
var buf bytes.Buffer
|
||||||
|
printReconcileReport(&buf, result, []string{"2026-04"})
|
||||||
|
got := buf.String()
|
||||||
|
|
||||||
|
if !strings.Contains(got, "[REVIEW]") {
|
||||||
|
t.Errorf("expected '[REVIEW]' annotation for review-confidence tx, got:\n%s", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
54
go/internal/services/membership/loader.go
Normal file
54
go/internal/services/membership/loader.go
Normal file
@@ -0,0 +1,54 @@
|
|||||||
|
package membership
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"fuj-management/go/internal/domain/reconcile"
|
||||||
|
)
|
||||||
|
|
||||||
|
// ErrIOPending is returned by stub loader methods until the M4 IO layer lands.
|
||||||
|
var ErrIOPending = errors.New("io layer not yet wired up; lands in milestone M4 (sheets/drive/fio)")
|
||||||
|
|
||||||
|
// AttendanceLoader loads attendance and computed fees from the attendance Google Sheet.
|
||||||
|
type AttendanceLoader interface {
|
||||||
|
LoadAdults(ctx context.Context) (members []reconcile.Member, sortedMonths []string, err error)
|
||||||
|
LoadJuniors(ctx context.Context) (members []reconcile.Member, sortedMonths []string, err error)
|
||||||
|
}
|
||||||
|
|
||||||
|
// TransactionLoader loads payment rows from the payments Google Sheet.
|
||||||
|
type TransactionLoader interface {
|
||||||
|
LoadTransactions(ctx context.Context) ([]reconcile.Transaction, error)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ExceptionLoader loads manual fee overrides from the exceptions sheet tab.
|
||||||
|
type ExceptionLoader interface {
|
||||||
|
LoadExceptions(ctx context.Context) (map[reconcile.ExceptionKey]reconcile.Exception, error)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Sources is the aggregate interface required by ReconcileReport.
|
||||||
|
type Sources interface {
|
||||||
|
AttendanceLoader
|
||||||
|
TransactionLoader
|
||||||
|
ExceptionLoader
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewStubSources returns a Sources whose every method returns ErrIOPending.
|
||||||
|
func NewStubSources() Sources { return stubSources{} }
|
||||||
|
|
||||||
|
type stubSources struct{}
|
||||||
|
|
||||||
|
func (stubSources) LoadAdults(_ context.Context) ([]reconcile.Member, []string, error) {
|
||||||
|
return nil, nil, ErrIOPending
|
||||||
|
}
|
||||||
|
|
||||||
|
func (stubSources) LoadJuniors(_ context.Context) ([]reconcile.Member, []string, error) {
|
||||||
|
return nil, nil, ErrIOPending
|
||||||
|
}
|
||||||
|
|
||||||
|
func (stubSources) LoadTransactions(_ context.Context) ([]reconcile.Transaction, error) {
|
||||||
|
return nil, ErrIOPending
|
||||||
|
}
|
||||||
|
|
||||||
|
func (stubSources) LoadExceptions(_ context.Context) (map[reconcile.ExceptionKey]reconcile.Exception, error) {
|
||||||
|
return nil, ErrIOPending
|
||||||
|
}
|
||||||
30
go/internal/services/membership/reconcile.go
Normal file
30
go/internal/services/membership/reconcile.go
Normal file
@@ -0,0 +1,30 @@
|
|||||||
|
package membership
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"io"
|
||||||
|
|
||||||
|
domreconcile "fuj-management/go/internal/domain/reconcile"
|
||||||
|
)
|
||||||
|
|
||||||
|
// ReconcileReport loads attendance, transactions, and exceptions via s, runs
|
||||||
|
// the three-phase reconciliation, and writes the balance report to out.
|
||||||
|
// Returns ErrIOPending until real loaders are injected in M4.
|
||||||
|
func ReconcileReport(ctx context.Context, s Sources, defaultYear int, out io.Writer) error {
|
||||||
|
members, sortedMonths, err := s.LoadAdults(ctx)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
txns, err := s.LoadTransactions(ctx)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
exceptions, err := s.LoadExceptions(ctx)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
result := domreconcile.Reconcile(members, sortedMonths, txns, exceptions, defaultYear)
|
||||||
|
printReconcileReport(out, result, sortedMonths)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
72
go/internal/services/membership/reconcile_test.go
Normal file
72
go/internal/services/membership/reconcile_test.go
Normal file
@@ -0,0 +1,72 @@
|
|||||||
|
package membership
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"fuj-management/go/internal/domain/reconcile"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
type fakeSources struct {
|
||||||
|
members []reconcile.Member
|
||||||
|
months []string
|
||||||
|
txns []reconcile.Transaction
|
||||||
|
exceptions map[reconcile.ExceptionKey]reconcile.Exception
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f fakeSources) LoadAdults(_ context.Context) ([]reconcile.Member, []string, error) {
|
||||||
|
return f.members, f.months, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f fakeSources) LoadJuniors(_ context.Context) ([]reconcile.Member, []string, error) {
|
||||||
|
return nil, nil, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f fakeSources) LoadTransactions(_ context.Context) ([]reconcile.Transaction, error) {
|
||||||
|
return f.txns, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f fakeSources) LoadExceptions(_ context.Context) (map[reconcile.ExceptionKey]reconcile.Exception, error) {
|
||||||
|
return f.exceptions, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestReconcileReport(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
s := fakeSources{
|
||||||
|
members: []reconcile.Member{
|
||||||
|
{Name: "Alice", Tier: "A", Fees: map[string]reconcile.FeeData{
|
||||||
|
"2026-04": {Expected: 700, Attendance: 3},
|
||||||
|
}},
|
||||||
|
},
|
||||||
|
months: []string{"2026-04"},
|
||||||
|
txns: []reconcile.Transaction{
|
||||||
|
{
|
||||||
|
Date: "2026-04-10", Amount: 700, Person: "Alice", Purpose: "2026-04",
|
||||||
|
Sender: "Alice Bank", Message: "fee",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
exceptions: map[reconcile.ExceptionKey]reconcile.Exception{},
|
||||||
|
}
|
||||||
|
|
||||||
|
var buf bytes.Buffer
|
||||||
|
if err := ReconcileReport(context.Background(), s, 2026, &buf); err != nil {
|
||||||
|
t.Fatalf("unexpected error: %v", err)
|
||||||
|
}
|
||||||
|
got := buf.String()
|
||||||
|
if !strings.Contains(got, "PAYMENT RECONCILIATION REPORT") {
|
||||||
|
t.Error("missing report header")
|
||||||
|
}
|
||||||
|
if !strings.Contains(got, "OK") {
|
||||||
|
t.Errorf("expected 'OK' for fully-paid Alice, got:\n%s", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestReconcileReportStubErrors(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
var buf bytes.Buffer
|
||||||
|
err := ReconcileReport(context.Background(), NewStubSources(), 2026, &buf)
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("expected error from stub, got nil")
|
||||||
|
}
|
||||||
|
}
|
||||||
477
go/internal/services/membership/sources.go
Normal file
477
go/internal/services/membership/sources.go
Normal file
@@ -0,0 +1,477 @@
|
|||||||
|
package membership
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"fmt"
|
||||||
|
"fuj-management/go/internal/config"
|
||||||
|
"fuj-management/go/internal/domain/czech"
|
||||||
|
"fuj-management/go/internal/domain/fees"
|
||||||
|
"fuj-management/go/internal/domain/matching"
|
||||||
|
"fuj-management/go/internal/domain/reconcile"
|
||||||
|
"fuj-management/go/internal/io/attendance"
|
||||||
|
"fuj-management/go/internal/io/cache"
|
||||||
|
"fuj-management/go/internal/io/drive"
|
||||||
|
"fuj-management/go/internal/io/sheets"
|
||||||
|
"sort"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Attendance CSV column indices (mirrors COL_* in scripts/attendance.py)
|
||||||
|
const (
|
||||||
|
colName = 0
|
||||||
|
colTier = 1
|
||||||
|
firstDateCol = 3
|
||||||
|
)
|
||||||
|
|
||||||
|
// adultMergedMonths mirrors ADULT_MERGED_MONTHS in scripts/attendance.py.
|
||||||
|
// Source month → target month (source attendance accumulated into target).
|
||||||
|
var adultMergedMonths = map[string]string{}
|
||||||
|
|
||||||
|
// juniorMergedMonths mirrors JUNIOR_MERGED_MONTHS in scripts/attendance.py.
|
||||||
|
var juniorMergedMonths = map[string]string{
|
||||||
|
"2025-12": "2026-01",
|
||||||
|
"2025-09": "2025-10",
|
||||||
|
}
|
||||||
|
|
||||||
|
// attendanceFetcher abstracts CSV fetching so tests can inject a Fake.
|
||||||
|
type attendanceFetcher interface {
|
||||||
|
FetchAdults(ctx context.Context) ([][]string, error)
|
||||||
|
FetchJuniors(ctx context.Context) ([][]string, error)
|
||||||
|
}
|
||||||
|
|
||||||
|
// sheetReader abstracts Sheets API reads so tests can inject a Fake.
|
||||||
|
type sheetReader interface {
|
||||||
|
GetValues(ctx context.Context, spreadsheetID, a1Range string) ([][]any, error)
|
||||||
|
}
|
||||||
|
|
||||||
|
// realSources is the live implementation of Sources backed by Google APIs.
|
||||||
|
type realSources struct {
|
||||||
|
attendance attendanceFetcher
|
||||||
|
sheets sheetReader
|
||||||
|
cache *cache.FileCache
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewSources builds a Sources backed by real Google Sheets and Drive APIs.
|
||||||
|
// Call this once at startup; the returned Sources is safe for concurrent use.
|
||||||
|
func NewSources(ctx context.Context, cfg config.Config) (Sources, error) {
|
||||||
|
driveCli, err := drive.New(ctx, cfg.CredentialsPath, cfg.DriveTimeout)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("drive client: %w", err)
|
||||||
|
}
|
||||||
|
sheetsCli, err := sheets.New(ctx, cfg.CredentialsPath, cfg.DriveTimeout)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("sheets client: %w", err)
|
||||||
|
}
|
||||||
|
attendanceCli := attendance.New(nil, config.AttendanceSheetID, config.AttendanceAdultSheetGID, config.JuniorSheetGID)
|
||||||
|
fc := cache.New(driveCli, cfg.CacheDir, config.CacheSheetMap, cfg.CacheTTL, cfg.CacheAPICheckTTL)
|
||||||
|
|
||||||
|
return &realSources{
|
||||||
|
attendance: attendanceCli,
|
||||||
|
sheets: sheetsCli,
|
||||||
|
cache: fc,
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// LoadAdults fetches adult attendance (cached) and returns reconcile.Members for all tiers.
|
||||||
|
func (s *realSources) LoadAdults(ctx context.Context) ([]reconcile.Member, []string, error) {
|
||||||
|
rows, err := cache.Get(ctx, s.cache, "attendance_regular", s.attendance.FetchAdults)
|
||||||
|
if err != nil {
|
||||||
|
return nil, nil, fmt.Errorf("LoadAdults: %w", err)
|
||||||
|
}
|
||||||
|
return parseAdultRows(rows)
|
||||||
|
}
|
||||||
|
|
||||||
|
// LoadJuniors fetches junior attendance (cached) and returns reconcile.Members for juniors.
|
||||||
|
func (s *realSources) LoadJuniors(ctx context.Context) ([]reconcile.Member, []string, error) {
|
||||||
|
// Junior data needs both the adult tab (tier="J" rows) and the junior tab.
|
||||||
|
adultRows, err := cache.Get(ctx, s.cache, "attendance_regular", s.attendance.FetchAdults)
|
||||||
|
if err != nil {
|
||||||
|
return nil, nil, fmt.Errorf("LoadJuniors (adult tab): %w", err)
|
||||||
|
}
|
||||||
|
juniorRows, err := cache.Get(ctx, s.cache, "attendance_juniors", s.attendance.FetchJuniors)
|
||||||
|
if err != nil {
|
||||||
|
return nil, nil, fmt.Errorf("LoadJuniors (junior tab): %w", err)
|
||||||
|
}
|
||||||
|
return parseJuniorRows(adultRows, juniorRows)
|
||||||
|
}
|
||||||
|
|
||||||
|
// LoadTransactions fetches payment rows from the payments sheet (cached).
|
||||||
|
func (s *realSources) LoadTransactions(ctx context.Context) ([]reconcile.Transaction, error) {
|
||||||
|
rows, err := cache.Get(ctx, s.cache, "payments_transactions",
|
||||||
|
func(ctx context.Context) ([][]any, error) {
|
||||||
|
return s.sheets.GetValues(ctx, config.PaymentsSheetID, "A1:Z")
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("LoadTransactions: %w", err)
|
||||||
|
}
|
||||||
|
return parseTransactionRows(rows)
|
||||||
|
}
|
||||||
|
|
||||||
|
// LoadExceptions fetches the exceptions tab (cached).
|
||||||
|
func (s *realSources) LoadExceptions(ctx context.Context) (map[reconcile.ExceptionKey]reconcile.Exception, error) {
|
||||||
|
rows, err := cache.Get(ctx, s.cache, "exceptions_dict",
|
||||||
|
func(ctx context.Context) ([][]any, error) {
|
||||||
|
return s.sheets.GetValues(ctx, config.PaymentsSheetID, "'exceptions'!A2:D")
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("LoadExceptions: %w", err)
|
||||||
|
}
|
||||||
|
return parseExceptionRows(rows), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// Attendance CSV parsing (ports scripts/attendance.py)
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
// parseDates returns (columnIndex, YYYY-MM) pairs for all date columns.
|
||||||
|
// Ports scripts/attendance.py parse_dates + strftime("%Y-%m").
|
||||||
|
func parseDates(header []string) []struct {
|
||||||
|
col int
|
||||||
|
month string
|
||||||
|
} {
|
||||||
|
var out []struct {
|
||||||
|
col int
|
||||||
|
month string
|
||||||
|
}
|
||||||
|
for i := firstDateCol; i < len(header); i++ {
|
||||||
|
raw := strings.TrimSpace(header[i])
|
||||||
|
if raw == "" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
var dt time.Time
|
||||||
|
var err error
|
||||||
|
for _, fmt_ := range []string{"02.01.2006", "01/02/2006"} {
|
||||||
|
dt, err = time.Parse(fmt_, raw)
|
||||||
|
if err == nil {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
out = append(out, struct {
|
||||||
|
col int
|
||||||
|
month string
|
||||||
|
}{col: i, month: dt.Format("2006-01")})
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
// groupByMonth groups column indices by YYYY-MM, applying merged month mapping.
|
||||||
|
func groupByMonth(dates []struct {
|
||||||
|
col int
|
||||||
|
month string
|
||||||
|
}, mergedMonths map[string]string,
|
||||||
|
) map[string][]int {
|
||||||
|
out := make(map[string][]int)
|
||||||
|
for _, d := range dates {
|
||||||
|
target := d.month
|
||||||
|
if v, ok := mergedMonths[d.month]; ok {
|
||||||
|
target = v
|
||||||
|
}
|
||||||
|
out[target] = append(out[target], d.col)
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
// countTrue counts how many cells in the given columns have the value "TRUE" (case-insensitive).
|
||||||
|
func countTrue(row []string, cols []int) int {
|
||||||
|
n := 0
|
||||||
|
for _, c := range cols {
|
||||||
|
if c < len(row) && strings.EqualFold(strings.TrimSpace(row[c]), "true") {
|
||||||
|
n++
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return n
|
||||||
|
}
|
||||||
|
|
||||||
|
// parseAdultRows converts raw CSV rows to []reconcile.Member.
|
||||||
|
// Includes all tiers; fee is 0 for non-A tiers (reconcile filters downstream).
|
||||||
|
// Ports scripts/attendance.py get_members_with_fees.
|
||||||
|
func parseAdultRows(rows [][]string) ([]reconcile.Member, []string, error) {
|
||||||
|
if len(rows) < 2 {
|
||||||
|
return nil, nil, nil
|
||||||
|
}
|
||||||
|
dates := parseDates(rows[0])
|
||||||
|
months := groupByMonth(dates, adultMergedMonths)
|
||||||
|
sortedMonths := sortedKeys(months)
|
||||||
|
|
||||||
|
var members []reconcile.Member
|
||||||
|
for _, row := range rows[1:] {
|
||||||
|
if len(row) == 0 {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
first := strings.TrimSpace(row[colName])
|
||||||
|
if strings.Contains(strings.ToLower(first), "# last line") {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
if strings.HasPrefix(first, "#") || first == "" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if strings.ToLower(first) == "jméno" || strings.ToLower(first) == "name" || strings.ToLower(first) == "jmeno" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
tier := ""
|
||||||
|
if len(row) > colTier {
|
||||||
|
tier = strings.ToUpper(strings.TrimSpace(row[colTier]))
|
||||||
|
}
|
||||||
|
|
||||||
|
feeMap := make(map[string]reconcile.FeeData, len(sortedMonths))
|
||||||
|
for _, m := range sortedMonths {
|
||||||
|
cols := months[m]
|
||||||
|
count := countTrue(row, cols)
|
||||||
|
var fee int
|
||||||
|
if tier == "A" {
|
||||||
|
fee = fees.CalculateFee(count, m)
|
||||||
|
}
|
||||||
|
feeMap[m] = reconcile.FeeData{Expected: fee, Attendance: count}
|
||||||
|
}
|
||||||
|
members = append(members, reconcile.Member{Name: first, Tier: tier, Fees: feeMap})
|
||||||
|
}
|
||||||
|
return members, sortedMonths, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// parseJuniorRows builds junior members by merging tier-J rows from the adult tab
|
||||||
|
// with the junior sheet, then calling CalculateJuniorFee.
|
||||||
|
// Ports scripts/attendance.py get_junior_members_with_fees.
|
||||||
|
func parseJuniorRows(adultRows, juniorRows [][]string) ([]reconcile.Member, []string, error) {
|
||||||
|
if len(adultRows) < 2 || len(juniorRows) < 2 {
|
||||||
|
return nil, nil, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
mainDates := parseDates(adultRows[0])
|
||||||
|
juniorDates := parseDates(juniorRows[0])
|
||||||
|
mainMonths := groupByMonth(mainDates, juniorMergedMonths)
|
||||||
|
jrMonths := groupByMonth(juniorDates, juniorMergedMonths)
|
||||||
|
|
||||||
|
allMonths := make(map[string]bool)
|
||||||
|
for m := range mainMonths {
|
||||||
|
allMonths[m] = true
|
||||||
|
}
|
||||||
|
for m := range jrMonths {
|
||||||
|
allMonths[m] = true
|
||||||
|
}
|
||||||
|
sortedMonths := sortedKeys(allMonths)
|
||||||
|
|
||||||
|
type counts struct{ adult, junior int }
|
||||||
|
merged := make(map[string]*struct {
|
||||||
|
tier string
|
||||||
|
months map[string]counts
|
||||||
|
})
|
||||||
|
|
||||||
|
// Tier-J rows from adult tab
|
||||||
|
for _, row := range adultRows[1:] {
|
||||||
|
if len(row) == 0 {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
first := strings.TrimSpace(row[colName])
|
||||||
|
if strings.Contains(strings.ToLower(first), "# last line") {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
if strings.HasPrefix(first, "#") || first == "" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
tier := ""
|
||||||
|
if len(row) > colTier {
|
||||||
|
tier = strings.ToUpper(strings.TrimSpace(row[colTier]))
|
||||||
|
}
|
||||||
|
if tier != "J" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if _, ok := merged[first]; !ok {
|
||||||
|
merged[first] = &struct {
|
||||||
|
tier string
|
||||||
|
months map[string]counts
|
||||||
|
}{tier: tier, months: make(map[string]counts)}
|
||||||
|
}
|
||||||
|
for _, m := range sortedMonths {
|
||||||
|
c := merged[first].months[m]
|
||||||
|
c.adult += countTrue(row, mainMonths[m])
|
||||||
|
merged[first].months[m] = c
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// All non-X rows from junior tab
|
||||||
|
for _, row := range juniorRows[1:] {
|
||||||
|
if len(row) == 0 {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
first := strings.TrimSpace(row[colName])
|
||||||
|
fl := strings.ToLower(first)
|
||||||
|
if strings.Contains(fl, "# treneri") || strings.Contains(fl, "# trenéři") {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
if strings.HasPrefix(first, "#") || first == "" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
tier := ""
|
||||||
|
if len(row) > colTier {
|
||||||
|
tier = strings.ToUpper(strings.TrimSpace(row[colTier]))
|
||||||
|
}
|
||||||
|
if tier == "X" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if _, ok := merged[first]; !ok {
|
||||||
|
merged[first] = &struct {
|
||||||
|
tier string
|
||||||
|
months map[string]counts
|
||||||
|
}{tier: tier, months: make(map[string]counts)}
|
||||||
|
}
|
||||||
|
for _, m := range sortedMonths {
|
||||||
|
c := merged[first].months[m]
|
||||||
|
c.junior += countTrue(row, jrMonths[m])
|
||||||
|
merged[first].months[m] = c
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
var members []reconcile.Member
|
||||||
|
for name, data := range merged {
|
||||||
|
feeMap := make(map[string]reconcile.FeeData, len(sortedMonths))
|
||||||
|
for _, m := range sortedMonths {
|
||||||
|
c := data.months[m]
|
||||||
|
total := c.adult + c.junior
|
||||||
|
exp := fees.CalculateJuniorFee(total, m)
|
||||||
|
fee := 0
|
||||||
|
if !exp.Unknown {
|
||||||
|
fee = exp.Value
|
||||||
|
}
|
||||||
|
feeMap[m] = reconcile.FeeData{Expected: fee, Attendance: total}
|
||||||
|
}
|
||||||
|
members = append(members, reconcile.Member{Name: name, Tier: data.tier, Fees: feeMap})
|
||||||
|
}
|
||||||
|
return members, sortedMonths, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// Payments sheet row parsing (ports scripts/match_payments.py fetch_sheet_data)
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
func parseTransactionRows(rows [][]any) ([]reconcile.Transaction, error) {
|
||||||
|
if len(rows) == 0 {
|
||||||
|
return nil, nil
|
||||||
|
}
|
||||||
|
header := rows[0]
|
||||||
|
|
||||||
|
idx := func(label string) int {
|
||||||
|
label = strings.ToLower(strings.TrimSpace(label))
|
||||||
|
for i, h := range header {
|
||||||
|
if strings.ToLower(strings.TrimSpace(fmt.Sprint(h))) == label {
|
||||||
|
return i
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return -1
|
||||||
|
}
|
||||||
|
idxDate := idx("date")
|
||||||
|
idxAmount := idx("amount")
|
||||||
|
idxPerson := idx("person")
|
||||||
|
idxPurpose := idx("purpose")
|
||||||
|
idxInferred := idx("inferred amount")
|
||||||
|
idxSender := idx("sender")
|
||||||
|
idxMessage := idx("message")
|
||||||
|
|
||||||
|
for _, label := range []string{"date", "amount", "person", "purpose"} {
|
||||||
|
if idx(label) == -1 {
|
||||||
|
return nil, fmt.Errorf("payments sheet missing required column %q", label)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
getVal := func(row []any, i int) string {
|
||||||
|
if i < 0 || i >= len(row) {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
return fmt.Sprint(row[i])
|
||||||
|
}
|
||||||
|
|
||||||
|
var txns []reconcile.Transaction
|
||||||
|
for _, row := range rows[1:] {
|
||||||
|
dateStr := matching.FormatDate(getVal(row, idxDate))
|
||||||
|
amountRaw := row[idxAmount]
|
||||||
|
if idxAmount < 0 || idxAmount >= len(row) {
|
||||||
|
amountRaw = ""
|
||||||
|
}
|
||||||
|
amount := parseFloat(amountRaw)
|
||||||
|
|
||||||
|
var inferredAmount *float64
|
||||||
|
if iv := getVal(row, idxInferred); iv != "" && iv != "<nil>" {
|
||||||
|
if f := parseFloat(iv); f != 0 {
|
||||||
|
inferredAmount = &f
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
txns = append(txns, reconcile.Transaction{
|
||||||
|
Date: dateStr,
|
||||||
|
Amount: amount,
|
||||||
|
Person: getVal(row, idxPerson),
|
||||||
|
Purpose: getVal(row, idxPurpose),
|
||||||
|
InferredAmount: inferredAmount,
|
||||||
|
Sender: getVal(row, idxSender),
|
||||||
|
Message: getVal(row, idxMessage),
|
||||||
|
})
|
||||||
|
}
|
||||||
|
return txns, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func parseFloat(v any) float64 {
|
||||||
|
switch x := v.(type) {
|
||||||
|
case float64:
|
||||||
|
return x
|
||||||
|
case float32:
|
||||||
|
return float64(x)
|
||||||
|
case int:
|
||||||
|
return float64(x)
|
||||||
|
case int64:
|
||||||
|
return float64(x)
|
||||||
|
case string:
|
||||||
|
f, _ := strconv.ParseFloat(strings.TrimSpace(x), 64)
|
||||||
|
return f
|
||||||
|
}
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// Exceptions tab parsing (ports scripts/match_payments.py fetch_exceptions)
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
func parseExceptionRows(rows [][]any) map[reconcile.ExceptionKey]reconcile.Exception {
|
||||||
|
out := make(map[reconcile.ExceptionKey]reconcile.Exception)
|
||||||
|
for _, row := range rows {
|
||||||
|
if len(row) < 3 {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
name := strings.TrimSpace(fmt.Sprint(row[0]))
|
||||||
|
if strings.ToLower(name) == "name" || strings.HasPrefix(strings.ToLower(name), "name") {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
period := strings.TrimSpace(fmt.Sprint(row[1]))
|
||||||
|
amountStr := fmt.Sprint(row[2])
|
||||||
|
amount, err := strconv.Atoi(strings.TrimSpace(amountStr))
|
||||||
|
if err != nil {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
note := ""
|
||||||
|
if len(row) > 3 {
|
||||||
|
note = strings.TrimSpace(fmt.Sprint(row[3]))
|
||||||
|
}
|
||||||
|
key := reconcile.ExceptionKey{
|
||||||
|
Name: czech.Normalize(name),
|
||||||
|
Period: czech.Normalize(period),
|
||||||
|
}
|
||||||
|
out[key] = reconcile.Exception{Amount: amount, Note: note}
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// Helpers
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
func sortedKeys[V any](m map[string]V) []string {
|
||||||
|
keys := make([]string, 0, len(m))
|
||||||
|
for k := range m {
|
||||||
|
keys = append(keys, k)
|
||||||
|
}
|
||||||
|
sort.Strings(keys)
|
||||||
|
return keys
|
||||||
|
}
|
||||||
198
go/internal/services/membership/sources_test.go
Normal file
198
go/internal/services/membership/sources_test.go
Normal file
@@ -0,0 +1,198 @@
|
|||||||
|
package membership
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"fuj-management/go/internal/config"
|
||||||
|
"fuj-management/go/internal/io/attendance"
|
||||||
|
"fuj-management/go/internal/io/cache"
|
||||||
|
"fuj-management/go/internal/io/drive"
|
||||||
|
"fuj-management/go/internal/io/sheets"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// buildSources wires a realSources with in-memory fakes and a no-TTL cache.
|
||||||
|
func buildSources(t *testing.T, att *attendance.Fake, sh *sheets.Fake) *realSources {
|
||||||
|
t.Helper()
|
||||||
|
dir := t.TempDir()
|
||||||
|
d := &drive.Fake{Times: map[string]string{
|
||||||
|
config.AttendanceSheetID: "t1",
|
||||||
|
config.PaymentsSheetID: "t1",
|
||||||
|
}}
|
||||||
|
fc := cache.New(d, dir, config.CacheSheetMap, 0, 0)
|
||||||
|
return &realSources{attendance: att, sheets: sh, cache: fc}
|
||||||
|
}
|
||||||
|
|
||||||
|
var minimalAdultCSV = [][]string{
|
||||||
|
{"Jméno", "Tier", "", "", "01.09.2025", "08.09.2025"},
|
||||||
|
{"Alice", "A", "", "", "TRUE", "TRUE"},
|
||||||
|
{"Bob", "A", "", "", "TRUE", "FALSE"},
|
||||||
|
{"# last line"},
|
||||||
|
}
|
||||||
|
|
||||||
|
// minimalJuniorCSV has dates in October because the junior merged-month map sends
|
||||||
|
// 2025-09 → 2025-10, so two columns for 01.10.2025 and 08.10.2025 land in "2025-10".
|
||||||
|
var minimalJuniorCSV = [][]string{
|
||||||
|
{"Jméno", "Tier", "", "", "01.10.2025", "08.10.2025"},
|
||||||
|
{"Charlie", "J", "", "", "TRUE", "TRUE"},
|
||||||
|
{"# Trenéři"},
|
||||||
|
{"Coach", "X", "", "", "FALSE", "FALSE"},
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestLoadAdults(t *testing.T) {
|
||||||
|
s := buildSources(t, &attendance.Fake{Adults: minimalAdultCSV}, &sheets.Fake{})
|
||||||
|
|
||||||
|
members, months, err := s.LoadAdults(context.Background())
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
// adultMergedMonths is empty so 2025-09 stays as-is
|
||||||
|
if len(months) != 1 || months[0] != "2025-09" {
|
||||||
|
t.Errorf("unexpected months: %v", months)
|
||||||
|
}
|
||||||
|
if len(members) != 2 {
|
||||||
|
t.Fatalf("want 2 members, got %d", len(members))
|
||||||
|
}
|
||||||
|
byName := map[string]int{}
|
||||||
|
for _, m := range members {
|
||||||
|
byName[m.Name] = m.Fees["2025-09"].Attendance
|
||||||
|
}
|
||||||
|
if byName["Alice"] != 2 {
|
||||||
|
t.Errorf("Alice: want 2 sessions, got %d", byName["Alice"])
|
||||||
|
}
|
||||||
|
if byName["Bob"] != 1 {
|
||||||
|
t.Errorf("Bob: want 1 session, got %d", byName["Bob"])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestLoadAdults_Fee(t *testing.T) {
|
||||||
|
s := buildSources(t, &attendance.Fake{Adults: minimalAdultCSV}, &sheets.Fake{})
|
||||||
|
members, _, err := s.LoadAdults(context.Background())
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
byName := map[string]int{}
|
||||||
|
for _, m := range members {
|
||||||
|
byName[m.Name] = m.Fees["2025-09"].Expected
|
||||||
|
}
|
||||||
|
// 2 sessions in 2025-09 → AdultFeeMonthlyRate["2025-09"] = 750
|
||||||
|
if byName["Alice"] != 750 {
|
||||||
|
t.Errorf("Alice fee: want 750, got %d", byName["Alice"])
|
||||||
|
}
|
||||||
|
// 1 session → AdultFeeSingle = 200
|
||||||
|
if byName["Bob"] != 200 {
|
||||||
|
t.Errorf("Bob fee: want 200, got %d", byName["Bob"])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestLoadJuniors(t *testing.T) {
|
||||||
|
s := buildSources(t,
|
||||||
|
&attendance.Fake{Adults: minimalAdultCSV, Juniors: minimalJuniorCSV},
|
||||||
|
&sheets.Fake{})
|
||||||
|
|
||||||
|
members, months, err := s.LoadJuniors(context.Background())
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(months) == 0 {
|
||||||
|
t.Fatal("want months, got none")
|
||||||
|
}
|
||||||
|
found := false
|
||||||
|
for _, m := range members {
|
||||||
|
if m.Name == "Charlie" {
|
||||||
|
found = true
|
||||||
|
// Charlie has 2 sessions in 2025-10 (October dates in junior CSV)
|
||||||
|
if m.Fees["2025-10"].Attendance != 2 {
|
||||||
|
t.Errorf("Charlie 2025-10 attendance: want 2, got %d", m.Fees["2025-10"].Attendance)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !found {
|
||||||
|
t.Error("Charlie not found in juniors")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestLoadTransactions(t *testing.T) {
|
||||||
|
// Sheets fake keyed by "<spreadsheetID>/<range>" — use the real constant.
|
||||||
|
paymentsKey := config.PaymentsSheetID + "/A1:Z"
|
||||||
|
sh := &sheets.Fake{Values: map[string][][]any{
|
||||||
|
paymentsKey: {
|
||||||
|
{"Date", "Amount", "manual fix", "Person", "Purpose", "Inferred Amount", "Sender", "VS", "Message", "Bank ID", "Sync ID"},
|
||||||
|
{"2026-04-01", 700.0, "", "Alice", "2026-04", "", "Alice Bank", "", "fee", "", "abc"},
|
||||||
|
{"2026-05-01", 500.0, "", "", "", "", "Bob Bank", "", "platba", "", "def"},
|
||||||
|
},
|
||||||
|
}}
|
||||||
|
s := buildSources(t, &attendance.Fake{}, sh)
|
||||||
|
|
||||||
|
txns, err := s.LoadTransactions(context.Background())
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(txns) != 2 {
|
||||||
|
t.Fatalf("want 2 transactions, got %d", len(txns))
|
||||||
|
}
|
||||||
|
if txns[0].Person != "Alice" {
|
||||||
|
t.Errorf("txn[0].Person: want Alice, got %q", txns[0].Person)
|
||||||
|
}
|
||||||
|
if txns[0].Amount != 700 {
|
||||||
|
t.Errorf("txn[0].Amount: want 700, got %v", txns[0].Amount)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestLoadExceptions(t *testing.T) {
|
||||||
|
excKey := config.PaymentsSheetID + "/'exceptions'!A2:D"
|
||||||
|
sh := &sheets.Fake{Values: map[string][][]any{
|
||||||
|
excKey: {
|
||||||
|
{"Alice", "2026-04", 350, "reduced"},
|
||||||
|
},
|
||||||
|
}}
|
||||||
|
s := buildSources(t, &attendance.Fake{}, sh)
|
||||||
|
|
||||||
|
exc, err := s.LoadExceptions(context.Background())
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(exc) != 1 {
|
||||||
|
t.Fatalf("want 1 exception, got %d", len(exc))
|
||||||
|
}
|
||||||
|
for k, v := range exc {
|
||||||
|
if v.Amount != 350 {
|
||||||
|
t.Errorf("exception amount: want 350, got %d (key=%v)", v.Amount, k)
|
||||||
|
}
|
||||||
|
if v.Note != "reduced" {
|
||||||
|
t.Errorf("exception note: want 'reduced', got %q", v.Note)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TTL smoke test: second call within TTL must not call fetch again.
|
||||||
|
func TestLoadAdults_CacheHit(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
d := &drive.Fake{Times: map[string]string{config.AttendanceSheetID: "t1"}}
|
||||||
|
fc := cache.New(d, dir, config.CacheSheetMap, time.Minute, time.Minute)
|
||||||
|
|
||||||
|
calls := 0
|
||||||
|
att := &countingFetcher{rows: minimalAdultCSV, calls: &calls}
|
||||||
|
s := &realSources{attendance: att, sheets: &sheets.Fake{}, cache: fc}
|
||||||
|
|
||||||
|
if _, _, err := s.LoadAdults(context.Background()); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, _, err := s.LoadAdults(context.Background()); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if calls != 1 {
|
||||||
|
t.Errorf("want 1 fetch (cache hit on 2nd call), got %d", calls)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
type countingFetcher struct {
|
||||||
|
rows [][]string
|
||||||
|
calls *int
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *countingFetcher) FetchAdults(_ context.Context) ([][]string, error) {
|
||||||
|
*f.calls++
|
||||||
|
return f.rows, nil
|
||||||
|
}
|
||||||
|
func (f *countingFetcher) FetchJuniors(_ context.Context) ([][]string, error) { return nil, nil }
|
||||||
27
go/internal/services/membership/stub_test.go
Normal file
27
go/internal/services/membership/stub_test.go
Normal file
@@ -0,0 +1,27 @@
|
|||||||
|
package membership
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestStubLoaderReturnsErrIOPending(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
s := NewStubSources()
|
||||||
|
|
||||||
|
_, _, err := s.LoadAdults(context.Background())
|
||||||
|
if !errors.Is(err, ErrIOPending) {
|
||||||
|
t.Errorf("LoadAdults: want ErrIOPending, got %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
_, err = s.LoadTransactions(context.Background())
|
||||||
|
if !errors.Is(err, ErrIOPending) {
|
||||||
|
t.Errorf("LoadTransactions: want ErrIOPending, got %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
_, err = s.LoadExceptions(context.Background())
|
||||||
|
if !errors.Is(err, ErrIOPending) {
|
||||||
|
t.Errorf("LoadExceptions: want ErrIOPending, got %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
128
go/tests/fixtures/README.md
vendored
Normal file
128
go/tests/fixtures/README.md
vendored
Normal file
@@ -0,0 +1,128 @@
|
|||||||
|
# Parity Fixtures
|
||||||
|
|
||||||
|
Captured outputs from the live Python implementation used as ground truth for
|
||||||
|
the Go parity test suite. All 98 files are committed and PII-free.
|
||||||
|
|
||||||
|
## Directory layout
|
||||||
|
|
||||||
|
```
|
||||||
|
fixtures/
|
||||||
|
pure/
|
||||||
|
normalize/ # scripts.czech_utils.normalize
|
||||||
|
parse_month_references/ # scripts.czech_utils.parse_month_references
|
||||||
|
calculate_fee/ # scripts.attendance.calculate_fee
|
||||||
|
calculate_junior_fee/ # scripts.attendance.calculate_junior_fee
|
||||||
|
parse_czk_amount/ # scripts.infer_payments.parse_czk_amount
|
||||||
|
generate_sync_id/ # scripts.sync_fio_to_sheets.generate_sync_id
|
||||||
|
build_name_variants/ # scripts.match_payments._build_name_variants
|
||||||
|
match_members/ # scripts.match_payments.match_members
|
||||||
|
infer_transaction_details/ # scripts.match_payments.infer_transaction_details
|
||||||
|
format_date/ # scripts.match_payments.format_date
|
||||||
|
reconcile/ # scripts.match_payments.reconcile (10 branch-coverage cases)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Fixture format
|
||||||
|
|
||||||
|
One JSON object per file:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"case": "range_wrap_nov_to_jan",
|
||||||
|
"func": "scripts.czech_utils.parse_month_references",
|
||||||
|
"captured_at": "2026-05-06",
|
||||||
|
"input": { "text": "...", "default_year": 2026 },
|
||||||
|
"output": { "months": ["2025-11", "2025-12", "2026-01"] }
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
`captured_at` is date-only so same-day re-runs produce byte-identical files.
|
||||||
|
|
||||||
|
### Amount type envelope
|
||||||
|
|
||||||
|
Four fields carry a type envelope to distinguish Python `int` / `float` / `None`:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{"type": "int", "value": 750}
|
||||||
|
{"type": "float", "value": 750.0}
|
||||||
|
{"type": "string", "value": "..."}
|
||||||
|
{"type": "none"}
|
||||||
|
```
|
||||||
|
|
||||||
|
Fields that use envelopes: `generate_sync_id.tx.amount`, `parse_czk_amount.val`,
|
||||||
|
`format_date.val`, `infer_transaction_details.tx.date`.
|
||||||
|
|
||||||
|
### Reconcile member format
|
||||||
|
|
||||||
|
Reconcile input members use a named dict to allow consistent PII scrubbing:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{"name": "Member_d035d9f9", "tier": "A", "fees": {"2026-01": [750, 3]}}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Running the parity tests
|
||||||
|
|
||||||
|
```bash
|
||||||
|
make go-parity # run all parity tests
|
||||||
|
make go-test-all # unit tests + parity tests
|
||||||
|
```
|
||||||
|
|
||||||
|
Or directly:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd go && go test -tags=parity ./tests/parity/...
|
||||||
|
cd go && go test -tags=parity -v -run TestReconcileParity ./tests/parity/reconcile/
|
||||||
|
```
|
||||||
|
|
||||||
|
## Refresh workflow
|
||||||
|
|
||||||
|
Regenerate the entire corpus from the live Python implementation:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
make capture-fixtures
|
||||||
|
git diff go/tests/fixtures/ # review changes before committing
|
||||||
|
```
|
||||||
|
|
||||||
|
To refresh a single function:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
PYTHONPATH=scripts:. python3 scripts/capture_fixtures.py --func normalize --all \
|
||||||
|
| while IFS= read -r line; do
|
||||||
|
id=$(echo "$line" | python3 -c "import sys,json; print(json.load(sys.stdin)['case'])")
|
||||||
|
echo "$line" | python3 scripts/scrub_fixtures.py \
|
||||||
|
> go/tests/fixtures/pure/normalize/${id}.json
|
||||||
|
done
|
||||||
|
```
|
||||||
|
|
||||||
|
## When to refresh
|
||||||
|
|
||||||
|
- A ported function is intentionally changed to match updated Python behaviour.
|
||||||
|
- A new Czech declension or fee tier is added to the Python implementation.
|
||||||
|
- A new reconcile code path needs fixture coverage.
|
||||||
|
|
||||||
|
**Do not refresh to silence a failing parity test** without first confirming that
|
||||||
|
the Python behaviour is the correct reference. A parity failure means either the
|
||||||
|
Go port diverges or the Python implementation changed — diagnose before regenerating.
|
||||||
|
|
||||||
|
## PII scrubbing audit
|
||||||
|
|
||||||
|
No real member names should appear in committed fixtures. Before committing any
|
||||||
|
regenerated fixtures, verify with:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Replace with names from the real roster to check:
|
||||||
|
git ls-files go/tests/fixtures | xargs grep -l "Real Name Here" | head
|
||||||
|
```
|
||||||
|
|
||||||
|
The scrubber applies deterministic SHA-256 pseudonyms (`Member_<8hex>`) to all
|
||||||
|
PII fields. `match_members` and `infer_transaction_details` fixtures use a
|
||||||
|
synthetic roster of fictional names and are exempt from field-key scrubbing;
|
||||||
|
verify that no real roster names appear in their `member_names` arrays.
|
||||||
|
|
||||||
|
## Adding a new fixture
|
||||||
|
|
||||||
|
1. Add a seed to `scripts/_fixture_seeds.py` under `SEEDS[("func_name", "case_id")]`.
|
||||||
|
2. Add `In`/`Out` struct fields to `go/tests/parity/parityio.go` if the function
|
||||||
|
is new.
|
||||||
|
3. Run the single-file capture recipe above and review the diff.
|
||||||
|
4. The parity test picks up new fixtures automatically — no test code changes needed
|
||||||
|
(unless the function itself is new).
|
||||||
15
go/tests/fixtures/pure/build_name_variants/common_diacritics.json
vendored
Normal file
15
go/tests/fixtures/pure/build_name_variants/common_diacritics.json
vendored
Normal file
@@ -0,0 +1,15 @@
|
|||||||
|
{
|
||||||
|
"case": "common_diacritics",
|
||||||
|
"func": "scripts.match_payments._build_name_variants",
|
||||||
|
"captured_at": "2026-05-06",
|
||||||
|
"input": {
|
||||||
|
"full_name": "Alžběta Testovická"
|
||||||
|
},
|
||||||
|
"output": {
|
||||||
|
"variants": [
|
||||||
|
"alzbeta testovicka",
|
||||||
|
"testovicka",
|
||||||
|
"alzbeta"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
15
go/tests/fixtures/pure/build_name_variants/full_name_no_nick.json
vendored
Normal file
15
go/tests/fixtures/pure/build_name_variants/full_name_no_nick.json
vendored
Normal file
@@ -0,0 +1,15 @@
|
|||||||
|
{
|
||||||
|
"case": "full_name_no_nick",
|
||||||
|
"func": "scripts.match_payments._build_name_variants",
|
||||||
|
"captured_at": "2026-05-06",
|
||||||
|
"input": {
|
||||||
|
"full_name": "Jan Novák"
|
||||||
|
},
|
||||||
|
"output": {
|
||||||
|
"variants": [
|
||||||
|
"jan novak",
|
||||||
|
"novak",
|
||||||
|
"jan"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
11
go/tests/fixtures/pure/build_name_variants/short_name_filtered.json
vendored
Normal file
11
go/tests/fixtures/pure/build_name_variants/short_name_filtered.json
vendored
Normal file
@@ -0,0 +1,11 @@
|
|||||||
|
{
|
||||||
|
"case": "short_name_filtered",
|
||||||
|
"func": "scripts.match_payments._build_name_variants",
|
||||||
|
"captured_at": "2026-05-06",
|
||||||
|
"input": {
|
||||||
|
"full_name": "Jo"
|
||||||
|
},
|
||||||
|
"output": {
|
||||||
|
"variants": []
|
||||||
|
}
|
||||||
|
}
|
||||||
13
go/tests/fixtures/pure/build_name_variants/single_word.json
vendored
Normal file
13
go/tests/fixtures/pure/build_name_variants/single_word.json
vendored
Normal file
@@ -0,0 +1,13 @@
|
|||||||
|
{
|
||||||
|
"case": "single_word",
|
||||||
|
"func": "scripts.match_payments._build_name_variants",
|
||||||
|
"captured_at": "2026-05-06",
|
||||||
|
"input": {
|
||||||
|
"full_name": "Jáchym"
|
||||||
|
},
|
||||||
|
"output": {
|
||||||
|
"variants": [
|
||||||
|
"jachym"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
16
go/tests/fixtures/pure/build_name_variants/three_word_name.json
vendored
Normal file
16
go/tests/fixtures/pure/build_name_variants/three_word_name.json
vendored
Normal file
@@ -0,0 +1,16 @@
|
|||||||
|
{
|
||||||
|
"case": "three_word_name",
|
||||||
|
"func": "scripts.match_payments._build_name_variants",
|
||||||
|
"captured_at": "2026-05-06",
|
||||||
|
"input": {
|
||||||
|
"full_name": "Jan Tomášek (Honza)"
|
||||||
|
},
|
||||||
|
"output": {
|
||||||
|
"variants": [
|
||||||
|
"jan tomasek",
|
||||||
|
"honza",
|
||||||
|
"tomasek",
|
||||||
|
"jan"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
16
go/tests/fixtures/pure/build_name_variants/with_nickname.json
vendored
Normal file
16
go/tests/fixtures/pure/build_name_variants/with_nickname.json
vendored
Normal file
@@ -0,0 +1,16 @@
|
|||||||
|
{
|
||||||
|
"case": "with_nickname",
|
||||||
|
"func": "scripts.match_payments._build_name_variants",
|
||||||
|
"captured_at": "2026-05-06",
|
||||||
|
"input": {
|
||||||
|
"full_name": "František Vrbík (Štrúdl)"
|
||||||
|
},
|
||||||
|
"output": {
|
||||||
|
"variants": [
|
||||||
|
"frantisek vrbik",
|
||||||
|
"strudl",
|
||||||
|
"vrbik",
|
||||||
|
"frantisek"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
12
go/tests/fixtures/pure/calculate_fee/one_session.json
vendored
Normal file
12
go/tests/fixtures/pure/calculate_fee/one_session.json
vendored
Normal file
@@ -0,0 +1,12 @@
|
|||||||
|
{
|
||||||
|
"case": "one_session",
|
||||||
|
"func": "scripts.attendance.calculate_fee",
|
||||||
|
"captured_at": "2026-05-06",
|
||||||
|
"input": {
|
||||||
|
"attendance_count": 1,
|
||||||
|
"month_key": "2026-01"
|
||||||
|
},
|
||||||
|
"output": {
|
||||||
|
"fee": 200
|
||||||
|
}
|
||||||
|
}
|
||||||
12
go/tests/fixtures/pure/calculate_fee/three_sessions_known_rate.json
vendored
Normal file
12
go/tests/fixtures/pure/calculate_fee/three_sessions_known_rate.json
vendored
Normal file
@@ -0,0 +1,12 @@
|
|||||||
|
{
|
||||||
|
"case": "three_sessions_known_rate",
|
||||||
|
"func": "scripts.attendance.calculate_fee",
|
||||||
|
"captured_at": "2026-05-06",
|
||||||
|
"input": {
|
||||||
|
"attendance_count": 3,
|
||||||
|
"month_key": "2026-02"
|
||||||
|
},
|
||||||
|
"output": {
|
||||||
|
"fee": 750
|
||||||
|
}
|
||||||
|
}
|
||||||
12
go/tests/fixtures/pure/calculate_fee/two_sessions_default_fallback.json
vendored
Normal file
12
go/tests/fixtures/pure/calculate_fee/two_sessions_default_fallback.json
vendored
Normal file
@@ -0,0 +1,12 @@
|
|||||||
|
{
|
||||||
|
"case": "two_sessions_default_fallback",
|
||||||
|
"func": "scripts.attendance.calculate_fee",
|
||||||
|
"captured_at": "2026-05-06",
|
||||||
|
"input": {
|
||||||
|
"attendance_count": 2,
|
||||||
|
"month_key": "2099-01"
|
||||||
|
},
|
||||||
|
"output": {
|
||||||
|
"fee": 700
|
||||||
|
}
|
||||||
|
}
|
||||||
12
go/tests/fixtures/pure/calculate_fee/two_sessions_known_rate.json
vendored
Normal file
12
go/tests/fixtures/pure/calculate_fee/two_sessions_known_rate.json
vendored
Normal file
@@ -0,0 +1,12 @@
|
|||||||
|
{
|
||||||
|
"case": "two_sessions_known_rate",
|
||||||
|
"func": "scripts.attendance.calculate_fee",
|
||||||
|
"captured_at": "2026-05-06",
|
||||||
|
"input": {
|
||||||
|
"attendance_count": 2,
|
||||||
|
"month_key": "2026-01"
|
||||||
|
},
|
||||||
|
"output": {
|
||||||
|
"fee": 750
|
||||||
|
}
|
||||||
|
}
|
||||||
12
go/tests/fixtures/pure/calculate_fee/two_sessions_reduced_march.json
vendored
Normal file
12
go/tests/fixtures/pure/calculate_fee/two_sessions_reduced_march.json
vendored
Normal file
@@ -0,0 +1,12 @@
|
|||||||
|
{
|
||||||
|
"case": "two_sessions_reduced_march",
|
||||||
|
"func": "scripts.attendance.calculate_fee",
|
||||||
|
"captured_at": "2026-05-06",
|
||||||
|
"input": {
|
||||||
|
"attendance_count": 2,
|
||||||
|
"month_key": "2026-03"
|
||||||
|
},
|
||||||
|
"output": {
|
||||||
|
"fee": 350
|
||||||
|
}
|
||||||
|
}
|
||||||
12
go/tests/fixtures/pure/calculate_fee/zero_sessions.json
vendored
Normal file
12
go/tests/fixtures/pure/calculate_fee/zero_sessions.json
vendored
Normal file
@@ -0,0 +1,12 @@
|
|||||||
|
{
|
||||||
|
"case": "zero_sessions",
|
||||||
|
"func": "scripts.attendance.calculate_fee",
|
||||||
|
"captured_at": "2026-05-06",
|
||||||
|
"input": {
|
||||||
|
"attendance_count": 0,
|
||||||
|
"month_key": "2026-01"
|
||||||
|
},
|
||||||
|
"output": {
|
||||||
|
"fee": 0
|
||||||
|
}
|
||||||
|
}
|
||||||
13
go/tests/fixtures/pure/calculate_junior_fee/one_session_unknown.json
vendored
Normal file
13
go/tests/fixtures/pure/calculate_junior_fee/one_session_unknown.json
vendored
Normal file
@@ -0,0 +1,13 @@
|
|||||||
|
{
|
||||||
|
"case": "one_session_unknown",
|
||||||
|
"func": "scripts.attendance.calculate_junior_fee",
|
||||||
|
"captured_at": "2026-05-06",
|
||||||
|
"input": {
|
||||||
|
"attendance_count": 1,
|
||||||
|
"month_key": "2026-01"
|
||||||
|
},
|
||||||
|
"output": {
|
||||||
|
"value": 0,
|
||||||
|
"unknown": true
|
||||||
|
}
|
||||||
|
}
|
||||||
13
go/tests/fixtures/pure/calculate_junior_fee/two_sessions_default.json
vendored
Normal file
13
go/tests/fixtures/pure/calculate_junior_fee/two_sessions_default.json
vendored
Normal file
@@ -0,0 +1,13 @@
|
|||||||
|
{
|
||||||
|
"case": "two_sessions_default",
|
||||||
|
"func": "scripts.attendance.calculate_junior_fee",
|
||||||
|
"captured_at": "2026-05-06",
|
||||||
|
"input": {
|
||||||
|
"attendance_count": 2,
|
||||||
|
"month_key": "2026-01"
|
||||||
|
},
|
||||||
|
"output": {
|
||||||
|
"value": 500,
|
||||||
|
"unknown": false
|
||||||
|
}
|
||||||
|
}
|
||||||
13
go/tests/fixtures/pure/calculate_junior_fee/two_sessions_default_fallback.json
vendored
Normal file
13
go/tests/fixtures/pure/calculate_junior_fee/two_sessions_default_fallback.json
vendored
Normal file
@@ -0,0 +1,13 @@
|
|||||||
|
{
|
||||||
|
"case": "two_sessions_default_fallback",
|
||||||
|
"func": "scripts.attendance.calculate_junior_fee",
|
||||||
|
"captured_at": "2026-05-06",
|
||||||
|
"input": {
|
||||||
|
"attendance_count": 2,
|
||||||
|
"month_key": "2099-06"
|
||||||
|
},
|
||||||
|
"output": {
|
||||||
|
"value": 500,
|
||||||
|
"unknown": false
|
||||||
|
}
|
||||||
|
}
|
||||||
13
go/tests/fixtures/pure/calculate_junior_fee/two_sessions_reduced_march.json
vendored
Normal file
13
go/tests/fixtures/pure/calculate_junior_fee/two_sessions_reduced_march.json
vendored
Normal file
@@ -0,0 +1,13 @@
|
|||||||
|
{
|
||||||
|
"case": "two_sessions_reduced_march",
|
||||||
|
"func": "scripts.attendance.calculate_junior_fee",
|
||||||
|
"captured_at": "2026-05-06",
|
||||||
|
"input": {
|
||||||
|
"attendance_count": 2,
|
||||||
|
"month_key": "2026-03"
|
||||||
|
},
|
||||||
|
"output": {
|
||||||
|
"value": 250,
|
||||||
|
"unknown": false
|
||||||
|
}
|
||||||
|
}
|
||||||
13
go/tests/fixtures/pure/calculate_junior_fee/two_sessions_reduced_sep.json
vendored
Normal file
13
go/tests/fixtures/pure/calculate_junior_fee/two_sessions_reduced_sep.json
vendored
Normal file
@@ -0,0 +1,13 @@
|
|||||||
|
{
|
||||||
|
"case": "two_sessions_reduced_sep",
|
||||||
|
"func": "scripts.attendance.calculate_junior_fee",
|
||||||
|
"captured_at": "2026-05-06",
|
||||||
|
"input": {
|
||||||
|
"attendance_count": 2,
|
||||||
|
"month_key": "2025-09"
|
||||||
|
},
|
||||||
|
"output": {
|
||||||
|
"value": 250,
|
||||||
|
"unknown": false
|
||||||
|
}
|
||||||
|
}
|
||||||
13
go/tests/fixtures/pure/calculate_junior_fee/zero_sessions.json
vendored
Normal file
13
go/tests/fixtures/pure/calculate_junior_fee/zero_sessions.json
vendored
Normal file
@@ -0,0 +1,13 @@
|
|||||||
|
{
|
||||||
|
"case": "zero_sessions",
|
||||||
|
"func": "scripts.attendance.calculate_junior_fee",
|
||||||
|
"captured_at": "2026-05-06",
|
||||||
|
"input": {
|
||||||
|
"attendance_count": 0,
|
||||||
|
"month_key": "2026-01"
|
||||||
|
},
|
||||||
|
"output": {
|
||||||
|
"value": 0,
|
||||||
|
"unknown": false
|
||||||
|
}
|
||||||
|
}
|
||||||
14
go/tests/fixtures/pure/format_date/empty_string.json
vendored
Normal file
14
go/tests/fixtures/pure/format_date/empty_string.json
vendored
Normal file
@@ -0,0 +1,14 @@
|
|||||||
|
{
|
||||||
|
"case": "empty_string",
|
||||||
|
"func": "scripts.match_payments.format_date",
|
||||||
|
"captured_at": "2026-05-06",
|
||||||
|
"input": {
|
||||||
|
"val": {
|
||||||
|
"type": "string",
|
||||||
|
"value": ""
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"output": {
|
||||||
|
"date": ""
|
||||||
|
}
|
||||||
|
}
|
||||||
13
go/tests/fixtures/pure/format_date/none_value.json
vendored
Normal file
13
go/tests/fixtures/pure/format_date/none_value.json
vendored
Normal file
@@ -0,0 +1,13 @@
|
|||||||
|
{
|
||||||
|
"case": "none_value",
|
||||||
|
"func": "scripts.match_payments.format_date",
|
||||||
|
"captured_at": "2026-05-06",
|
||||||
|
"input": {
|
||||||
|
"val": {
|
||||||
|
"type": "none"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"output": {
|
||||||
|
"date": ""
|
||||||
|
}
|
||||||
|
}
|
||||||
14
go/tests/fixtures/pure/format_date/serial_float.json
vendored
Normal file
14
go/tests/fixtures/pure/format_date/serial_float.json
vendored
Normal file
@@ -0,0 +1,14 @@
|
|||||||
|
{
|
||||||
|
"case": "serial_float",
|
||||||
|
"func": "scripts.match_payments.format_date",
|
||||||
|
"captured_at": "2026-05-06",
|
||||||
|
"input": {
|
||||||
|
"val": {
|
||||||
|
"type": "float",
|
||||||
|
"value": 46027.5
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"output": {
|
||||||
|
"date": "2026-01-05"
|
||||||
|
}
|
||||||
|
}
|
||||||
14
go/tests/fixtures/pure/format_date/serial_float_exact.json
vendored
Normal file
14
go/tests/fixtures/pure/format_date/serial_float_exact.json
vendored
Normal file
@@ -0,0 +1,14 @@
|
|||||||
|
{
|
||||||
|
"case": "serial_float_exact",
|
||||||
|
"func": "scripts.match_payments.format_date",
|
||||||
|
"captured_at": "2026-05-06",
|
||||||
|
"input": {
|
||||||
|
"val": {
|
||||||
|
"type": "float",
|
||||||
|
"value": 45957.0
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"output": {
|
||||||
|
"date": "2025-10-27"
|
||||||
|
}
|
||||||
|
}
|
||||||
14
go/tests/fixtures/pure/format_date/serial_int.json
vendored
Normal file
14
go/tests/fixtures/pure/format_date/serial_int.json
vendored
Normal file
@@ -0,0 +1,14 @@
|
|||||||
|
{
|
||||||
|
"case": "serial_int",
|
||||||
|
"func": "scripts.match_payments.format_date",
|
||||||
|
"captured_at": "2026-05-06",
|
||||||
|
"input": {
|
||||||
|
"val": {
|
||||||
|
"type": "int",
|
||||||
|
"value": 46027
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"output": {
|
||||||
|
"date": "2026-01-05"
|
||||||
|
}
|
||||||
|
}
|
||||||
14
go/tests/fixtures/pure/format_date/string_iso.json
vendored
Normal file
14
go/tests/fixtures/pure/format_date/string_iso.json
vendored
Normal file
@@ -0,0 +1,14 @@
|
|||||||
|
{
|
||||||
|
"case": "string_iso",
|
||||||
|
"func": "scripts.match_payments.format_date",
|
||||||
|
"captured_at": "2026-05-06",
|
||||||
|
"input": {
|
||||||
|
"val": {
|
||||||
|
"type": "string",
|
||||||
|
"value": "2026-01-15"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"output": {
|
||||||
|
"date": "2026-01-15"
|
||||||
|
}
|
||||||
|
}
|
||||||
14
go/tests/fixtures/pure/format_date/string_non_iso.json
vendored
Normal file
14
go/tests/fixtures/pure/format_date/string_non_iso.json
vendored
Normal file
@@ -0,0 +1,14 @@
|
|||||||
|
{
|
||||||
|
"case": "string_non_iso",
|
||||||
|
"func": "scripts.match_payments.format_date",
|
||||||
|
"captured_at": "2026-05-06",
|
||||||
|
"input": {
|
||||||
|
"val": {
|
||||||
|
"type": "string",
|
||||||
|
"value": "garbage"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"output": {
|
||||||
|
"date": "garbage"
|
||||||
|
}
|
||||||
|
}
|
||||||
22
go/tests/fixtures/pure/generate_sync_id/empty_fields.json
vendored
Normal file
22
go/tests/fixtures/pure/generate_sync_id/empty_fields.json
vendored
Normal file
@@ -0,0 +1,22 @@
|
|||||||
|
{
|
||||||
|
"case": "empty_fields",
|
||||||
|
"func": "scripts.sync_fio_to_sheets.generate_sync_id",
|
||||||
|
"captured_at": "2026-05-06",
|
||||||
|
"input": {
|
||||||
|
"tx": {
|
||||||
|
"date": "2026-03-01",
|
||||||
|
"amount": {
|
||||||
|
"type": "float",
|
||||||
|
"value": 0.0
|
||||||
|
},
|
||||||
|
"currency": "CZK",
|
||||||
|
"sender": "",
|
||||||
|
"vs": "",
|
||||||
|
"message": "",
|
||||||
|
"bank_id": ""
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"output": {
|
||||||
|
"sync_id": "80d5f2762dbe807adde8dab64c3f3f00936ceafc75d4ceba232b08c09bb71c60"
|
||||||
|
}
|
||||||
|
}
|
||||||
22
go/tests/fixtures/pure/generate_sync_id/integer_amount.json
vendored
Normal file
22
go/tests/fixtures/pure/generate_sync_id/integer_amount.json
vendored
Normal file
@@ -0,0 +1,22 @@
|
|||||||
|
{
|
||||||
|
"case": "integer_amount",
|
||||||
|
"func": "scripts.sync_fio_to_sheets.generate_sync_id",
|
||||||
|
"captured_at": "2026-05-06",
|
||||||
|
"input": {
|
||||||
|
"tx": {
|
||||||
|
"date": "2026-01-15",
|
||||||
|
"amount": {
|
||||||
|
"type": "int",
|
||||||
|
"value": 750
|
||||||
|
},
|
||||||
|
"currency": "CZK",
|
||||||
|
"sender": "Member_9b16314c",
|
||||||
|
"vs": "864722",
|
||||||
|
"message": "pausal leden",
|
||||||
|
"bank_id": "983770300"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"output": {
|
||||||
|
"sync_id": "155e983a0a3a11210e19728c427395f6681ee5d2a0ef3b60438e6efeaf3775df"
|
||||||
|
}
|
||||||
|
}
|
||||||
22
go/tests/fixtures/pure/generate_sync_id/large_amount.json
vendored
Normal file
22
go/tests/fixtures/pure/generate_sync_id/large_amount.json
vendored
Normal file
@@ -0,0 +1,22 @@
|
|||||||
|
{
|
||||||
|
"case": "large_amount",
|
||||||
|
"func": "scripts.sync_fio_to_sheets.generate_sync_id",
|
||||||
|
"captured_at": "2026-05-06",
|
||||||
|
"input": {
|
||||||
|
"tx": {
|
||||||
|
"date": "2025-10-05",
|
||||||
|
"amount": {
|
||||||
|
"type": "float",
|
||||||
|
"value": 2100.0
|
||||||
|
},
|
||||||
|
"currency": "CZK",
|
||||||
|
"sender": "Member_bd5eb92a",
|
||||||
|
"vs": "110515",
|
||||||
|
"message": "FUJ treninky",
|
||||||
|
"bank_id": "609470745"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"output": {
|
||||||
|
"sync_id": "639d98f8ab8e6954b7e4d31508936cc4366ee0281eebc860338585cdeda43ae3"
|
||||||
|
}
|
||||||
|
}
|
||||||
21
go/tests/fixtures/pure/generate_sync_id/missing_currency.json
vendored
Normal file
21
go/tests/fixtures/pure/generate_sync_id/missing_currency.json
vendored
Normal file
@@ -0,0 +1,21 @@
|
|||||||
|
{
|
||||||
|
"case": "missing_currency",
|
||||||
|
"func": "scripts.sync_fio_to_sheets.generate_sync_id",
|
||||||
|
"captured_at": "2026-05-06",
|
||||||
|
"input": {
|
||||||
|
"tx": {
|
||||||
|
"date": "2026-02-01",
|
||||||
|
"amount": {
|
||||||
|
"type": "float",
|
||||||
|
"value": 500.0
|
||||||
|
},
|
||||||
|
"sender": "Member_32a79b03",
|
||||||
|
"vs": "720261",
|
||||||
|
"message": "trenink",
|
||||||
|
"bank_id": "072657565"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"output": {
|
||||||
|
"sync_id": "8bd2cc2c2e6b376ad2d2501f72ee5d987fdca37662c4be0b9bb5345dcb28553d"
|
||||||
|
}
|
||||||
|
}
|
||||||
22
go/tests/fixtures/pure/generate_sync_id/typical_float_amount.json
vendored
Normal file
22
go/tests/fixtures/pure/generate_sync_id/typical_float_amount.json
vendored
Normal file
@@ -0,0 +1,22 @@
|
|||||||
|
{
|
||||||
|
"case": "typical_float_amount",
|
||||||
|
"func": "scripts.sync_fio_to_sheets.generate_sync_id",
|
||||||
|
"captured_at": "2026-05-06",
|
||||||
|
"input": {
|
||||||
|
"tx": {
|
||||||
|
"date": "2026-01-15",
|
||||||
|
"amount": {
|
||||||
|
"type": "float",
|
||||||
|
"value": 750.0
|
||||||
|
},
|
||||||
|
"currency": "CZK",
|
||||||
|
"sender": "Member_9b16314c",
|
||||||
|
"vs": "864722",
|
||||||
|
"message": "pausal leden",
|
||||||
|
"bank_id": "983770300"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"output": {
|
||||||
|
"sync_id": "155e983a0a3a11210e19728c427395f6681ee5d2a0ef3b60438e6efeaf3775df"
|
||||||
|
}
|
||||||
|
}
|
||||||
36
go/tests/fixtures/pure/infer_transaction_details/member_in_message.json
vendored
Normal file
36
go/tests/fixtures/pure/infer_transaction_details/member_in_message.json
vendored
Normal file
@@ -0,0 +1,36 @@
|
|||||||
|
{
|
||||||
|
"case": "member_in_message",
|
||||||
|
"func": "scripts.match_payments.infer_transaction_details",
|
||||||
|
"captured_at": "2026-05-06",
|
||||||
|
"input": {
|
||||||
|
"tx": {
|
||||||
|
"sender": "Test Payer",
|
||||||
|
"message": "alzbeta testovicka leden 2026",
|
||||||
|
"user_id": "",
|
||||||
|
"date": {
|
||||||
|
"type": "string",
|
||||||
|
"value": "2026-01-15"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"member_names": [
|
||||||
|
"Alžběta Testovická",
|
||||||
|
"Tomáš Fiktivný (Tov)",
|
||||||
|
"Pavel Smutný (Štrúdl)",
|
||||||
|
"Jana Nováková",
|
||||||
|
"Adam Novák"
|
||||||
|
],
|
||||||
|
"default_year": 2026
|
||||||
|
},
|
||||||
|
"output": {
|
||||||
|
"matches": [
|
||||||
|
{
|
||||||
|
"name": "Alžběta Testovická",
|
||||||
|
"confidence": "auto"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"months": [
|
||||||
|
"2026-01"
|
||||||
|
],
|
||||||
|
"search_text": "Test Payer alzbeta testovicka leden 2026 "
|
||||||
|
}
|
||||||
|
}
|
||||||
36
go/tests/fixtures/pure/infer_transaction_details/member_in_sender.json
vendored
Normal file
36
go/tests/fixtures/pure/infer_transaction_details/member_in_sender.json
vendored
Normal file
@@ -0,0 +1,36 @@
|
|||||||
|
{
|
||||||
|
"case": "member_in_sender",
|
||||||
|
"func": "scripts.match_payments.infer_transaction_details",
|
||||||
|
"captured_at": "2026-05-06",
|
||||||
|
"input": {
|
||||||
|
"tx": {
|
||||||
|
"sender": "Tomáš Fiktivný",
|
||||||
|
"message": "FUJ trenink",
|
||||||
|
"user_id": "",
|
||||||
|
"date": {
|
||||||
|
"type": "string",
|
||||||
|
"value": "2026-02-01"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"member_names": [
|
||||||
|
"Alžběta Testovická",
|
||||||
|
"Tomáš Fiktivný (Tov)",
|
||||||
|
"Pavel Smutný (Štrúdl)",
|
||||||
|
"Jana Nováková",
|
||||||
|
"Adam Novák"
|
||||||
|
],
|
||||||
|
"default_year": 2026
|
||||||
|
},
|
||||||
|
"output": {
|
||||||
|
"matches": [
|
||||||
|
{
|
||||||
|
"name": "Tomáš Fiktivný (Tov)",
|
||||||
|
"confidence": "auto"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"months": [
|
||||||
|
"2026-02"
|
||||||
|
],
|
||||||
|
"search_text": "Tomáš Fiktivný FUJ trenink "
|
||||||
|
}
|
||||||
|
}
|
||||||
36
go/tests/fixtures/pure/infer_transaction_details/month_fallback_from_date.json
vendored
Normal file
36
go/tests/fixtures/pure/infer_transaction_details/month_fallback_from_date.json
vendored
Normal file
@@ -0,0 +1,36 @@
|
|||||||
|
{
|
||||||
|
"case": "month_fallback_from_date",
|
||||||
|
"func": "scripts.match_payments.infer_transaction_details",
|
||||||
|
"captured_at": "2026-05-06",
|
||||||
|
"input": {
|
||||||
|
"tx": {
|
||||||
|
"sender": "Alžběta Testovická",
|
||||||
|
"message": "platba",
|
||||||
|
"user_id": "",
|
||||||
|
"date": {
|
||||||
|
"type": "string",
|
||||||
|
"value": "2026-03-15"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"member_names": [
|
||||||
|
"Alžběta Testovická",
|
||||||
|
"Tomáš Fiktivný (Tov)",
|
||||||
|
"Pavel Smutný (Štrúdl)",
|
||||||
|
"Jana Nováková",
|
||||||
|
"Adam Novák"
|
||||||
|
],
|
||||||
|
"default_year": 2026
|
||||||
|
},
|
||||||
|
"output": {
|
||||||
|
"matches": [
|
||||||
|
{
|
||||||
|
"name": "Alžběta Testovická",
|
||||||
|
"confidence": "auto"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"months": [
|
||||||
|
"2026-03"
|
||||||
|
],
|
||||||
|
"search_text": "Alžběta Testovická platba "
|
||||||
|
}
|
||||||
|
}
|
||||||
28
go/tests/fixtures/pure/infer_transaction_details/no_member_no_month.json
vendored
Normal file
28
go/tests/fixtures/pure/infer_transaction_details/no_member_no_month.json
vendored
Normal file
@@ -0,0 +1,28 @@
|
|||||||
|
{
|
||||||
|
"case": "no_member_no_month",
|
||||||
|
"func": "scripts.match_payments.infer_transaction_details",
|
||||||
|
"captured_at": "2026-05-06",
|
||||||
|
"input": {
|
||||||
|
"tx": {
|
||||||
|
"sender": "Unknown Person",
|
||||||
|
"message": "random text",
|
||||||
|
"user_id": "",
|
||||||
|
"date": {
|
||||||
|
"type": "none"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"member_names": [
|
||||||
|
"Alžběta Testovická",
|
||||||
|
"Tomáš Fiktivný (Tov)",
|
||||||
|
"Pavel Smutný (Štrúdl)",
|
||||||
|
"Jana Nováková",
|
||||||
|
"Adam Novák"
|
||||||
|
],
|
||||||
|
"default_year": 2026
|
||||||
|
},
|
||||||
|
"output": {
|
||||||
|
"matches": [],
|
||||||
|
"months": [],
|
||||||
|
"search_text": "Unknown Person random text "
|
||||||
|
}
|
||||||
|
}
|
||||||
36
go/tests/fixtures/pure/infer_transaction_details/serial_date.json
vendored
Normal file
36
go/tests/fixtures/pure/infer_transaction_details/serial_date.json
vendored
Normal file
@@ -0,0 +1,36 @@
|
|||||||
|
{
|
||||||
|
"case": "serial_date",
|
||||||
|
"func": "scripts.match_payments.infer_transaction_details",
|
||||||
|
"captured_at": "2026-05-06",
|
||||||
|
"input": {
|
||||||
|
"tx": {
|
||||||
|
"sender": "Jana Nováková",
|
||||||
|
"message": "leden",
|
||||||
|
"user_id": "",
|
||||||
|
"date": {
|
||||||
|
"type": "float",
|
||||||
|
"value": 46027.0
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"member_names": [
|
||||||
|
"Alžběta Testovická",
|
||||||
|
"Tomáš Fiktivný (Tov)",
|
||||||
|
"Pavel Smutný (Štrúdl)",
|
||||||
|
"Jana Nováková",
|
||||||
|
"Adam Novák"
|
||||||
|
],
|
||||||
|
"default_year": 2026
|
||||||
|
},
|
||||||
|
"output": {
|
||||||
|
"matches": [
|
||||||
|
{
|
||||||
|
"name": "Jana Nováková",
|
||||||
|
"confidence": "auto"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"months": [
|
||||||
|
"2026-01"
|
||||||
|
],
|
||||||
|
"search_text": "Jana Nováková leden "
|
||||||
|
}
|
||||||
|
}
|
||||||
18
go/tests/fixtures/pure/match_members/common_surname_no_match.json
vendored
Normal file
18
go/tests/fixtures/pure/match_members/common_surname_no_match.json
vendored
Normal file
@@ -0,0 +1,18 @@
|
|||||||
|
{
|
||||||
|
"case": "common_surname_no_match",
|
||||||
|
"func": "scripts.match_payments.match_members",
|
||||||
|
"captured_at": "2026-05-06",
|
||||||
|
"input": {
|
||||||
|
"text": "novak leden",
|
||||||
|
"member_names": [
|
||||||
|
"Alžběta Testovická",
|
||||||
|
"Tomáš Fiktivný (Tov)",
|
||||||
|
"Pavel Smutný (Štrúdl)",
|
||||||
|
"Jana Nováková",
|
||||||
|
"Adam Novák"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"output": {
|
||||||
|
"matches": []
|
||||||
|
}
|
||||||
|
}
|
||||||
23
go/tests/fixtures/pure/match_members/exact_full_name.json
vendored
Normal file
23
go/tests/fixtures/pure/match_members/exact_full_name.json
vendored
Normal file
@@ -0,0 +1,23 @@
|
|||||||
|
{
|
||||||
|
"case": "exact_full_name",
|
||||||
|
"func": "scripts.match_payments.match_members",
|
||||||
|
"captured_at": "2026-05-06",
|
||||||
|
"input": {
|
||||||
|
"text": "platba od alzbeta testovicka leden",
|
||||||
|
"member_names": [
|
||||||
|
"Alžběta Testovická",
|
||||||
|
"Tomáš Fiktivný (Tov)",
|
||||||
|
"Pavel Smutný (Štrúdl)",
|
||||||
|
"Jana Nováková",
|
||||||
|
"Adam Novák"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"output": {
|
||||||
|
"matches": [
|
||||||
|
{
|
||||||
|
"name": "Alžběta Testovická",
|
||||||
|
"confidence": "auto"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
23
go/tests/fixtures/pure/match_members/first_and_last.json
vendored
Normal file
23
go/tests/fixtures/pure/match_members/first_and_last.json
vendored
Normal file
@@ -0,0 +1,23 @@
|
|||||||
|
{
|
||||||
|
"case": "first_and_last",
|
||||||
|
"func": "scripts.match_payments.match_members",
|
||||||
|
"captured_at": "2026-05-06",
|
||||||
|
"input": {
|
||||||
|
"text": "jan nový payment tomas fiktivny",
|
||||||
|
"member_names": [
|
||||||
|
"Alžběta Testovická",
|
||||||
|
"Tomáš Fiktivný (Tov)",
|
||||||
|
"Pavel Smutný (Štrúdl)",
|
||||||
|
"Jana Nováková",
|
||||||
|
"Adam Novák"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"output": {
|
||||||
|
"matches": [
|
||||||
|
{
|
||||||
|
"name": "Tomáš Fiktivný (Tov)",
|
||||||
|
"confidence": "auto"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
23
go/tests/fixtures/pure/match_members/nickname_match.json
vendored
Normal file
23
go/tests/fixtures/pure/match_members/nickname_match.json
vendored
Normal file
@@ -0,0 +1,23 @@
|
|||||||
|
{
|
||||||
|
"case": "nickname_match",
|
||||||
|
"func": "scripts.match_payments.match_members",
|
||||||
|
"captured_at": "2026-05-06",
|
||||||
|
"input": {
|
||||||
|
"text": "payment from strudl",
|
||||||
|
"member_names": [
|
||||||
|
"Alžběta Testovická",
|
||||||
|
"Tomáš Fiktivný (Tov)",
|
||||||
|
"Pavel Smutný (Štrúdl)",
|
||||||
|
"Jana Nováková",
|
||||||
|
"Adam Novák"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"output": {
|
||||||
|
"matches": [
|
||||||
|
{
|
||||||
|
"name": "Pavel Smutný (Štrúdl)",
|
||||||
|
"confidence": "auto"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
18
go/tests/fixtures/pure/match_members/no_match.json
vendored
Normal file
18
go/tests/fixtures/pure/match_members/no_match.json
vendored
Normal file
@@ -0,0 +1,18 @@
|
|||||||
|
{
|
||||||
|
"case": "no_match",
|
||||||
|
"func": "scripts.match_payments.match_members",
|
||||||
|
"captured_at": "2026-05-06",
|
||||||
|
"input": {
|
||||||
|
"text": "xyz platba",
|
||||||
|
"member_names": [
|
||||||
|
"Alžběta Testovická",
|
||||||
|
"Tomáš Fiktivný (Tov)",
|
||||||
|
"Pavel Smutný (Štrúdl)",
|
||||||
|
"Jana Nováková",
|
||||||
|
"Adam Novák"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"output": {
|
||||||
|
"matches": []
|
||||||
|
}
|
||||||
|
}
|
||||||
23
go/tests/fixtures/pure/match_members/review_lastname_only.json
vendored
Normal file
23
go/tests/fixtures/pure/match_members/review_lastname_only.json
vendored
Normal file
@@ -0,0 +1,23 @@
|
|||||||
|
{
|
||||||
|
"case": "review_lastname_only",
|
||||||
|
"func": "scripts.match_payments.match_members",
|
||||||
|
"captured_at": "2026-05-06",
|
||||||
|
"input": {
|
||||||
|
"text": "testovicka leden",
|
||||||
|
"member_names": [
|
||||||
|
"Alžběta Testovická",
|
||||||
|
"Tomáš Fiktivný (Tov)",
|
||||||
|
"Pavel Smutný (Štrúdl)",
|
||||||
|
"Jana Nováková",
|
||||||
|
"Adam Novák"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"output": {
|
||||||
|
"matches": [
|
||||||
|
{
|
||||||
|
"name": "Alžběta Testovická",
|
||||||
|
"confidence": "review"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
27
go/tests/fixtures/pure/match_members/two_members_exact.json
vendored
Normal file
27
go/tests/fixtures/pure/match_members/two_members_exact.json
vendored
Normal file
@@ -0,0 +1,27 @@
|
|||||||
|
{
|
||||||
|
"case": "two_members_exact",
|
||||||
|
"func": "scripts.match_payments.match_members",
|
||||||
|
"captured_at": "2026-05-06",
|
||||||
|
"input": {
|
||||||
|
"text": "pavel smutny a alzbeta testovicka",
|
||||||
|
"member_names": [
|
||||||
|
"Alžběta Testovická",
|
||||||
|
"Tomáš Fiktivný (Tov)",
|
||||||
|
"Pavel Smutný (Štrúdl)",
|
||||||
|
"Jana Nováková",
|
||||||
|
"Adam Novák"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"output": {
|
||||||
|
"matches": [
|
||||||
|
{
|
||||||
|
"name": "Alžběta Testovická",
|
||||||
|
"confidence": "auto"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "Pavel Smutný (Štrúdl)",
|
||||||
|
"confidence": "auto"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
11
go/tests/fixtures/pure/normalize/czech_basic.json
vendored
Normal file
11
go/tests/fixtures/pure/normalize/czech_basic.json
vendored
Normal file
@@ -0,0 +1,11 @@
|
|||||||
|
{
|
||||||
|
"case": "czech_basic",
|
||||||
|
"func": "scripts.czech_utils.normalize",
|
||||||
|
"captured_at": "2026-05-06",
|
||||||
|
"input": {
|
||||||
|
"text": "štefan čakrtový"
|
||||||
|
},
|
||||||
|
"output": {
|
||||||
|
"text": "stefan cakrtovy"
|
||||||
|
}
|
||||||
|
}
|
||||||
11
go/tests/fixtures/pure/normalize/czech_full_set.json
vendored
Normal file
11
go/tests/fixtures/pure/normalize/czech_full_set.json
vendored
Normal file
@@ -0,0 +1,11 @@
|
|||||||
|
{
|
||||||
|
"case": "czech_full_set",
|
||||||
|
"func": "scripts.czech_utils.normalize",
|
||||||
|
"captured_at": "2026-05-06",
|
||||||
|
"input": {
|
||||||
|
"text": "áčďéěíňóřšťůúýžÁČĎÉĚÍŇÓŘŠŤŮÚÝŽ"
|
||||||
|
},
|
||||||
|
"output": {
|
||||||
|
"text": "acdeeinorstuuyzacdeeinorstuuyz"
|
||||||
|
}
|
||||||
|
}
|
||||||
11
go/tests/fixtures/pure/normalize/digits_symbols.json
vendored
Normal file
11
go/tests/fixtures/pure/normalize/digits_symbols.json
vendored
Normal file
@@ -0,0 +1,11 @@
|
|||||||
|
{
|
||||||
|
"case": "digits_symbols",
|
||||||
|
"func": "scripts.czech_utils.normalize",
|
||||||
|
"captured_at": "2026-05-06",
|
||||||
|
"input": {
|
||||||
|
"text": "FUJ2026! +3"
|
||||||
|
},
|
||||||
|
"output": {
|
||||||
|
"text": "fuj2026! +3"
|
||||||
|
}
|
||||||
|
}
|
||||||
11
go/tests/fixtures/pure/normalize/empty_string.json
vendored
Normal file
11
go/tests/fixtures/pure/normalize/empty_string.json
vendored
Normal file
@@ -0,0 +1,11 @@
|
|||||||
|
{
|
||||||
|
"case": "empty_string",
|
||||||
|
"func": "scripts.czech_utils.normalize",
|
||||||
|
"captured_at": "2026-05-06",
|
||||||
|
"input": {
|
||||||
|
"text": ""
|
||||||
|
},
|
||||||
|
"output": {
|
||||||
|
"text": ""
|
||||||
|
}
|
||||||
|
}
|
||||||
11
go/tests/fixtures/pure/normalize/mixed_case.json
vendored
Normal file
11
go/tests/fixtures/pure/normalize/mixed_case.json
vendored
Normal file
@@ -0,0 +1,11 @@
|
|||||||
|
{
|
||||||
|
"case": "mixed_case",
|
||||||
|
"func": "scripts.czech_utils.normalize",
|
||||||
|
"captured_at": "2026-05-06",
|
||||||
|
"input": {
|
||||||
|
"text": "Henrietta OTTOVÁ"
|
||||||
|
},
|
||||||
|
"output": {
|
||||||
|
"text": "henrietta ottova"
|
||||||
|
}
|
||||||
|
}
|
||||||
11
go/tests/fixtures/pure/normalize/simple_ascii.json
vendored
Normal file
11
go/tests/fixtures/pure/normalize/simple_ascii.json
vendored
Normal file
@@ -0,0 +1,11 @@
|
|||||||
|
{
|
||||||
|
"case": "simple_ascii",
|
||||||
|
"func": "scripts.czech_utils.normalize",
|
||||||
|
"captured_at": "2026-05-06",
|
||||||
|
"input": {
|
||||||
|
"text": "hello world"
|
||||||
|
},
|
||||||
|
"output": {
|
||||||
|
"text": "hello world"
|
||||||
|
}
|
||||||
|
}
|
||||||
11
go/tests/fixtures/pure/normalize/with_parens.json
vendored
Normal file
11
go/tests/fixtures/pure/normalize/with_parens.json
vendored
Normal file
@@ -0,0 +1,11 @@
|
|||||||
|
{
|
||||||
|
"case": "with_parens",
|
||||||
|
"func": "scripts.czech_utils.normalize",
|
||||||
|
"captured_at": "2026-05-06",
|
||||||
|
"input": {
|
||||||
|
"text": "Pavel Smutný (Štrúdl)"
|
||||||
|
},
|
||||||
|
"output": {
|
||||||
|
"text": "pavel smutny (strudl)"
|
||||||
|
}
|
||||||
|
}
|
||||||
14
go/tests/fixtures/pure/parse_czk_amount/czech_comma_decimal.json
vendored
Normal file
14
go/tests/fixtures/pure/parse_czk_amount/czech_comma_decimal.json
vendored
Normal file
@@ -0,0 +1,14 @@
|
|||||||
|
{
|
||||||
|
"case": "czech_comma_decimal",
|
||||||
|
"func": "scripts.infer_payments.parse_czk_amount",
|
||||||
|
"captured_at": "2026-05-06",
|
||||||
|
"input": {
|
||||||
|
"val": {
|
||||||
|
"type": "string",
|
||||||
|
"value": "1.500,00"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"output": {
|
||||||
|
"amount": 1500.0
|
||||||
|
}
|
||||||
|
}
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user