Compare commits

...

22 Commits

Author SHA1 Message Date
2b7eff14c4 chore: CHANGELOG and progress tracker for M5.2
All checks were successful
Deploy to K8s / deploy (push) Successful in 7s
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-07 21:02:54 +02:00
7d48e8f607 feat(go): M5.2 — HTTP handlers for /api/adults, /api/juniors, /api/payments, /api/version
All checks were successful
Deploy to K8s / deploy (push) Successful in 10s
- Add web/api/handler.go: Handler struct wiring Sources+Config into ServeAdults,
  ServeJuniors, ServePayments, ServeVersion
- Add web/api/build_common.go: getMonthLabels, groupRawPaymentsByPerson, settledBalance,
  domain-to-wire converters, ensureSlice generic helper
- Add web/api/build_adults.go: buildAdultsResponse + buildAdultMemberRow mirroring
  scripts/views.py:build_adults_view_model
- Add web/api/build_juniors.go: buildJuniorsResponse + buildJuniorMemberRow mirroring
  scripts/views.py:build_juniors_view_model, including "?" sentinel and :NJ,MA breakdown
- Add web/api/build_payments.go: buildPaymentsResponse with Unmatched/Unknown bucket
- Extend reconcile.FeeData/MonthData with IsUnknown, JuniorAttendance, AdultAttendance
- Extend reconcile.Transaction with ManualFix, VS, BankID, SyncID for raw_payments wire field
- Export membership.AdultMergedMonths and JuniorMergedMonths
- Update sources.go to propagate new FeeData fields and parse extra transaction columns
- Wire sources+cfg into web.Run; register /api/* routes via Go 1.22 method+path patterns
- Fix pre-existing gofumpt formatting in fio_test.go and fio_table.go

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-07 20:13:38 +02:00
be4ecef20f Merge pull request 'feat(go): M5.1 — hand-author /api/* wire types + JSON Schemas' (#16) from feat/go-m5-1-api-structs into main
All checks were successful
Deploy to K8s / deploy (push) Successful in 6s
Reviewed-on: #16
2026-05-07 17:50:55 +00:00
da5b82fcdb chore: CHANGELOG and progress tracker for M5.1
All checks were successful
Deploy to K8s / deploy (push) Successful in 6s
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-07 17:38:00 +02:00
f253e3fcb1 feat(go): M5.1 — hand-author /api/* wire types + JSON Schemas
All checks were successful
Deploy to K8s / deploy (push) Successful in 7s
Add internal/web/api package with Go structs for every /api/X route:
AdultsResponse, JuniorsResponse, PaymentsResponse, VersionResponse.
All fields carry explicit json: tags matching the Python view-model keys.

Key design choices:
- member_data / month_labels / raw_payments are nested objects (not
  the pre-serialised JSON strings used in Jinja templates)
- Expected{Value int; Unknown bool} with custom MarshalJSON emits int
  or the string "?" for junior single-attendance months
- RawTransaction covers the full 11-column payments sheet row

schemagen_test.go reflects all four response types via
github.com/invopop/jsonschema and golden-compares against committed
schemas in tests/fixtures/api-schema/. The JSONSchema() method on
Expected lives in the test file so the prod binary has no jsonschema
dependency.

Closes M5.1 in docs/plans/2026-05-03-2349-go-backend-rewrite-progress.md.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-07 17:36:46 +02:00
59223c0da4 chore: CHANGELOG for Python view-model extraction
All checks were successful
Deploy to K8s / deploy (push) Successful in 8s
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-07 15:27:17 +02:00
32a16ff50d Merge pull request 'refactor(app): extract view-model builders into scripts/views.py' (#15) from feat/m5-python-views-extraction into main
All checks were successful
Deploy to K8s / deploy (push) Successful in 5s
Reviewed-on: #15
2026-05-07 13:26:42 +00:00
2eec51bb34 fix(app): restore missing import re needed by qr_code route
All checks were successful
Deploy to K8s / deploy (push) Successful in 6s
Accidentally removed when moving group_payments_by_person to views.py;
re.match in qr_code caused a 500 on every QR request.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-07 15:24:40 +02:00
b562ce3201 refactor(app): extract view-model builders into scripts/views.py
All checks were successful
Deploy to K8s / deploy (push) Successful in 8s
Pull 350+ lines of inline per-row computation out of adults_view,
juniors_view, and payments into three pure builder functions with no
Flask globals or IO dependencies. Route handlers now contain only
cache/IO calls and a single render_template. No behaviour change —
all 27 tests pass.

Also moves get_month_labels, group_payments_by_person, and
adapt_junior_members out of app.py. Prep for /api/* shadow endpoints
(M5 Go parity).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-07 15:22:12 +02:00
f0de300292 chore: CHANGELOG for --print-fio-table, debug logging, and date parser fix
All checks were successful
Deploy to K8s / deploy (push) Successful in 9s
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-07 14:13:56 +02:00
2164e99866 Merge pull request 'feat(go): add --print-fio-table debug flag to fuj sync' (#14) from feat/fuj-sync-print-fio-table into main
All checks were successful
Deploy to K8s / deploy (push) Successful in 9s
Reviewed-on: #14
2026-05-07 12:13:19 +00:00
b41b8ef29c fix(go/fio): accept 2-digit year format in transparent date parser
All checks were successful
Deploy to K8s / deploy (push) Successful in 9s
Fio's transparent account page now serves dates as DD.MM.YY (e.g.
07.05.26) rather than the previously expected 4-digit-year format.
Extends parseCzechDate to try all eight layout variants: padded and
non-padded, dot and slash separators, 4-digit and 2-digit years.

Go maps 2-digit year 00-68 → 2000-2068, so 26 → 2026.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-07 14:12:34 +02:00
80db33945d chore: add make go-sync-debug target
All checks were successful
Deploy to K8s / deploy (push) Successful in 6s
Wraps `LOG_LEVEL=DEBUG ./bin/fuj sync -dry-run -print-fio-table -days N`
behind a single make target. Default DAYS=30, override with
`make go-sync-debug DAYS=90`. Builds the Go binary first via go-build.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-07 14:01:46 +02:00
f87adeff9f feat(go/fio): debug logging via slog at LOG_LEVEL=DEBUG
All checks were successful
Deploy to K8s / deploy (push) Successful in 8s
Wires slog.SetDefault to honour LOG_LEVEL in all CLI commands and adds
debug logs on the Fio fetch path so a silent "fetched 0 transaction(s)"
can be diagnosed without code changes:

- fio.New: which client variant (api/transparent) was selected
- apiClient: GET URL (token redacted as ****), HTTP status, body bytes,
  parsed transaction count
- transparentClient: GET URL, HTTP status, body bytes, plus parser
  stats (raw rows from second table, kept, dropped_bad_date,
  dropped_nonpositive_amount)

Also suppresses the --print-fio-table block when zero transactions were
fetched, so the bare header no longer prints under that condition.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-07 13:59:22 +02:00
a7cf45fc95 feat(go): add --print-fio-table flag to fuj sync --dry-run
All checks were successful
Deploy to K8s / deploy (push) Successful in 7s
Prints an aligned tabwriter table of every Fio transaction in the
look-back window, with a STATUS column showing NEW (would be appended)
or DUP (already in sheet). Only fires when --dry-run is also set, so
it can't affect real syncs. Refactors Sync ID computation into a single
pre-pass shared by both the table printer and the row builder.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-07 13:49:42 +02:00
f0a0f79475 Merge pull request 'feat(go): IO layer behind interfaces (M4)' (#13) from feat/m4-io-layer into main
All checks were successful
Deploy to K8s / deploy (push) Successful in 10s
Reviewed-on: #13
2026-05-07 08:48:54 +00:00
fcb83691f5 fix(go/fio): nested-table early exit + non-padded date parsing
All checks were successful
Deploy to K8s / deploy (push) Successful in 9s
extractSecondTableRows tracked a boolean inTarget flag and exited on
the first </table> token while inside the target. Any nested <table>
(e.g. pagination markup in the real Fio page) would cause an early
return before reading any data rows, explaining the 0-transaction report.
Fixed by tracking targetDepth instead: depth increments on every <table>
inside the target and we only return when it reaches 0 again.

parseCzechDate also only tried zero-padded layouts ("02.01.2006").
The real Fio transparent page emits non-padded dates ("7.5.2026");
added "2.1.2006" and "2/1/2006" as the preferred layouts.

Also adds a dry-run diagnostic line ("fetched N transaction(s) from Fio")
so the fetch vs dedup split is visible without reading logs.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-07 10:47:54 +02:00
8275db1a63 fix(go/fio): nil http client panic in fio.New
All checks were successful
Deploy to K8s / deploy (push) Successful in 10s
When token is empty, New falls back to transparentClient with the
caller-supplied hc. main.go passes nil, so the first Do() call panicked.
Default to http.DefaultClient when hc is nil.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-07 10:36:20 +02:00
36a28a40d2 feat(go): add --dry-run to fuj sync
All checks were successful
Deploy to K8s / deploy (push) Successful in 18s
Mirror fuj infer's read-only mode: SyncOpts.DryRun skips WriteHeader,
AppendValues, and SortByDateColumn, printing one "Dry run: would …"
line per planned operation instead. ID-dedup still runs so the output
reflects exactly what the next real sync would write.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-07 10:33:55 +02:00
6465e2a221 feat(go): IO layer behind interfaces (M4)
All checks were successful
Deploy to K8s / deploy (push) Successful in 11s
- io/attendance: CSV-over-public-URL client + Fake for adult/junior tabs
- io/drive: Drive v3 modifiedTime client + Fake
- io/sheets: Sheets v4 client (GetValues/AppendValues/BatchUpdateValues/
  WriteHeader/SortByDateColumn) + Fake with call-capture
- io/cache: Drive-modifiedTime-gated FileCache; two TTL knobs; atomic
  writes; generic Get[T]; Python-compatible JSON format; Flush()
- io/fio: Client interface backed by Fio REST API (apiClient) and HTML
  scraper (transparentClient); Fake; testdata fixtures
- membership/sources: NewSources wires attendance CSV + Sheets + cache
  into LoadAdults/LoadJuniors/LoadTransactions/LoadExceptions; Czech
  month parsing + merged-month maps
- banksync: SyncToSheets (SHA-256 dedup, optional sort) and
  InferPayments ([?] review prefix, dry-run) — tested with fakes
- cmd/fuj: sync and infer subcommands wired; fees and reconcile use
  real NewSources; go.mod gains google.golang.org/api + x/net
- gofumpt extra-rules applied across all packages; lint clean

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-07 01:05:59 +02:00
7afd12d9a5 chore: tick M3.1–M3.6 in progress tracker + CHANGELOG entry
All checks were successful
Deploy to K8s / deploy (push) Successful in 6s
Merges SHA 57518a8 (PR #12).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-06 23:34:00 +02:00
57518a8a68 Merge pull request 'feat(go): fixture capture + characterization framework (M3)' (#12) from feat/m3-fixture-capture into main
All checks were successful
Deploy to K8s / deploy (push) Successful in 8s
Reviewed-on: #12
2026-05-06 21:29:48 +00:00
71 changed files with 6688 additions and 465 deletions

View File

@@ -1,5 +1,53 @@
# Changelog
## 2026-05-07 20:13 CEST — feat(go): M5.2 — HTTP handlers for /api/adults, /api/juniors, /api/payments, /api/version
- `web/api/handler.go`: `Handler` struct + `ServeAdults`, `ServeJuniors`, `ServePayments`, `ServeVersion` using `membership.Sources`.
- `web/api/build_{adults,juniors,payments,common}.go`: ports of `scripts/views.py` view-model builders; `buildJuniorMemberRow` handles `"?"` sentinel, `:NJ,MA` breakdown, unknown-month skip.
- Extended `reconcile.FeeData`/`MonthData` with `IsUnknown`, `JuniorAttendance`, `AdultAttendance`; `Transaction` with `ManualFix`, `VS`, `BankID`, `SyncID`.
- `sources.go` exports `AdultMergedMonths`/`JuniorMergedMonths`; parses new FeeData and transaction columns.
- `web/server.go` + `cmd/fuj/main.go` wired to register `/api/*` routes.
- PR #17.
## 2026-05-07 17:37 CEST — feat(go): M5.1 — /api/* wire types + JSON Schemas
- New `go/internal/web/api/` package: `AdultsResponse`, `JuniorsResponse`, `PaymentsResponse`, `VersionResponse` with explicit `json:` tags matching Python view-model keys.
- `Expected{Value int; Unknown bool}` custom `MarshalJSON` emits integer or `"?"` for junior single-attendance months.
- `schemagen_test.go` golden-tests four JSON Schemas committed to `go/tests/fixtures/api-schema/`. `JSONSchema()` on `Expected` lives in the test file — production binary has no jsonschema dep.
- PR #16.
## 2026-05-07 15:26 CEST — refactor(app): extract view-model builders into scripts/views.py
- Pulled ~350 lines of inline per-row computation out of `adults_view`, `juniors_view`, and `payments` into three pure functions in `scripts/views.py`: `build_adults_view_model`, `build_juniors_view_model`, `build_payments_view_model`.
- Moved `get_month_labels`, `group_payments_by_person`, `adapt_junior_members` from `app.py` to `scripts/views.py`. Route handlers now ~25 lines each.
- Hotfixed missing `import re` that caused 500 on `/qr` after the refactor.
- No behaviour change; all 27 tests pass. Prep for `/api/*` shadow endpoints (M5).
## 2026-05-07 14:13 CEST — feat(go): --print-fio-table + Fio debug logging + date parser fix
- Added `--print-fio-table` flag to `fuj sync --dry-run`: prints an aligned table of every Fio transaction in the window with `STATUS=NEW/DUP`, using `text/tabwriter`. Key files: `go/internal/services/banksync/fio_table.go`, `sync.go`, `cmd/fuj/main.go`.
- Added `LOG_LEVEL=DEBUG` debug logging on the Fio fetch path: client variant selected, full GET URL (token redacted on API path), HTTP status, body bytes, and per-parse drop-reason counters (`raw_rows`, `kept`, `dropped_bad_date`, `dropped_nonpositive_amount`). Key files: `go/internal/io/fio/{client,api,transparent}.go`.
- Fixed `parseCzechDate` to accept `DD.MM.YY` (2-digit year) in addition to the 4-digit variant — Fio's transparent page now serves this format. Key file: `go/internal/io/fio/transparent.go`.
- Added `make go-sync-debug [DAYS=N]` Makefile target (default 30 days).
## 2026-05-07 10:32 CEST — feat(go): --dry-run for fuj sync
- `SyncOpts.DryRun bool` added; when true, `SyncToSheets` prints planned writes (`would write header row`, `would append date=… amount=… sender=…`, `would sort by date`) and returns without calling `WriteHeader`, `AppendValues`, or `SortByDateColumn`.
- `fuj sync --dry-run` flag wired in `cmd/fuj/main.go`; mirrors existing `fuj infer --dry-run` behaviour.
- `TestSyncToSheets_DryRun` added to banksync test suite.
## 2026-05-07 01:03 CEST — feat(go/M4): IO layer behind interfaces
- `go/internal/io/attendance`: CSV-over-public-URL client + `Fake` for both adult and junior tabs.
- `go/internal/io/drive`: thin Drive v3 wrapper for `modifiedTime` reads + `Fake`.
- `go/internal/io/sheets`: Sheets v4 client (`GetValues`, `AppendValues`, `BatchUpdateValues`, `WriteHeader`, `SortByDateColumn`) + `Fake` with call-capture for assertions.
- `go/internal/io/cache`: Drive-modifiedTime-gated `FileCache` with two TTL knobs, atomic writes, and generic `Get[T]`; Python-compatible JSON format; `Flush()` support.
- `go/internal/io/fio`: `Client` interface backed by Fio REST API (`apiClient`) and HTML-scraper (`transparentClient`); `Fake` for tests. Fixtures in `testdata/`.
- `go/internal/services/membership/sources.go`: `NewSources` wires attendance CSV + Sheets + cache into `LoadAdults`, `LoadJuniors`, `LoadTransactions`, `LoadExceptions`. Includes Czech month/merged-month parsing logic.
- `go/internal/services/banksync`: `SyncToSheets` (dedup via SHA-256 Sync ID, optional sort) and `InferPayments` (name-match + `[?]` review prefix, dry-run) — fully tested with fakes.
- `go/cmd/fuj/main.go`: `sync` and `infer` subcommands wired to real clients; `fees` and `reconcile` now use real `NewSources`.
- All packages lint-clean (golangci-lint v1.64.8, gofumpt extra-rules).
## 2026-05-06 23:25 CEST — feat(go/M3): fixture capture + parity test framework
- `scripts/capture_fixtures.py`: dispatcher CLI that calls each ported function with seeded inputs and emits captured output as JSON fixtures.
@@ -24,6 +72,7 @@
- `scripts/infer_payments.py`: union adults + junior rosters so junior-only members are visible to the matcher.
- Root cause: `get_members_with_fees()` reads only the adults sheet; junior-only kids like Jáchym Kubík were absent from `member_names`, causing the exact-match short-circuit to never fire and a different adult sharing the first name to win via fuzzy review.
- Two regression tests added to `tests/test_match_members.py`.
## 2026-05-06 16:05 CEST — feat(go/M2.10): port domain/reconcile.Reconcile
- New `go/internal/domain/reconcile` package porting the three-phase payment allocation from `scripts/match_payments.py reconcile()`.

View File

@@ -1,4 +1,4 @@
.PHONY: help fees match web web-py web-debug web-go go-build go-test go-test-all go-parity go-run go-lint capture-fixtures image run sync sync-2026 test test-v docs
.PHONY: help fees match web web-py web-debug web-go go-build go-test go-test-all go-parity go-run go-sync-debug go-lint capture-fixtures image run sync sync-2026 test test-v docs
export PYTHONPATH := scripts:$(PYTHONPATH)
VENV := .venv
@@ -27,6 +27,7 @@ help:
@echo " make go-parity - Run Go parity tests (requires -tags=parity fixture corpus)"
@echo " make go-test-all - Run both unit and parity tests"
@echo " make go-lint - Run golangci-lint on Go code"
@echo " make go-sync-debug [DAYS=N] - Dry-run Go sync with Fio debug logs and txn table (default DAYS=30)"
@echo " make capture-fixtures - Regenerate parity fixture corpus from live Python"
@echo " make image - Build Python OCI container image"
@echo " make run - Run the built Python Docker image locally"
@@ -91,6 +92,10 @@ capture-fixtures: $(PYTHON)
go-run: go-build
./$(GO_BIN) $(ARGS)
DAYS ?= 30
go-sync-debug: go-build
LOG_LEVEL=DEBUG ./$(GO_BIN) sync -dry-run -print-fio-table -days $(DAYS)
go-lint:
cd $(GO_SRC) && golangci-lint run ./...

424
app.py
View File

@@ -21,8 +21,14 @@ from config import (
ATTENDANCE_SHEET_ID, PAYMENTS_SHEET_ID, JUNIOR_SHEET_GID,
BANK_ACCOUNT, CREDENTIALS_PATH,
)
from attendance import get_members_with_fees, get_junior_members_with_fees, ADULT_MERGED_MONTHS, JUNIOR_MERGED_MONTHS
from match_payments import reconcile, fetch_sheet_data, fetch_exceptions, normalize, canonical_member_key
from attendance import get_members_with_fees, get_junior_members_with_fees
from match_payments import reconcile, fetch_sheet_data, fetch_exceptions
from views import (
build_adults_view_model,
build_juniors_view_model,
build_payments_view_model,
adapt_junior_members,
)
from cache_utils import get_sheet_modified_time, read_cache, write_cache, _LAST_CHECKED, flush_cache
from sync_fio_to_sheets import sync_to_sheets
from infer_payments import infer_payments
@@ -38,44 +44,6 @@ def get_cached_data(cache_key, sheet_id, fetch_func, *args, serialize=None, dese
write_cache(cache_key, mod_time, serialize(data) if serialize else data)
return data
def get_month_labels(sorted_months, merged_months):
labels = {}
for m in sorted_months:
dt = datetime.strptime(m, "%Y-%m")
# Find which months were merged into m (e.g. 2026-01 is merged into 2026-02)
merged_in = sorted([k for k, v in merged_months.items() if v == m])
if merged_in:
all_dts = [datetime.strptime(x, "%Y-%m") for x in sorted(merged_in + [m])]
years = {d.year for d in all_dts}
if len(years) > 1:
parts = [d.strftime("%b %Y") for d in all_dts]
labels[m] = "+".join(parts)
else:
parts = [d.strftime("%b") for d in all_dts]
labels[m] = f"{'+'.join(parts)} {dt.strftime('%Y')}"
else:
labels[m] = dt.strftime("%b %Y")
return labels
def group_payments_by_person(transactions, member_names=None):
canonical_by_key = (
{canonical_member_key(n): n for n in member_names} if member_names else {}
)
grouped = {}
for tx in transactions:
person = str(tx.get("person", "")).strip()
if not person:
continue
for p in person.split(","):
p = re.sub(r"\[\?\]\s*", "", p).strip()
if not p:
continue
key = canonical_by_key.get(canonical_member_key(p), p)
grouped.setdefault(key, []).append(tx)
for rows in grouped.values():
rows.sort(key=lambda t: str(t.get("date", "")), reverse=True)
return grouped
def warmup_cache():
"""Pre-fetch all cached data so first request is fast."""
logger = logging.getLogger(__name__)
@@ -181,381 +149,77 @@ def adults_view():
attendance_url = f"https://docs.google.com/spreadsheets/d/{ATTENDANCE_SHEET_ID}/edit"
payments_url = f"https://docs.google.com/spreadsheets/d/{PAYMENTS_SHEET_ID}/edit"
credentials_path = CREDENTIALS_PATH
members_data = get_cached_data("attendance_regular", ATTENDANCE_SHEET_ID, get_members_with_fees)
record_step("fetch_members")
if not members_data:
return "No data."
members, sorted_months = members_data
transactions = get_cached_data("payments_transactions", PAYMENTS_SHEET_ID, fetch_sheet_data, PAYMENTS_SHEET_ID, credentials_path)
record_step("fetch_payments")
exceptions = get_cached_data(
"exceptions_dict", PAYMENTS_SHEET_ID, fetch_exceptions,
PAYMENTS_SHEET_ID, credentials_path,
serialize=lambda d: [[list(k), v] for k, v in d.items()],
deserialize=lambda c: {tuple(k): v for k, v in c},
)
"exceptions_dict", PAYMENTS_SHEET_ID, fetch_exceptions,
PAYMENTS_SHEET_ID, credentials_path,
serialize=lambda d: [[list(k), v] for k, v in d.items()],
deserialize=lambda c: {tuple(k): v for k, v in c},
)
record_step("fetch_exceptions")
result = reconcile(members, sorted_months, transactions, exceptions)
record_step("reconcile")
month_labels = get_month_labels(sorted_months, ADULT_MERGED_MONTHS)
adult_names = sorted([name for name, tier, _ in members if tier == "A"])
current_month = datetime.now().strftime("%Y-%m")
monthly_totals = {m: {"expected": 0, "paid": 0} for m in sorted_months}
formatted_results = []
for name in adult_names:
data = result["members"][name]
row = {"name": name, "months": [], "balance": data["total_balance"], "unpaid_periods": "", "raw_unpaid_periods": ""}
unpaid_months = []
raw_unpaid_months = []
for m in sorted_months:
mdata = data["months"].get(m, {"expected": 0, "original_expected": 0, "attendance_count": 0, "paid": 0, "exception": None})
expected = mdata.get("expected", 0)
original_expected = mdata.get("original_expected", 0)
count = mdata.get("attendance_count", 0)
paid = int(mdata.get("paid", 0))
exception_info = mdata.get("exception", None)
monthly_totals[m]["expected"] += expected
monthly_totals[m]["paid"] += paid
override_amount = exception_info["amount"] if exception_info else None
if override_amount is not None and override_amount != original_expected:
is_overridden = True
fee_display = f"{override_amount} ({original_expected}) CZK ({count})" if count > 0 else f"{override_amount} ({original_expected}) CZK"
else:
is_overridden = False
fee_display = f"{expected} CZK ({count})" if count > 0 else f"{expected} CZK"
status = "empty"
cell_text = "-"
amount_to_pay = 0
if expected > 0:
amount_to_pay = max(0, expected - paid)
if paid >= expected:
status = "ok"
cell_text = f"{paid}/{fee_display}"
elif paid > 0:
status = "partial"
cell_text = f"{paid}/{fee_display}"
if m < current_month:
unpaid_months.append(month_labels[m])
raw_unpaid_months.append(datetime.strptime(m, "%Y-%m").strftime("%m/%Y"))
else:
status = "unpaid"
cell_text = f"0/{fee_display}"
if m < current_month:
unpaid_months.append(month_labels[m])
raw_unpaid_months.append(datetime.strptime(m, "%Y-%m").strftime("%m/%Y"))
elif paid > 0:
status = "surplus"
cell_text = f"PAID {paid}"
else:
cell_text = "-"
amount_to_pay = 0
if expected > 0 or paid > 0:
tooltip = f"Received: {paid}, Expected: {expected}"
else:
tooltip = ""
row["months"].append({
"text": cell_text,
"overridden": is_overridden,
"status": status,
"amount": amount_to_pay,
"month": month_labels[m],
"raw_month": m,
"tooltip": tooltip
})
# Balance = sum of (paid - expected) for past months only; current/future months ignored.
settled_balance = 0
for m, mdata in data["months"].items():
if m >= current_month:
continue
exp = mdata.get("expected", 0)
if isinstance(exp, int):
settled_balance += int(mdata.get("paid", 0)) - exp
payable_amount = max(0, -settled_balance)
row["unpaid_periods"] = ", ".join(unpaid_months)
row["raw_unpaid_periods"] = "+".join(raw_unpaid_months)
row["balance"] = settled_balance
row["payable_amount"] = payable_amount
formatted_results.append(row)
formatted_totals = []
for m in sorted_months:
t = monthly_totals[m]
status = "empty"
if t["expected"] > 0 or t["paid"] > 0:
if t["paid"] == t["expected"]:
status = "ok"
elif t["paid"] < t["expected"]:
status = "unpaid"
else:
status = "surplus"
formatted_totals.append({
"text": f"{t['paid']} / {t['expected']} CZK",
"status": status
})
def settled_balance(name):
data = result["members"][name]
total = 0
for m, mdata in data["months"].items():
if m >= current_month:
continue
exp = mdata.get("expected", 0)
if isinstance(exp, int):
total += int(mdata.get("paid", 0)) - exp
return total
credits = sorted([{"name": n, "amount": settled_balance(n)} for n in adult_names if settled_balance(n) > 0], key=lambda x: x["name"])
debts = sorted([{"name": n, "amount": abs(settled_balance(n))} for n in adult_names if settled_balance(n) < 0], key=lambda x: x["name"])
unmatched = result["unmatched"]
import json
raw_payments_by_person = group_payments_by_person(transactions, [name for name, _, _ in members])
record_step("process_data")
return render_template(
"adults.html",
months=[month_labels[m] for m in sorted_months],
raw_months=sorted_months,
results=formatted_results,
totals=formatted_totals,
member_data=json.dumps(result["members"]),
month_labels_json=json.dumps(month_labels),
raw_payments_json=json.dumps(raw_payments_by_person),
credits=credits,
debts=debts,
unmatched=unmatched,
vm = build_adults_view_model(
members, sorted_months, result, transactions,
datetime.now().strftime("%Y-%m"),
attendance_url=attendance_url,
payments_url=payments_url,
bank_account=BANK_ACCOUNT,
current_month=current_month
)
record_step("process_data")
return render_template("adults.html", **vm)
@app.route("/juniors")
def juniors_view():
attendance_url = f"https://docs.google.com/spreadsheets/d/{ATTENDANCE_SHEET_ID}/edit#gid={JUNIOR_SHEET_GID}"
payments_url = f"https://docs.google.com/spreadsheets/d/{PAYMENTS_SHEET_ID}/edit"
credentials_path = CREDENTIALS_PATH
junior_members_data = get_cached_data("attendance_juniors", ATTENDANCE_SHEET_ID, get_junior_members_with_fees)
record_step("fetch_junior_members")
if not junior_members_data:
return "No data."
junior_members, sorted_months = junior_members_data
transactions = get_cached_data("payments_transactions", PAYMENTS_SHEET_ID, fetch_sheet_data, PAYMENTS_SHEET_ID, credentials_path)
record_step("fetch_payments")
exceptions = get_cached_data(
"exceptions_dict", PAYMENTS_SHEET_ID, fetch_exceptions,
PAYMENTS_SHEET_ID, credentials_path,
serialize=lambda d: [[list(k), v] for k, v in d.items()],
deserialize=lambda c: {tuple(k): v for k, v in c},
)
"exceptions_dict", PAYMENTS_SHEET_ID, fetch_exceptions,
PAYMENTS_SHEET_ID, credentials_path,
serialize=lambda d: [[list(k), v] for k, v in d.items()],
deserialize=lambda c: {tuple(k): v for k, v in c},
)
record_step("fetch_exceptions")
# Adapt junior tuple format (name, tier, {month: (fee, total_count, adult_count, junior_count)})
# to what match_payments expects: (name, tier, {month: (expected_fee, attendance_count)})
adapted_members = []
for name, tier, fees_dict in junior_members:
adapted_fees = {}
for m, fee_data in fees_dict.items():
if len(fee_data) == 4:
fee, total_count, _, _ = fee_data
adapted_fees[m] = (fee, total_count)
else:
fee, count = fee_data
adapted_fees[m] = (fee, count)
adapted_members.append((name, tier, adapted_fees))
adapted_members = adapt_junior_members(junior_members)
result = reconcile(adapted_members, sorted_months, transactions, exceptions)
record_step("reconcile")
# Format month labels
month_labels = get_month_labels(sorted_months, JUNIOR_MERGED_MONTHS)
junior_names = sorted([name for name, tier, _ in adapted_members])
junior_members_dict = {name: fees_dict for name, _, fees_dict in junior_members}
current_month = datetime.now().strftime("%Y-%m")
monthly_totals = {m: {"expected": 0, "paid": 0} for m in sorted_months}
formatted_results = []
for name in junior_names:
data = result["members"][name]
row = {"name": name, "months": [], "balance": data["total_balance"], "unpaid_periods": "", "raw_unpaid_periods": ""}
unpaid_months = []
raw_unpaid_months = []
for m in sorted_months:
mdata = data["months"].get(m, {"expected": 0, "original_expected": 0, "attendance_count": 0, "paid": 0, "exception": None})
expected = mdata.get("expected", 0)
original_expected = mdata.get("original_expected", 0)
count = mdata.get("attendance_count", 0)
paid = int(mdata.get("paid", 0))
exception_info = mdata.get("exception", None)
if expected != "?" and isinstance(expected, int):
monthly_totals[m]["expected"] += expected
monthly_totals[m]["paid"] += paid
orig_fee_data = junior_members_dict.get(name, {}).get(m)
adult_count = 0
junior_count = 0
if orig_fee_data and len(orig_fee_data) == 4:
_, _, adult_count, junior_count = orig_fee_data
breakdown = ""
if adult_count > 0 and junior_count > 0:
breakdown = f":{junior_count}J,{adult_count}A"
elif junior_count > 0:
breakdown = f":{junior_count}J"
elif adult_count > 0:
breakdown = f":{adult_count}A"
count_str = f" ({count}{breakdown})" if count > 0 else ""
override_amount = exception_info["amount"] if exception_info else None
if override_amount is not None and override_amount != original_expected:
is_overridden = True
fee_display = f"{override_amount} ({original_expected}) CZK{count_str}"
else:
is_overridden = False
fee_display = f"{expected} CZK{count_str}"
status = "empty"
cell_text = "-"
amount_to_pay = 0
if expected == "?" or (isinstance(expected, int) and expected > 0):
if expected == "?":
status = "empty"
cell_text = f"?{count_str}"
elif paid >= expected:
status = "ok"
cell_text = f"{paid}/{fee_display}"
elif paid > 0:
status = "partial"
cell_text = f"{paid}/{fee_display}"
amount_to_pay = expected - paid
if m < current_month:
unpaid_months.append(month_labels[m])
raw_unpaid_months.append(datetime.strptime(m, "%Y-%m").strftime("%m/%Y"))
else:
status = "unpaid"
cell_text = f"0/{fee_display}"
amount_to_pay = expected
if m < current_month:
unpaid_months.append(month_labels[m])
raw_unpaid_months.append(datetime.strptime(m, "%Y-%m").strftime("%m/%Y"))
elif paid > 0:
status = "surplus"
cell_text = f"PAID {paid}"
if (isinstance(expected, int) and expected > 0) or paid > 0:
tooltip = f"Received: {paid}, Expected: {expected}"
else:
tooltip = ""
row["months"].append({
"text": cell_text,
"overridden": is_overridden,
"status": status,
"amount": amount_to_pay,
"month": month_labels[m],
"raw_month": m,
"tooltip": tooltip
})
# Balance = sum of (paid - expected) for past months only; current/future months ignored.
settled_balance = 0
for m, mdata in data["months"].items():
if m >= current_month:
continue
exp = mdata.get("expected", 0)
if isinstance(exp, int):
settled_balance += int(mdata.get("paid", 0)) - exp
payable_amount = max(0, -settled_balance)
row["unpaid_periods"] = ", ".join(unpaid_months)
row["raw_unpaid_periods"] = "+".join(raw_unpaid_months)
row["balance"] = settled_balance
row["payable_amount"] = payable_amount
formatted_results.append(row)
formatted_totals = []
for m in sorted_months:
t = monthly_totals[m]
status = "empty"
if t["expected"] > 0 or t["paid"] > 0:
if t["paid"] == t["expected"]:
status = "ok"
elif t["paid"] < t["expected"]:
status = "unpaid"
else:
status = "surplus"
formatted_totals.append({
"text": f"{t['paid']} / {t['expected']} CZK",
"status": status
})
# Format credits and debts
def junior_settled_balance(name):
data = result["members"][name]
total = 0
for m, mdata in data["months"].items():
if m >= current_month:
continue
exp = mdata.get("expected", 0)
if isinstance(exp, int):
total += int(mdata.get("paid", 0)) - exp
return total
junior_all_names = [name for name, _, _ in adapted_members]
credits = sorted([{"name": n, "amount": junior_settled_balance(n)} for n in junior_all_names if junior_settled_balance(n) > 0], key=lambda x: x["name"])
debts = sorted([{"name": n, "amount": abs(junior_settled_balance(n))} for n in junior_all_names if junior_settled_balance(n) < 0], key=lambda x: x["name"])
unmatched = result["unmatched"]
raw_payments_by_person = group_payments_by_person(transactions, [name for name, _, _ in adapted_members])
import json
record_step("process_data")
return render_template(
"juniors.html",
months=[month_labels[m] for m in sorted_months],
raw_months=sorted_months,
results=formatted_results,
totals=formatted_totals,
member_data=json.dumps(result["members"]),
month_labels_json=json.dumps(month_labels),
raw_payments_json=json.dumps(raw_payments_by_person),
credits=credits,
debts=debts,
unmatched=unmatched,
vm = build_juniors_view_model(
junior_members, adapted_members, sorted_months, result, transactions,
datetime.now().strftime("%Y-%m"),
attendance_url=attendance_url,
payments_url=payments_url,
bank_account=BANK_ACCOUNT,
current_month=current_month
)
record_step("process_data")
return render_template("juniors.html", **vm)
@app.route("/payments")
def payments():
attendance_url = f"https://docs.google.com/spreadsheets/d/{ATTENDANCE_SHEET_ID}/edit"
payments_url = f"https://docs.google.com/spreadsheets/d/{PAYMENTS_SHEET_ID}/edit"
credentials_path = CREDENTIALS_PATH
transactions = get_cached_data("payments_transactions", PAYMENTS_SHEET_ID, fetch_sheet_data, PAYMENTS_SHEET_ID, credentials_path)
record_step("fetch_payments")
@@ -567,23 +231,13 @@ def payments():
if juniors_data:
member_names.extend(name for name, _, _ in juniors_data[0])
grouped = group_payments_by_person(transactions, member_names)
# payments page also groups unmatched rows under a fallback key
for tx in transactions:
if not str(tx.get("person", "")).strip():
grouped.setdefault("Unmatched / Unknown", []).append(tx)
for rows in grouped.values():
rows.sort(key=lambda t: str(t.get("date", "")), reverse=True)
sorted_people = sorted(grouped.keys())
record_step("process_data")
return render_template(
"payments.html",
grouped_payments=grouped,
sorted_people=sorted_people,
vm = build_payments_view_model(
transactions, member_names,
attendance_url=attendance_url,
payments_url=payments_url
payments_url=payments_url,
)
record_step("process_data")
return render_template("payments.html", **vm)
@app.route("/qr")
def qr_code():

View File

@@ -2,9 +2,9 @@
Companion to [2026-05-03-2349-go-backend-rewrite.md](2026-05-03-2349-go-backend-rewrite.md).
**Current milestone:** M3Fixture capture + characterization framework
**Current milestone:** M4IO layer behind interfaces
**Started:** 2026-05-04
**Last updated:** 2026-05-06
**Last updated:** 2026-05-07
## How to use
@@ -65,14 +65,14 @@ Each task: port the function, write Go unit tests for fresh cases, hook into the
Goal: deterministic, PII-free fixture corpus that drives parity tests. Runs in parallel with M2 (M3.1/M3.2 unblocks M2.1).
- [x] **M3.1** `scripts/capture_fixtures.py` — pure-function output dumper. Reads inputs from stdin / argv, prints `{"input":..., "output":...}` JSON
- [x] **M3.2** `scripts/scrub_fixtures.py` — replaces names with `Member_<8hex>` (deterministic per name); scrambles sender/account/VS/bank_id with stable bijection; preserves dates, amounts, exception keys
- [x] **M3.3** Capture pure-fn fixtures for M2.1M2.9 (run helper + scrubber, commit to `tests/fixtures/pure/<func>/<case>.json`)
- [x] **M3.4** Capture ~10 reconcile fixtures spanning every code path: greedy, proportional (float remainder), even-split, out-of-window credit, exception override, `other:` purpose, junior `"?"`, multi-person comma-split, multi-month range, unmatched. Commit to `tests/fixtures/reconcile/`
- [x] **M3.5** Hook fixtures into Tier-1 test runner with `-tags=parity` build constraint
- [x] **M3.6** Document fixture-refresh workflow in `tests/fixtures/README.md` (what to do when sheet schema changes)
- [x] **M3.1** `scripts/capture_fixtures.py` — pure-function output dumper. Reads inputs from stdin / argv, prints `{"input":..., "output":...}` JSON — `57518a8`
- [x] **M3.2** `scripts/scrub_fixtures.py` — replaces names with `Member_<8hex>` (deterministic per name); scrambles sender/account/VS/bank_id with stable bijection; preserves dates, amounts, exception keys — `57518a8`
- [x] **M3.3** Capture pure-fn fixtures for M2.1M2.9 (run helper + scrubber, commit to `tests/fixtures/pure/<func>/<case>.json`) — `57518a8`
- [x] **M3.4** Capture ~10 reconcile fixtures spanning every code path: greedy, proportional (float remainder), even-split, out-of-window credit, exception override, `other:` purpose, junior `"?"`, multi-person comma-split, multi-month range, unmatched. Commit to `tests/fixtures/reconcile/` — `57518a8`
- [x] **M3.5** Hook fixtures into Tier-1 test runner with `-tags=parity` build constraint — `57518a8`
- [x] **M3.6** Document fixture-refresh workflow in `tests/fixtures/README.md` (what to do when sheet schema changes) — `57518a8`
**Gate:** ✅ `tests/fixtures/` populated (98 files); `make go-parity` green; `make go-lint` (parity tag) clean; raw `tmp/*.json` confirmed gitignored.
**Gate:** ✅ `tests/fixtures/` populated (98 files); `make go-parity` green; `make go-lint` (parity tag) clean; raw `tmp/*.json` confirmed gitignored. Merged as `57518a8`.
---
@@ -80,16 +80,16 @@ Goal: deterministic, PII-free fixture corpus that drives parity tests. Runs in p
Goal: every external IO (Sheets, Drive, Fio, file cache) accessed through a narrow Go interface with both a real and a fake implementation.
- [ ] **M4.1** Design IO interfaces (`SheetsClient`, `DriveClient`, `FioClient`, `FileCache`) + in-memory fakes seeded from M3 fixtures
- [ ] **M4.2** `internal/io/sheets` — Google client (read + append + batchUpdate); integration test against a separate test sheet (NOT prod)
- [ ] **M4.3** `internal/io/drive` — Drive `modifiedTime` client + integration test
- [ ] **M4.4** `internal/io/fio` — API JSON impl (token-based); parses by hardcoded `column0..column22` indices matching [fio_utils.py](scripts/fio_utils.py)
- [ ] **M4.5** `internal/io/fio` — transparent-page HTML scraper using `golang.org/x/net/html` token visitor; targets the **second** `<table class="table">`
- [ ] **M4.6** `internal/io/cache` — FileCache with `modifiedTime` gating + two TTL knobs + atomic writes (`os.Rename`)
- [ ] **M4.7** `services/banksync.SyncToSheets` + `fuj sync` subcommand
- [ ] **M4.8** `services/banksync.InferPayments` + `fuj infer [--dry-run]` subcommand
- [x] **M4.1** Design IO interfaces (`SheetsClient`, `DriveClient`, `FioClient`, `FileCache`) + in-memory fakes seeded from M3 fixtures
- [x] **M4.2** `internal/io/sheets` — Google client (read + append + batchUpdate); fake with call-capture
- [x] **M4.3** `internal/io/drive` — Drive `modifiedTime` client + fake
- [x] **M4.4** `internal/io/fio` — API JSON impl (token-based); parses by hardcoded `column0..column22` indices matching [fio_utils.py](scripts/fio_utils.py)
- [x] **M4.5** `internal/io/fio` — transparent-page HTML scraper using `golang.org/x/net/html` token visitor; targets the **second** `<table class="table">`
- [x] **M4.6** `internal/io/cache` — FileCache with `modifiedTime` gating + two TTL knobs + atomic writes (`os.Rename`)
- [x] **M4.7** `services/banksync.SyncToSheets` + `fuj sync` subcommand
- [x] **M4.8** `services/banksync.InferPayments` + `fuj infer [--dry-run]` subcommand; `NewSources` wires all IO into fees+reconcile
**Gate:** `go test -tags=integration ./internal/io/...` round-trips against test sheet; default-tag tests run on fakes.
**Gate:** ✅ Fakes-only unit tests; `make go-test` + `make go-lint` both green. Live smoke test deferred to first real sync run.
---
@@ -97,8 +97,8 @@ Goal: every external IO (Sheets, Drive, Fio, file cache) accessed through a narr
Goal: byte-equal JSON between Python and Go for every route. This is the parity contract.
- [ ] **M5.1** Hand-author Go structs for `/api/adults`, `/api/juniors`, `/api/payments`, `/api/version` with explicit `json:` tags matching Python keys; emit JSON Schemas via `github.com/invopop/jsonschema` to `tests/fixtures/api-schema/`
- [ ] **M5.2** Implement Go handlers for `/api/*` routes composing `services/*` results into the JSON structs
- [x] **M5.1** Hand-author Go structs for `/api/adults`, `/api/juniors`, `/api/payments`, `/api/version` with explicit `json:` tags matching Python keys; emit JSON Schemas via `github.com/invopop/jsonschema` to `tests/fixtures/api-schema/` — `f253e3f`
- [x] **M5.2** Implement Go handlers for `/api/*` routes composing `services/*` results into the JSON structs — `7d48e8f`
- [ ] **M5.3** Add Python `/api/X` shadow endpoints in [app.py](app.py): `jsonify(view_model_dict)` — no transformation
- [ ] **M5.4** Build `cmd/parity/main.go`: hits both backends' `/api/X`, normalizes allowlist (`render_time.total`, `build_meta`), prints `cmp.Diff`. Add `make parity` target
@@ -155,4 +155,5 @@ Goal: Go is the one true backend.
(Add entries as you go. Format: `YYYY-MM-DD — short note`.)
- 2026-05-04 — Plan approved. Versioning policy: latest stable for Go and all libs at the time M1 starts. Frontends explicitly allowed to diverge between Python and Go; only the JSON API contract is parity-locked. No reverse proxy — both backends run on different ports via `make web-py` / `make web-go`.
- 2026-05-07 — M4 complete. Chose fakes-only unit tests (no live integration tests) and CSV-via-public-URL for attendance (no Sheets API auth required for read-only). golangci-lint gofumpt extra-rules differ slightly from standalone gofumpt; used `golangci-lint run --fix --enable-only gofumpt` to auto-resolve formatting.
- 2026-05-04 — M1 complete. Dockerfile base changed from `distroless/static:nonroot` → `alpine:3` for debuggability (can tighten later). CLI dispatcher uses stdlib `flag`; module path `fuj-management/go`. golangci-lint v1 embedded gofumpt merges all imports into one group (no stdlib/local split) — accepted as the project style.

View File

@@ -0,0 +1,313 @@
# Plan: Go rewrite — M4 IO layer behind interfaces
Companion to [2026-05-03-2349-go-backend-rewrite.md](2026-05-03-2349-go-backend-rewrite.md)
and [2026-05-03-2349-go-backend-rewrite-progress.md](2026-05-03-2349-go-backend-rewrite-progress.md).
## Context
M1M3 are merged: skeleton + tooling, every pure-domain function ported and
parity-tested against PII-scrubbed fixtures, and the `fuj fees` / `fuj
reconcile` subcommands wired but stubbed (`membership.NewStubSources()`
returns `ErrIOPending` for every loader). M4's job is to replace that stub
with real IO: read attendance CSVs, read the payments sheet + exceptions
tab, fetch Drive `modifiedTime` for cache gating, fetch Fio bank
transactions, and append/update rows on the payments sheet — all behind
narrow Go interfaces that have in-memory fakes for tests.
Once M4 lands, `fuj fees`, `fuj reconcile`, `fuj sync`, and `fuj infer` all
work end-to-end against the real Google Sheets and the real Fio account, and
M5 can start porting the JSON API on top of that IO.
User-confirmed scope choices for this milestone:
- **No live integration tests.** Fakes-only at unit level; live
verification deferred to manual smoke during M7.
- **Three PRs** (sheets/drive/cache → fio/sync → infer), one per major
area, each independently reviewable.
- **Attendance stays on CSV-via-public-URL** — matches Python, no extra
service-account grant needed.
## Approach
### Layering
```
internal/io/ ← raw, narrow clients (one per external system)
sheets/ ← typed wrapper around google.golang.org/api/sheets/v4
drive/ ← Drive v3, only ModifiedTime
attendance/ ← CSV-via-public-URL fetcher (no auth, no Sheets API)
fio/ ← FioClient interface + apiClient + transparentClient
cache/ ← FileCache: modifiedTime gate + two-TTL fallback + atomic write
internal/services/membership/ ← already exists; M4 adds adapters that satisfy
AttendanceLoader / TransactionLoader / ExceptionLoader
by composing io/sheets + io/drive + io/cache + io/attendance.
internal/services/banksync/ ← new: SyncToSheets (M4.7) + InferPayments (M4.8)
composing fio + sheets + attendance loaders.
```
The existing interfaces in [go/internal/services/membership/loader.go](../../go/internal/services/membership/loader.go)
(`AttendanceLoader`, `TransactionLoader`, `ExceptionLoader`, `Sources`) are
the seam — M4 adds a `NewSources(cfg config.Config) (Sources, error)`
constructor next to `NewStubSources()`, and `cmd/fuj/main.go` swaps the
stub for it.
### Auth — service-account only
Drop the OAuth+`token.pickle` path entirely (the production already uses a
service account; the fallback only existed because the original Python
script ran from a developer laptop). Sheets and Drive both authenticate via
`option.WithCredentialsFile(cfg.CredentialsPath)` plus
`option.WithScopes(...)`. Single shared `*http.Client` per backend with a
10s timeout (matches `DRIVE_TIMEOUT`).
### Cache shape
Match Python's wire format so the `tmp/*_cache.json` directory is shared
safely while both backends run side-by-side:
```json
{ "modifiedTime": "<RFC3339>", "data": <list|object>, "cachedAt": "<RFC3339>" }
```
Improvements over Python:
- Atomic write: marshal → `os.WriteFile(path+".tmp", ..., 0o600)`
`os.Rename`. Python's plain truncate-write stays as-is until M8.
- The two TTLs (`CacheTTL` and `CacheAPICheckTTL`) live in `config.Config`
already; only the `CacheDir` field is new.
The four cache keys mirror Python's `CACHE_SHEET_MAP`:
`attendance_regular`, `attendance_juniors`, `exceptions_dict`,
`payments_transactions` → maps to either `AttendanceSheetID` or
`PaymentsSheetID`.
When Drive fails, fall back to a synthetic key
`fmt.Sprintf("ttl-5m-%d", time.Now().Unix()/300)` so cache still keys
deterministically per 5-min bucket (same as Python).
### Fio: two impls behind one interface
```go
type Client interface {
FetchTransactions(ctx context.Context, from, to time.Time) ([]Transaction, error)
}
```
`apiClient` (when `cfg.FioAPIToken != ""`) hits
`https://fioapi.fio.cz/v1/rest/periods/{token}/{from}/{to}/transactions.json`,
unmarshals via a typed struct, and maps `column0..column22` to fields per
[scripts/fio_utils.py](../../scripts/fio_utils.py:90). Negative-amount rows
dropped (matches Python).
`transparentClient` (fallback) GETs
`https://ib.fio.cz/ib/transparent?a={accountNum}&f={DD.MM.YYYY}&t={DD.MM.YYYY}`
and walks the response with `golang.org/x/net/html` token visitor, counting
`<table class="table">` tags and grabbing rows from the **second** one
(skipping `<thead>`). `bank_id`, `currency`, `user_id`, `sender_account`
are empty (matches Python — known limitation).
`accountNum` is derived from `cfg.BankAccount` by stripping the IBAN prefix
(`CZ85 2010 0000 0028 0035 9168``2800359168`); add a small helper in
`config` for this since both the API URL and the transparent URL need it.
### Fakes
In-memory fakes live next to each real impl: `sheets/fake.go`,
`drive/fake.go`, `fio/fake.go`, `attendance/fake.go`,
`cache/fake.go` (a passthrough). All exported as `Fake` so tests do
`sheets.NewFake(rows)` and inject. The membership-adapter tests use these
fakes plus a couple of new raw-bytes fixtures under
`go/internal/io/<pkg>/testdata/`:
- `sheets/testdata/payments_minimal.json` — 2D-string array shaped like
`values.get` would return.
- `sheets/testdata/exceptions_minimal.json` — same, for the exceptions tab.
- `attendance/testdata/adults_minimal.csv` — small adult attendance CSV.
- `attendance/testdata/juniors_minimal.csv` — small junior CSV.
- `fio/testdata/api_response.json` — captured Fio API JSON shape.
- `fio/testdata/transparent.html` — captured transparent-page HTML.
Existing M3 domain fixtures under `go/tests/fixtures/` stay where they are
and continue to drive parity tests; they aren't reused for IO-layer tests
because they're at the wrong layer (post-parse domain types).
## Tasks (mapped to tracker)
Same 8 sub-milestones as the tracker, grouped into 3 PRs.
### PR 1 — sheets / drive / cache + membership wiring (M4.1, M4.2, M4.3, M4.6)
1. **Add deps** in [go/go.mod](../../go/go.mod):
`google.golang.org/api/{sheets/v4,drive/v3,option}`,
`golang.org/x/oauth2/google` (transitively pulled), `golang.org/x/net/html`.
2. **`internal/io/sheets/`**:
- `client.go``Client` struct holding `*sheets.Service`; methods
`GetValues(ctx, spreadsheetID, a1Range string) ([][]any, error)`,
`AppendValues(ctx, spreadsheetID, a1Range string, rows [][]any) error`,
`BatchUpdateValues(ctx, spreadsheetID, updates []ValueRange) error`,
`SortByColumn(ctx, spreadsheetID, sheetGID int64, columnIndex int) error`.
- `fake.go` — exported `Fake` with seedable `Values map[string][][]any`.
3. **`internal/io/drive/`**:
- `client.go``Client.ModifiedTime(ctx, fileID string) (string, error)`
using `drive.New(...).Files.Get(fileID).Fields("modifiedTime").SupportsAllDrives(true)`.
- `fake.go` with seedable `Times map[string]string`.
4. **`internal/io/attendance/`** (new — public-URL CSV):
- `client.go``Client.FetchAdults(ctx) ([][]string, error)` and
`FetchJuniors(ctx) ([][]string, error)` using `http.Get` on
`https://docs.google.com/spreadsheets/d/{ID}/export?format=csv&gid={GID}`,
decoded via `encoding/csv`.
- Add `AttendanceAdultSheetGID = "0"` constant in `internal/config`.
5. **`internal/io/cache/`**:
- `filecache.go``FileCache` with `Get(ctx, key string, fetch func(ctx) (any, error)) (any, error)`
wired through `Drive.ModifiedTime` and the two TTL knobs. Atomic write
via tmp-file + rename.
- Cache key → sheet ID map mirrors Python's `CACHE_SHEET_MAP`.
6. **`internal/services/membership/sources.go`** (new file in existing
package):
- `realSources struct { sheets *sheets.Client; drive *drive.Client; attendance *attendance.Client; cache *cache.FileCache }`.
- Constructor `NewSources(ctx, cfg) (Sources, error)` builds all clients.
- `LoadAdults` reads cached attendance CSV, runs through
`domain/fees.CalculateFee` + merged-month logic (port of
[scripts/attendance.py](../../scripts/attendance.py:170)
`get_members_with_fees`), returns `[]reconcile.Member`.
- `LoadTransactions` reads payments sheet rows via cache, parses to
`[]reconcile.Transaction` (port of
[match_payments.py:208](../../scripts/match_payments.py:208)
`fetch_sheet_data`).
- `LoadExceptions` reads `'exceptions'!A2:D` via cache, builds
`map[ExceptionKey]Exception` (port of `match_payments.py:266`).
7. **Add `LoadJuniors`** to the `AttendanceLoader` interface (Python infer
pulls both adult + junior member lists; needed for M4.8).
8. **Wire into [cmd/fuj/main.go](../../go/cmd/fuj/main.go)**: replace
`membership.NewStubSources()` in `feesCmd` and `reconcileCmd` with
`membership.NewSources(ctx, cfg)`.
9. **Tests** (default tag, no live IO):
- `sheets/client_test.go`, `drive/client_test.go`,
`cache/filecache_test.go` — exercise fakes + parsing logic with
testdata fixtures.
- `membership/sources_test.go` — adapter tests with sheets/drive/cache
fakes verify CSV→Member, rows→Transaction, exceptions tab → map.
10. **Config additions**: `CacheDir` (default `tmp` relative to `$PWD`,
overridable via `CACHE_DIR` env), `DriveTimeout` (default 10s).
11. **Manual verification**: `make go-build && go run ./cmd/fuj fees` and
`... reconcile` print real reports against the live sheet (with valid
`.secret/...credentials.json`).
12. CHANGELOG entry; tick M4.1, M4.2, M4.3, M4.6 in the progress tracker.
### PR 2 — fio + bank sync (M4.4, M4.5, M4.7)
1. **`internal/io/fio/`**:
- `client.go``Client` interface, `Transaction` struct.
- `api.go``apiClient` impl + URL builder + JSON struct definitions
for `accountStatement.transactionList.transaction[].column{N}.value`.
- `transparent.go``transparentClient` impl using
`golang.org/x/net/html` token visitor; helper functions
`parseCzechAmount` (NBSP/space strip + comma→dot) and
`parseCzechDate` (DD.MM.YYYY / DD/MM/YYYY).
- `fake.go`.
- `New(cfg) Client` chooses impl based on `cfg.FioAPIToken`.
- `accountNum(iban)` helper in `internal/config` strips IBAN prefix.
2. **`internal/services/banksync/sync.go`** (new package):
- `SyncToSheets(ctx, cfg, fio Client, sheets *sheets.Client, opts SyncOpts) (added int, err error)`.
- Reads existing rows via `sheets.GetValues(... "A1:K")`, validates
header against `COLUMN_LABELS`, writes header if missing, builds
`existingIDs` from column K (`Sync ID`).
- Computes date window: explicit `from`/`to` or `now - days*24h` (default 30d).
- For each fetched tx, computes `domain/synch.GenerateSyncID`, skips if
present, otherwise builds row in COLUMN_LABELS order with empty
manual/person/purpose/inferred slots.
- `sheets.AppendValues(... "A2", rows)`.
- Optional sort: `sheets.SortByColumn(... gid, 0)` — sheet GID resolved
once via `spreadsheets.Get`.
3. **Wire `fuj sync` subcommand** in `cmd/fuj/main.go`:
- Flags: `--days N` (default 30), `--from YYYY-MM-DD`, `--to YYYY-MM-DD`,
`--sort` (default true matching `make sync-2026`).
- Replace the M4-stub error path.
4. **Tests** (default tag): `banksync/sync_test.go` with fakes — verify
header insertion, dedup against existing sync IDs, multi-row append,
sort call.
5. **Manual verification**: dry-run sync against the real Fio account in a
throwaway test sheet; or visually verify `--from --to` window in stdout
with a no-write flag (only if cheap to add — otherwise skip per the
"no live integration tests" decision).
6. CHANGELOG entry; tick M4.4, M4.5, M4.7.
### PR 3 — infer (M4.8)
1. **`internal/services/banksync/infer.go`**:
- `InferPayments(ctx, cfg, sheets *sheets.Client, attendanceLoader, juniorLoader, opts InferOpts) (updated int, err error)`.
- Reads payments sheet `A1:Z` with case-insensitive header lookup.
- Required columns: `Person, Purpose, Inferred Amount`. Optional input:
`Date, Amount, Sender, Message, VS, manual fix`.
- Skip rule (matches [scripts/infer_payments.py:127](../../scripts/infer_payments.py:127)):
non-empty `manual fix` OR `Person` OR `Purpose` → leave row alone.
- Member list = union of `LoadAdults` + `LoadJuniors` deduped via
`domain/matching.CanonicalKey` (already exists from M2).
- For each empty row: build tx dict, call
`domain/matching.InferTransactionDetails`, prefix `[?] ` if
confidence == "review", emit a `ValueRange` update with R1C1 range
`R{i}C{personCol+1}:R{i}C{amountCol+1}`.
- Single `sheets.BatchUpdateValues` call for all updates.
2. **Wire `fuj infer` subcommand**: flags `--dry-run` (prints planned
updates, no API write).
3. **Tests** (default tag): `banksync/infer_test.go` — fixture rows,
verify skip rule, verify `[?]` prefix on review matches, verify
batchUpdate payload shape, verify `--dry-run` is no-op.
4. CHANGELOG entry; tick M4.8 → milestone gate ✅.
## Critical files
To modify:
- [go/internal/services/membership/loader.go](../../go/internal/services/membership/loader.go) — add `LoadJuniors` to `AttendanceLoader`, add `NewSources`.
- [go/cmd/fuj/main.go](../../go/cmd/fuj/main.go) — swap stub for real sources, add `sync`/`infer` subcommands.
- [go/internal/config/config.go](../../go/internal/config/config.go) — add `CacheDir`, `DriveTimeout`, `AttendanceAdultSheetGID` constant, IBAN→account-num helper.
- [go/go.mod](../../go/go.mod) / `go.sum` — google APIs + `x/net/html`.
- [docs/plans/2026-05-03-2349-go-backend-rewrite-progress.md](2026-05-03-2349-go-backend-rewrite-progress.md) — tick M4.x boxes after each PR.
- [CHANGELOG.md](../../CHANGELOG.md) — entry per PR.
To create:
- `go/internal/io/{sheets,drive,attendance,fio,cache}/{client,fake,*_test}.go`
- `go/internal/io/{sheets,attendance,fio}/testdata/*`
- `go/internal/services/membership/sources.go` (+ `sources_test.go`)
- `go/internal/services/banksync/{sync,infer}.go` (+ tests)
## Reused existing helpers
- `domain/fees.CalculateFee` / `CalculateJuniorFee` — fee math (M2.3, M2.4).
- `domain/matching.{BuildNameVariants,MatchMembers,InferTransactionDetails,FormatDate,CanonicalKey}` — match logic (M2.7M2.9).
- `domain/synch.GenerateSyncID` — dedup hash (M2.6).
- `domain/reconcile.{Member,Transaction,Exception,ExceptionKey}` — domain types.
- `domain/czech.{Normalize,ParseMonthReferences}` — used inside the
attendance/exceptions parsers.
- `domain/money.ParseCZK` — for parsing transparent-scrape amounts.
## Verification
End-to-end checks once all three PRs land:
1. `make go-build && make go-lint && make go-test` — clean.
2. `make go-parity` — M3 fixtures still pass (no domain regressions).
3. `./bin/fuj fees` — prints adult fee report matching Python `make fees`
(visual diff acceptable for now; byte-equality enforced in M5).
4. `./bin/fuj reconcile` — prints balance report comparable to
[scripts/match_payments.py](../../scripts/match_payments.py) `print_balance_report`.
5. `./bin/fuj sync --days 7` — appends new Fio rows to the payments sheet
(run with a real but recent date window; verify by counting added rows
and confirming no duplicates on a second run).
6. `./bin/fuj infer --dry-run` — prints planned Person/Purpose/Inferred
Amount updates without modifying the sheet. Then `./bin/fuj infer`
applies them; second run is a no-op (skip rule).
7. **Cache check**: delete `tmp/*_cache.json`, run `fuj fees`, verify file
appears with `modifiedTime` matching Drive. Re-run within 5 min;
verify no Drive call (debug log).
8. **Cross-process cache safety**: while `make web-py` is running, run
`fuj reconcile`; verify Python's cache file isn't corrupted and Go
reads the same data.
Gate (per tracker):
> `go test -tags=integration ./internal/io/...` round-trips against test sheet; default-tag tests run on fakes.
Per the user's scope decision, **the integration-test gate is downgraded
to "default-tag tests on fakes" only**. Live verification is deferred to
manual smoke during M7's parallel-run watch period. The progress tracker's
M4 gate line will be amended in PR 1.

View File

@@ -0,0 +1,36 @@
# Plan: add `--dry-run` to `fuj sync`
## Context
`fuj infer` already supports `--dry-run` (it builds the planned `BatchUpdateValues`
operations, prints them, and skips the actual write — see
`go/internal/services/banksync/infer.go:136-156` and the
`Dry run: would update N row(s).` line in `go/cmd/fuj/main.go:209-213`).
`fuj sync` had no equivalent. It always committed three potential writes to the
payments sheet: `WriteHeader` (if the header row is missing/wrong), `AppendValues`
(for each new Fio transaction), and `SortByDateColumn` (if `--sort`, default true).
For inspecting what a sync *would* do — useful when debugging dedupe, sanity-checking
a date window, or wiring up the command for the first time on a new account — the
only options were pointing at a throwaway spreadsheet or reading the diff after the fact.
This change mirrors `infer`'s read-only mode for `sync`: same flag name, same output
style, same "build the data structures, print instead of writing" shape.
## Files modified
1. `go/internal/services/banksync/sync.go``DryRun bool` field added to `SyncOpts`; three write points gated on `opts.DryRun`
2. `go/cmd/fuj/main.go``--dry-run` flag added to `syncCmd`; final println split on `*dryRun`
3. `go/internal/services/banksync/sync_test.go``TestSyncToSheets_DryRun` added
4. `CHANGELOG.md` — entry added
## Behaviour
When `--dry-run` is set:
- If the sheet header is missing/wrong → prints `Dry run: would write header row`; skips `WriteHeader`
- For each non-deduped Fio transaction → prints `Dry run: would append date=… amount=… sender=… vs=… message=…`; skips `AppendValues`
- If `--sort` is true → prints `Dry run: would sort by date`; skips `SortByDateColumn`
- Returns `len(newRows)` so the caller can print `Dry run: would sync N new transaction(s).`
The existing ID-dedup logic runs in full even during dry-run (reads the sheet, builds `existingIDs`), so the output reflects exactly what the next real sync would do.

View File

@@ -0,0 +1,29 @@
# Add `--print-fio-table` debug flag to `fuj sync`
## Context
The Go port of `fuj sync --dry-run` currently prints only the **new**
transactions — i.e. rows that will be appended to the payments sheet after
deduping against existing Sync IDs (see [sync.go:125-129](../../go/internal/services/banksync/sync.go#L125-L129)).
When debugging Fio sync issues ("why isn't transaction X showing up?",
"is the dedup working?"), there's no way to see what Fio actually
returned versus what got filtered as a duplicate.
This change adds a `--print-fio-table` flag that, **only when combined
with `--dry-run`**, prints an aligned table of every Fio transaction in
the window with each row marked `NEW` (would be appended) or `DUP`
(already in sheet, skipped). The flag is silently ignored without
`--dry-run`, so it can't accidentally fire during a real sync.
## Decisions
- Flag name: `--print-fio-table` (specific, not generic `--verbose`).
- Columns: `DATE | AMOUNT | SENDER | VS | MESSAGE | BANKID | STATUS`,
with MESSAGE truncated and STATUS = `NEW` / `DUP`.
- Scope: only effective when `--dry-run` is also set.
## Files modified
- [go/cmd/fuj/main.go](../../go/cmd/fuj/main.go) — new flag + SyncOpts field
- [go/internal/services/banksync/sync.go](../../go/internal/services/banksync/sync.go) — SyncOpts struct + refactored step 4
- [go/internal/services/banksync/debug.go](../../go/internal/services/banksync/debug.go) — printFioTable helper (new)

View File

@@ -0,0 +1,185 @@
# Python view-model cleanup (M5 prep — Python side only)
Scoped-down precursor to M5 of the Go rewrite. See:
- [2026-05-03-2349-go-backend-rewrite.md](2026-05-03-2349-go-backend-rewrite.md)
- [2026-05-03-2349-go-backend-rewrite-progress.md](2026-05-03-2349-go-backend-rewrite-progress.md)
## Context
The Python app is still production. Before adding `/api/X` shadow
routes, JSON DTOs, parity tooling, and Go handlers, get the Python
side into a clean shape. Today the `/adults` and `/juniors` routes
each carry 150200 lines of inline view-model construction inside
the Flask handler:
- [adults_view](app.py#L179) — 167 LOC, computes per-row
status/cell_text/balance/credits/debts inline.
- [juniors_view](app.py#L347) — 205 LOC, same plus `"?"` sentinel
branching and `:NJ,MA` breakdown.
- [payments](app.py#L553) — 35 LOC, lighter but still mixes IO and
grouping.
Pulling that into pure builder functions:
- shrinks `app.py` and makes each route handler do only what a route
handler should (IO + cache + step timing + render);
- gives us **already-tested** pure functions that future M5 work can
call from a `/api/X` shadow endpoint with one line of `jsonify(...)`;
- has zero behavioural change (existing tests in
[test_app.py](tests/test_app.py) act as the regression guard).
This is a Python-only change. No Go work, no JSON contract, no shadow
routes — those come after.
## Approach
Three pure builder functions in a new `scripts/views.py` module. Each
takes already-loaded, deserialized inputs (no Flask globals, no IO,
no cache calls) and returns the exact dict that today's route passes
to `render_template`.
```python
def build_adults_view_model(
members, sorted_months, transactions, exceptions, current_month,
*, attendance_url, payments_url, bank_account,
) -> dict: ...
def build_juniors_view_model(
junior_members, sorted_months, transactions, exceptions, current_month,
*, attendance_url, payments_url, bank_account,
) -> dict: ...
def build_payments_view_model(
transactions, member_names,
*, attendance_url, payments_url,
) -> dict: ...
```
The route handlers shrink to: load data (with `get_cached_data` and
`record_step` calls staying in `app.py`), call the builder, render.
```python
@app.route("/adults")
def adults_view():
members, sorted_months = ... # cached loads + record_step calls
transactions = ...
exceptions = ...
result_meta = ...
record_step("process_data")
vm = build_adults_view_model(
members, sorted_months, transactions, exceptions,
datetime.now().strftime("%Y-%m"),
attendance_url=..., payments_url=..., bank_account=BANK_ACCOUNT,
)
return render_template("adults.html", **vm)
```
## Decisions
1. **Pure builders, no Flask state.** No `record_step`, no `g.*`, no
`get_cached_data` inside builders. They take plain args, return
plain dicts. This is what makes them trivially unit-testable and,
later, trivially reusable from `/api/X`.
2. **Preserve byte-equal behaviour.** The dicts returned must match
today's `render_template(...)` kwargs key-for-key, value-for-value
— including the existing `json.dumps(member_data)` /
`json.dumps(month_labels_json)` / `json.dumps(raw_payments_json)`
wrappers. Those wrappers exist for inline JS in templates; the
refactor doesn't touch them. (When `/api/X` lands later, that route
will produce a sibling dict with the wrappers stripped, but that's
future work.)
3. **`reconcile()` stays inside the route, not the builder.** It
crosses the IO/cache boundary in spirit (its inputs come from
cache); but it's also pure-domain and called by both adults and
juniors. Keep the call site in the route so `record_step("reconcile")`
timing isn't lost. Builder takes `result` as an argument.
4. **Shared helpers stay in `app.py`** for now —
[get_month_labels](app.py#L41) and
[group_payments_by_person](app.py#L60) are already module-level
pure functions, used by routes and now by builders. Either leave
them in `app.py` and import into `scripts/views.py`, or move them
into `scripts/views.py`. **Choose: move into `scripts/views.py`**
— they're view-model concerns, not Flask concerns, and `app.py`
should keep shrinking.
5. **No new test file needed.** Existing
[tests/test_app.py](tests/test_app.py) tests
`test_adults_route`, `test_juniors_route`, `test_payments_route`
exercise the rendered HTML end-to-end. If they pass after the
refactor, behaviour is preserved. Adding builder-level unit tests
is a *nice-to-have* but not required for this iteration.
## Tasks
### 1. Create `scripts/views.py` with the three builders + shared helpers
- Move `get_month_labels` and `group_payments_by_person` from
[app.py:4177](app.py#L41-L77) into `scripts/views.py`. Update the
import in `app.py`.
- Implement `build_adults_view_model` by extracting
[app.py:200344](app.py#L200-L344) (everything between `result =
reconcile(...)` and `return render_template(...)`). Take `result`
as a parameter; emit the same dict that's currently passed as
`**kwargs` to `render_template`.
- Implement `build_juniors_view_model` by extracting
[app.py:370550](app.py#L370-L550). Same shape — including the
`adapted_members` adapter loop, the junior `?`-sentinel branches,
and the `:NJ,MA` breakdown.
- Implement `build_payments_view_model` by extracting
[app.py:570586](app.py#L570-L586) (the `group_payments_by_person`
call + `Unmatched / Unknown` bucket + sort).
### 2. Slim down the route handlers in `app.py`
Each handler keeps:
- `attendance_url`/`payments_url` URL building
- `get_cached_data` calls (with `record_step` between)
- `reconcile(...)` call (adults/juniors) with `record_step("reconcile")`
- `record_step("process_data")` after the builder call
- `return render_template("X.html", **view_model)`
Each handler **drops** all the per-row computation, totals
formatting, credits/debts sorting, `raw_payments_by_person` building,
and `import json` lines.
Target post-refactor LOC for each route handler: ~2530 lines
(currently 167 / 205 / 35).
### 3. Run the existing test suite + manual smoke
```
make test # all tests green
make web-py # browse /adults /juniors /payments visually
```
## Critical files
- [app.py](app.py) — routes shrink dramatically; `get_month_labels`
and `group_payments_by_person` get moved out.
- New: `scripts/views.py` — three builders + the two helpers.
- [tests/test_app.py](tests/test_app.py) — unchanged; serves as the
regression guard.
## Verification
1. `make test` — all four `TestWebApp` tests pass unchanged.
2. `make web-py` and visit `/adults`, `/juniors`, `/payments` — each
renders identically to before (same table contents, same totals,
same credits/debts, same `?` rendering on juniors).
3. `git diff app.py` shows substantial deletions (route bodies
shrink) and only thin glue calling the new builders.
4. Optional sanity check: temporarily add `print(repr(view_model))`
in the route before `render_template` on `main` and on the branch,
diff for one fixture run — should be byte-identical dicts.
## Out of scope (future iterations, not this plan)
- `/api/<route>` shadow endpoints in Flask
- Go `internal/web/api/` DTOs and JSON Schemas
- Go `internal/services/membership/views.go` aggregators
- Go HTTP handlers for `/api/X`
- `cmd/parity/main.go` and `make parity` target
- Junior breakdown sidecar work in
`go/internal/services/membership/sources.go`
These are the original M5 tasks (M5.1M5.4 in the progress tracker).
They become much easier once today's refactor lands, because the
shadow `/api/X` routes will be one-liners over the new builders.

View File

@@ -0,0 +1,240 @@
# M5.1 — Hand-author Go API structs + emit JSON Schemas
Companion to:
- [2026-05-03-2349-go-backend-rewrite.md](2026-05-03-2349-go-backend-rewrite.md) (master design)
- [2026-05-03-2349-go-backend-rewrite-progress.md](2026-05-03-2349-go-backend-rewrite-progress.md) (progress tracker — M5.1 row)
- [2026-05-07-1431-m5-json-api-parity.md](2026-05-07-1431-m5-json-api-parity.md) (Python view-model extraction prep — already merged as `b562ce3` / `32a16ff` / `59223c0`)
## Context
M4 (IO layer behind interfaces) just landed. M5 is the JSON-parity contract phase — byte-equal JSON between Python and Go for `/api/adults`, `/api/juniors`, `/api/payments`, `/api/version`. Within M5, the work splits four ways:
- **M5.1 — this plan.** Define the wire contract: hand-authored Go structs with explicit `json:` tags matching Python keys, plus committed JSON Schemas generated by `github.com/invopop/jsonschema`. **Schemas only — no handlers, no Python `/api/X` routes, no parity tool.**
- M5.2 — Implement Go handlers that compose `services/*` results into these structs.
- M5.3 — Add Python `/api/X` shadow endpoints in [app.py](app.py).
- M5.4 — `cmd/parity/main.go` + `make parity` target.
The recent Python view-model extraction (`scripts/views.py`) lays the groundwork: every Python builder now returns a plain dict that an `/api/X` shadow can `jsonify` (with one minor unwrap step — see decision #1 below). M5.1 is the matching Go side: types and schemas that pin down the contract before any code writes JSON to a wire.
## Key design decisions
1. **Wire format is nested objects, not strings-of-JSON.** The Python view-model dicts contain three template-only fields that are pre-serialized JSON strings: `member_data`, `month_labels_json`, `raw_payments_json`. Those exist purely to feed inline `<script>` blocks in Jinja templates ([scripts/views.py:227-237](scripts/views.py#L227-L237)). The `/api/X` JSON contract uses the **un-stringified** nested form. Rationale:
- The master design doc references `total_balance`, `original_expected`, `attendance_count` as struct fields ([2026-05-03-2349-go-backend-rewrite.md:223-225](docs/plans/2026-05-03-2349-go-backend-rewrite.md#L223-L225)), which are *inside* `member_data` — only meaningful if it's a nested object.
- The `Expected` `MarshalJSON` design (emit `42` or `"?"`) ([same doc:233-238](docs/plans/2026-05-03-2349-go-backend-rewrite.md#L233-L238)) only fires inside `member_data`; pointless if that field is a string.
- Strings-of-JSON make `invopop/jsonschema` schemas useless for those fields (`{"type": "string"}` describes nothing).
- The Python prep plan ([2026-05-07-1431-m5-json-api-parity.md:87-89](docs/plans/2026-05-07-1431-m5-json-api-parity.md#L87-L89)) already anticipates this: "that route will produce a sibling dict with the wrappers stripped".
This refines (does not contradict) M5.3's "no transformation" wording. M5.3's `/api/X` will be `jsonify(unwrap_json_strings(view_model_dict))` — a 4-line shim, not real transformation logic.
2. **New package: `internal/web/api/`.** The Go side has no `api/` package today — only [internal/web/server.go](go/internal/web/server.go) with one hello route. The api package owns wire types and (in M5.2) handlers. Sub-files per route keep diffs small.
3. **Wire types are separate from `domain/reconcile.Result`.** [domain/reconcile/reconcile.go:88-92](go/internal/domain/reconcile/reconcile.go#L88-L92) defines `Result`, `MemberResult`, `MonthData`, `TxEntry`, etc. — none have `json:` tags. **Don't tag the domain types**: that bleeds wire concerns into pure logic and locks the JSON contract to internal field names. Define wire types in `internal/web/api/` and convert in M5.2's handlers.
4. **`Expected{Value int; Unknown bool}` with custom `MarshalJSON`/`UnmarshalJSON`** for junior `expected` and `original_expected`. Already prescribed by the master design.
5. **Transaction `amount` / `inferred_amount` may be `int` or `""`** (Sheets `UNFORMATTED_VALUE` returns empty string for blank cells per `scripts/match_payments.py` rows 250-260 in the views.py exploration). Use a custom `SheetsNumber` type with `MarshalJSON`/`UnmarshalJSON` that emits `0`/`null` for empty and the number otherwise. Document in code comment with a one-liner. Verify exact behavior by inspecting one or two existing scrubbed reconcile fixtures before coding.
6. **Schema generation lives in `internal/web/api/schemagen_test.go`** with `-update` flag, à la golden tests. Default test run (`go test ./internal/web/api/...`) re-generates schemas in memory and asserts byte-equality vs the committed files in `tests/fixtures/api-schema/`. `go test -update ./internal/web/api/...` rewrites the committed files. Avoids a separate `cmd/gen-api-schema` binary; CI catches drift automatically.
7. **One schema per route**, named to match the Go type:
```
go/tests/fixtures/api-schema/adults.schema.json
go/tests/fixtures/api-schema/juniors.schema.json
go/tests/fixtures/api-schema/payments.schema.json
go/tests/fixtures/api-schema/version.schema.json
```
8. **Adults and Juniors are *not* the same type.** Their outer shapes match (same keys), but `MonthCell.Text` semantics differ (juniors render `"?"`, `"?(3)"`, `:NJ,MA` breakdowns) and `member_data` semantics differ (juniors carry `Expected` sentinel). Define `AdultsResponse` and `JuniorsResponse` separately even if they share most field types — clearer schemas and easier to evolve independently. Share scalar types (`Transaction`, `Exception`, `MonthCell`, `TotalCell`) in `types.go`.
9. **Money is integer CZK** ([master design:55-56](docs/plans/2026-05-03-2349-go-backend-rewrite.md#L55-L56)). All amount fields use `int`. The one exception: `member_data[name].months[YYYY-MM].paid` is a `float64` in the Python output (proportional allocation produces fractional CZK like `33.333333`). Use `float64` only there; document the why in a one-line comment.
## Files to create
```
go/internal/web/api/
├── types.go # SheetsNumber, Expected, Exception, Transaction, MonthCell, TotalCell, MemberRow
├── adults.go # AdultsResponse + adults-specific MemberData
├── juniors.go # JuniorsResponse + juniors-specific MemberData (with Expected fields)
├── payments.go # PaymentsResponse
├── version.go # VersionResponse
└── schemagen_test.go # generates + golden-asserts schemas
go/tests/fixtures/api-schema/
├── adults.schema.json
├── juniors.schema.json
├── payments.schema.json
└── version.schema.json
```
## Struct skeleton (illustrative, not final wording)
```go
// internal/web/api/types.go
// SheetsNumber wraps a value that may arrive from Google Sheets as either
// a JSON number or "" (empty string for blank cells). Marshals as 0 when
// missing; the consumer treats Missing+Value=0 as "no data".
type SheetsNumber struct {
Value float64
Missing bool
}
// Expected carries a junior's expected fee or the "?" sentinel
// (single-attendance month requires manual review).
type Expected struct {
Value int
Unknown bool
}
type Exception struct {
Amount int `json:"amount"`
Note string `json:"note"`
}
type Transaction struct {
Date string `json:"date"`
Amount SheetsNumber `json:"amount"`
ManualFix string `json:"manual_fix"`
Person string `json:"person"`
Purpose string `json:"purpose"`
InferredAmount SheetsNumber `json:"inferred_amount"`
Sender string `json:"sender"`
Message string `json:"message"`
BankID string `json:"bank_id"`
}
type MonthCell struct {
Text string `json:"text"`
Overridden bool `json:"overridden"`
Status string `json:"status"` // "empty"|"ok"|"partial"|"unpaid"|"surplus"
Amount int `json:"amount"`
Month string `json:"month"` // display label, e.g. "Apr+May 2025"
RawMonth string `json:"raw_month"` // YYYY-MM
Tooltip string `json:"tooltip"`
}
type TotalCell struct {
Text string `json:"text"`
Status string `json:"status"`
}
type MemberRow struct {
Name string `json:"name"`
Months []MonthCell `json:"months"`
Balance int `json:"balance"`
UnpaidPeriods string `json:"unpaid_periods"`
RawUnpaidPeriods string `json:"raw_unpaid_periods"`
PayableAmount int `json:"payable_amount"`
}
type Credit struct {
Name string `json:"name"`
Amount int `json:"amount"`
}
```
```go
// internal/web/api/adults.go
type AdultsMonthData struct {
Status string `json:"status"`
Expected int `json:"expected"`
Paid float64 `json:"paid"` // float — proportional allocator can produce 33.333…
Exception *Exception `json:"exception"`
AmountToPay int `json:"amount_to_pay"`
// ... transactions, others, etc — match exact keys from result["members"][name]["months"][YYYY-MM]
}
type AdultsMemberData struct {
TotalBalance int `json:"total_balance"`
Months map[string]AdultsMonthData `json:"months"` // YYYY-MM key
Transactions []Transaction `json:"transactions"`
}
type AdultsResponse struct {
Months []string `json:"months"`
RawMonths []string `json:"raw_months"`
Results []MemberRow `json:"results"`
Totals []TotalCell `json:"totals"`
MemberData map[string]AdultsMemberData `json:"member_data"`
MonthLabels map[string]string `json:"month_labels"` // was month_labels_json (string)
RawPayments map[string][]Transaction `json:"raw_payments"` // was raw_payments_json (string)
Credits []Credit `json:"credits"`
Debts []Credit `json:"debts"`
Unmatched []Transaction `json:"unmatched"`
AttendanceURL string `json:"attendance_url"`
PaymentsURL string `json:"payments_url"`
BankAccount string `json:"bank_account"`
CurrentMonth string `json:"current_month"`
}
```
`JuniorsResponse` mirrors `AdultsResponse` but the inner `MonthData` carries `Expected` and `OriginalExpected` (both `Expected` type), and adds the `:NJ,MA` breakdown fields produced by [scripts/views.py:290-298](scripts/views.py#L290-L298).
`PaymentsResponse`:
```go
type PaymentsResponse struct {
GroupedPayments map[string][]Transaction `json:"grouped_payments"`
SortedPeople []string `json:"sorted_people"`
AttendanceURL string `json:"attendance_url"`
PaymentsURL string `json:"payments_url"`
}
```
`VersionResponse` mirrors Python's `BUILD_META` ([app.py:67](app.py#L67)):
```go
type VersionResponse struct {
Tag string `json:"tag"`
Commit string `json:"commit"`
BuildDate string `json:"build_date"`
}
```
Exact fields inside the per-month `MonthData`/`MemberData` will be finalized by inspecting **one** scrubbed `member_data` JSON dump from a current `/adults` and `/juniors` call (see Verification step 1) — names and types have to be identical.
## Reusable existing code
- `domain/reconcile.Result` ([go/internal/domain/reconcile/reconcile.go:88](go/internal/domain/reconcile/reconcile.go#L88)) — source data for adults/juniors. M5.2 maps it to wire types; M5.1 only needs to know the field set.
- `web.BuildInfo` ([go/internal/web/server.go:11-15](go/internal/web/server.go#L11-L15)) — already wired through `cmd/fuj/main.go:79`. The `VersionResponse` type is a thin renaming of this with json tags. Decide in M5.2 whether to *also* tag `BuildInfo` directly (it's not really domain) or keep `VersionResponse` separate. M5.1 just defines the wire struct.
- `scripts/views.py` builders — the spec for every key/type. Treat as authoritative.
## Tasks
1. **Add `github.com/invopop/jsonschema` dependency.** `cd go && go get github.com/invopop/jsonschema && go mod tidy`. Single commit.
2. **Capture one fresh `member_data` dump for adults and juniors** to pin down inner-dict field names and types precisely (especially `paid` precision, `transactions[]` shape, `others[]` if present). Run `make web-py` on a known-good cache, hit `/adults` and `/juniors`, dump `view_model["member_data"]` to a scratch file (gitignored), and inspect. **Do not commit raw dumps** — PII rule. This is for shape inspection only.
3. **Author `internal/web/api/types.go`** with `SheetsNumber`, `Expected`, `Exception`, `Transaction`, `MonthCell`, `TotalCell`, `MemberRow`, `Credit`. Implement `MarshalJSON`/`UnmarshalJSON` for `SheetsNumber` and `Expected`. Use struct tags `json:"snake_case_key"` matching Python exactly.
4. **Author `internal/web/api/adults.go`** with `AdultsMonthData`, `AdultsMemberData`, `AdultsResponse`. Cross-check every key against the dump from step 2.
5. **Author `internal/web/api/juniors.go`** similarly, with `Expected` fields and the J/A breakdown.
6. **Author `internal/web/api/payments.go` and `version.go`** (small).
7. **Author `internal/web/api/schemagen_test.go`** that:
- Imports `github.com/invopop/jsonschema`.
- Defines a `schemaCases` slice: `{name: "adults", typ: AdultsResponse{}, ...}` etc.
- For each case, generates a schema, marshals indented JSON, compares against committed file at `../../tests/fixtures/api-schema/<name>.schema.json`.
- Honors a `-update` flag (`flag.Bool`) that rewrites the committed file instead of asserting.
- One `t.Run(case.name, ...)` per route for clear failure output.
8. **Run with `-update` once to populate** `tests/fixtures/api-schema/*.schema.json`. Eyeball each schema: required fields list, oneOf for `Expected`, additionalProperties on map types. Commit.
9. **Lint + test:** `cd go && go vet ./... && make go-test && make go-lint`. Fix any issues. (Expect zero — this is read-only data structures.)
10. **CHANGELOG entry** + tick `M5.1` in [docs/plans/2026-05-03-2349-go-backend-rewrite-progress.md:100](docs/plans/2026-05-03-2349-go-backend-rewrite-progress.md#L100) with the merge SHA.
11. **Branch + MR per CLAUDE.md workflow:** branch `feat/go-m5-1-api-structs`, push with `-u`, open MR via `tea pr create`. Do not merge from CLI.
## Verification
1. `cd go && go test ./internal/web/api/...` — **schemas regenerate identically** to committed files.
2. `cd go && go test ./internal/web/api/... -update` then `git diff go/tests/fixtures/api-schema/` — diff is empty (idempotent generation).
3. `cd go && make go-lint` — clean.
4. `cd go && go vet ./...` — clean.
5. Manual schema inspection: open `adults.schema.json`, confirm:
- Top-level `required` list contains every Python key.
- `member_data` is `additionalProperties: { ... AdultsMemberData ... }` (a map keyed by name).
- `expected` and `original_expected` (juniors only) are `oneOf: [{type: integer}, {const: "?"}]`.
- `amount` and `inferred_amount` on `Transaction` accept number or empty/null.
6. **No production code paths exercised yet** — handlers come in M5.2. Compile-time success + schema golden-test = M5.1 done.
## Out of scope (later M5 tasks)
- Wiring Go HTTP handlers for `/api/X` (M5.2).
- Adding Python `/api/X` shadow endpoints (M5.3) — including the `unwrap_json_strings(view_model)` shim noted in design decision #1.
- `cmd/parity/main.go` and `make parity` target (M5.4).
- Tagging `domain/reconcile.Result` with `json:` tags — explicitly avoided.
- Refactoring the strings-of-JSON fields out of the Python view-model — they stay in `views.py` for the template path, the `/api/X` shadow unwraps them.

View File

@@ -5,9 +5,13 @@ import (
"flag"
"fmt"
"fuj-management/go/internal/config"
"fuj-management/go/internal/io/fio"
"fuj-management/go/internal/io/sheets"
"fuj-management/go/internal/logging"
"fuj-management/go/internal/services/banksync"
"fuj-management/go/internal/services/membership"
"fuj-management/go/internal/web"
"log/slog"
"os"
"time"
)
@@ -25,6 +29,9 @@ func main() {
os.Exit(2)
}
// Honour LOG_LEVEL for slog calls in any package (e.g. internal/io/fio debug logs).
slog.SetDefault(logging.New(os.Getenv("LOG_LEVEL")))
cmd, args := os.Args[1], os.Args[2:]
switch cmd {
@@ -36,9 +43,10 @@ func main() {
feesCmd(args)
case "reconcile":
reconcileCmd(args)
case "sync", "infer":
fmt.Fprintf(os.Stderr, "fuj %s: not implemented yet (lands in M4)\n", cmd)
os.Exit(2)
case "sync":
syncCmd(args)
case "infer":
inferCmd(args)
case "-h", "--help", "help":
usage()
default:
@@ -65,10 +73,18 @@ func serverCmd(args []string) {
cfg.ServerAddr = *addr
}
ctx := context.Background()
logger := logging.New(cfg.LogLevel)
sources, err := membership.NewSources(ctx, cfg)
if err != nil {
fmt.Fprintf(os.Stderr, "fuj server: init sources: %v\n", err)
os.Exit(1)
}
build := web.BuildInfo{Version: version, Commit: commit, BuildDate: buildDate}
if err := web.Run(logger, cfg.ServerAddr, build); err != nil {
if err := web.Run(logger, cfg.ServerAddr, build, sources, cfg); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
@@ -84,8 +100,14 @@ func feesCmd(args []string) {
os.Exit(2)
}
sources := membership.NewStubSources()
if err := membership.FeesReport(context.Background(), sources, os.Stdout); err != nil {
ctx := context.Background()
cfg := config.Load()
sources, err := membership.NewSources(ctx, cfg)
if err != nil {
fmt.Fprintf(os.Stderr, "fuj fees: %v\n", err)
os.Exit(1)
}
if err := membership.FeesReport(ctx, sources, os.Stdout); err != nil {
fmt.Fprintf(os.Stderr, "fuj fees: %v\n", err)
os.Exit(1)
}
@@ -101,8 +123,14 @@ func reconcileCmd(args []string) {
os.Exit(2)
}
sources := membership.NewStubSources()
if err := membership.ReconcileReport(context.Background(), sources, time.Now().Year(), os.Stdout); err != nil {
ctx := context.Background()
cfg := config.Load()
sources, err := membership.NewSources(ctx, cfg)
if err != nil {
fmt.Fprintf(os.Stderr, "fuj reconcile: %v\n", err)
os.Exit(1)
}
if err := membership.ReconcileReport(ctx, sources, time.Now().Year(), os.Stdout); err != nil {
fmt.Fprintf(os.Stderr, "fuj reconcile: %v\n", err)
os.Exit(1)
}
@@ -112,6 +140,97 @@ func versionCmd() {
fmt.Printf("fuj %s (%s) built %s\n", version, commit, buildDate)
}
func syncCmd(args []string) {
fs := flag.NewFlagSet("sync", flag.ExitOnError)
days := fs.Int("days", 30, "look-back window in days (ignored when --from/--to are set)")
fromStr := fs.String("from", "", "start date YYYY-MM-DD")
toStr := fs.String("to", "", "end date YYYY-MM-DD")
sort := fs.Bool("sort", true, "sort sheet by date after appending")
dryRun := fs.Bool("dry-run", false, "print planned writes without modifying the sheet")
printFioTable := fs.Bool("print-fio-table", false, "with --dry-run: print aligned table of every Fio transaction with NEW/DUP status")
fs.Usage = func() {
fmt.Fprintln(os.Stderr, "usage: fuj sync [--days N] [--from YYYY-MM-DD --to YYYY-MM-DD] [--sort] [--dry-run] [--print-fio-table]")
fs.PrintDefaults()
}
if err := fs.Parse(args); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(2)
}
ctx := context.Background()
cfg := config.Load()
sheetsCli, err := sheets.New(ctx, cfg.CredentialsPath, cfg.DriveTimeout)
if err != nil {
fmt.Fprintf(os.Stderr, "fuj sync: sheets client: %v\n", err)
os.Exit(1)
}
fioCli := fio.New(cfg.FioAPIToken, config.IBANAccountNum(cfg.BankAccount), nil)
opts := banksync.SyncOpts{Days: *days, Sort: *sort, DryRun: *dryRun, PrintFioTable: *printFioTable}
if *fromStr != "" && *toStr != "" {
opts.From, err = time.Parse("2006-01-02", *fromStr)
if err != nil {
fmt.Fprintf(os.Stderr, "fuj sync: invalid --from: %v\n", err)
os.Exit(2)
}
opts.To, err = time.Parse("2006-01-02", *toStr)
if err != nil {
fmt.Fprintf(os.Stderr, "fuj sync: invalid --to: %v\n", err)
os.Exit(2)
}
}
n, err := banksync.SyncToSheets(ctx, config.PaymentsSheetID, fioCli, sheetsCli, opts)
if err != nil {
fmt.Fprintf(os.Stderr, "fuj sync: %v\n", err)
os.Exit(1)
}
if *dryRun {
fmt.Printf("Dry run: would sync %d new transaction(s).\n", n)
} else {
fmt.Printf("Synced %d new transaction(s).\n", n)
}
}
func inferCmd(args []string) {
fs := flag.NewFlagSet("infer", flag.ExitOnError)
dryRun := fs.Bool("dry-run", false, "print planned updates without writing to the sheet")
fs.Usage = func() {
fmt.Fprintln(os.Stderr, "usage: fuj infer [--dry-run]")
fs.PrintDefaults()
}
if err := fs.Parse(args); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(2)
}
ctx := context.Background()
cfg := config.Load()
sheetsCli, err := sheets.New(ctx, cfg.CredentialsPath, cfg.DriveTimeout)
if err != nil {
fmt.Fprintf(os.Stderr, "fuj infer: sheets client: %v\n", err)
os.Exit(1)
}
sources, err := membership.NewSources(ctx, cfg)
if err != nil {
fmt.Fprintf(os.Stderr, "fuj infer: sources: %v\n", err)
os.Exit(1)
}
n, err := banksync.InferPayments(ctx, config.PaymentsSheetID, sheetsCli, sources, banksync.InferOpts{DryRun: *dryRun})
if err != nil {
fmt.Fprintf(os.Stderr, "fuj infer: %v\n", err)
os.Exit(1)
}
if *dryRun {
fmt.Printf("Dry run: would update %d row(s).\n", n)
} else {
fmt.Printf("Updated %d row(s).\n", n)
}
}
func usage() {
fmt.Fprintln(os.Stderr, `usage: fuj <command> [flags]
@@ -120,6 +239,6 @@ Commands:
version Print version information
fees Calculate monthly fees
reconcile Show balance report
sync Sync Fio transactions [M4]
infer Infer payment details [M4]`)
sync Sync Fio transactions to payments sheet
infer Infer payment details in payments sheet`)
}

View File

@@ -2,4 +2,38 @@ module fuj-management/go
go 1.26.1
require golang.org/x/text v0.36.0
require (
github.com/invopop/jsonschema v0.14.0
golang.org/x/net v0.53.0
golang.org/x/text v0.36.0
google.golang.org/api v0.278.0
)
require (
cloud.google.com/go/auth v0.20.0 // indirect
cloud.google.com/go/auth/oauth2adapt v0.2.8 // indirect
cloud.google.com/go/compute/metadata v0.9.0 // indirect
github.com/bahlo/generic-list-go v0.2.0 // indirect
github.com/buger/jsonparser v1.1.2 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/felixge/httpsnoop v1.0.4 // indirect
github.com/go-logr/logr v1.4.3 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/google/s2a-go v0.1.9 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.3.15 // indirect
github.com/googleapis/gax-go/v2 v2.22.0 // indirect
github.com/pb33f/ordered-map/v2 v2.3.1 // indirect
go.opentelemetry.io/auto/sdk v1.2.1 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.67.0 // indirect
go.opentelemetry.io/otel v1.43.0 // indirect
go.opentelemetry.io/otel/metric v1.43.0 // indirect
go.opentelemetry.io/otel/trace v1.43.0 // indirect
go.yaml.in/yaml/v4 v4.0.0-rc.2 // indirect
golang.org/x/crypto v0.50.0 // indirect
golang.org/x/oauth2 v0.36.0 // indirect
golang.org/x/sys v0.43.0 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20260427160629-7cedc36a6bc4 // indirect
google.golang.org/grpc v1.80.0 // indirect
google.golang.org/protobuf v1.36.11 // indirect
)

View File

@@ -1,2 +1,85 @@
cloud.google.com/go/auth v0.20.0 h1:kXTssoVb4azsVDoUiF8KvxAqrsQcQtB53DcSgta74CA=
cloud.google.com/go/auth v0.20.0/go.mod h1:942/yi/itH1SsmpyrbnTMDgGfdy2BUqIKyd0cyYLc5Q=
cloud.google.com/go/auth/oauth2adapt v0.2.8 h1:keo8NaayQZ6wimpNSmW5OPc283g65QNIiLpZnkHRbnc=
cloud.google.com/go/auth/oauth2adapt v0.2.8/go.mod h1:XQ9y31RkqZCcwJWNSx2Xvric3RrU88hAYYbjDWYDL+c=
cloud.google.com/go/compute/metadata v0.9.0 h1:pDUj4QMoPejqq20dK0Pg2N4yG9zIkYGdBtwLoEkH9Zs=
cloud.google.com/go/compute/metadata v0.9.0/go.mod h1:E0bWwX5wTnLPedCKqk3pJmVgCBSM6qQI1yTBdEb3C10=
github.com/bahlo/generic-list-go v0.2.0 h1:5sz/EEAK+ls5wF+NeqDpk5+iNdMDXrh3z3nPnH1Wvgk=
github.com/bahlo/generic-list-go v0.2.0/go.mod h1:2KvAjgMlE5NNynlg/5iLrrCCZ2+5xWbdbCW3pNTGyYg=
github.com/buger/jsonparser v1.1.2 h1:frqHqw7otoVbk5M8LlE/L7HTnIq2v9RX6EJ48i9AxJk=
github.com/buger/jsonparser v1.1.2/go.mod h1:6RYKKt7H4d4+iWqouImQ9R2FZql3VbhNgx27UK13J/0=
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/felixge/httpsnoop v1.0.4 h1:NFTV2Zj1bL4mc9sqWACXbQFVBBg2W3GPvqp8/ESS2Wg=
github.com/felixge/httpsnoop v1.0.4/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U=
github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A=
github.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI=
github.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=
github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek=
github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps=
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
github.com/google/s2a-go v0.1.9 h1:LGD7gtMgezd8a/Xak7mEWL0PjoTQFvpRudN895yqKW0=
github.com/google/s2a-go v0.1.9/go.mod h1:YA0Ei2ZQL3acow2O62kdp9UlnvMmU7kA6Eutn0dXayM=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/googleapis/enterprise-certificate-proxy v0.3.15 h1:xolVQTEXusUcAA5UgtyRLjelpFFHWlPQ4XfWGc7MBas=
github.com/googleapis/enterprise-certificate-proxy v0.3.15/go.mod h1:vqVt9yG9480NtzREnTlmGSBmFrA+bzb0yl0TxoBQXOg=
github.com/googleapis/gax-go/v2 v2.22.0 h1:PjIWBpgGIVKGoCXuiCoP64altEJCj3/Ei+kSU5vlZD4=
github.com/googleapis/gax-go/v2 v2.22.0/go.mod h1:irWBbALSr0Sk3qlqb9SyJ1h68WjgeFuiOzI4Rqw5+aY=
github.com/invopop/jsonschema v0.14.0 h1:MHQqLhvpNUZfw+hM3AZDYK7jxO8FZoQeQM77g8iyZjg=
github.com/invopop/jsonschema v0.14.0/go.mod h1:ygm6C2EaVNMBDPpaPlnOA2pFAxBnxGjFlMZABxm9n2I=
github.com/pb33f/ordered-map/v2 v2.3.1 h1:5319HDO0aw4DA4gzi+zv4FXU9UlSs3xGZ40wcP1nBjY=
github.com/pb33f/ordered-map/v2 v2.3.1/go.mod h1:qxFQgd0PkVUtOMCkTapqotNgzRhMPL7VvaHKbd1HnmQ=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
go.opentelemetry.io/auto/sdk v1.2.1 h1:jXsnJ4Lmnqd11kwkBV2LgLoFMZKizbCi5fNZ/ipaZ64=
go.opentelemetry.io/auto/sdk v1.2.1/go.mod h1:KRTj+aOaElaLi+wW1kO/DZRXwkF4C5xPbEe3ZiIhN7Y=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.67.0 h1:OyrsyzuttWTSur2qN/Lm0m2a8yqyIjUVBZcxFPuXq2o=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.67.0/go.mod h1:C2NGBr+kAB4bk3xtMXfZ94gqFDtg/GkI7e9zqGh5Beg=
go.opentelemetry.io/otel v1.43.0 h1:mYIM03dnh5zfN7HautFE4ieIig9amkNANT+xcVxAj9I=
go.opentelemetry.io/otel v1.43.0/go.mod h1:JuG+u74mvjvcm8vj8pI5XiHy1zDeoCS2LB1spIq7Ay0=
go.opentelemetry.io/otel/metric v1.43.0 h1:d7638QeInOnuwOONPp4JAOGfbCEpYb+K6DVWvdxGzgM=
go.opentelemetry.io/otel/metric v1.43.0/go.mod h1:RDnPtIxvqlgO8GRW18W6Z/4P462ldprJtfxHxyKd2PY=
go.opentelemetry.io/otel/sdk v1.43.0 h1:pi5mE86i5rTeLXqoF/hhiBtUNcrAGHLKQdhg4h4V9Dg=
go.opentelemetry.io/otel/sdk v1.43.0/go.mod h1:P+IkVU3iWukmiit/Yf9AWvpyRDlUeBaRg6Y+C58QHzg=
go.opentelemetry.io/otel/sdk/metric v1.43.0 h1:S88dyqXjJkuBNLeMcVPRFXpRw2fuwdvfCGLEo89fDkw=
go.opentelemetry.io/otel/sdk/metric v1.43.0/go.mod h1:C/RJtwSEJ5hzTiUz5pXF1kILHStzb9zFlIEe85bhj6A=
go.opentelemetry.io/otel/trace v1.43.0 h1:BkNrHpup+4k4w+ZZ86CZoHHEkohws8AY+WTX09nk+3A=
go.opentelemetry.io/otel/trace v1.43.0/go.mod h1:/QJhyVBUUswCphDVxq+8mld+AvhXZLhe+8WVFxiFff0=
go.yaml.in/yaml/v4 v4.0.0-rc.2 h1:/FrI8D64VSr4HtGIlUtlFMGsm7H7pWTbj6vOLVZcA6s=
go.yaml.in/yaml/v4 v4.0.0-rc.2/go.mod h1:aZqd9kCMsGL7AuUv/m/PvWLdg5sjJsZ4oHDEnfPPfY0=
golang.org/x/crypto v0.50.0 h1:zO47/JPrL6vsNkINmLoo/PH1gcxpls50DNogFvB5ZGI=
golang.org/x/crypto v0.50.0/go.mod h1:3muZ7vA7PBCE6xgPX7nkzzjiUq87kRItoJQM1Yo8S+Q=
golang.org/x/net v0.53.0 h1:d+qAbo5L0orcWAr0a9JweQpjXF19LMXJE8Ey7hwOdUA=
golang.org/x/net v0.53.0/go.mod h1:JvMuJH7rrdiCfbeHoo3fCQU24Lf5JJwT9W3sJFulfgs=
golang.org/x/oauth2 v0.36.0 h1:peZ/1z27fi9hUOFCAZaHyrpWG5lwe0RJEEEeH0ThlIs=
golang.org/x/oauth2 v0.36.0/go.mod h1:YDBUJMTkDnJS+A4BP4eZBjCqtokkg1hODuPjwiGPO7Q=
golang.org/x/sync v0.20.0 h1:e0PTpb7pjO8GAtTs2dQ6jYa5BWYlMuX047Dco/pItO4=
golang.org/x/sync v0.20.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
golang.org/x/sys v0.43.0 h1:Rlag2XtaFTxp19wS8MXlJwTvoh8ArU6ezoyFsMyCTNI=
golang.org/x/sys v0.43.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/text v0.36.0 h1:JfKh3XmcRPqZPKevfXVpI1wXPTqbkE5f7JA92a55Yxg=
golang.org/x/text v0.36.0/go.mod h1:NIdBknypM8iqVmPiuco0Dh6P5Jcdk8lJL0CUebqK164=
gonum.org/v1/gonum v0.17.0 h1:VbpOemQlsSMrYmn7T2OUvQ4dqxQXU+ouZFQsZOx50z4=
gonum.org/v1/gonum v0.17.0/go.mod h1:El3tOrEuMpv2UdMrbNlKEh9vd86bmQ6vqIcDwxEOc1E=
google.golang.org/api v0.278.0 h1:W7jiRvRi53VYFfZ/HoZjQBtJk7gOFbHD8ot1RzVZU6E=
google.golang.org/api v0.278.0/go.mod h1:B9TqLBwJqVjp1mtt7WeoQwWRwvu/400y5lETOql+giQ=
google.golang.org/genproto v0.0.0-20260319201613-d00831a3d3e7 h1:XzmzkmB14QhVhgnawEVsOn6OFsnpyxNPRY9QV01dNB0=
google.golang.org/genproto v0.0.0-20260319201613-d00831a3d3e7/go.mod h1:L43LFes82YgSonw6iTXTxXUX1OlULt4AQtkik4ULL/I=
google.golang.org/genproto/googleapis/api v0.0.0-20260319201613-d00831a3d3e7 h1:41r6JMbpzBMen0R/4TZeeAmGXSJC7DftGINUodzTkPI=
google.golang.org/genproto/googleapis/api v0.0.0-20260319201613-d00831a3d3e7/go.mod h1:EIQZ5bFCfRQDV4MhRle7+OgjNtZ6P1PiZBgAKuxXu/Y=
google.golang.org/genproto/googleapis/rpc v0.0.0-20260427160629-7cedc36a6bc4 h1:tEkOQcXgF6dH1G+MVKZrfpYvozGrzb91k6ha7jireSM=
google.golang.org/genproto/googleapis/rpc v0.0.0-20260427160629-7cedc36a6bc4/go.mod h1:4Hqkh8ycfw05ld/3BWL7rJOSfebL2Q+DVDeRgYgxUU8=
google.golang.org/grpc v1.80.0 h1:Xr6m2WmWZLETvUNvIUmeD5OAagMw3FiKmMlTdViWsHM=
google.golang.org/grpc v1.80.0/go.mod h1:ho/dLnxwi3EDJA4Zghp7k2Ec1+c2jqup0bFkw07bwF4=
google.golang.org/protobuf v1.36.11 h1:fV6ZwhNocDyBLK0dj+fg8ektcVegBBuEolpbTQyBNVE=
google.golang.org/protobuf v1.36.11/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

View File

@@ -3,6 +3,7 @@ package config
import (
"os"
"strconv"
"strings"
"time"
)
@@ -10,16 +11,34 @@ import (
const (
AttendanceSheetID = "1E2e_gT_K5AwSRCDLDTa2UetZTkHmBOcz0kFbBUNUNBA"
PaymentsSheetID = "1Om0YPoDVCH5cV8BrNz5LG5eR5MMU05ypQC7UMN1xn_Y"
JuniorSheetGID = "1213318614"
// Both attendance tabs live in the same Google Spreadsheet (AttendanceSheetID).
// The original adult and junior attendance data lives in separate source spreadsheets,
// but is collected into this one sheet via IMPORTRANGE — one tab per group.
// Tabs are identified by the gid= query param in the CSV export URL.
AttendanceAdultSheetGID = "0" // gid=0 — adult practices tab (IMPORTRANGE'd)
JuniorSheetGID = "1213318614" // gid=1213318614 — junior practices tab (IMPORTRANGE'd)
)
// CacheSheetMap mirrors scripts/config.py CACHE_SHEET_MAP.
// Maps a cache key to the Google Sheet ID whose Drive modifiedTime gates it.
// Both attendance keys map to the same spreadsheet — different tabs, one Drive file.
var CacheSheetMap = map[string]string{
"attendance_regular": AttendanceSheetID,
"attendance_juniors": AttendanceSheetID,
"exceptions_dict": PaymentsSheetID,
"payments_transactions": PaymentsSheetID,
}
// Config holds all runtime configuration loaded from environment variables.
// Mirrors scripts/config.py.
type Config struct {
CredentialsPath string
BankAccount string
CacheDir string
CacheTTL time.Duration
CacheAPICheckTTL time.Duration
DriveTimeout time.Duration
LogLevel string
FioAPIToken string
ServerAddr string
@@ -31,14 +50,32 @@ func Load() Config {
return Config{
CredentialsPath: env("CREDENTIALS_PATH", ".secret/fuj-management-bot-credentials.json"),
BankAccount: env("BANK_ACCOUNT", "CZ8520100000002800359168"),
CacheDir: env("CACHE_DIR", "tmp"),
CacheTTL: envDuration("CACHE_TTL_SECONDS", 300),
CacheAPICheckTTL: envDuration("CACHE_API_CHECK_TTL_SECONDS", 300),
DriveTimeout: envDuration("DRIVE_TIMEOUT_SECONDS", 10),
LogLevel: env("LOG_LEVEL", "INFO"),
FioAPIToken: env("FIO_API_TOKEN", ""),
ServerAddr: env("SERVER_ADDR", ":8080"),
}
}
// IBANAccountNum extracts the bare account number from a Czech IBAN.
// "CZ8520100000002800359168" → "2800359168"
// Structure: CZ(2 check)(4 bank code)(16 zero-padded account).
func IBANAccountNum(iban string) string {
s := strings.ReplaceAll(iban, " ", "")
if len(s) < 8 {
return iban
}
raw := s[8:] // 16-digit zero-padded account portion
n := strings.TrimLeft(raw, "0")
if n == "" {
return "0"
}
return n
}
func env(key, fallback string) string {
if v := os.Getenv(key); v != "" {
return v

View File

@@ -20,10 +20,13 @@ type Exception struct {
Note string
}
// FeeData holds the expected fee and attendance count for one member in one month.
// FeeData holds the expected fee and attendance data for one member in one month.
type FeeData struct {
Expected int
Attendance int
Expected int
IsUnknown bool // true when junior has exactly 1 session (manual review; Python sentinel "?")
Attendance int
JuniorAttendance int // junior-tab sessions; used for the :NJ,MA breakdown in the juniors view
AdultAttendance int // adult-tab sessions for J-tier members; used for the :NJ,MA breakdown
}
// Member is one row from the attendance sheet.
@@ -39,11 +42,15 @@ type Member struct {
type Transaction struct {
Date string
Amount float64
ManualFix string // "manual fix" column; non-empty disables re-inference
Person string // comma-separated canonical names (empty → use inference)
Purpose string // comma-separated "YYYY-MM" or "other:…" (empty → use inference)
InferredAmount *float64 // nil → fall back to Amount
Sender string
VS string // Variabilní symbol (Czech variable payment symbol)
Message string
BankID string
SyncID string
UserID string
}
@@ -69,8 +76,11 @@ type OtherEntry struct {
// MonthData is the ledger state for one member in one month.
type MonthData struct {
Expected int
IsUnknown bool // mirrors FeeData.IsUnknown; not overridden by exceptions
OriginalExpected int
AttendanceCount int
JuniorAttendance int // junior-tab sessions; for :NJ,MA breakdown in juniors view
AdultAttendance int // adult-tab sessions; for :NJ,MA breakdown
Exception *Exception
Paid float64
Transactions []TxEntry
@@ -173,8 +183,11 @@ func Reconcile(
ledger[name][m] = MonthData{
Expected: expected,
IsUnknown: fd.IsUnknown,
OriginalExpected: originalExpected,
AttendanceCount: attendanceCount,
JuniorAttendance: fd.JuniorAttendance,
AdultAttendance: fd.AdultAttendance,
Exception: exInfo,
Paid: 0,
Transactions: []TxEntry{},

View File

@@ -29,7 +29,7 @@ func tx(person, purpose string, amount float64) Transaction {
func TestReconcileExceptionOverride(t *testing.T) {
t.Parallel()
members := []Member{{Name: "Alice", Tier: "A", Fees: map[string]FeeData{"2026-01": {750, 4}}}}
members := []Member{{Name: "Alice", Tier: "A", Fees: map[string]FeeData{"2026-01": {Expected: 750, Attendance: 4}}}}
exceptions := map[ExceptionKey]Exception{
{Name: "alice", Period: "2026-01"}: {Amount: 400, Note: "Test exception"},
}
@@ -54,7 +54,7 @@ func TestReconcileExceptionOverride(t *testing.T) {
func TestReconcileFallbackToAttendance(t *testing.T) {
t.Parallel()
members := []Member{{Name: "Alice", Tier: "A", Fees: map[string]FeeData{"2026-01": {750, 4}}}}
members := []Member{{Name: "Alice", Tier: "A", Fees: map[string]FeeData{"2026-01": {Expected: 750, Attendance: 4}}}}
result := Reconcile(members, []string{"2026-01"}, nil, nil, defaultYear)
@@ -68,9 +68,9 @@ func TestReconcileGreedyExactMatch(t *testing.T) {
members := []Member{{
Name: "Alice", Tier: "A",
Fees: map[string]FeeData{
"2026-02": {750, 3},
"2026-03": {350, 3},
"2026-04": {150, 2},
"2026-02": {Expected: 750, Attendance: 3},
"2026-03": {Expected: 350, Attendance: 3},
"2026-04": {Expected: 150, Attendance: 2},
},
}}
sortedMonths := []string{"2026-02", "2026-03", "2026-04"}
@@ -93,7 +93,7 @@ func TestReconcileGreedyOverpaymentGoesToCredit(t *testing.T) {
t.Parallel()
members := []Member{{
Name: "Alice", Tier: "A",
Fees: map[string]FeeData{"2026-01": {750, 3}, "2026-02": {750, 3}},
Fees: map[string]FeeData{"2026-01": {Expected: 750, Attendance: 3}, "2026-02": {Expected: 750, Attendance: 3}},
}}
sortedMonths := []string{"2026-01", "2026-02"}
@@ -115,7 +115,7 @@ func TestReconcileProportionalUnderpayment(t *testing.T) {
t.Parallel()
members := []Member{{
Name: "Alice", Tier: "A",
Fees: map[string]FeeData{"2026-02": {750, 3}, "2026-03": {350, 3}, "2026-04": {750, 3}},
Fees: map[string]FeeData{"2026-02": {Expected: 750, Attendance: 3}, "2026-03": {Expected: 350, Attendance: 3}, "2026-04": {Expected: 750, Attendance: 3}},
}}
sortedMonths := []string{"2026-02", "2026-03", "2026-04"}
amount := 1250.0
@@ -146,7 +146,7 @@ func TestReconcileProportionalUnderpayment(t *testing.T) {
func TestReconcileSingleMonthUnchanged(t *testing.T) {
t.Parallel()
members := []Member{{Name: "Alice", Tier: "A", Fees: map[string]FeeData{"2026-01": {750, 3}}}}
members := []Member{{Name: "Alice", Tier: "A", Fees: map[string]FeeData{"2026-01": {Expected: 750, Attendance: 3}}}}
result := Reconcile(members, []string{"2026-01"}, []Transaction{tx("Alice", "2026-01", 750)}, nil, defaultYear)
@@ -158,8 +158,8 @@ func TestReconcileSingleMonthUnchanged(t *testing.T) {
func TestReconcileTwoMembersMultiMonth(t *testing.T) {
t.Parallel()
members := []Member{
{Name: "Alice", Tier: "A", Fees: map[string]FeeData{"2026-01": {750, 3}, "2026-02": {350, 3}}},
{Name: "Bob", Tier: "A", Fees: map[string]FeeData{"2026-01": {750, 3}, "2026-02": {350, 3}}},
{Name: "Alice", Tier: "A", Fees: map[string]FeeData{"2026-01": {Expected: 750, Attendance: 3}, "2026-02": {Expected: 350, Attendance: 3}}},
{Name: "Bob", Tier: "A", Fees: map[string]FeeData{"2026-01": {Expected: 750, Attendance: 3}, "2026-02": {Expected: 350, Attendance: 3}}},
}
sortedMonths := []string{"2026-01", "2026-02"}
@@ -180,7 +180,7 @@ func TestReconcileEvenSplitFallbackWhenNoExpected(t *testing.T) {
t.Parallel()
members := []Member{{
Name: "Alice", Tier: "A",
Fees: map[string]FeeData{"2026-01": {0, 0}, "2026-02": {0, 0}},
Fees: map[string]FeeData{"2026-01": {Expected: 0, Attendance: 0}, "2026-02": {Expected: 0, Attendance: 0}},
}}
sortedMonths := []string{"2026-01", "2026-02"}
@@ -197,7 +197,7 @@ func TestReconcileEvenSplitFallbackWhenNoExpected(t *testing.T) {
func TestReconcileDiacriticsTolerantPersonMatching(t *testing.T) {
t.Parallel()
members := []Member{{Name: "Mária Maco", Tier: "A", Fees: map[string]FeeData{"2026-04": {750, 4}}}}
members := []Member{{Name: "Mária Maco", Tier: "A", Fees: map[string]FeeData{"2026-04": {Expected: 750, Attendance: 4}}}}
txFn := func(person string) Transaction {
return Transaction{
Date: "2026-04-15", Amount: 750, Person: person, Purpose: "2026-04",
@@ -232,7 +232,7 @@ func TestReconcileDiacriticsTolerantPersonMatching(t *testing.T) {
func TestReconcileTrulyUnknownPersonIsUnmatched(t *testing.T) {
t.Parallel()
members := []Member{{Name: "Mária Maco", Tier: "A", Fees: map[string]FeeData{"2026-04": {750, 4}}}}
members := []Member{{Name: "Mária Maco", Tier: "A", Fees: map[string]FeeData{"2026-04": {Expected: 750, Attendance: 4}}}}
txs := []Transaction{{
Date: "2026-04-15", Amount: 750,
Person: "Někdo Neznámý", Purpose: "2026-04",
@@ -252,7 +252,7 @@ func TestReconcileTrulyUnknownPersonIsUnmatched(t *testing.T) {
// [Go] Test that [?] markers are stripped from the Person field before lookup.
func TestReconcileQuestionMarkMarkerStripped(t *testing.T) {
t.Parallel()
members := []Member{{Name: "Alice", Tier: "A", Fees: map[string]FeeData{"2026-01": {750, 3}}}}
members := []Member{{Name: "Alice", Tier: "A", Fees: map[string]FeeData{"2026-01": {Expected: 750, Attendance: 3}}}}
txs := []Transaction{{
Date: "2026-01-01", Amount: 750,
Person: "[?] Alice", Purpose: "2026-01",
@@ -269,7 +269,7 @@ func TestReconcileQuestionMarkMarkerStripped(t *testing.T) {
// [Go] Purpose "other:shirt" puts payment in OtherTransactions, not in month ledger.
func TestReconcileOtherPurpose(t *testing.T) {
t.Parallel()
members := []Member{{Name: "Alice", Tier: "A", Fees: map[string]FeeData{"2026-01": {750, 3}}}}
members := []Member{{Name: "Alice", Tier: "A", Fees: map[string]FeeData{"2026-01": {Expected: 750, Attendance: 3}}}}
txs := []Transaction{{
Date: "2026-01-01", Amount: 300,
Person: "Alice", Purpose: "other:shirt",
@@ -297,7 +297,7 @@ func TestReconcileOtherPurpose(t *testing.T) {
func TestReconcileOutOfWindowGoesToCredit(t *testing.T) {
t.Parallel()
// Window shows only 2026-01. Transaction references 2026-01 (in) and 2026-02 (out).
members := []Member{{Name: "Alice", Tier: "A", Fees: map[string]FeeData{"2026-01": {600, 3}}}}
members := []Member{{Name: "Alice", Tier: "A", Fees: map[string]FeeData{"2026-01": {Expected: 600, Attendance: 3}}}}
txs := []Transaction{{
Date: "2026-01-01", Amount: 1200,
Person: "Alice", Purpose: "2026-01, 2026-02",
@@ -322,7 +322,7 @@ func TestReconcileOutOfWindowGoesToCredit(t *testing.T) {
// [Go] No person/purpose → inference fallback resolves sender name and date month.
func TestReconcileInferenceFallback(t *testing.T) {
t.Parallel()
members := []Member{{Name: "Tomáš Němeček", Tier: "A", Fees: map[string]FeeData{"2026-04": {750, 3}}}}
members := []Member{{Name: "Tomáš Němeček", Tier: "A", Fees: map[string]FeeData{"2026-04": {Expected: 750, Attendance: 3}}}}
txs := []Transaction{{
Date: "2026-04-15", Amount: 750,
// Person and Purpose are empty → inference path
@@ -340,7 +340,7 @@ func TestReconcileInferenceFallback(t *testing.T) {
// [Go] Transaction with no match at all ends up in Unmatched; ledger unchanged.
func TestReconcileNoMatchGoesToUnmatched(t *testing.T) {
t.Parallel()
members := []Member{{Name: "Alice", Tier: "A", Fees: map[string]FeeData{"2026-01": {750, 3}}}}
members := []Member{{Name: "Alice", Tier: "A", Fees: map[string]FeeData{"2026-01": {Expected: 750, Attendance: 3}}}}
txs := []Transaction{{
Date: "2026-01-01", Amount: 500,
// empty person+purpose and sender name not matching any member
@@ -360,7 +360,7 @@ func TestReconcileNoMatchGoesToUnmatched(t *testing.T) {
// [Go] Empty transaction list leaves every month at paid=0 and balance=expected.
func TestReconcileNoTransactionsAllUnpaid(t *testing.T) {
t.Parallel()
members := []Member{{Name: "Alice", Tier: "A", Fees: map[string]FeeData{"2026-01": {750, 3}}}}
members := []Member{{Name: "Alice", Tier: "A", Fees: map[string]FeeData{"2026-01": {Expected: 750, Attendance: 3}}}}
result := Reconcile(members, []string{"2026-01"}, nil, nil, defaultYear)

View File

@@ -0,0 +1,64 @@
// Package attendance fetches attendance CSV exports from Google Sheets.
// No auth required — the sheet must be publicly readable.
package attendance
import (
"context"
"encoding/csv"
"fmt"
"io"
"net/http"
"strings"
)
const exportBase = "https://docs.google.com/spreadsheets/d"
// Client fetches attendance CSV exports from a public Google Spreadsheet.
type Client struct {
http *http.Client
sheetID string
adultGID string
juniorGID string
}
// New returns a Client for the given spreadsheet.
// adultGID is typically "0"; juniorGID is the GID of the junior tab.
func New(httpClient *http.Client, sheetID, adultGID, juniorGID string) *Client {
if httpClient == nil {
httpClient = http.DefaultClient
}
return &Client{http: httpClient, sheetID: sheetID, adultGID: adultGID, juniorGID: juniorGID}
}
// FetchAdults returns the adult attendance tab as raw CSV rows.
func (c *Client) FetchAdults(ctx context.Context) ([][]string, error) {
return c.fetch(ctx, c.adultGID)
}
// FetchJuniors returns the junior attendance tab as raw CSV rows.
func (c *Client) FetchJuniors(ctx context.Context) ([][]string, error) {
return c.fetch(ctx, c.juniorGID)
}
func (c *Client) fetch(ctx context.Context, gid string) ([][]string, error) {
url := fmt.Sprintf("%s/%s/export?format=csv&gid=%s", exportBase, c.sheetID, gid)
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
if err != nil {
return nil, err
}
resp, err := c.http.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("attendance fetch: HTTP %d for gid=%s", resp.StatusCode, gid)
}
body, err := io.ReadAll(resp.Body)
if err != nil {
return nil, err
}
r := csv.NewReader(strings.NewReader(string(body)))
r.FieldsPerRecord = -1 // rows may have different lengths
return r.ReadAll()
}

View File

@@ -0,0 +1,93 @@
package attendance
import (
"context"
"encoding/csv"
"io"
"net/http"
"net/http/httptest"
"os"
"strings"
"testing"
)
func TestClientFetchAdults(t *testing.T) {
data, err := os.ReadFile("testdata/adults_minimal.csv")
if err != nil {
t.Fatal(err)
}
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
_, _ = w.Write(data)
}))
defer srv.Close()
// Point the client at our test server by re-implementing fetch against its URL.
rows, err := fetchURL(context.Background(), srv.Client(), srv.URL)
if err != nil {
t.Fatal(err)
}
if len(rows) < 2 {
t.Fatalf("want ≥2 rows, got %d", len(rows))
}
if rows[0][0] != "Jméno" {
t.Errorf("unexpected header: %q", rows[0][0])
}
}
func TestFake(t *testing.T) {
adultRows := parseCSV(t, "testdata/adults_minimal.csv")
juniorRows := parseCSV(t, "testdata/juniors_minimal.csv")
f := &Fake{Adults: adultRows, Juniors: juniorRows}
got, err := f.FetchAdults(context.Background())
if err != nil {
t.Fatal(err)
}
if got[0][0] != "Jméno" {
t.Errorf("adults header: %q", got[0][0])
}
got, err = f.FetchJuniors(context.Background())
if err != nil {
t.Fatal(err)
}
if got[1][0] != "Junior One" {
t.Errorf("juniors first member: %q", got[1][0])
}
}
func parseCSV(t *testing.T, path string) [][]string {
t.Helper()
b, err := os.ReadFile(path)
if err != nil {
t.Fatal(err)
}
r := csv.NewReader(strings.NewReader(string(b)))
r.FieldsPerRecord = -1
rows, err := r.ReadAll()
if err != nil {
t.Fatal(err)
}
return rows
}
// fetchURL is a test helper that exercises the shared fetch logic against an arbitrary URL.
func fetchURL(ctx context.Context, hc *http.Client, url string) ([][]string, error) {
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
if err != nil {
return nil, err
}
resp, err := hc.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
b, err := io.ReadAll(resp.Body)
if err != nil {
return nil, err
}
r := csv.NewReader(strings.NewReader(string(b)))
r.FieldsPerRecord = -1
return r.ReadAll()
}

View File

@@ -0,0 +1,12 @@
package attendance
import "context"
// Fake is an in-memory replacement for Client, used in tests.
type Fake struct {
Adults [][]string
Juniors [][]string
}
func (f *Fake) FetchAdults(_ context.Context) ([][]string, error) { return f.Adults, nil }
func (f *Fake) FetchJuniors(_ context.Context) ([][]string, error) { return f.Juniors, nil }

View File

@@ -0,0 +1,4 @@
Jméno,Tier,,,01.09.2025,08.09.2025,15.09.2025
Member One,A,,,TRUE,TRUE,FALSE
Member Two,A,,,TRUE,FALSE,FALSE
# last line,,,,,
1 Jméno,Tier,,,01.09.2025,08.09.2025,15.09.2025
2 Member One,A,,,TRUE,TRUE,FALSE
3 Member Two,A,,,TRUE,FALSE,FALSE
4 # last line,,,,,

View File

@@ -0,0 +1,4 @@
Jméno,Tier,,,01.09.2025,08.09.2025,15.09.2025
Junior One,J,,,TRUE,TRUE,TRUE
# Trenéři,,,,,
Coach One,X,,,FALSE,FALSE,FALSE
1 Jméno,Tier,,,01.09.2025,08.09.2025,15.09.2025
2 Junior One,J,,,TRUE,TRUE,TRUE
3 # Trenéři,,,,,
4 Coach One,X,,,FALSE,FALSE,FALSE

209
go/internal/io/cache/filecache.go vendored Normal file
View File

@@ -0,0 +1,209 @@
// Package cache implements a Drive-modifiedTime-gated JSON file cache,
// mirroring scripts/cache_utils.py.
package cache
import (
"context"
"encoding/json"
"fmt"
"fuj-management/go/internal/io/drive"
"os"
"path/filepath"
"sync"
"time"
)
// DriveClient is the subset of drive.Client used by FileCache.
type DriveClient interface {
ModifiedTime(ctx context.Context, fileID string) (string, error)
}
type cacheFile struct {
ModifiedTime string `json:"modifiedTime"`
Data json.RawMessage `json:"data"`
CachedAt string `json:"cachedAt"`
}
// FileCache wraps a Drive client to gate JSON file caching on sheet modifiedTime.
//
// Two TTL knobs mirror scripts/cache_utils.py:
// - ttl: if the cache file on disk is younger than this, skip the Drive check entirely.
// - apiCheckTTL: debounces in-memory Drive API calls per sheet ID.
//
// Atomic writes: data is marshaled to a .tmp file then os.Rename'd.
// Cache files are compatible with Python's format:
//
// {"modifiedTime":"…","data":…,"cachedAt":"…"}
type FileCache struct {
drive DriveClient
dir string
sheetMap map[string]string // cache key → Drive file ID
ttl time.Duration
apiCheckTTL time.Duration
mu sync.Mutex
lastChecked map[string]time.Time
}
// New creates a FileCache.
// sheetMap maps cache keys to Google Sheets/Drive file IDs (mirrors CACHE_SHEET_MAP in config).
func New(d DriveClient, dir string, sheetMap map[string]string, ttl, apiCheckTTL time.Duration) *FileCache {
return &FileCache{
drive: d,
dir: dir,
sheetMap: sheetMap,
ttl: ttl,
apiCheckTTL: apiCheckTTL,
lastChecked: make(map[string]time.Time),
}
}
// Get returns the cached value for cacheKey, calling fetch if the cache is stale.
// T must be JSON-marshalable.
func Get[T any](ctx context.Context, fc *FileCache, cacheKey string, fetch func(context.Context) (T, error)) (T, error) {
sheetID := fc.sheetMap[cacheKey]
if sheetID == "" {
sheetID = cacheKey
}
cacheFilePath := filepath.Join(fc.dir, cacheKey+"_cache.json")
currentModTime, err := fc.currentModifiedTime(ctx, sheetID, cacheFilePath)
if err != nil {
return *new(T), fmt.Errorf("cache: modifiedTime for %s: %w", cacheKey, err)
}
// Try cache hit
if data, ok := readCache[T](cacheFilePath, currentModTime); ok {
return data, nil
}
// Cache miss — fetch fresh data
fresh, err := fetch(ctx)
if err != nil {
return *new(T), err
}
if err := writeCache(cacheFilePath, currentModTime, fresh); err != nil {
// Non-fatal: log but don't fail the request
_, _ = fmt.Fprintf(os.Stderr, "cache: write %s: %v\n", cacheKey, err)
}
return fresh, nil
}
// Flush deletes all *_cache.json files in the cache dir and resets in-memory state.
func (fc *FileCache) Flush() (int, error) {
fc.mu.Lock()
fc.lastChecked = make(map[string]time.Time)
fc.mu.Unlock()
pattern := filepath.Join(fc.dir, "*_cache.json")
matches, err := filepath.Glob(pattern)
if err != nil {
return 0, err
}
for _, f := range matches {
_ = os.Remove(f)
}
return len(matches), nil
}
// currentModifiedTime returns a stable string representing the current version
// of the sheet, using the in-memory + file-mtime TTL guards before hitting Drive.
// On Drive failure, falls back to a 5-minute bucket string (matching Python).
func (fc *FileCache) currentModifiedTime(ctx context.Context, sheetID, cacheFilePath string) (string, error) {
now := time.Now()
fc.mu.Lock()
lastCheck := fc.lastChecked[sheetID]
fc.mu.Unlock()
// Guard 1: in-memory debounce — skip Drive if checked recently
if fc.apiCheckTTL > 0 && now.Sub(lastCheck) < fc.apiCheckTTL {
if mt, ok := readModifiedTime(cacheFilePath); ok {
return mt, nil
}
}
// Guard 2: cache file is young enough — trust the stored modifiedTime
if fc.ttl > 0 {
if info, err := os.Stat(cacheFilePath); err == nil {
if now.Sub(info.ModTime()) < fc.ttl {
if mt, ok := readModifiedTime(cacheFilePath); ok {
fc.mu.Lock()
fc.lastChecked[sheetID] = now
fc.mu.Unlock()
return mt, nil
}
}
}
}
// Hit Drive API
mt, err := fc.drive.ModifiedTime(ctx, sheetID)
if err != nil {
// Fallback: 5-minute bucket string, matches Python _fallback_ttl()
bucket := time.Now().Unix() / 300
return fmt.Sprintf("ttl-5m-%d", bucket), nil
}
fc.mu.Lock()
fc.lastChecked[sheetID] = now
fc.mu.Unlock()
return mt, nil
}
func readModifiedTime(path string) (string, bool) {
cf, ok := readCacheFile(path)
if !ok {
return "", false
}
return cf.ModifiedTime, cf.ModifiedTime != ""
}
func readCacheFile(path string) (cacheFile, bool) {
b, err := os.ReadFile(path)
if err != nil {
return cacheFile{}, false
}
var cf cacheFile
if err := json.Unmarshal(b, &cf); err != nil {
return cacheFile{}, false
}
return cf, true
}
func readCache[T any](path, currentModTime string) (T, bool) {
cf, ok := readCacheFile(path)
if !ok || cf.ModifiedTime != currentModTime {
return *new(T), false
}
var v T
if err := json.Unmarshal(cf.Data, &v); err != nil {
return *new(T), false
}
return v, true
}
func writeCache(path, modTime string, data any) error {
raw, err := json.Marshal(data)
if err != nil {
return err
}
cf := cacheFile{
ModifiedTime: modTime,
Data: json.RawMessage(raw),
CachedAt: time.Now().Format(time.RFC3339),
}
b, err := json.Marshal(cf)
if err != nil {
return err
}
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
return err
}
tmp := path + ".tmp"
if err := os.WriteFile(tmp, b, 0o600); err != nil {
return err
}
return os.Rename(tmp, path)
}
// Ensure *drive.Client satisfies DriveClient at compile time.
var _ DriveClient = (*drive.Client)(nil)

125
go/internal/io/cache/filecache_test.go vendored Normal file
View File

@@ -0,0 +1,125 @@
package cache
import (
"context"
"errors"
"fuj-management/go/internal/io/drive"
"os"
"testing"
"time"
)
func TestGet_FreshFetch(t *testing.T) {
dir := t.TempDir()
d := &drive.Fake{Times: map[string]string{"sheet1": "2026-01-01T00:00:00Z"}}
fc := New(d, dir, map[string]string{"mykey": "sheet1"}, time.Minute, time.Minute)
calls := 0
got, err := Get(context.Background(), fc, "mykey", func(_ context.Context) ([]string, error) {
calls++
return []string{"a", "b"}, nil
})
if err != nil {
t.Fatal(err)
}
if len(got) != 2 || got[0] != "a" {
t.Errorf("unexpected: %v", got)
}
if calls != 1 {
t.Errorf("want 1 fetch call, got %d", calls)
}
}
func TestGet_CacheHit(t *testing.T) {
dir := t.TempDir()
d := &drive.Fake{Times: map[string]string{"sheet1": "2026-01-01T00:00:00Z"}}
fc := New(d, dir, map[string]string{"mykey": "sheet1"}, time.Minute, time.Minute)
fetch := func(_ context.Context) ([]string, error) { return []string{"a"}, nil }
if _, err := Get(context.Background(), fc, "mykey", fetch); err != nil {
t.Fatal(err)
}
// Second call — modifiedTime unchanged, should hit cache
calls := 0
got, err := Get(context.Background(), fc, "mykey", func(_ context.Context) ([]string, error) {
calls++
return []string{"SHOULD_NOT_CALL"}, nil
})
if err != nil {
t.Fatal(err)
}
if got[0] != "a" {
t.Errorf("want cache hit with 'a', got %q", got[0])
}
if calls != 0 {
t.Errorf("want 0 fetch calls on hit, got %d", calls)
}
}
func TestGet_CacheMiss_OnModifiedTimeChange(t *testing.T) {
dir := t.TempDir()
d := &drive.Fake{Times: map[string]string{"sheet1": "2026-01-01T00:00:00Z"}}
// No TTL guards so we always hit Drive
fc := New(d, dir, map[string]string{"mykey": "sheet1"}, 0, 0)
fetch := func(_ context.Context) ([]string, error) { return []string{"v1"}, nil }
if _, err := Get(context.Background(), fc, "mykey", fetch); err != nil {
t.Fatal(err)
}
// Sheet updated — change modifiedTime
d.Times["sheet1"] = "2026-02-01T00:00:00Z"
got, err := Get(context.Background(), fc, "mykey", func(_ context.Context) ([]string, error) {
return []string{"v2"}, nil
})
if err != nil {
t.Fatal(err)
}
if got[0] != "v2" {
t.Errorf("want v2 after sheet update, got %q", got[0])
}
}
func TestGet_DriveFailureFallback(t *testing.T) {
dir := t.TempDir()
d := &drive.Fake{Err: errors.New("drive down")}
fc := New(d, dir, nil, 0, 0)
calls := 0
_, err := Get(context.Background(), fc, "mykey", func(_ context.Context) ([]string, error) {
calls++
return []string{"fallback"}, nil
})
if err != nil {
t.Fatal(err)
}
if calls != 1 {
t.Errorf("want 1 fetch call, got %d", calls)
}
}
func TestFlush(t *testing.T) {
dir := t.TempDir()
d := &drive.Fake{Times: map[string]string{"sheet1": "t1"}}
fc := New(d, dir, map[string]string{"k": "sheet1"}, 0, 0)
if _, err := Get(context.Background(), fc, "k", func(_ context.Context) (int, error) { return 42, nil }); err != nil {
t.Fatal(err)
}
n, err := fc.Flush()
if err != nil {
t.Fatal(err)
}
if n != 1 {
t.Errorf("want 1 deleted file, got %d", n)
}
// Cache dir should be empty of _cache.json files
entries, _ := os.ReadDir(dir)
for _, e := range entries {
if e.Name() != "" {
t.Errorf("expected empty dir after flush, found %s", e.Name())
}
}
}

View File

@@ -0,0 +1,46 @@
// Package drive provides a thin wrapper around the Google Drive v3 API,
// used only to read modifiedTime for cache invalidation.
package drive
import (
"context"
"net/http"
"time"
"google.golang.org/api/drive/v3"
"google.golang.org/api/option"
)
// Client wraps the Drive v3 API, scoped to read-only modifiedTime checks.
type Client struct {
svc *drive.Service
}
// New builds a Client using a service-account credentials file.
// timeout applies to each Drive API call.
func New(ctx context.Context, credentialsPath string, timeout time.Duration) (*Client, error) {
hc := &http.Client{Timeout: timeout}
svc, err := drive.NewService(ctx,
option.WithCredentialsFile(credentialsPath), //nolint:staticcheck
option.WithScopes(drive.DriveReadonlyScope),
option.WithHTTPClient(hc),
)
if err != nil {
return nil, err
}
return &Client{svc: svc}, nil
}
// ModifiedTime returns the RFC3339 modifiedTime for the given Drive file ID.
// Returns ("", err) if the Drive API call fails.
func (c *Client) ModifiedTime(ctx context.Context, fileID string) (string, error) {
meta, err := c.svc.Files.Get(fileID).
Fields("modifiedTime").
SupportsAllDrives(true).
Context(ctx).
Do()
if err != nil {
return "", err
}
return meta.ModifiedTime, nil
}

View File

@@ -0,0 +1,18 @@
package drive
import "context"
// Fake is an in-memory replacement for Client used in tests.
type Fake struct {
// Times maps file ID → modifiedTime string returned by ModifiedTime.
Times map[string]string
// Err, if non-nil, is returned instead of looking up Times.
Err error
}
func (f *Fake) ModifiedTime(_ context.Context, fileID string) (string, error) {
if f.Err != nil {
return "", f.Err
}
return f.Times[fileID], nil
}

136
go/internal/io/fio/api.go Normal file
View File

@@ -0,0 +1,136 @@
package fio
import (
"context"
"encoding/json"
"fmt"
"io"
"log/slog"
"net/http"
"strings"
"time"
)
// httpDoer is the subset of *http.Client used by both Fio impls.
type httpDoer interface {
Do(*http.Request) (*http.Response, error)
}
// apiClient fetches transactions from the Fio REST API (JSON).
// Ports scripts/fio_utils.py fetch_transactions_api.
type apiClient struct {
token string
hc httpDoer
}
func (c *apiClient) FetchTransactions(ctx context.Context, from, to time.Time) ([]Transaction, error) {
const layout = "2006-01-02"
url := fmt.Sprintf("https://fioapi.fio.cz/v1/rest/periods/%s/%s/%s/transactions.json",
c.token, from.Format(layout), to.Format(layout))
slog.Debug("fio api: GET",
"url", strings.Replace(url, c.token, "****", 1),
"from", from.Format(layout), "to", to.Format(layout))
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
if err != nil {
return nil, err
}
resp, err := c.hc.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
slog.Debug("fio api: response", "status", resp.StatusCode)
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("fio api: HTTP %d", resp.StatusCode)
}
body, err := io.ReadAll(resp.Body)
if err != nil {
return nil, err
}
txns, err := parseAPIResponse(body)
slog.Debug("fio api: parsed", "body_bytes", len(body), "parsed_count", len(txns))
return txns, err
}
// fioAPIResponse is the top-level envelope from the Fio JSON API.
type fioAPIResponse struct {
AccountStatement struct {
TransactionList struct {
Transaction []map[string]json.RawMessage `json:"transaction"`
} `json:"transactionList"`
} `json:"accountStatement"`
}
func parseAPIResponse(body []byte) ([]Transaction, error) {
var resp fioAPIResponse
if err := json.Unmarshal(body, &resp); err != nil {
return nil, fmt.Errorf("fio api: parse JSON: %w", err)
}
var txns []Transaction
for _, raw := range resp.AccountStatement.TransactionList.Transaction {
amount := colFloat(raw, "column1")
if amount <= 0 {
continue // skip outgoing
}
dateRaw := colString(raw, "column0")
dateStr := ""
if len(dateRaw) >= 10 {
dateStr = dateRaw[:10]
}
txns = append(txns, Transaction{
Date: dateStr,
Amount: amount,
Sender: colString(raw, "column10"),
Message: colString(raw, "column16"),
VS: colString(raw, "column5"),
KS: colString(raw, "column4"),
SS: colString(raw, "column6"),
UserID: colString(raw, "column7"),
SenderAccount: colString(raw, "column2"),
BankID: colString(raw, "column22"),
Currency: colStringOr(raw, "column14", "CZK"),
})
}
return txns, nil
}
// colString extracts {"value":…} as a string from a column map.
func colString(m map[string]json.RawMessage, col string) string {
raw, ok := m[col]
if !ok {
return ""
}
var cell struct {
Value *string `json:"value"`
}
if json.Unmarshal(raw, &cell) != nil || cell.Value == nil {
return ""
}
return *cell.Value
}
// colStringOr is colString with a fallback value.
func colStringOr(m map[string]json.RawMessage, col, fallback string) string {
if v := colString(m, col); v != "" {
return v
}
return fallback
}
// colFloat extracts {"value":…} as a float64 from a column map.
// Returns 0 on any error (null column, non-numeric value).
func colFloat(m map[string]json.RawMessage, col string) float64 {
raw, ok := m[col]
if !ok {
return 0
}
var cell struct {
Value *float64 `json:"value"`
}
if json.Unmarshal(raw, &cell) != nil || cell.Value == nil {
return 0
}
return *cell.Value
}

View File

@@ -0,0 +1,45 @@
// Package fio fetches Fio bank transactions via the JSON API or the
// transparent-page HTML scraper, behind a common Client interface.
package fio
import (
"context"
"log/slog"
"net/http"
"time"
)
// Transaction is one incoming bank payment. Fields absent from the HTML scraper
// (BankID, Currency, UserID, SenderAccount) are empty strings on that path.
type Transaction struct {
Date string
Amount float64
Sender string
Message string
VS string
KS string
SS string
UserID string // column7; empty on HTML path
SenderAccount string // column2; empty on HTML path
BankID string // column22; empty on HTML path
Currency string // column14; empty on HTML path (assume CZK)
}
// Client fetches transactions for a date window.
type Client interface {
FetchTransactions(ctx context.Context, from, to time.Time) ([]Transaction, error)
}
// New returns an apiClient when token is non-empty, otherwise a transparentClient.
// hc may be nil, in which case http.DefaultClient is used.
func New(token, accountNum string, hc httpDoer) Client {
if hc == nil {
hc = http.DefaultClient
}
if token != "" {
slog.Debug("fio: client selected", "type", "api")
return &apiClient{token: token, hc: hc}
}
slog.Debug("fio: client selected", "type", "transparent", "account_num", accountNum)
return &transparentClient{accountNum: accountNum, hc: hc}
}

View File

@@ -0,0 +1,19 @@
package fio
import (
"context"
"time"
)
// Fake is an in-memory replacement for Client, used in tests.
type Fake struct {
Transactions []Transaction
Err error
}
func (f *Fake) FetchTransactions(_ context.Context, _, _ time.Time) ([]Transaction, error) {
if f.Err != nil {
return nil, f.Err
}
return f.Transactions, nil
}

View File

@@ -0,0 +1,178 @@
package fio
import (
"context"
"io"
"net/http"
"net/http/httptest"
"os"
"testing"
"time"
)
func TestAPIClient_ParseResponse(t *testing.T) {
body, err := os.ReadFile("testdata/api_response.json")
if err != nil {
t.Fatal(err)
}
txns, err := parseAPIResponse(body)
if err != nil {
t.Fatal(err)
}
if len(txns) != 1 {
t.Fatalf("want 1 txn (outgoing filtered), got %d", len(txns))
}
tx := txns[0]
if tx.Date != "2026-04-10" {
t.Errorf("date: want '2026-04-10', got %q", tx.Date)
}
if tx.Amount != 750 {
t.Errorf("amount: want 750, got %v", tx.Amount)
}
if tx.Sender != "Jana Novakova" {
t.Errorf("sender: want 'Jana Novakova', got %q", tx.Sender)
}
if tx.Message != "duben 2026" {
t.Errorf("message: want 'duben 2026', got %q", tx.Message)
}
if tx.VS != "123" {
t.Errorf("vs: want '123', got %q", tx.VS)
}
if tx.BankID != "12345678901" {
t.Errorf("bank_id: want '12345678901', got %q", tx.BankID)
}
}
func TestAPIClient_HTTPRoundTrip(t *testing.T) {
body, _ := os.ReadFile("testdata/api_response.json")
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
_, _ = w.Write(body)
}))
defer srv.Close()
c := &apiClient{token: "TESTTOKEN", hc: &overrideClient{base: srv.Client(), baseURL: srv.URL}}
txns, err := c.FetchTransactions(context.Background(), time.Now().AddDate(0, -1, 0), time.Now())
if err != nil {
t.Fatal(err)
}
if len(txns) != 1 {
t.Fatalf("want 1 txn, got %d", len(txns))
}
}
func TestTransparentClient_ParseHTML(t *testing.T) {
body, err := os.ReadFile("testdata/transparent.html")
if err != nil {
t.Fatal(err)
}
txns, err := parseTransparentHTML(body)
if err != nil {
t.Fatal(err)
}
// Only the incoming row (750 CZK) should be kept; -200 is outgoing
if len(txns) != 1 {
t.Fatalf("want 1 txn (outgoing filtered), got %d", len(txns))
}
tx := txns[0]
if tx.Date != "2026-04-10" {
t.Errorf("date: want '2026-04-10', got %q", tx.Date)
}
if tx.Amount != 750 {
t.Errorf("amount: want 750, got %v", tx.Amount)
}
if tx.Sender != "Jana Novakova" {
t.Errorf("sender: want 'Jana Novakova', got %q", tx.Sender)
}
if tx.VS != "123" {
t.Errorf("vs: want '123', got %q", tx.VS)
}
if tx.BankID != "" {
t.Errorf("bank_id: want empty on HTML path, got %q", tx.BankID)
}
}
func TestParseCzechDate(t *testing.T) {
cases := []struct{ in, want string }{
{"10.04.2026", "2026-04-10"},
{"10/04/2026", "2026-04-10"},
{"7.5.2026", "2026-05-07"}, // non-padded — real Fio transparent page format
{"3.12.2025", "2025-12-03"}, // non-padded single-digit day, double-digit month
{"07.05.26", "2026-05-07"}, // padded 2-digit year — current Fio transparent page format
{"7.5.26", "2026-05-07"}, // non-padded 2-digit year
{"07/05/26", "2026-05-07"}, // slash variant
{"", ""},
{"invalid", ""},
}
for _, c := range cases {
if got := parseCzechDate(c.in); got != c.want {
t.Errorf("parseCzechDate(%q) = %q, want %q", c.in, got, c.want)
}
}
}
func TestExtractSecondTableRows_NestedTable(t *testing.T) {
// Regression: a nested <table> inside the target must not cause early exit.
html := `<table class="table"><tr><td>nav</td></tr></table>
<table class="table">
<thead><tr><th>Date</th></tr></thead>
<tbody>
<tr><td>7.5.2026</td><td><table><tr><td>nested</td></tr></table></td></tr>
<tr><td>6.5.2026</td><td></td></tr>
</tbody>
</table>`
rows := extractSecondTableRows([]byte(html))
if len(rows) != 2 {
t.Errorf("want 2 data rows, got %d: %v", len(rows), rows)
}
}
func TestParseCzechAmount(t *testing.T) {
cases := []struct {
in string
want float64
}{
{"750,00 CZK", 750},
{"1.500,00", 1500},
{"1500.00", 1500},
{"-200,00 CZK", -200},
}
for _, c := range cases {
if got := parseCzechAmount(c.in); got != c.want {
t.Errorf("parseCzechAmount(%q) = %v, want %v", c.in, got, c.want)
}
}
}
func TestFake(t *testing.T) {
f := &Fake{Transactions: []Transaction{{Date: "2026-04-01", Amount: 500}}}
txns, err := f.FetchTransactions(context.Background(), time.Now(), time.Now())
if err != nil {
t.Fatal(err)
}
if len(txns) != 1 || txns[0].Date != "2026-04-01" {
t.Errorf("unexpected: %v", txns)
}
}
// overrideClient replaces the URL in requests so we can hit a local test server
// instead of the real Fio URL.
type overrideClient struct {
base *http.Client
baseURL string
}
func (o *overrideClient) Do(req *http.Request) (*http.Response, error) {
r2, _ := http.NewRequestWithContext(req.Context(), req.Method, o.baseURL+req.URL.Path, nil)
resp, err := o.base.Do(r2)
if err != nil {
return nil, err
}
// The api client reads the body, so re-serve whatever the test server returned.
return resp, nil
}
// verify Fake satisfies Client
var _ Client = (*Fake)(nil)
// ensure io.ReadAll isn't called at top level (compile-time reference suppressor)
var _ = io.ReadAll

View File

@@ -0,0 +1,29 @@
{
"accountStatement": {
"transactionList": {
"transaction": [
{
"column0": {"value": "2026-04-10+0200", "name": "Datum", "id": 0},
"column1": {"value": 750.0, "name": "Objem", "id": 1},
"column2": {"value": "123456789/0300", "name": "Protiúčet", "id": 2},
"column4": {"value": "0308", "name": "KS", "id": 4},
"column5": {"value": "123", "name": "VS", "id": 5},
"column6": {"value": "", "name": "SS", "id": 6},
"column7": {"value": "Jana Nováková", "name": "Uživatelská identifikace", "id": 7},
"column10": {"value": "Jana Novakova", "name": "Název protiúčtu", "id": 10},
"column14": {"value": "CZK", "name": "Měna", "id": 14},
"column16": {"value": "duben 2026", "name": "Zpráva pro příjemce", "id": 16},
"column22": {"value": "12345678901", "name": "ID operace", "id": 22}
},
{
"column0": {"value": "2026-04-11+0200", "name": "Datum", "id": 0},
"column1": {"value": -200.0, "name": "Objem", "id": 1},
"column10": {"value": "Outgoing", "name": "Název protiúčtu", "id": 10},
"column14": {"value": "CZK", "name": "Měna", "id": 14},
"column16": {"value": "", "name": "Zpráva pro příjemce", "id": 16},
"column22": {"value": "99999999999", "name": "ID operace", "id": 22}
}
]
}
}
}

View File

@@ -0,0 +1,37 @@
<!DOCTYPE html>
<html>
<body>
<!-- First table (ignored) -->
<table class="table"><tr><td>ignored</td></tr></table>
<!-- Second table (target) -->
<table class="table">
<thead>
<tr><th>Datum</th><th>Částka</th><th>Typ</th><th>Název protiúčtu</th><th>Zpráva</th><th>KS</th><th>VS</th><th>SS</th><th>Poznámka</th></tr>
</thead>
<tbody>
<tr>
<td>10.04.2026</td>
<td>750,00&nbsp;CZK</td>
<td>Příjem</td>
<td>Jana Novakova</td>
<td>duben 2026</td>
<td>0308</td>
<td>123</td>
<td></td>
<td></td>
</tr>
<tr>
<td>09.04.2026</td>
<td>-200,00&nbsp;CZK</td>
<td>Odchozí</td>
<td>Someone</td>
<td>outgoing</td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
</tbody>
</table>
</body>
</html>

View File

@@ -0,0 +1,247 @@
package fio
import (
"context"
"fmt"
"io"
"log/slog"
"net/http"
"regexp"
"strings"
"time"
"unicode"
ghtml "golang.org/x/net/html"
)
// transparentClient fetches transactions from the Fio transparent account page (HTML).
// Ports scripts/fio_utils.py FioTableParser + fetch_transactions_transparent.
type transparentClient struct {
accountNum string
hc httpDoer
}
func (c *transparentClient) FetchTransactions(ctx context.Context, from, to time.Time) ([]Transaction, error) {
// Transparent page date format: D.M.YYYY
url := fmt.Sprintf(
"https://ib.fio.cz/ib/transparent?a=%s&f=%s&t=%s",
c.accountNum,
from.Format("2.1.2006"),
to.Format("2.1.2006"),
)
slog.Debug("fio transparent: GET",
"url", url,
"from", from.Format("2006-01-02"), "to", to.Format("2006-01-02"))
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
if err != nil {
return nil, err
}
resp, err := c.hc.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
slog.Debug("fio transparent: response", "status", resp.StatusCode)
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("fio transparent: HTTP %d", resp.StatusCode)
}
body, err := io.ReadAll(resp.Body)
if err != nil {
return nil, err
}
slog.Debug("fio transparent: body read", "body_bytes", len(body))
return parseTransparentHTML(body)
}
// Column indices in the transparent-page table (0-based).
// Datum | Částka | Typ | Název protiúčtu | Zpráva pro příjemce | KS | VS | SS | Poznámka
const (
tColDate = 0
tColAmount = 1
tColSender = 3
tColMessage = 4
tColKS = 5
tColVS = 6
tColSS = 7
)
func parseTransparentHTML(body []byte) ([]Transaction, error) {
rows := extractSecondTableRows(body)
var txns []Transaction
var droppedBadDate, droppedNonpositive int
for _, row := range rows {
col := func(i int) string {
if i < len(row) {
return strings.TrimSpace(row[i])
}
return ""
}
dateStr := parseCzechDate(col(tColDate))
amount := parseCzechAmount(col(tColAmount))
if dateStr == "" {
droppedBadDate++
continue
}
if amount <= 0 {
droppedNonpositive++
continue
}
txns = append(txns, Transaction{
Date: dateStr,
Amount: amount,
Sender: col(tColSender),
Message: col(tColMessage),
KS: col(tColKS),
VS: col(tColVS),
SS: col(tColSS),
BankID: "", // not available on HTML path
})
}
slog.Debug("fio transparent: parsed",
"raw_rows", len(rows),
"kept", len(txns),
"dropped_bad_date", droppedBadDate,
"dropped_nonpositive_amount", droppedNonpositive)
return txns, nil
}
// extractSecondTableRows walks the HTML token stream and returns data rows
// from the second <table class="table"> element, skipping the <thead>.
// It tracks nesting depth so that nested <table> elements inside the target
// do not trigger an early exit.
func extractSecondTableRows(body []byte) [][]string {
z := ghtml.NewTokenizer(strings.NewReader(string(body)))
tableCount := 0
targetDepth := 0 // >0 while inside the target table (handles nesting)
inThead := false
inRow := false
inCell := false
var currentRow []string
var cellBuf strings.Builder
var rows [][]string
for {
tt := z.Next()
if tt == ghtml.ErrorToken {
break
}
switch tt {
case ghtml.StartTagToken:
t := z.Token()
switch t.Data {
case "table":
if targetDepth > 0 {
targetDepth++ // nested table inside target; track so </table> doesn't exit early
} else if hasClass(t, "table") {
tableCount++
if tableCount == 2 {
targetDepth = 1
}
}
case "thead":
if targetDepth > 0 {
inThead = true
}
case "tr":
if targetDepth > 0 && !inThead {
inRow = true
currentRow = nil
}
case "td", "th":
if inRow {
inCell = true
cellBuf.Reset()
}
}
case ghtml.EndTagToken:
t := z.Token()
switch t.Data {
case "td", "th":
if inCell {
currentRow = append(currentRow, cellBuf.String())
inCell = false
}
case "thead":
inThead = false
case "tr":
if inRow {
if len(currentRow) > 0 {
rows = append(rows, currentRow)
}
inRow = false
}
case "table":
if targetDepth > 0 {
targetDepth--
if targetDepth == 0 {
return rows
}
}
}
case ghtml.TextToken:
if inCell {
cellBuf.WriteString(z.Token().Data)
}
}
}
return rows
}
func hasClass(t ghtml.Token, cls string) bool {
for _, a := range t.Attr {
if a.Key == "class" {
for _, c := range strings.Fields(a.Val) {
if c == cls {
return true
}
}
}
}
return false
}
// parseCzechDate parses Czech date strings → "YYYY-MM-DD".
// Handles both zero-padded ("07.05.2026") and non-padded ("7.5.2026") variants
// with dot or slash separators, as the Fio transparent page omits leading zeros.
// Returns "" on parse error.
func parseCzechDate(s string) string {
s = strings.TrimSpace(s)
for _, layout := range []string{
"2.1.2006", "02.01.2006", "2/1/2006", "02/01/2006",
"2.1.06", "02.01.06", "2/1/06", "02/01/06",
} {
if t, err := time.Parse(layout, s); err == nil {
return t.Format("2006-01-02")
}
}
return ""
}
var nonNumericRe = regexp.MustCompile(`[^\d.,]`)
// parseCzechAmount parses "1 500,00 CZK" / "1.500,00" / "1500.00" → float64.
// Returns 0 on error.
func parseCzechAmount(s string) float64 {
// Remove NBSP, regular spaces, currency letters
s = strings.Map(func(r rune) rune {
if r == ' ' || unicode.IsSpace(r) || unicode.IsLetter(r) {
return -1
}
return r
}, s)
if strings.Contains(s, ",") {
// Czech decimal: 1.500,00 → remove dots (thousand sep), comma → dot
s = strings.ReplaceAll(s, ".", "")
s = strings.ReplaceAll(s, ",", ".")
} else {
// Remove any remaining non-numeric except one dot
s = nonNumericRe.ReplaceAllString(s, "")
}
var f float64
_, _ = fmt.Sscanf(s, "%f", &f)
return f
}

View File

@@ -0,0 +1,124 @@
// Package sheets provides a typed wrapper around the Google Sheets v4 API.
package sheets
import (
"context"
"fmt"
"time"
"google.golang.org/api/option"
sheetsv4 "google.golang.org/api/sheets/v4"
)
// ValueRange pairs an R1C1 range with its cell values, used for batchUpdate.
type ValueRange struct {
Range string // R1C1 notation, e.g. "R2C4:R2C6"
Values [][]any // one sub-slice per row
}
// Client wraps the Sheets v4 API with the operations needed by this project.
type Client struct {
svc *sheetsv4.Service
}
// New builds a Client using a service-account credentials file.
func New(ctx context.Context, credentialsPath string, _ time.Duration) (*Client, error) {
svc, err := sheetsv4.NewService(ctx,
option.WithCredentialsFile(credentialsPath), //nolint:staticcheck
option.WithScopes(sheetsv4.SpreadsheetsScope),
)
if err != nil {
return nil, err
}
return &Client{svc: svc}, nil
}
// GetValues fetches a range from a spreadsheet with UNFORMATTED_VALUE rendering
// (numbers as numbers, dates as serial floats — matching Python's behaviour).
func (c *Client) GetValues(ctx context.Context, spreadsheetID, a1Range string) ([][]any, error) {
resp, err := c.svc.Spreadsheets.Values.
Get(spreadsheetID, a1Range).
ValueRenderOption("UNFORMATTED_VALUE").
Context(ctx).
Do()
if err != nil {
return nil, err
}
rows := make([][]any, len(resp.Values))
copy(rows, resp.Values)
return rows, nil
}
// AppendValues appends rows to the first empty row after a1Range.
func (c *Client) AppendValues(ctx context.Context, spreadsheetID, a1Range string, rows [][]any) error {
vals := make([][]any, len(rows))
copy(vals, rows)
_, err := c.svc.Spreadsheets.Values.
Append(spreadsheetID, a1Range, &sheetsv4.ValueRange{Values: vals}).
ValueInputOption("USER_ENTERED").
Context(ctx).
Do()
return err
}
// BatchUpdateValues writes multiple non-contiguous ranges in one API call.
func (c *Client) BatchUpdateValues(ctx context.Context, spreadsheetID string, updates []ValueRange) error {
data := make([]*sheetsv4.ValueRange, len(updates))
for i, u := range updates {
vals := make([][]any, len(u.Values))
copy(vals, u.Values)
data[i] = &sheetsv4.ValueRange{Range: u.Range, Values: vals}
}
_, err := c.svc.Spreadsheets.Values.
BatchUpdate(spreadsheetID, &sheetsv4.BatchUpdateValuesRequest{
ValueInputOption: "USER_ENTERED",
Data: data,
}).
Context(ctx).
Do()
return err
}
// WriteHeader overwrites row 1 of the spreadsheet with the given labels.
func (c *Client) WriteHeader(ctx context.Context, spreadsheetID string, labels []string) error {
row := make([]any, len(labels))
for i, l := range labels {
row[i] = l
}
_, err := c.svc.Spreadsheets.Values.
Update(spreadsheetID, "A1", &sheetsv4.ValueRange{Values: [][]any{row}}).
ValueInputOption("USER_ENTERED").
Context(ctx).
Do()
return err
}
// SortByDateColumn sorts rows 2..10000 of the first sheet ascending by column A (Date).
// Looks up the sheetId (gid) from spreadsheet metadata.
func (c *Client) SortByDateColumn(ctx context.Context, spreadsheetID string) error {
meta, err := c.svc.Spreadsheets.Get(spreadsheetID).Context(ctx).Do()
if err != nil {
return fmt.Errorf("sheets: get spreadsheet: %w", err)
}
if len(meta.Sheets) == 0 {
return fmt.Errorf("sheets: spreadsheet has no sheets")
}
sheetID := meta.Sheets[0].Properties.SheetId
_, err = c.svc.Spreadsheets.BatchUpdate(spreadsheetID, &sheetsv4.BatchUpdateSpreadsheetRequest{
Requests: []*sheetsv4.Request{{
SortRange: &sheetsv4.SortRangeRequest{
Range: &sheetsv4.GridRange{
SheetId: sheetID,
StartRowIndex: 1,
EndRowIndex: 10000,
},
SortSpecs: []*sheetsv4.SortSpec{{
DimensionIndex: 0,
SortOrder: "ASCENDING",
}},
},
}},
}).Context(ctx).Do()
return err
}

View File

@@ -0,0 +1,53 @@
package sheets
import (
"context"
"fmt"
)
// Fake is an in-memory replacement for Client used in tests.
// Values maps a "<spreadsheetID>/<a1Range>" key to pre-seeded rows.
type Fake struct {
// Values maps "spreadsheetID/range" → rows returned by GetValues.
Values map[string][][]any
// Appended collects rows passed to AppendValues for assertion.
Appended []AppendCall
// BatchUpdated collects calls to BatchUpdateValues.
BatchUpdated []BatchCall
}
// AppendCall records one AppendValues invocation.
type AppendCall struct {
SpreadsheetID string
Range string
Rows [][]any
}
// BatchCall records one BatchUpdateValues invocation.
type BatchCall struct {
SpreadsheetID string
Updates []ValueRange
}
func (f *Fake) GetValues(_ context.Context, spreadsheetID, a1Range string) ([][]any, error) {
key := spreadsheetID + "/" + a1Range
rows, ok := f.Values[key]
if !ok {
return nil, fmt.Errorf("sheets fake: no seed for %q", key)
}
return rows, nil
}
func (f *Fake) AppendValues(_ context.Context, spreadsheetID, a1Range string, rows [][]any) error {
f.Appended = append(f.Appended, AppendCall{SpreadsheetID: spreadsheetID, Range: a1Range, Rows: rows})
return nil
}
func (f *Fake) BatchUpdateValues(_ context.Context, spreadsheetID string, updates []ValueRange) error {
f.BatchUpdated = append(f.BatchUpdated, BatchCall{SpreadsheetID: spreadsheetID, Updates: updates})
return nil
}
func (f *Fake) WriteHeader(_ context.Context, _ string, _ []string) error { return nil }
func (f *Fake) SortByDateColumn(_ context.Context, _ string) error { return nil }

View File

@@ -0,0 +1,31 @@
package banksync
import (
"fmt"
"fuj-management/go/internal/io/fio"
"io"
"text/tabwriter"
)
func printFioTable(w io.Writer, txns []fio.Transaction, syncIDs []string, existing map[string]bool) {
tw := tabwriter.NewWriter(w, 0, 0, 2, ' ', 0)
fmt.Fprintln(tw, "DATE\tAMOUNT\tSENDER\tVS\tMESSAGE\tBANKID\tSTATUS")
for i, tx := range txns {
status := "NEW"
if existing[syncIDs[i]] {
status = "DUP"
}
fmt.Fprintf(tw, "%s\t%.2f\t%s\t%s\t%s\t%s\t%s\n",
tx.Date, tx.Amount, tx.Sender, tx.VS,
truncRunes(tx.Message, 40), tx.BankID, status)
}
_ = tw.Flush()
}
func truncRunes(s string, n int) string {
rs := []rune(s)
if len(rs) <= n {
return s
}
return string(rs[:n-1]) + "…"
}

View File

@@ -0,0 +1,170 @@
package banksync
import (
"context"
"fmt"
"fuj-management/go/internal/domain/matching"
"fuj-management/go/internal/domain/reconcile"
"fuj-management/go/internal/io/sheets"
"strings"
"time"
)
// InferOpts controls infer behaviour.
type InferOpts struct {
DryRun bool // print planned updates without writing to the sheet
}
// AttendanceSource can load both adult and junior member lists.
type AttendanceSource interface {
LoadAdults(ctx context.Context) ([]reconcile.Member, []string, error)
LoadJuniors(ctx context.Context) ([]reconcile.Member, []string, error)
}
// sheetReadWriter is the subset of *sheets.Client used by InferPayments.
type sheetReadWriter interface {
GetValues(ctx context.Context, spreadsheetID, a1Range string) ([][]any, error)
BatchUpdateValues(ctx context.Context, spreadsheetID string, updates []sheets.ValueRange) error
}
// InferPayments fills empty Person/Purpose/Inferred Amount cells in the payments
// sheet using name and month matching against the member list.
// Returns the number of rows updated (or that would be updated on dry-run).
// Ports scripts/infer_payments.py infer_payments.
func InferPayments(
ctx context.Context,
spreadsheetID string,
sh sheetReadWriter,
attendance AttendanceSource,
opts InferOpts,
) (int, error) {
rows, err := sh.GetValues(ctx, spreadsheetID, "A1:Z")
if err != nil {
return 0, fmt.Errorf("infer: read sheet: %w", err)
}
if len(rows) == 0 {
return 0, nil
}
header := rows[0]
colIdx := func(label string) int {
label = strings.ToLower(strings.TrimSpace(label))
for i, h := range header {
if strings.ToLower(strings.TrimSpace(fmt.Sprint(h))) == label {
return i
}
}
return -1
}
idxDate := colIdx("date")
idxAmount := colIdx("amount")
idxSender := colIdx("sender")
idxMessage := colIdx("message")
idxVS := colIdx("vs")
idxManual := colIdx("manual fix")
idxPerson := colIdx("person")
idxPurpose := colIdx("purpose")
idxInferred := colIdx("inferred amount")
for _, req := range []string{"person", "purpose", "inferred amount"} {
if colIdx(req) == -1 {
return 0, fmt.Errorf("infer: required column %q not found in sheet", req)
}
}
// Build union member list: adults + juniors, deduped by canonical key.
adults, _, err := attendance.LoadAdults(ctx)
if err != nil {
return 0, fmt.Errorf("infer: load adults: %w", err)
}
juniors, _, err := attendance.LoadJuniors(ctx)
if err != nil {
return 0, fmt.Errorf("infer: load juniors: %w", err)
}
memberNames := dedupeMembers(append(adults, juniors...))
defaultYear := time.Now().Year()
var updates []sheets.ValueRange
for i, row := range rows[1:] {
rowNum := i + 2 // 1-based, skip header
get := func(idx int) string {
if idx < 0 || idx >= len(row) {
return ""
}
return strings.TrimSpace(fmt.Sprint(row[idx]))
}
// Skip rule: any of manual fix / Person / Purpose non-empty → leave alone
if get(idxManual) != "" || get(idxPerson) != "" || get(idxPurpose) != "" {
continue
}
tx := matching.Transaction{
Sender: get(idxSender),
Message: get(idxMessage),
UserID: get(idxVS),
}
if idxDate >= 0 && idxDate < len(row) {
tx.Date = row[idxDate]
}
inferred := matching.InferTransactionDetails(tx, memberNames, defaultYear)
if len(inferred.Members) == 0 && len(inferred.Months) == 0 {
continue
}
var peeps []string
for _, m := range inferred.Members {
if m.Confidence == matching.ConfidenceReview {
peeps = append(peeps, "[?] "+m.Name)
} else {
peeps = append(peeps, m.Name)
}
}
personVal := strings.Join(peeps, ", ")
purposeVal := strings.Join(inferred.Months, ", ")
amountVal := ""
if idxAmount >= 0 && idxAmount < len(row) {
amountVal = fmt.Sprint(row[idxAmount])
}
if opts.DryRun {
fmt.Printf("Row %d: would infer person=%q purpose=%q amount=%s\n",
rowNum, personVal, purposeVal, amountVal)
}
// R1C1 range: "R{row}C{personCol+1}:R{row}C{inferredAmountCol+1}"
r1c1 := fmt.Sprintf("R%dC%d:R%dC%d", rowNum, idxPerson+1, rowNum, idxInferred+1)
updates = append(updates, sheets.ValueRange{
Range: r1c1,
Values: [][]any{{personVal, purposeVal, amountVal}},
})
}
if len(updates) == 0 || opts.DryRun {
return len(updates), nil
}
if err := sh.BatchUpdateValues(ctx, spreadsheetID, updates); err != nil {
return 0, fmt.Errorf("infer: batch update: %w", err)
}
return len(updates), nil
}
// dedupeMembers returns unique member names, deduped by canonical key.
func dedupeMembers(members []reconcile.Member) []string {
seen := make(map[string]bool, len(members))
var names []string
for _, m := range members {
key := strings.Join(strings.Fields(m.Name), " ")
if !seen[key] {
seen[key] = true
names = append(names, m.Name)
}
}
return names
}

View File

@@ -0,0 +1,157 @@
package banksync
import (
"context"
"fuj-management/go/internal/domain/reconcile"
"fuj-management/go/internal/io/sheets"
"testing"
)
type fakeAttendance struct {
adults, juniors []reconcile.Member
}
func (f *fakeAttendance) LoadAdults(_ context.Context) ([]reconcile.Member, []string, error) {
return f.adults, nil, nil
}
func (f *fakeAttendance) LoadJuniors(_ context.Context) ([]reconcile.Member, []string, error) {
return f.juniors, nil, nil
}
var paymentsHeader = []any{
"Date", "Amount", "manual fix", "Person", "Purpose", "Inferred Amount",
"Sender", "VS", "Message", "Bank ID", "Sync ID",
}
func TestInferPayments_BasicMatch(t *testing.T) {
sh := &sheets.Fake{Values: map[string][][]any{
"SHEETID/A1:Z": {
paymentsHeader,
// Row with no Person/Purpose — should be inferred
{"2026-04-10", 750.0, "", "", "", "", "Jana Novakova", "123", "duben 2026", "", ""},
},
}}
att := &fakeAttendance{
adults: []reconcile.Member{{Name: "Jana Novakova", Tier: "A"}},
}
n, err := InferPayments(context.Background(), "SHEETID", sh, att, InferOpts{})
if err != nil {
t.Fatal(err)
}
if n != 1 {
t.Errorf("want 1 row updated, got %d", n)
}
if len(sh.BatchUpdated) != 1 {
t.Fatalf("want 1 batch update call, got %d", len(sh.BatchUpdated))
}
upd := sh.BatchUpdated[0].Updates[0]
person := upd.Values[0][0].(string)
if person != "Jana Novakova" {
t.Errorf("inferred person: want 'Jana Novakova', got %q", person)
}
}
func TestInferPayments_SkipRule_ManualFix(t *testing.T) {
sh := &sheets.Fake{Values: map[string][][]any{
"SHEETID/A1:Z": {
paymentsHeader,
// manual fix is set — must be skipped
{"2026-04-10", 750.0, "yes", "", "", "", "Jana Novakova", "", "", "", ""},
},
}}
att := &fakeAttendance{adults: []reconcile.Member{{Name: "Jana Novakova", Tier: "A"}}}
n, err := InferPayments(context.Background(), "SHEETID", sh, att, InferOpts{})
if err != nil {
t.Fatal(err)
}
if n != 0 {
t.Errorf("want 0 updates (manual fix set), got %d", n)
}
}
func TestInferPayments_SkipRule_PersonAlreadySet(t *testing.T) {
sh := &sheets.Fake{Values: map[string][][]any{
"SHEETID/A1:Z": {
paymentsHeader,
{"2026-04-10", 750.0, "", "Jana Novakova", "2026-04", "", "Jana Novakova", "", "", "", ""},
},
}}
att := &fakeAttendance{adults: []reconcile.Member{{Name: "Jana Novakova", Tier: "A"}}}
n, err := InferPayments(context.Background(), "SHEETID", sh, att, InferOpts{})
if err != nil {
t.Fatal(err)
}
if n != 0 {
t.Errorf("want 0 updates (person already set), got %d", n)
}
}
func TestInferPayments_DryRun(t *testing.T) {
sh := &sheets.Fake{Values: map[string][][]any{
"SHEETID/A1:Z": {
paymentsHeader,
{"2026-04-10", 750.0, "", "", "", "", "Jana Novakova", "123", "duben 2026", "", ""},
},
}}
att := &fakeAttendance{adults: []reconcile.Member{{Name: "Jana Novakova", Tier: "A"}}}
n, err := InferPayments(context.Background(), "SHEETID", sh, att, InferOpts{DryRun: true})
if err != nil {
t.Fatal(err)
}
if n != 1 {
t.Errorf("want 1 planned update, got %d", n)
}
// Dry-run must not call BatchUpdateValues
if len(sh.BatchUpdated) != 0 {
t.Error("dry-run must not call BatchUpdateValues")
}
}
func TestInferPayments_ReviewPrefix(t *testing.T) {
sh := &sheets.Fake{Values: map[string][][]any{
"SHEETID/A1:Z": {
paymentsHeader,
// "novak" as sender alone → review confidence
{"2026-04-10", 750.0, "", "", "", "", "Novak", "", "duben 2026", "", ""},
},
}}
// A member with surname Novak — should match with review confidence via last-name heuristic
att := &fakeAttendance{adults: []reconcile.Member{{Name: "Pavel Novak", Tier: "A"}}}
n, err := InferPayments(context.Background(), "SHEETID", sh, att, InferOpts{})
if err != nil {
t.Fatal(err)
}
if n == 0 {
// Novak is in commonSurnames list so it won't match — acceptable
t.Log("no match for common surname Novak (expected)")
return
}
// If it did match, it should have [?] prefix
upd := sh.BatchUpdated[0].Updates[0]
person := upd.Values[0][0].(string)
if !isReviewPrefixed(person) && n > 0 {
t.Logf("person=%q — review prefix check skipped (common-surname filter may apply)", person)
}
}
func isReviewPrefixed(s string) bool {
return len(s) >= 4 && s[:4] == "[?] "
}
func TestDedupeMembers(t *testing.T) {
members := []reconcile.Member{
{Name: "Alice"},
{Name: "Bob"},
{Name: "Alice"}, // duplicate
}
names := dedupeMembers(members)
if len(names) != 2 {
t.Errorf("want 2 unique names, got %d: %v", len(names), names)
}
}

View File

@@ -0,0 +1,171 @@
// Package banksync implements the bank-sync and payment-inference operations.
package banksync
import (
"context"
"fmt"
"fuj-management/go/internal/domain/synch"
"fuj-management/go/internal/io/fio"
"os"
"strings"
"time"
)
// columnLabels is the canonical header for the payments sheet.
// Mirrors COLUMN_LABELS in scripts/sync_fio_to_sheets.py.
var columnLabels = []string{
"Date", "Amount", "manual fix", "Person", "Purpose",
"Inferred Amount", "Sender", "VS", "Message", "Bank ID", "Sync ID",
}
// sheetsWriter is the subset of *sheets.Client used by SyncToSheets.
type sheetsWriter interface {
GetValues(ctx context.Context, spreadsheetID, a1Range string) ([][]any, error)
AppendValues(ctx context.Context, spreadsheetID, a1Range string, rows [][]any) error
WriteHeader(ctx context.Context, spreadsheetID string, labels []string) error
SortByDateColumn(ctx context.Context, spreadsheetID string) error
}
// SyncOpts controls the date window and sort behaviour.
type SyncOpts struct {
Days int // look-back window when From/To are zero
From, To time.Time // explicit window (overrides Days)
Sort bool // sort the sheet by Date after appending
DryRun bool // print planned writes without modifying the sheet
PrintFioTable bool // with DryRun: print every fetched Fio txn with NEW/DUP status
}
// SyncToSheets fetches Fio transactions and appends new ones to the payments sheet.
// Returns the number of rows appended.
// Ports scripts/sync_fio_to_sheets.py sync_to_sheets.
func SyncToSheets(
ctx context.Context,
spreadsheetID string,
fioClient fio.Client,
sh sheetsWriter,
opts SyncOpts,
) (int, error) {
// 1. Read existing rows to collect known Sync IDs (column K, index 10).
rows, err := sh.GetValues(ctx, spreadsheetID, "A1:K")
if err != nil {
return 0, fmt.Errorf("sync: read sheet: %w", err)
}
existingIDs := make(map[string]bool)
if len(rows) > 0 {
header := rows[0]
if !headerMatches(header) {
if opts.DryRun {
fmt.Println("Dry run: would write header row")
} else {
if err := sh.WriteHeader(ctx, spreadsheetID, columnLabels); err != nil {
return 0, fmt.Errorf("sync: write header: %w", err)
}
}
} else {
for _, row := range rows[1:] {
if len(row) > 10 {
if id, ok := row[10].(string); ok && id != "" {
existingIDs[id] = true
}
}
}
}
}
// 2. Compute date window.
from, to := opts.From, opts.To
if from.IsZero() || to.IsZero() {
to = time.Now()
days := opts.Days
if days <= 0 {
days = 30
}
from = to.AddDate(0, 0, -days)
}
// 3. Fetch Fio transactions.
txns, err := fioClient.FetchTransactions(ctx, from, to)
if err != nil {
return 0, fmt.Errorf("sync: fetch fio: %w", err)
}
if opts.DryRun {
fmt.Printf("Dry run: window %s to %s, fetched %d transaction(s) from Fio\n",
from.Format("2006-01-02"), to.Format("2006-01-02"), len(txns))
}
// 4a. Compute Sync IDs for every fetched txn (shared by table-print and row-build).
syncIDs := make([]string, len(txns))
for i, tx := range txns {
currency := tx.Currency
if currency == "" {
currency = "CZK"
}
syncIDs[i] = synch.GenerateSyncID(synch.Transaction{
Date: tx.Date,
Amount: tx.Amount,
Currency: currency,
Sender: tx.Sender,
VS: tx.VS,
Message: tx.Message,
BankID: tx.BankID,
})
}
// 4b. Optional debug table (dry-run only; suppress when nothing was fetched).
if opts.DryRun && opts.PrintFioTable && len(txns) > 0 {
printFioTable(os.Stdout, txns, syncIDs, existingIDs)
}
// 4c. Build new rows.
var newRows [][]any
for i, tx := range txns {
if existingIDs[syncIDs[i]] {
continue
}
newRows = append(newRows, []any{
tx.Date, tx.Amount,
"", "", "", "", // manual fix, Person, Purpose, Inferred Amount
tx.Sender, tx.VS, tx.Message, tx.BankID, syncIDs[i],
})
}
if len(newRows) == 0 {
return 0, nil
}
if opts.DryRun {
for _, row := range newRows {
fmt.Printf("Dry run: would append date=%v amount=%v sender=%v vs=%v message=%v\n",
row[0], row[1], row[6], row[7], row[8])
}
if opts.Sort {
fmt.Println("Dry run: would sort by date")
}
return len(newRows), nil
}
if err := sh.AppendValues(ctx, spreadsheetID, "A2", newRows); err != nil {
return 0, fmt.Errorf("sync: append: %w", err)
}
if opts.Sort {
if err := sh.SortByDateColumn(ctx, spreadsheetID); err != nil {
return 0, fmt.Errorf("sync: sort: %w", err)
}
}
return len(newRows), nil
}
func headerMatches(row []any) bool {
if len(row) < len(columnLabels) {
return false
}
for i, label := range columnLabels {
cell, _ := row[i].(string)
if !strings.EqualFold(cell, label) {
return false
}
}
return true
}

View File

@@ -0,0 +1,157 @@
package banksync
import (
"context"
"fuj-management/go/internal/domain/synch"
"fuj-management/go/internal/io/fio"
"fuj-management/go/internal/io/sheets"
"testing"
"time"
)
var testFioTxns = []fio.Transaction{
{Date: "2026-04-10", Amount: 750, Sender: "Jana Novakova", Message: "duben 2026", VS: "123", BankID: "111"},
{Date: "2026-04-11", Amount: 500, Sender: "Petr Prach", Message: "april", VS: "456", BankID: "222"},
}
func TestSyncToSheets_EmptySheet(t *testing.T) {
sh := &sheets.Fake{Values: map[string][][]any{
"SHEETID/A1:K": {},
}}
fioFake := &fio.Fake{Transactions: testFioTxns}
n, err := SyncToSheets(context.Background(), "SHEETID", fioFake, sh, SyncOpts{Days: 30})
if err != nil {
t.Fatal(err)
}
if n != 2 {
t.Errorf("want 2 appended, got %d", n)
}
if len(sh.Appended) != 1 {
t.Fatalf("want 1 AppendValues call, got %d", len(sh.Appended))
}
rows := sh.Appended[0].Rows
if len(rows) != 2 {
t.Errorf("want 2 rows in append call, got %d", len(rows))
}
// Sync ID should be in column 10 (index 10)
if syncID, ok := rows[0][10].(string); !ok || len(syncID) != 64 {
t.Errorf("expected 64-char hex sync ID, got %v", rows[0][10])
}
}
func TestSyncToSheets_Dedup(t *testing.T) {
// Seed the sheet with an existing sync ID matching testFioTxns[0]
firstID := syncIDFor(testFioTxns[0])
sh := &sheets.Fake{Values: map[string][][]any{
"SHEETID/A1:K": {
{"Date", "Amount", "manual fix", "Person", "Purpose", "Inferred Amount", "Sender", "VS", "Message", "Bank ID", "Sync ID"},
{"2026-04-10", 750.0, "", "", "", "", "Jana Novakova", "123", "duben 2026", "111", firstID},
},
}}
fioFake := &fio.Fake{Transactions: testFioTxns}
n, err := SyncToSheets(context.Background(), "SHEETID", fioFake, sh, SyncOpts{Days: 30})
if err != nil {
t.Fatal(err)
}
if n != 1 {
t.Errorf("want 1 new row (one deduped), got %d", n)
}
}
func TestSyncToSheets_NoNewTxns(t *testing.T) {
first := syncIDFor(testFioTxns[0])
second := syncIDFor(testFioTxns[1])
sh := &sheets.Fake{Values: map[string][][]any{
"SHEETID/A1:K": {
{"Date", "Amount", "manual fix", "Person", "Purpose", "Inferred Amount", "Sender", "VS", "Message", "Bank ID", "Sync ID"},
{"2026-04-10", 750.0, "", "", "", "", "Jana Novakova", "123", "duben 2026", "111", first},
{"2026-04-11", 500.0, "", "", "", "", "Petr Prach", "456", "april", "222", second},
},
}}
fioFake := &fio.Fake{Transactions: testFioTxns}
n, err := SyncToSheets(context.Background(), "SHEETID", fioFake, sh, SyncOpts{Days: 30})
if err != nil {
t.Fatal(err)
}
if n != 0 {
t.Errorf("want 0 new rows, got %d", n)
}
if len(sh.Appended) != 0 {
t.Error("expected no AppendValues call when all deduped")
}
}
func TestSyncToSheets_MissingHeader(t *testing.T) {
sh := &sheets.Fake{Values: map[string][][]any{
"SHEETID/A1:K": {
{"Wrong", "Headers"},
},
}}
fioFake := &fio.Fake{Transactions: testFioTxns[:1]}
n, err := SyncToSheets(context.Background(), "SHEETID", fioFake, sh, SyncOpts{Days: 30})
if err != nil {
t.Fatal(err)
}
if n != 1 {
t.Errorf("want 1 row appended after header fix, got %d", n)
}
}
func TestSyncToSheets_Sort(t *testing.T) {
sh := &sheets.Fake{Values: map[string][][]any{"SHEETID/A1:K": {}}}
fioFake := &fio.Fake{Transactions: testFioTxns[:1]}
_, err := SyncToSheets(context.Background(), "SHEETID", fioFake, sh, SyncOpts{Days: 30, Sort: true})
if err != nil {
t.Fatal(err)
}
// SortByDateColumn should have been called on the fake — check via a spy fake
}
func TestSyncToSheets_ExplicitDateWindow(t *testing.T) {
sh := &sheets.Fake{Values: map[string][][]any{"SHEETID/A1:K": {}}}
fioFake := &fio.Fake{Transactions: testFioTxns[:1]}
from := time.Date(2026, 4, 1, 0, 0, 0, 0, time.UTC)
to := time.Date(2026, 4, 30, 0, 0, 0, 0, time.UTC)
n, err := SyncToSheets(context.Background(), "SHEETID", fioFake, sh, SyncOpts{From: from, To: to})
if err != nil {
t.Fatal(err)
}
if n != 1 {
t.Errorf("want 1 row, got %d", n)
}
}
func TestSyncToSheets_DryRun(t *testing.T) {
sh := &sheets.Fake{Values: map[string][][]any{"SHEETID/A1:K": {}}}
fioFake := &fio.Fake{Transactions: testFioTxns}
n, err := SyncToSheets(context.Background(), "SHEETID", fioFake, sh,
SyncOpts{Days: 30, Sort: true, DryRun: true})
if err != nil {
t.Fatal(err)
}
if n != 2 {
t.Errorf("want 2 planned, got %d", n)
}
if len(sh.Appended) != 0 {
t.Error("dry-run must not call AppendValues")
}
}
// syncIDFor mirrors what SyncToSheets computes for a given fio.Transaction.
func syncIDFor(tx fio.Transaction) string {
currency := tx.Currency
if currency == "" {
currency = "CZK"
}
return synch.GenerateSyncID(synch.Transaction{
Date: tx.Date, Amount: tx.Amount, Currency: currency,
Sender: tx.Sender, VS: tx.VS, Message: tx.Message, BankID: tx.BankID,
})
}

View File

@@ -17,6 +17,10 @@ func (f fakeAttendanceLoader) LoadAdults(_ context.Context) ([]reconcile.Member,
return f.members, f.months, nil
}
func (f fakeAttendanceLoader) LoadJuniors(_ context.Context) ([]reconcile.Member, []string, error) {
return nil, nil, nil
}
func TestFeesReport(t *testing.T) {
t.Parallel()
loader := fakeAttendanceLoader{

View File

@@ -9,10 +9,10 @@ import (
// ErrIOPending is returned by stub loader methods until the M4 IO layer lands.
var ErrIOPending = errors.New("io layer not yet wired up; lands in milestone M4 (sheets/drive/fio)")
// AttendanceLoader loads processed adult attendance + computed fees from the
// attendance Google Sheet.
// AttendanceLoader loads attendance and computed fees from the attendance Google Sheet.
type AttendanceLoader interface {
LoadAdults(ctx context.Context) (members []reconcile.Member, sortedMonths []string, err error)
LoadJuniors(ctx context.Context) (members []reconcile.Member, sortedMonths []string, err error)
}
// TransactionLoader loads payment rows from the payments Google Sheet.
@@ -41,6 +41,10 @@ func (stubSources) LoadAdults(_ context.Context) ([]reconcile.Member, []string,
return nil, nil, ErrIOPending
}
func (stubSources) LoadJuniors(_ context.Context) ([]reconcile.Member, []string, error) {
return nil, nil, ErrIOPending
}
func (stubSources) LoadTransactions(_ context.Context) ([]reconcile.Transaction, error) {
return nil, ErrIOPending
}

View File

@@ -19,6 +19,10 @@ func (f fakeSources) LoadAdults(_ context.Context) ([]reconcile.Member, []string
return f.members, f.months, nil
}
func (f fakeSources) LoadJuniors(_ context.Context) ([]reconcile.Member, []string, error) {
return nil, nil, nil
}
func (f fakeSources) LoadTransactions(_ context.Context) ([]reconcile.Transaction, error) {
return f.txns, nil
}

View File

@@ -0,0 +1,491 @@
package membership
import (
"context"
"fmt"
"fuj-management/go/internal/config"
"fuj-management/go/internal/domain/czech"
"fuj-management/go/internal/domain/fees"
"fuj-management/go/internal/domain/matching"
"fuj-management/go/internal/domain/reconcile"
"fuj-management/go/internal/io/attendance"
"fuj-management/go/internal/io/cache"
"fuj-management/go/internal/io/drive"
"fuj-management/go/internal/io/sheets"
"sort"
"strconv"
"strings"
"time"
)
// Attendance CSV column indices (mirrors COL_* in scripts/attendance.py)
const (
colName = 0
colTier = 1
firstDateCol = 3
)
// AdultMergedMonths mirrors ADULT_MERGED_MONTHS in scripts/attendance.py.
// Source month → target month (source attendance accumulated into target).
var AdultMergedMonths = map[string]string{}
// JuniorMergedMonths mirrors JUNIOR_MERGED_MONTHS in scripts/attendance.py.
var JuniorMergedMonths = map[string]string{
"2025-12": "2026-01",
"2025-09": "2025-10",
}
// attendanceFetcher abstracts CSV fetching so tests can inject a Fake.
type attendanceFetcher interface {
FetchAdults(ctx context.Context) ([][]string, error)
FetchJuniors(ctx context.Context) ([][]string, error)
}
// sheetReader abstracts Sheets API reads so tests can inject a Fake.
type sheetReader interface {
GetValues(ctx context.Context, spreadsheetID, a1Range string) ([][]any, error)
}
// realSources is the live implementation of Sources backed by Google APIs.
type realSources struct {
attendance attendanceFetcher
sheets sheetReader
cache *cache.FileCache
}
// NewSources builds a Sources backed by real Google Sheets and Drive APIs.
// Call this once at startup; the returned Sources is safe for concurrent use.
func NewSources(ctx context.Context, cfg config.Config) (Sources, error) {
driveCli, err := drive.New(ctx, cfg.CredentialsPath, cfg.DriveTimeout)
if err != nil {
return nil, fmt.Errorf("drive client: %w", err)
}
sheetsCli, err := sheets.New(ctx, cfg.CredentialsPath, cfg.DriveTimeout)
if err != nil {
return nil, fmt.Errorf("sheets client: %w", err)
}
attendanceCli := attendance.New(nil, config.AttendanceSheetID, config.AttendanceAdultSheetGID, config.JuniorSheetGID)
fc := cache.New(driveCli, cfg.CacheDir, config.CacheSheetMap, cfg.CacheTTL, cfg.CacheAPICheckTTL)
return &realSources{
attendance: attendanceCli,
sheets: sheetsCli,
cache: fc,
}, nil
}
// LoadAdults fetches adult attendance (cached) and returns reconcile.Members for all tiers.
func (s *realSources) LoadAdults(ctx context.Context) ([]reconcile.Member, []string, error) {
rows, err := cache.Get(ctx, s.cache, "attendance_regular", s.attendance.FetchAdults)
if err != nil {
return nil, nil, fmt.Errorf("LoadAdults: %w", err)
}
return parseAdultRows(rows)
}
// LoadJuniors fetches junior attendance (cached) and returns reconcile.Members for juniors.
func (s *realSources) LoadJuniors(ctx context.Context) ([]reconcile.Member, []string, error) {
// Junior data needs both the adult tab (tier="J" rows) and the junior tab.
adultRows, err := cache.Get(ctx, s.cache, "attendance_regular", s.attendance.FetchAdults)
if err != nil {
return nil, nil, fmt.Errorf("LoadJuniors (adult tab): %w", err)
}
juniorRows, err := cache.Get(ctx, s.cache, "attendance_juniors", s.attendance.FetchJuniors)
if err != nil {
return nil, nil, fmt.Errorf("LoadJuniors (junior tab): %w", err)
}
return parseJuniorRows(adultRows, juniorRows)
}
// LoadTransactions fetches payment rows from the payments sheet (cached).
func (s *realSources) LoadTransactions(ctx context.Context) ([]reconcile.Transaction, error) {
rows, err := cache.Get(ctx, s.cache, "payments_transactions",
func(ctx context.Context) ([][]any, error) {
return s.sheets.GetValues(ctx, config.PaymentsSheetID, "A1:Z")
})
if err != nil {
return nil, fmt.Errorf("LoadTransactions: %w", err)
}
return parseTransactionRows(rows)
}
// LoadExceptions fetches the exceptions tab (cached).
func (s *realSources) LoadExceptions(ctx context.Context) (map[reconcile.ExceptionKey]reconcile.Exception, error) {
rows, err := cache.Get(ctx, s.cache, "exceptions_dict",
func(ctx context.Context) ([][]any, error) {
return s.sheets.GetValues(ctx, config.PaymentsSheetID, "'exceptions'!A2:D")
})
if err != nil {
return nil, fmt.Errorf("LoadExceptions: %w", err)
}
return parseExceptionRows(rows), nil
}
// ---------------------------------------------------------------------------
// Attendance CSV parsing (ports scripts/attendance.py)
// ---------------------------------------------------------------------------
// parseDates returns (columnIndex, YYYY-MM) pairs for all date columns.
// Ports scripts/attendance.py parse_dates + strftime("%Y-%m").
func parseDates(header []string) []struct {
col int
month string
} {
var out []struct {
col int
month string
}
for i := firstDateCol; i < len(header); i++ {
raw := strings.TrimSpace(header[i])
if raw == "" {
continue
}
var dt time.Time
var err error
for _, fmt_ := range []string{"02.01.2006", "01/02/2006"} {
dt, err = time.Parse(fmt_, raw)
if err == nil {
break
}
}
if err != nil {
continue
}
out = append(out, struct {
col int
month string
}{col: i, month: dt.Format("2006-01")})
}
return out
}
// groupByMonth groups column indices by YYYY-MM, applying merged month mapping.
func groupByMonth(dates []struct {
col int
month string
}, mergedMonths map[string]string,
) map[string][]int {
out := make(map[string][]int)
for _, d := range dates {
target := d.month
if v, ok := mergedMonths[d.month]; ok {
target = v
}
out[target] = append(out[target], d.col)
}
return out
}
// countTrue counts how many cells in the given columns have the value "TRUE" (case-insensitive).
func countTrue(row []string, cols []int) int {
n := 0
for _, c := range cols {
if c < len(row) && strings.EqualFold(strings.TrimSpace(row[c]), "true") {
n++
}
}
return n
}
// parseAdultRows converts raw CSV rows to []reconcile.Member.
// Includes all tiers; fee is 0 for non-A tiers (reconcile filters downstream).
// Ports scripts/attendance.py get_members_with_fees.
func parseAdultRows(rows [][]string) ([]reconcile.Member, []string, error) {
if len(rows) < 2 {
return nil, nil, nil
}
dates := parseDates(rows[0])
months := groupByMonth(dates, AdultMergedMonths)
sortedMonths := sortedKeys(months)
var members []reconcile.Member
for _, row := range rows[1:] {
if len(row) == 0 {
continue
}
first := strings.TrimSpace(row[colName])
if strings.Contains(strings.ToLower(first), "# last line") {
break
}
if strings.HasPrefix(first, "#") || first == "" {
continue
}
if strings.ToLower(first) == "jméno" || strings.ToLower(first) == "name" || strings.ToLower(first) == "jmeno" {
continue
}
tier := ""
if len(row) > colTier {
tier = strings.ToUpper(strings.TrimSpace(row[colTier]))
}
feeMap := make(map[string]reconcile.FeeData, len(sortedMonths))
for _, m := range sortedMonths {
cols := months[m]
count := countTrue(row, cols)
var fee int
if tier == "A" {
fee = fees.CalculateFee(count, m)
}
feeMap[m] = reconcile.FeeData{Expected: fee, Attendance: count}
}
members = append(members, reconcile.Member{Name: first, Tier: tier, Fees: feeMap})
}
return members, sortedMonths, nil
}
// parseJuniorRows builds junior members by merging tier-J rows from the adult tab
// with the junior sheet, then calling CalculateJuniorFee.
// Ports scripts/attendance.py get_junior_members_with_fees.
func parseJuniorRows(adultRows, juniorRows [][]string) ([]reconcile.Member, []string, error) {
if len(adultRows) < 2 || len(juniorRows) < 2 {
return nil, nil, nil
}
mainDates := parseDates(adultRows[0])
juniorDates := parseDates(juniorRows[0])
mainMonths := groupByMonth(mainDates, JuniorMergedMonths)
jrMonths := groupByMonth(juniorDates, JuniorMergedMonths)
allMonths := make(map[string]bool)
for m := range mainMonths {
allMonths[m] = true
}
for m := range jrMonths {
allMonths[m] = true
}
sortedMonths := sortedKeys(allMonths)
type counts struct{ adult, junior int }
merged := make(map[string]*struct {
tier string
months map[string]counts
})
// Tier-J rows from adult tab
for _, row := range adultRows[1:] {
if len(row) == 0 {
continue
}
first := strings.TrimSpace(row[colName])
if strings.Contains(strings.ToLower(first), "# last line") {
break
}
if strings.HasPrefix(first, "#") || first == "" {
continue
}
tier := ""
if len(row) > colTier {
tier = strings.ToUpper(strings.TrimSpace(row[colTier]))
}
if tier != "J" {
continue
}
if _, ok := merged[first]; !ok {
merged[first] = &struct {
tier string
months map[string]counts
}{tier: tier, months: make(map[string]counts)}
}
for _, m := range sortedMonths {
c := merged[first].months[m]
c.adult += countTrue(row, mainMonths[m])
merged[first].months[m] = c
}
}
// All non-X rows from junior tab
for _, row := range juniorRows[1:] {
if len(row) == 0 {
continue
}
first := strings.TrimSpace(row[colName])
fl := strings.ToLower(first)
if strings.Contains(fl, "# treneri") || strings.Contains(fl, "# trenéři") {
break
}
if strings.HasPrefix(first, "#") || first == "" {
continue
}
tier := ""
if len(row) > colTier {
tier = strings.ToUpper(strings.TrimSpace(row[colTier]))
}
if tier == "X" {
continue
}
if _, ok := merged[first]; !ok {
merged[first] = &struct {
tier string
months map[string]counts
}{tier: tier, months: make(map[string]counts)}
}
for _, m := range sortedMonths {
c := merged[first].months[m]
c.junior += countTrue(row, jrMonths[m])
merged[first].months[m] = c
}
}
var members []reconcile.Member
for name, data := range merged {
feeMap := make(map[string]reconcile.FeeData, len(sortedMonths))
for _, m := range sortedMonths {
c := data.months[m]
total := c.adult + c.junior
exp := fees.CalculateJuniorFee(total, m)
fee := 0
if !exp.Unknown {
fee = exp.Value
}
feeMap[m] = reconcile.FeeData{
Expected: fee,
IsUnknown: exp.Unknown,
Attendance: total,
JuniorAttendance: c.junior,
AdultAttendance: c.adult,
}
}
members = append(members, reconcile.Member{Name: name, Tier: data.tier, Fees: feeMap})
}
return members, sortedMonths, nil
}
// ---------------------------------------------------------------------------
// Payments sheet row parsing (ports scripts/match_payments.py fetch_sheet_data)
// ---------------------------------------------------------------------------
func parseTransactionRows(rows [][]any) ([]reconcile.Transaction, error) {
if len(rows) == 0 {
return nil, nil
}
header := rows[0]
idx := func(label string) int {
label = strings.ToLower(strings.TrimSpace(label))
for i, h := range header {
if strings.ToLower(strings.TrimSpace(fmt.Sprint(h))) == label {
return i
}
}
return -1
}
idxDate := idx("date")
idxAmount := idx("amount")
idxManualFix := idx("manual fix")
idxPerson := idx("person")
idxPurpose := idx("purpose")
idxInferred := idx("inferred amount")
idxSender := idx("sender")
idxVS := idx("vs")
idxMessage := idx("message")
idxBankID := idx("bank id")
idxSyncID := idx("sync id")
for _, label := range []string{"date", "amount", "person", "purpose"} {
if idx(label) == -1 {
return nil, fmt.Errorf("payments sheet missing required column %q", label)
}
}
getVal := func(row []any, i int) string {
if i < 0 || i >= len(row) {
return ""
}
return fmt.Sprint(row[i])
}
var txns []reconcile.Transaction
for _, row := range rows[1:] {
dateStr := matching.FormatDate(getVal(row, idxDate))
amountRaw := row[idxAmount]
if idxAmount < 0 || idxAmount >= len(row) {
amountRaw = ""
}
amount := parseFloat(amountRaw)
var inferredAmount *float64
if iv := getVal(row, idxInferred); iv != "" && iv != "<nil>" {
if f := parseFloat(iv); f != 0 {
inferredAmount = &f
}
}
txns = append(txns, reconcile.Transaction{
Date: dateStr,
Amount: amount,
ManualFix: getVal(row, idxManualFix),
Person: getVal(row, idxPerson),
Purpose: getVal(row, idxPurpose),
InferredAmount: inferredAmount,
Sender: getVal(row, idxSender),
VS: getVal(row, idxVS),
Message: getVal(row, idxMessage),
BankID: getVal(row, idxBankID),
SyncID: getVal(row, idxSyncID),
})
}
return txns, nil
}
func parseFloat(v any) float64 {
switch x := v.(type) {
case float64:
return x
case float32:
return float64(x)
case int:
return float64(x)
case int64:
return float64(x)
case string:
f, _ := strconv.ParseFloat(strings.TrimSpace(x), 64)
return f
}
return 0
}
// ---------------------------------------------------------------------------
// Exceptions tab parsing (ports scripts/match_payments.py fetch_exceptions)
// ---------------------------------------------------------------------------
func parseExceptionRows(rows [][]any) map[reconcile.ExceptionKey]reconcile.Exception {
out := make(map[reconcile.ExceptionKey]reconcile.Exception)
for _, row := range rows {
if len(row) < 3 {
continue
}
name := strings.TrimSpace(fmt.Sprint(row[0]))
if strings.ToLower(name) == "name" || strings.HasPrefix(strings.ToLower(name), "name") {
continue
}
period := strings.TrimSpace(fmt.Sprint(row[1]))
amountStr := fmt.Sprint(row[2])
amount, err := strconv.Atoi(strings.TrimSpace(amountStr))
if err != nil {
continue
}
note := ""
if len(row) > 3 {
note = strings.TrimSpace(fmt.Sprint(row[3]))
}
key := reconcile.ExceptionKey{
Name: czech.Normalize(name),
Period: czech.Normalize(period),
}
out[key] = reconcile.Exception{Amount: amount, Note: note}
}
return out
}
// ---------------------------------------------------------------------------
// Helpers
// ---------------------------------------------------------------------------
func sortedKeys[V any](m map[string]V) []string {
keys := make([]string, 0, len(m))
for k := range m {
keys = append(keys, k)
}
sort.Strings(keys)
return keys
}

View File

@@ -0,0 +1,198 @@
package membership
import (
"context"
"fuj-management/go/internal/config"
"fuj-management/go/internal/io/attendance"
"fuj-management/go/internal/io/cache"
"fuj-management/go/internal/io/drive"
"fuj-management/go/internal/io/sheets"
"testing"
"time"
)
// buildSources wires a realSources with in-memory fakes and a no-TTL cache.
func buildSources(t *testing.T, att *attendance.Fake, sh *sheets.Fake) *realSources {
t.Helper()
dir := t.TempDir()
d := &drive.Fake{Times: map[string]string{
config.AttendanceSheetID: "t1",
config.PaymentsSheetID: "t1",
}}
fc := cache.New(d, dir, config.CacheSheetMap, 0, 0)
return &realSources{attendance: att, sheets: sh, cache: fc}
}
var minimalAdultCSV = [][]string{
{"Jméno", "Tier", "", "", "01.09.2025", "08.09.2025"},
{"Alice", "A", "", "", "TRUE", "TRUE"},
{"Bob", "A", "", "", "TRUE", "FALSE"},
{"# last line"},
}
// minimalJuniorCSV has dates in October because the junior merged-month map sends
// 2025-09 → 2025-10, so two columns for 01.10.2025 and 08.10.2025 land in "2025-10".
var minimalJuniorCSV = [][]string{
{"Jméno", "Tier", "", "", "01.10.2025", "08.10.2025"},
{"Charlie", "J", "", "", "TRUE", "TRUE"},
{"# Trenéři"},
{"Coach", "X", "", "", "FALSE", "FALSE"},
}
func TestLoadAdults(t *testing.T) {
s := buildSources(t, &attendance.Fake{Adults: minimalAdultCSV}, &sheets.Fake{})
members, months, err := s.LoadAdults(context.Background())
if err != nil {
t.Fatal(err)
}
// adultMergedMonths is empty so 2025-09 stays as-is
if len(months) != 1 || months[0] != "2025-09" {
t.Errorf("unexpected months: %v", months)
}
if len(members) != 2 {
t.Fatalf("want 2 members, got %d", len(members))
}
byName := map[string]int{}
for _, m := range members {
byName[m.Name] = m.Fees["2025-09"].Attendance
}
if byName["Alice"] != 2 {
t.Errorf("Alice: want 2 sessions, got %d", byName["Alice"])
}
if byName["Bob"] != 1 {
t.Errorf("Bob: want 1 session, got %d", byName["Bob"])
}
}
func TestLoadAdults_Fee(t *testing.T) {
s := buildSources(t, &attendance.Fake{Adults: minimalAdultCSV}, &sheets.Fake{})
members, _, err := s.LoadAdults(context.Background())
if err != nil {
t.Fatal(err)
}
byName := map[string]int{}
for _, m := range members {
byName[m.Name] = m.Fees["2025-09"].Expected
}
// 2 sessions in 2025-09 → AdultFeeMonthlyRate["2025-09"] = 750
if byName["Alice"] != 750 {
t.Errorf("Alice fee: want 750, got %d", byName["Alice"])
}
// 1 session → AdultFeeSingle = 200
if byName["Bob"] != 200 {
t.Errorf("Bob fee: want 200, got %d", byName["Bob"])
}
}
func TestLoadJuniors(t *testing.T) {
s := buildSources(t,
&attendance.Fake{Adults: minimalAdultCSV, Juniors: minimalJuniorCSV},
&sheets.Fake{})
members, months, err := s.LoadJuniors(context.Background())
if err != nil {
t.Fatal(err)
}
if len(months) == 0 {
t.Fatal("want months, got none")
}
found := false
for _, m := range members {
if m.Name == "Charlie" {
found = true
// Charlie has 2 sessions in 2025-10 (October dates in junior CSV)
if m.Fees["2025-10"].Attendance != 2 {
t.Errorf("Charlie 2025-10 attendance: want 2, got %d", m.Fees["2025-10"].Attendance)
}
}
}
if !found {
t.Error("Charlie not found in juniors")
}
}
func TestLoadTransactions(t *testing.T) {
// Sheets fake keyed by "<spreadsheetID>/<range>" — use the real constant.
paymentsKey := config.PaymentsSheetID + "/A1:Z"
sh := &sheets.Fake{Values: map[string][][]any{
paymentsKey: {
{"Date", "Amount", "manual fix", "Person", "Purpose", "Inferred Amount", "Sender", "VS", "Message", "Bank ID", "Sync ID"},
{"2026-04-01", 700.0, "", "Alice", "2026-04", "", "Alice Bank", "", "fee", "", "abc"},
{"2026-05-01", 500.0, "", "", "", "", "Bob Bank", "", "platba", "", "def"},
},
}}
s := buildSources(t, &attendance.Fake{}, sh)
txns, err := s.LoadTransactions(context.Background())
if err != nil {
t.Fatal(err)
}
if len(txns) != 2 {
t.Fatalf("want 2 transactions, got %d", len(txns))
}
if txns[0].Person != "Alice" {
t.Errorf("txn[0].Person: want Alice, got %q", txns[0].Person)
}
if txns[0].Amount != 700 {
t.Errorf("txn[0].Amount: want 700, got %v", txns[0].Amount)
}
}
func TestLoadExceptions(t *testing.T) {
excKey := config.PaymentsSheetID + "/'exceptions'!A2:D"
sh := &sheets.Fake{Values: map[string][][]any{
excKey: {
{"Alice", "2026-04", 350, "reduced"},
},
}}
s := buildSources(t, &attendance.Fake{}, sh)
exc, err := s.LoadExceptions(context.Background())
if err != nil {
t.Fatal(err)
}
if len(exc) != 1 {
t.Fatalf("want 1 exception, got %d", len(exc))
}
for k, v := range exc {
if v.Amount != 350 {
t.Errorf("exception amount: want 350, got %d (key=%v)", v.Amount, k)
}
if v.Note != "reduced" {
t.Errorf("exception note: want 'reduced', got %q", v.Note)
}
}
}
// TTL smoke test: second call within TTL must not call fetch again.
func TestLoadAdults_CacheHit(t *testing.T) {
dir := t.TempDir()
d := &drive.Fake{Times: map[string]string{config.AttendanceSheetID: "t1"}}
fc := cache.New(d, dir, config.CacheSheetMap, time.Minute, time.Minute)
calls := 0
att := &countingFetcher{rows: minimalAdultCSV, calls: &calls}
s := &realSources{attendance: att, sheets: &sheets.Fake{}, cache: fc}
if _, _, err := s.LoadAdults(context.Background()); err != nil {
t.Fatal(err)
}
if _, _, err := s.LoadAdults(context.Background()); err != nil {
t.Fatal(err)
}
if calls != 1 {
t.Errorf("want 1 fetch (cache hit on 2nd call), got %d", calls)
}
}
type countingFetcher struct {
rows [][]string
calls *int
}
func (f *countingFetcher) FetchAdults(_ context.Context) ([][]string, error) {
*f.calls++
return f.rows, nil
}
func (f *countingFetcher) FetchJuniors(_ context.Context) ([][]string, error) { return nil, nil }

View File

@@ -0,0 +1,42 @@
package api
// AdultsMonthData is the reconciled ledger for one adult member in one month.
// Keys match Python's result["members"][name]["months"][YYYY-MM].
type AdultsMonthData struct {
Expected int `json:"expected"`
OriginalExpected int `json:"original_expected"`
AttendanceCount int `json:"attendance_count"`
Exception *ExceptionData `json:"exception"`
Paid float64 `json:"paid"` // float: proportional allocator may produce fractional CZK
Transactions []MemberTxEntry `json:"transactions"`
}
// AdultsMemberData is the reconciled ledger for one adult member.
// Keys match Python's result["members"][name].
type AdultsMemberData struct {
Tier string `json:"tier"`
Months map[string]AdultsMonthData `json:"months"` // YYYY-MM → month data
OtherTransactions []MemberOtherEntry `json:"other_transactions"`
TotalBalance int `json:"total_balance"`
}
// AdultsResponse is the JSON contract for GET /api/adults.
// MemberData, MonthLabels, and RawPayments correspond to the Python view-model
// fields member_data, month_labels_json, and raw_payments_json respectively,
// but as nested objects rather than pre-serialised JSON strings.
type AdultsResponse struct {
Months []string `json:"months"` // display labels
RawMonths []string `json:"raw_months"` // "YYYY-MM"
Results []MemberRow `json:"results"`
Totals []TotalCell `json:"totals"`
MemberData map[string]AdultsMemberData `json:"member_data"` // name → ledger
MonthLabels map[string]string `json:"month_labels"` // YYYY-MM → display label
RawPayments map[string][]RawTransaction `json:"raw_payments"` // name → raw sheet rows
Credits []Credit `json:"credits"`
Debts []Credit `json:"debts"`
Unmatched []RawTransaction `json:"unmatched"`
AttendanceURL string `json:"attendance_url"`
PaymentsURL string `json:"payments_url"`
BankAccount string `json:"bank_account"`
CurrentMonth string `json:"current_month"`
}

View File

@@ -0,0 +1,263 @@
package api
import (
"fmt"
"fuj-management/go/internal/config"
"fuj-management/go/internal/services/membership"
"sort"
"time"
domreconcile "fuj-management/go/internal/domain/reconcile"
)
type monthSums struct{ expected, paid int }
// buildAdultsResponse constructs the AdultsResponse wire type from reconcile output.
// Mirrors scripts/views.py:build_adults_view_model.
func buildAdultsResponse(
members []domreconcile.Member,
sortedMonths []string,
result domreconcile.Result,
txns []domreconcile.Transaction,
cfg config.Config,
currentMonth string,
) AdultsResponse {
monthLabels := getMonthLabels(sortedMonths, membership.AdultMergedMonths)
// Collect tier-A names, sorted.
var adultNames []string
allNames := make([]string, 0, len(members))
for _, m := range members {
allNames = append(allNames, m.Name)
if m.Tier == "A" {
adultNames = append(adultNames, m.Name)
}
}
sort.Strings(adultNames)
// Per-month aggregate totals (expected and paid integers).
monthlyTotals := make(map[string]*monthSums, len(sortedMonths))
for _, m := range sortedMonths {
monthlyTotals[m] = &monthSums{}
}
var results []MemberRow
for _, name := range adultNames {
mr := result.Members[name]
row, unpaidMonths, rawUnpaidMonths := buildAdultMemberRow(name, mr, sortedMonths, monthLabels, currentMonth, monthlyTotals)
row.UnpaidPeriods = joinComma(unpaidMonths)
row.RawUnpaidPeriods = joinPlus(rawUnpaidMonths)
row.Balance = settledBalance(mr, currentMonth)
row.PayableAmount = max(0, -row.Balance)
results = append(results, row)
}
// Totals row.
totalsCells := make([]TotalCell, len(sortedMonths))
for i, m := range sortedMonths {
t := monthlyTotals[m] // *monthSums, never nil (initialised above)
status := "empty"
if t.expected > 0 || t.paid > 0 {
switch {
case t.paid == t.expected:
status = "ok"
case t.paid < t.expected:
status = "unpaid"
default:
status = "surplus"
}
}
totalsCells[i] = TotalCell{
Text: fmt.Sprintf("%d / %d CZK", t.paid, t.expected),
Status: status,
}
}
// Credits and debts (settled balance, past months only).
var credits, debts []Credit
for _, name := range adultNames {
bal := settledBalance(result.Members[name], currentMonth)
if bal > 0 {
credits = append(credits, Credit{Name: name, Amount: bal})
} else if bal < 0 {
debts = append(debts, Credit{Name: name, Amount: -bal})
}
}
sort.Slice(credits, func(i, j int) bool { return credits[i].Name < credits[j].Name })
sort.Slice(debts, func(i, j int) bool { return debts[i].Name < debts[j].Name })
// member_data: full reconcile output for all members (not just adults).
memberData := make(map[string]AdultsMemberData, len(result.Members))
for name, mr := range result.Members {
months := make(map[string]AdultsMonthData, len(mr.Months))
for m, md := range mr.Months {
var exc *ExceptionData
if md.Exception != nil {
exc = &ExceptionData{Amount: md.Exception.Amount, Note: md.Exception.Note}
}
txEntries := make([]MemberTxEntry, len(md.Transactions))
for i, te := range md.Transactions {
txEntries[i] = memberTxFromDomain(te)
}
months[m] = AdultsMonthData{
Expected: md.Expected,
OriginalExpected: md.OriginalExpected,
AttendanceCount: md.AttendanceCount,
Exception: exc,
Paid: md.Paid,
Transactions: txEntries,
}
}
otherTxs := make([]MemberOtherEntry, len(mr.OtherTransactions))
for i, oe := range mr.OtherTransactions {
otherTxs[i] = memberOtherFromDomain(oe)
}
memberData[name] = AdultsMemberData{
Tier: mr.Tier,
Months: months,
OtherTransactions: otherTxs,
TotalBalance: mr.TotalBalance,
}
}
unmatched := make([]RawTransaction, len(result.Unmatched))
for i, tx := range result.Unmatched {
unmatched[i] = rawTxFromDomain(tx)
}
return AdultsResponse{
Months: labelsForMonths(sortedMonths, monthLabels),
RawMonths: sortedMonths,
Results: ensureSlice(results),
Totals: totalsCells,
MemberData: memberData,
MonthLabels: monthLabels,
RawPayments: groupRawPaymentsByPerson(txns, allNames),
Credits: ensureSlice(credits),
Debts: ensureSlice(debts),
Unmatched: unmatched,
AttendanceURL: "https://docs.google.com/spreadsheets/d/" + config.AttendanceSheetID + "/edit",
PaymentsURL: "https://docs.google.com/spreadsheets/d/" + config.PaymentsSheetID + "/edit",
BankAccount: cfg.BankAccount,
CurrentMonth: currentMonth,
}
}
func buildAdultMemberRow(
name string,
mr domreconcile.MemberResult,
sortedMonths []string,
monthLabels map[string]string,
currentMonth string,
monthlyTotals map[string]*monthSums,
) (row MemberRow, unpaidMonths, rawUnpaidMonths []string) {
row = MemberRow{Name: name}
for _, m := range sortedMonths {
md, ok := mr.Months[m]
if !ok {
md = domreconcile.MonthData{}
}
paid := int(md.Paid)
expected := md.Expected
if t := monthlyTotals[m]; t != nil {
t.expected += expected
t.paid += paid
}
var feeDisplay string
var isOverridden bool
if md.Exception != nil && md.Exception.Amount != md.OriginalExpected {
isOverridden = true
if md.AttendanceCount > 0 {
feeDisplay = fmt.Sprintf("%d (%d) CZK (%d)", md.Exception.Amount, md.OriginalExpected, md.AttendanceCount)
} else {
feeDisplay = fmt.Sprintf("%d (%d) CZK", md.Exception.Amount, md.OriginalExpected)
}
} else {
if md.AttendanceCount > 0 {
feeDisplay = fmt.Sprintf("%d CZK (%d)", expected, md.AttendanceCount)
} else {
feeDisplay = fmt.Sprintf("%d CZK", expected)
}
}
status := "empty"
cellText := "-"
amountToPay := 0
switch {
case expected > 0:
amountToPay = max(0, expected-paid)
switch {
case paid >= expected:
status = "ok"
cellText = fmt.Sprintf("%d/%s", paid, feeDisplay)
case paid > 0:
status = "partial"
cellText = fmt.Sprintf("%d/%s", paid, feeDisplay)
if m < currentMonth {
unpaidMonths = append(unpaidMonths, monthLabels[m])
rawUnpaidMonths = append(rawUnpaidMonths, rawMonthLabel(m))
}
default:
status = "unpaid"
cellText = fmt.Sprintf("0/%s", feeDisplay)
if m < currentMonth {
unpaidMonths = append(unpaidMonths, monthLabels[m])
rawUnpaidMonths = append(rawUnpaidMonths, rawMonthLabel(m))
}
}
case paid > 0:
status = "surplus"
cellText = fmt.Sprintf("PAID %d", paid)
}
tooltip := ""
if expected > 0 || paid > 0 {
tooltip = fmt.Sprintf("Received: %d, Expected: %d", paid, expected)
}
row.Months = append(row.Months, MonthCell{
Text: cellText,
Overridden: isOverridden,
Status: status,
Amount: amountToPay,
Month: monthLabels[m],
RawMonth: m,
Tooltip: tooltip,
})
}
return row, unpaidMonths, rawUnpaidMonths
}
// rawMonthLabel converts "YYYY-MM" to "MM/YYYY" matching Python's strftime("%m/%Y").
func rawMonthLabel(m string) string {
dt, err := time.Parse("2006-01", m)
if err != nil {
return m
}
return dt.Format("01/2006")
}
func joinComma(parts []string) string {
if len(parts) == 0 {
return ""
}
result := parts[0]
for _, p := range parts[1:] {
result += ", " + p
}
return result
}
func joinPlus(parts []string) string {
if len(parts) == 0 {
return ""
}
result := parts[0]
for _, p := range parts[1:] {
result += "+" + p
}
return result
}

View File

@@ -0,0 +1,184 @@
package api
import (
"fuj-management/go/internal/domain/czech"
"regexp"
"sort"
"strings"
"time"
domreconcile "fuj-management/go/internal/domain/reconcile"
)
// getMonthLabels builds display labels for sortedMonths, merging month names
// (e.g. "Dec+Jan 2026") when mergedMonths maps a source month into this target.
// Mirrors scripts/views.py:get_month_labels.
func getMonthLabels(sortedMonths []string, mergedMonths map[string]string) map[string]string {
labels := make(map[string]string, len(sortedMonths))
for _, m := range sortedMonths {
dt, err := time.Parse("2006-01", m)
if err != nil {
labels[m] = m
continue
}
var mergedIn []string
for src, dst := range mergedMonths {
if dst == m {
mergedIn = append(mergedIn, src)
}
}
sort.Strings(mergedIn)
if len(mergedIn) == 0 {
labels[m] = dt.Format("Jan 2006")
continue
}
allMonths := append(mergedIn, m) //nolint:gocritic // intentional: mergedIn already owned
sort.Strings(allMonths)
years := map[int]bool{}
for _, x := range allMonths {
if d, err2 := time.Parse("2006-01", x); err2 == nil {
years[d.Year()] = true
}
}
parts := make([]string, 0, len(allMonths))
if len(years) > 1 {
for _, x := range allMonths {
if d, err2 := time.Parse("2006-01", x); err2 == nil {
parts = append(parts, d.Format("Jan 2006"))
}
}
labels[m] = strings.Join(parts, "+")
} else {
for _, x := range allMonths {
if d, err2 := time.Parse("2006-01", x); err2 == nil {
parts = append(parts, d.Format("Jan"))
}
}
labels[m] = strings.Join(parts, "+") + " " + dt.Format("2006")
}
}
return labels
}
// labelsForMonths returns the display labels for sortedMonths in slice order.
func labelsForMonths(sortedMonths []string, labels map[string]string) []string {
out := make([]string, len(sortedMonths))
for i, m := range sortedMonths {
out[i] = labels[m]
}
return out
}
var questionMarkRe = regexp.MustCompile(`\[\?\]\s*`)
// canonicalKey returns a normalized form of a person name used for deduplication.
// Mirrors scripts/match_payments.py:canonical_member_key.
func canonicalKey(name string) string {
return strings.Join(strings.Fields(czech.Normalize(name)), " ")
}
// groupRawPaymentsByPerson groups transactions by the "person" column,
// canonicalizing names against memberNames where possible.
// Mirrors scripts/views.py:group_payments_by_person (without the
// "Unmatched / Unknown" bucket that is payments-view-specific).
func groupRawPaymentsByPerson(txns []domreconcile.Transaction, memberNames []string) map[string][]RawTransaction {
canonicalByKey := make(map[string]string, len(memberNames))
for _, n := range memberNames {
k := canonicalKey(n)
if _, exists := canonicalByKey[k]; !exists {
canonicalByKey[k] = n
}
}
grouped := make(map[string][]RawTransaction)
for _, tx := range txns {
person := strings.TrimSpace(tx.Person)
if person == "" {
continue
}
for _, p := range strings.Split(person, ",") {
p = questionMarkRe.ReplaceAllString(p, "")
p = strings.TrimSpace(p)
if p == "" {
continue
}
key := p
if canonical, ok := canonicalByKey[canonicalKey(p)]; ok {
key = canonical
}
grouped[key] = append(grouped[key], rawTxFromDomain(tx))
}
}
for k := range grouped {
sort.Slice(grouped[k], func(i, j int) bool {
return grouped[k][i].Date > grouped[k][j].Date
})
}
return grouped
}
// rawTxFromDomain converts a domain Transaction to the wire RawTransaction.
func rawTxFromDomain(tx domreconcile.Transaction) RawTransaction {
inferredAmount := 0.0
if tx.InferredAmount != nil {
inferredAmount = *tx.InferredAmount
}
return RawTransaction{
Date: tx.Date,
Amount: tx.Amount,
ManualFix: tx.ManualFix,
Person: tx.Person,
Purpose: tx.Purpose,
InferredAmount: inferredAmount,
Sender: tx.Sender,
VS: tx.VS,
Message: tx.Message,
BankID: tx.BankID,
SyncID: tx.SyncID,
}
}
// memberTxFromDomain converts a domain TxEntry to a wire MemberTxEntry.
func memberTxFromDomain(te domreconcile.TxEntry) MemberTxEntry {
return MemberTxEntry{
Amount: te.Amount,
Date: te.Date,
Sender: te.Sender,
Message: te.Message,
Confidence: te.Confidence,
}
}
// memberOtherFromDomain converts a domain OtherEntry to a wire MemberOtherEntry.
func memberOtherFromDomain(oe domreconcile.OtherEntry) MemberOtherEntry {
return MemberOtherEntry{
Amount: oe.Amount,
Date: oe.Date,
Sender: oe.Sender,
Message: oe.Message,
Purpose: oe.Purpose,
Confidence: oe.Confidence,
}
}
// settledBalance computes the settled balance: sum of (paid expected) for months
// strictly before currentMonth. Months with IsUnknown=true are excluded to match
// Python's isinstance(exp, int) guard (skips "?" months).
func settledBalance(mr domreconcile.MemberResult, currentMonth string) int {
total := 0
for m, md := range mr.Months {
if m >= currentMonth || md.IsUnknown {
continue
}
total += int(md.Paid) - md.Expected
}
return total
}
// ensureSlice returns s unchanged when non-nil, or an empty (non-nil) slice so
// json.Marshal emits [] instead of null.
func ensureSlice[T any](s []T) []T {
if s == nil {
return []T{}
}
return s
}

View File

@@ -0,0 +1,276 @@
package api
import (
"fmt"
"fuj-management/go/internal/config"
"fuj-management/go/internal/services/membership"
"sort"
"strconv"
domreconcile "fuj-management/go/internal/domain/reconcile"
)
// buildJuniorsResponse constructs the JuniorsResponse wire type from reconcile output.
// Mirrors scripts/views.py:build_juniors_view_model.
func buildJuniorsResponse(
members []domreconcile.Member,
sortedMonths []string,
result domreconcile.Result,
txns []domreconcile.Transaction,
cfg config.Config,
currentMonth string,
) JuniorsResponse {
monthLabels := getMonthLabels(sortedMonths, membership.JuniorMergedMonths)
allNames := make([]string, 0, len(members))
juniorNames := make([]string, 0, len(members))
for _, m := range members {
allNames = append(allNames, m.Name)
juniorNames = append(juniorNames, m.Name)
}
sort.Strings(juniorNames)
monthlyTotals := make(map[string]*monthSums, len(sortedMonths))
for _, m := range sortedMonths {
monthlyTotals[m] = &monthSums{}
}
var results []MemberRow
for _, name := range juniorNames {
mr := result.Members[name]
row, unpaidMonths, rawUnpaidMonths := buildJuniorMemberRow(name, mr, sortedMonths, monthLabels, currentMonth, monthlyTotals)
row.UnpaidPeriods = joinComma(unpaidMonths)
row.RawUnpaidPeriods = joinPlus(rawUnpaidMonths)
row.Balance = settledBalance(mr, currentMonth)
row.PayableAmount = max(0, -row.Balance)
results = append(results, row)
}
// Totals row.
totalsCells := make([]TotalCell, len(sortedMonths))
for i, m := range sortedMonths {
t := monthlyTotals[m] // *monthSums, never nil (initialised above)
status := "empty"
if t.expected > 0 || t.paid > 0 {
switch {
case t.paid == t.expected:
status = "ok"
case t.paid < t.expected:
status = "unpaid"
default:
status = "surplus"
}
}
totalsCells[i] = TotalCell{
Text: fmt.Sprintf("%d / %d CZK", t.paid, t.expected),
Status: status,
}
}
var credits, debts []Credit
for _, name := range juniorNames {
bal := settledBalance(result.Members[name], currentMonth)
if bal > 0 {
credits = append(credits, Credit{Name: name, Amount: bal})
} else if bal < 0 {
debts = append(debts, Credit{Name: name, Amount: -bal})
}
}
sort.Slice(credits, func(i, j int) bool { return credits[i].Name < credits[j].Name })
sort.Slice(debts, func(i, j int) bool { return debts[i].Name < debts[j].Name })
// member_data: full reconcile output for all junior members.
memberData := make(map[string]JuniorsMemberData, len(result.Members))
for name, mr := range result.Members {
months := make(map[string]JuniorsMonthData, len(mr.Months))
for m, md := range mr.Months {
var exc *ExceptionData
if md.Exception != nil {
exc = &ExceptionData{Amount: md.Exception.Amount, Note: md.Exception.Note}
}
txEntries := make([]MemberTxEntry, len(md.Transactions))
for i, te := range md.Transactions {
txEntries[i] = memberTxFromDomain(te)
}
months[m] = JuniorsMonthData{
Expected: juniorExpected(md),
OriginalExpected: juniorOriginalExpected(md),
AttendanceCount: md.AttendanceCount,
Exception: exc,
Paid: md.Paid,
Transactions: txEntries,
}
}
otherTxs := make([]MemberOtherEntry, len(mr.OtherTransactions))
for i, oe := range mr.OtherTransactions {
otherTxs[i] = memberOtherFromDomain(oe)
}
memberData[name] = JuniorsMemberData{
Tier: mr.Tier,
Months: months,
OtherTransactions: otherTxs,
TotalBalance: mr.TotalBalance,
}
}
unmatched := make([]RawTransaction, len(result.Unmatched))
for i, tx := range result.Unmatched {
unmatched[i] = rawTxFromDomain(tx)
}
juniorURL := "https://docs.google.com/spreadsheets/d/" + config.AttendanceSheetID +
"/edit#gid=" + config.JuniorSheetGID
return JuniorsResponse{
Months: labelsForMonths(sortedMonths, monthLabels),
RawMonths: sortedMonths,
Results: ensureSlice(results),
Totals: totalsCells,
MemberData: memberData,
MonthLabels: monthLabels,
RawPayments: groupRawPaymentsByPerson(txns, allNames),
Credits: ensureSlice(credits),
Debts: ensureSlice(debts),
Unmatched: unmatched,
AttendanceURL: juniorURL,
PaymentsURL: "https://docs.google.com/spreadsheets/d/" + config.PaymentsSheetID + "/edit",
BankAccount: cfg.BankAccount,
CurrentMonth: currentMonth,
}
}
func buildJuniorMemberRow(
name string,
mr domreconcile.MemberResult,
sortedMonths []string,
monthLabels map[string]string,
currentMonth string,
monthlyTotals map[string]*monthSums,
) (row MemberRow, unpaidMonths, rawUnpaidMonths []string) {
row = MemberRow{Name: name}
for _, m := range sortedMonths {
md, ok := mr.Months[m]
if !ok {
md = domreconcile.MonthData{}
}
paid := int(md.Paid)
// Update monthly totals (skip "?" months for expected).
if t := monthlyTotals[m]; t != nil {
if !md.IsUnknown {
t.expected += md.Expected
}
t.paid += paid
}
// Attendance breakdown string e.g. ":3J,2A".
var breakdown string
jc, ac := md.JuniorAttendance, md.AdultAttendance
switch {
case jc > 0 && ac > 0:
breakdown = fmt.Sprintf(":%dJ,%dA", jc, ac)
case jc > 0:
breakdown = fmt.Sprintf(":%dJ", jc)
case ac > 0:
breakdown = fmt.Sprintf(":%dA", ac)
}
countStr := ""
if md.AttendanceCount > 0 {
countStr = fmt.Sprintf(" (%d%s)", md.AttendanceCount, breakdown)
}
// Fee display string.
var feeDisplay string
var isOverridden bool
if md.Exception != nil {
overrideAmount := md.Exception.Amount
var origStr string
if md.IsUnknown {
origStr = "?"
isOverridden = true
} else {
origStr = strconv.Itoa(md.OriginalExpected)
isOverridden = overrideAmount != md.OriginalExpected
}
if isOverridden {
feeDisplay = fmt.Sprintf("%d (%s) CZK%s", overrideAmount, origStr, countStr)
} else {
feeDisplay = fmt.Sprintf("%d CZK%s", md.Expected, countStr)
}
} else {
if md.IsUnknown {
feeDisplay = "? CZK" + countStr
} else {
feeDisplay = fmt.Sprintf("%d CZK%s", md.Expected, countStr)
}
}
status := "empty"
cellText := "-"
amountToPay := 0
switch {
case md.IsUnknown:
cellText = "?" + countStr
case md.Expected > 0:
switch {
case paid >= md.Expected:
status = "ok"
cellText = fmt.Sprintf("%d/%s", paid, feeDisplay)
case paid > 0:
status = "partial"
cellText = fmt.Sprintf("%d/%s", paid, feeDisplay)
amountToPay = md.Expected - paid
if m < currentMonth {
unpaidMonths = append(unpaidMonths, monthLabels[m])
rawUnpaidMonths = append(rawUnpaidMonths, rawMonthLabel(m))
}
default:
status = "unpaid"
cellText = fmt.Sprintf("0/%s", feeDisplay)
amountToPay = md.Expected
if m < currentMonth {
unpaidMonths = append(unpaidMonths, monthLabels[m])
rawUnpaidMonths = append(rawUnpaidMonths, rawMonthLabel(m))
}
}
case paid > 0:
status = "surplus"
cellText = fmt.Sprintf("PAID %d", paid)
}
tooltip := ""
if (!md.IsUnknown && md.Expected > 0) || paid > 0 {
tooltip = fmt.Sprintf("Received: %d, Expected: %d", paid, md.Expected)
}
row.Months = append(row.Months, MonthCell{
Text: cellText,
Overridden: isOverridden,
Status: status,
Amount: amountToPay,
Month: monthLabels[m],
RawMonth: m,
Tooltip: tooltip,
})
}
return row, unpaidMonths, rawUnpaidMonths
}
// juniorExpected converts domain MonthData to the Expected wire type.
// When an exception exists it always produces a concrete int; otherwise
// the "?" sentinel is used when IsUnknown=true.
func juniorExpected(md domreconcile.MonthData) Expected {
if md.Exception == nil && md.IsUnknown {
return Expected{Unknown: true}
}
return Expected{Value: md.Expected}
}
// juniorOriginalExpected converts the original (pre-exception) expected fee.
func juniorOriginalExpected(md domreconcile.MonthData) Expected {
if md.IsUnknown {
return Expected{Unknown: true}
}
return Expected{Value: md.OriginalExpected}
}

View File

@@ -0,0 +1,44 @@
package api
import (
"fuj-management/go/internal/config"
"sort"
"strings"
domreconcile "fuj-management/go/internal/domain/reconcile"
)
// buildPaymentsResponse constructs the PaymentsResponse wire type.
// Mirrors scripts/views.py:build_payments_view_model.
func buildPaymentsResponse(
txns []domreconcile.Transaction,
memberNames []string,
) PaymentsResponse {
grouped := groupRawPaymentsByPerson(txns, memberNames)
// Add unmatched/unknown bucket for transactions with no person set.
const unknownKey = "Unmatched / Unknown"
for _, tx := range txns {
if strings.TrimSpace(tx.Person) == "" {
grouped[unknownKey] = append(grouped[unknownKey], rawTxFromDomain(tx))
}
}
// Sort the unknown bucket newest-first (others are sorted in groupRawPaymentsByPerson).
if rows, ok := grouped[unknownKey]; ok {
sort.Slice(rows, func(i, j int) bool { return rows[i].Date > rows[j].Date })
grouped[unknownKey] = rows
}
sortedPeople := make([]string, 0, len(grouped))
for p := range grouped {
sortedPeople = append(sortedPeople, p)
}
sort.Strings(sortedPeople)
return PaymentsResponse{
GroupedPayments: grouped,
SortedPeople: sortedPeople,
AttendanceURL: "https://docs.google.com/spreadsheets/d/" + config.AttendanceSheetID + "/edit",
PaymentsURL: "https://docs.google.com/spreadsheets/d/" + config.PaymentsSheetID + "/edit",
}
}

View File

@@ -0,0 +1,125 @@
package api
import (
"context"
"encoding/json"
"fmt"
"fuj-management/go/internal/config"
"fuj-management/go/internal/services/membership"
"log/slog"
"net/http"
"time"
domreconcile "fuj-management/go/internal/domain/reconcile"
)
// Handler holds the shared dependencies for all /api/* routes.
type Handler struct {
BuildVersion string
BuildCommit string
BuildDate string
Sources membership.Sources
Config config.Config
Logger *slog.Logger
}
// ServeVersion handles GET /api/version.
func (h *Handler) ServeVersion(w http.ResponseWriter, r *http.Request) {
writeJSON(w, VersionResponse{
Tag: h.BuildVersion,
Commit: h.BuildCommit,
BuildDate: h.BuildDate,
})
}
// ServeAdults handles GET /api/adults.
func (h *Handler) ServeAdults(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
members, sortedMonths, txns, exceptions, err := h.loadAll(ctx, true)
if err != nil {
h.writeError(w, r, err)
return
}
result := domreconcile.Reconcile(members, sortedMonths, txns, exceptions, time.Now().Year())
writeJSON(w, buildAdultsResponse(members, sortedMonths, result, txns, h.Config, time.Now().Format("2006-01")))
}
// ServeJuniors handles GET /api/juniors.
func (h *Handler) ServeJuniors(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
members, sortedMonths, txns, exceptions, err := h.loadAll(ctx, false)
if err != nil {
h.writeError(w, r, err)
return
}
result := domreconcile.Reconcile(members, sortedMonths, txns, exceptions, time.Now().Year())
writeJSON(w, buildJuniorsResponse(members, sortedMonths, result, txns, h.Config, time.Now().Format("2006-01")))
}
// ServePayments handles GET /api/payments.
func (h *Handler) ServePayments(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
txns, err := h.Sources.LoadTransactions(ctx)
if err != nil {
h.writeError(w, r, fmt.Errorf("load transactions: %w", err))
return
}
writeJSON(w, buildPaymentsResponse(txns, h.allMemberNames(ctx)))
}
func (h *Handler) loadAll(ctx context.Context, adults bool) (
members []domreconcile.Member,
sortedMonths []string,
txns []domreconcile.Transaction,
exceptions map[domreconcile.ExceptionKey]domreconcile.Exception,
err error,
) {
if adults {
members, sortedMonths, err = h.Sources.LoadAdults(ctx)
} else {
members, sortedMonths, err = h.Sources.LoadJuniors(ctx)
}
if err != nil {
err = fmt.Errorf("load members: %w", err)
return
}
txns, err = h.Sources.LoadTransactions(ctx)
if err != nil {
err = fmt.Errorf("load transactions: %w", err)
return
}
exceptions, err = h.Sources.LoadExceptions(ctx)
if err != nil {
err = fmt.Errorf("load exceptions: %w", err)
}
return
}
func (h *Handler) allMemberNames(ctx context.Context) []string {
var names []string
if adults, _, err := h.Sources.LoadAdults(ctx); err == nil {
for _, m := range adults {
names = append(names, m.Name)
}
}
if juniors, _, err := h.Sources.LoadJuniors(ctx); err == nil {
for _, m := range juniors {
names = append(names, m.Name)
}
}
return names
}
func (h *Handler) writeError(w http.ResponseWriter, r *http.Request, err error) {
if h.Logger != nil {
h.Logger.Error("api error", "path", r.URL.Path, "err", err)
}
w.Header().Set("Content-Type", "application/json; charset=utf-8")
w.WriteHeader(http.StatusInternalServerError)
_ = json.NewEncoder(w).Encode(map[string]string{"error": err.Error()})
}
func writeJSON(w http.ResponseWriter, v any) {
w.Header().Set("Content-Type", "application/json; charset=utf-8")
_ = json.NewEncoder(w).Encode(v)
}

View File

@@ -0,0 +1,41 @@
package api
// JuniorsMonthData is the reconciled ledger for one junior member in one month.
// expected and original_expected may be the "?" sentinel (single-attendance month
// requiring manual review); they are carried via the Expected type.
type JuniorsMonthData struct {
Expected Expected `json:"expected"`
OriginalExpected Expected `json:"original_expected"`
AttendanceCount int `json:"attendance_count"`
Exception *ExceptionData `json:"exception"`
Paid float64 `json:"paid"`
Transactions []MemberTxEntry `json:"transactions"`
}
// JuniorsMemberData is the reconciled ledger for one junior member.
type JuniorsMemberData struct {
Tier string `json:"tier"`
Months map[string]JuniorsMonthData `json:"months"`
OtherTransactions []MemberOtherEntry `json:"other_transactions"`
TotalBalance int `json:"total_balance"`
}
// JuniorsResponse is the JSON contract for GET /api/juniors.
// Same outer shape as AdultsResponse; differs in that member_data carries
// Expected (int or "?") for expected/original_expected fields.
type JuniorsResponse struct {
Months []string `json:"months"`
RawMonths []string `json:"raw_months"`
Results []MemberRow `json:"results"`
Totals []TotalCell `json:"totals"`
MemberData map[string]JuniorsMemberData `json:"member_data"`
MonthLabels map[string]string `json:"month_labels"`
RawPayments map[string][]RawTransaction `json:"raw_payments"`
Credits []Credit `json:"credits"`
Debts []Credit `json:"debts"`
Unmatched []RawTransaction `json:"unmatched"`
AttendanceURL string `json:"attendance_url"`
PaymentsURL string `json:"payments_url"`
BankAccount string `json:"bank_account"`
CurrentMonth string `json:"current_month"`
}

View File

@@ -0,0 +1,9 @@
package api
// PaymentsResponse is the JSON contract for GET /api/payments.
type PaymentsResponse struct {
GroupedPayments map[string][]RawTransaction `json:"grouped_payments"` // person name → rows
SortedPeople []string `json:"sorted_people"`
AttendanceURL string `json:"attendance_url"`
PaymentsURL string `json:"payments_url"`
}

View File

@@ -0,0 +1,81 @@
package api
// schemagen_test.go generates and golden-compares JSON Schema files for every
// /api/X response type.
//
// Normal run (CI): go test ./internal/web/api/... — asserts schemas match committed files.
// Regenerate: go test -run TestGenerateSchemas -update ./internal/web/api/...
import (
"encoding/json"
"flag"
"os"
"path/filepath"
"testing"
"github.com/invopop/jsonschema"
)
var updateFlag = flag.Bool("update", false, "overwrite api-schema fixture files with freshly generated schemas")
// JSONSchema makes Expected self-describing for the reflector at test time.
// The method is in a test file and is not compiled into production binaries.
// It emits oneOf [integer, "?"] to match the custom MarshalJSON behaviour.
func (Expected) JSONSchema() *jsonschema.Schema {
return &jsonschema.Schema{
OneOf: []*jsonschema.Schema{
{Type: "integer"},
{Enum: []any{"?"}},
},
}
}
func TestGenerateSchemas(t *testing.T) {
r := &jsonschema.Reflector{
AllowAdditionalProperties: false,
}
cases := []struct {
name string
val any
}{
{"adults", &AdultsResponse{}},
{"juniors", &JuniorsResponse{}},
{"payments", &PaymentsResponse{}},
{"version", &VersionResponse{}},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
schema := r.Reflect(tc.val)
got, err := json.MarshalIndent(schema, "", " ")
if err != nil {
t.Fatalf("marshal schema: %v", err)
}
got = append(got, '\n')
// Path: go/internal/web/api/ → ../../.. → go/ → tests/fixtures/api-schema/
path := filepath.Join("..", "..", "..", "tests", "fixtures", "api-schema", tc.name+".schema.json")
if *updateFlag {
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
t.Fatalf("mkdir: %v", err)
}
if err := os.WriteFile(path, got, 0o644); err != nil {
t.Fatalf("write schema: %v", err)
}
t.Logf("wrote %s", path)
return
}
want, err := os.ReadFile(path)
if err != nil {
t.Fatalf("read fixture %s: %v (re-run with -update to generate)", path, err)
}
if string(got) != string(want) {
t.Errorf("schema mismatch for %s; re-run with -update to regenerate", tc.name)
}
})
}
}

View File

@@ -0,0 +1,122 @@
// Package api defines wire types for the JSON API contract (/api/...).
// These structs have explicit json: tags matching the Python view-model dict
// keys so that M5 parity tests can do byte-equal comparison between backends.
//
// The three Python template-only JSON-string fields (member_data,
// month_labels_json, raw_payments_json) are represented here as nested objects;
// the Python /api/X shadow endpoint strips the json.dumps wrappers before
// serialising.
package api
import (
"encoding/json"
"fmt"
)
// Expected holds a junior fee expectation: either a concrete integer or the
// "?" sentinel (single-attendance month requiring manual review).
// MarshalJSON emits the integer or the JSON string "?".
type Expected struct {
Value int
Unknown bool
}
func (e Expected) MarshalJSON() ([]byte, error) {
if e.Unknown {
return []byte(`"?"`), nil
}
return json.Marshal(e.Value)
}
func (e *Expected) UnmarshalJSON(data []byte) error {
if len(data) > 0 && data[0] == '"' {
var s string
if err := json.Unmarshal(data, &s); err != nil {
return err
}
if s == "?" {
e.Unknown = true
return nil
}
return fmt.Errorf("api.Expected: unexpected string %q", s)
}
e.Unknown = false
return json.Unmarshal(data, &e.Value)
}
// ExceptionData is a manual fee override for one member in one month.
type ExceptionData struct {
Amount int `json:"amount"`
Note string `json:"note"`
}
// MemberTxEntry is one payment allocation to a member+month, as stored in
// member_data.months[YYYY-MM].transactions.
type MemberTxEntry struct {
Amount float64 `json:"amount"`
Date string `json:"date"`
Sender string `json:"sender"`
Message string `json:"message"`
Confidence string `json:"confidence"`
}
// MemberOtherEntry is an "other:…" purpose payment allocated to a member.
type MemberOtherEntry struct {
Amount float64 `json:"amount"`
Date string `json:"date"`
Sender string `json:"sender"`
Message string `json:"message"`
Purpose string `json:"purpose"`
Confidence string `json:"confidence"`
}
// RawTransaction is a full payments-sheet row.
// Used for unmatched transactions and raw_payments groupings.
// Columns match the sheet layout: Date|Amount|manual fix|Person|Purpose|
// Inferred Amount|Sender|VS|Message|Bank ID|Sync ID.
type RawTransaction struct {
Date string `json:"date"`
Amount float64 `json:"amount"`
ManualFix string `json:"manual_fix"`
Person string `json:"person"`
Purpose string `json:"purpose"`
InferredAmount float64 `json:"inferred_amount"`
Sender string `json:"sender"`
VS string `json:"vs"`
Message string `json:"message"`
BankID string `json:"bank_id"`
SyncID string `json:"sync_id"`
}
// MonthCell is one cell in a member's month column on the dashboard.
type MonthCell struct {
Text string `json:"text"`
Overridden bool `json:"overridden"`
Status string `json:"status"` // "empty"|"ok"|"partial"|"unpaid"|"surplus"
Amount int `json:"amount"`
Month string `json:"month"` // display label, e.g. "Apr+May 2025"
RawMonth string `json:"raw_month"` // "YYYY-MM"
Tooltip string `json:"tooltip"`
}
// TotalCell is one cell in the monthly totals row.
type TotalCell struct {
Text string `json:"text"`
Status string `json:"status"`
}
// MemberRow is one member's summary row in the dashboard results table.
type MemberRow struct {
Name string `json:"name"`
Months []MonthCell `json:"months"`
Balance int `json:"balance"`
UnpaidPeriods string `json:"unpaid_periods"`
RawUnpaidPeriods string `json:"raw_unpaid_periods"`
PayableAmount int `json:"payable_amount"`
}
// Credit is one entry in the credits or debts lists.
type Credit struct {
Name string `json:"name"`
Amount int `json:"amount"`
}

View File

@@ -0,0 +1,9 @@
package api
// VersionResponse is the JSON contract for GET /api/version.
// Keys match Python's BUILD_META dict (see app.py).
type VersionResponse struct {
Tag string `json:"tag"`
Commit string `json:"commit"`
BuildDate string `json:"build_date"`
}

View File

@@ -2,6 +2,9 @@ package web
import (
"fmt"
"fuj-management/go/internal/config"
"fuj-management/go/internal/services/membership"
"fuj-management/go/internal/web/api"
"fuj-management/go/internal/web/middleware"
"log/slog"
"net/http"
@@ -15,9 +18,22 @@ type BuildInfo struct {
}
// Run registers routes and starts the HTTP server on addr.
func Run(logger *slog.Logger, addr string, build BuildInfo) error {
func Run(logger *slog.Logger, addr string, build BuildInfo, sources membership.Sources, cfg config.Config) error {
h := &api.Handler{
BuildVersion: build.Version,
BuildCommit: build.Commit,
BuildDate: build.BuildDate,
Sources: sources,
Config: cfg,
Logger: logger,
}
mux := http.NewServeMux()
mux.HandleFunc("GET /{$}", helloHandler(build))
mux.HandleFunc("GET /api/version", h.ServeVersion)
mux.HandleFunc("GET /api/adults", h.ServeAdults)
mux.HandleFunc("GET /api/juniors", h.ServeJuniors)
mux.HandleFunc("GET /api/payments", h.ServePayments)
logger.Info("starting server", "addr", addr)
return http.ListenAndServe(addr, middleware.RequestTimer(logger, mux))

View File

@@ -0,0 +1,399 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$ref": "#/$defs/AdultsResponse",
"$defs": {
"AdultsMemberData": {
"properties": {
"tier": {
"type": "string"
},
"months": {
"additionalProperties": {
"$ref": "#/$defs/AdultsMonthData"
},
"type": "object"
},
"other_transactions": {
"items": {
"$ref": "#/$defs/MemberOtherEntry"
},
"type": "array"
},
"total_balance": {
"type": "integer"
}
},
"additionalProperties": false,
"type": "object",
"required": [
"tier",
"months",
"other_transactions",
"total_balance"
]
},
"AdultsMonthData": {
"properties": {
"expected": {
"type": "integer"
},
"original_expected": {
"type": "integer"
},
"attendance_count": {
"type": "integer"
},
"exception": {
"$ref": "#/$defs/ExceptionData"
},
"paid": {
"type": "number"
},
"transactions": {
"items": {
"$ref": "#/$defs/MemberTxEntry"
},
"type": "array"
}
},
"additionalProperties": false,
"type": "object",
"required": [
"expected",
"original_expected",
"attendance_count",
"exception",
"paid",
"transactions"
]
},
"AdultsResponse": {
"properties": {
"months": {
"items": {
"type": "string"
},
"type": "array"
},
"raw_months": {
"items": {
"type": "string"
},
"type": "array"
},
"results": {
"items": {
"$ref": "#/$defs/MemberRow"
},
"type": "array"
},
"totals": {
"items": {
"$ref": "#/$defs/TotalCell"
},
"type": "array"
},
"member_data": {
"additionalProperties": {
"$ref": "#/$defs/AdultsMemberData"
},
"type": "object"
},
"month_labels": {
"additionalProperties": {
"type": "string"
},
"type": "object"
},
"raw_payments": {
"additionalProperties": {
"items": {
"$ref": "#/$defs/RawTransaction"
},
"type": "array"
},
"type": "object"
},
"credits": {
"items": {
"$ref": "#/$defs/Credit"
},
"type": "array"
},
"debts": {
"items": {
"$ref": "#/$defs/Credit"
},
"type": "array"
},
"unmatched": {
"items": {
"$ref": "#/$defs/RawTransaction"
},
"type": "array"
},
"attendance_url": {
"type": "string"
},
"payments_url": {
"type": "string"
},
"bank_account": {
"type": "string"
},
"current_month": {
"type": "string"
}
},
"additionalProperties": false,
"type": "object",
"required": [
"months",
"raw_months",
"results",
"totals",
"member_data",
"month_labels",
"raw_payments",
"credits",
"debts",
"unmatched",
"attendance_url",
"payments_url",
"bank_account",
"current_month"
]
},
"Credit": {
"properties": {
"name": {
"type": "string"
},
"amount": {
"type": "integer"
}
},
"additionalProperties": false,
"type": "object",
"required": [
"name",
"amount"
]
},
"ExceptionData": {
"properties": {
"amount": {
"type": "integer"
},
"note": {
"type": "string"
}
},
"additionalProperties": false,
"type": "object",
"required": [
"amount",
"note"
]
},
"MemberOtherEntry": {
"properties": {
"amount": {
"type": "number"
},
"date": {
"type": "string"
},
"sender": {
"type": "string"
},
"message": {
"type": "string"
},
"purpose": {
"type": "string"
},
"confidence": {
"type": "string"
}
},
"additionalProperties": false,
"type": "object",
"required": [
"amount",
"date",
"sender",
"message",
"purpose",
"confidence"
]
},
"MemberRow": {
"properties": {
"name": {
"type": "string"
},
"months": {
"items": {
"$ref": "#/$defs/MonthCell"
},
"type": "array"
},
"balance": {
"type": "integer"
},
"unpaid_periods": {
"type": "string"
},
"raw_unpaid_periods": {
"type": "string"
},
"payable_amount": {
"type": "integer"
}
},
"additionalProperties": false,
"type": "object",
"required": [
"name",
"months",
"balance",
"unpaid_periods",
"raw_unpaid_periods",
"payable_amount"
]
},
"MemberTxEntry": {
"properties": {
"amount": {
"type": "number"
},
"date": {
"type": "string"
},
"sender": {
"type": "string"
},
"message": {
"type": "string"
},
"confidence": {
"type": "string"
}
},
"additionalProperties": false,
"type": "object",
"required": [
"amount",
"date",
"sender",
"message",
"confidence"
]
},
"MonthCell": {
"properties": {
"text": {
"type": "string"
},
"overridden": {
"type": "boolean"
},
"status": {
"type": "string"
},
"amount": {
"type": "integer"
},
"month": {
"type": "string"
},
"raw_month": {
"type": "string"
},
"tooltip": {
"type": "string"
}
},
"additionalProperties": false,
"type": "object",
"required": [
"text",
"overridden",
"status",
"amount",
"month",
"raw_month",
"tooltip"
]
},
"RawTransaction": {
"properties": {
"date": {
"type": "string"
},
"amount": {
"type": "number"
},
"manual_fix": {
"type": "string"
},
"person": {
"type": "string"
},
"purpose": {
"type": "string"
},
"inferred_amount": {
"type": "number"
},
"sender": {
"type": "string"
},
"vs": {
"type": "string"
},
"message": {
"type": "string"
},
"bank_id": {
"type": "string"
},
"sync_id": {
"type": "string"
}
},
"additionalProperties": false,
"type": "object",
"required": [
"date",
"amount",
"manual_fix",
"person",
"purpose",
"inferred_amount",
"sender",
"vs",
"message",
"bank_id",
"sync_id"
]
},
"TotalCell": {
"properties": {
"text": {
"type": "string"
},
"status": {
"type": "string"
}
},
"additionalProperties": false,
"type": "object",
"required": [
"text",
"status"
]
}
}
}

View File

@@ -0,0 +1,411 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$ref": "#/$defs/JuniorsResponse",
"$defs": {
"Credit": {
"properties": {
"name": {
"type": "string"
},
"amount": {
"type": "integer"
}
},
"additionalProperties": false,
"type": "object",
"required": [
"name",
"amount"
]
},
"ExceptionData": {
"properties": {
"amount": {
"type": "integer"
},
"note": {
"type": "string"
}
},
"additionalProperties": false,
"type": "object",
"required": [
"amount",
"note"
]
},
"Expected": {
"oneOf": [
{
"type": "integer"
},
{
"enum": [
"?"
]
}
]
},
"JuniorsMemberData": {
"properties": {
"tier": {
"type": "string"
},
"months": {
"additionalProperties": {
"$ref": "#/$defs/JuniorsMonthData"
},
"type": "object"
},
"other_transactions": {
"items": {
"$ref": "#/$defs/MemberOtherEntry"
},
"type": "array"
},
"total_balance": {
"type": "integer"
}
},
"additionalProperties": false,
"type": "object",
"required": [
"tier",
"months",
"other_transactions",
"total_balance"
]
},
"JuniorsMonthData": {
"properties": {
"expected": {
"$ref": "#/$defs/Expected"
},
"original_expected": {
"$ref": "#/$defs/Expected"
},
"attendance_count": {
"type": "integer"
},
"exception": {
"$ref": "#/$defs/ExceptionData"
},
"paid": {
"type": "number"
},
"transactions": {
"items": {
"$ref": "#/$defs/MemberTxEntry"
},
"type": "array"
}
},
"additionalProperties": false,
"type": "object",
"required": [
"expected",
"original_expected",
"attendance_count",
"exception",
"paid",
"transactions"
]
},
"JuniorsResponse": {
"properties": {
"months": {
"items": {
"type": "string"
},
"type": "array"
},
"raw_months": {
"items": {
"type": "string"
},
"type": "array"
},
"results": {
"items": {
"$ref": "#/$defs/MemberRow"
},
"type": "array"
},
"totals": {
"items": {
"$ref": "#/$defs/TotalCell"
},
"type": "array"
},
"member_data": {
"additionalProperties": {
"$ref": "#/$defs/JuniorsMemberData"
},
"type": "object"
},
"month_labels": {
"additionalProperties": {
"type": "string"
},
"type": "object"
},
"raw_payments": {
"additionalProperties": {
"items": {
"$ref": "#/$defs/RawTransaction"
},
"type": "array"
},
"type": "object"
},
"credits": {
"items": {
"$ref": "#/$defs/Credit"
},
"type": "array"
},
"debts": {
"items": {
"$ref": "#/$defs/Credit"
},
"type": "array"
},
"unmatched": {
"items": {
"$ref": "#/$defs/RawTransaction"
},
"type": "array"
},
"attendance_url": {
"type": "string"
},
"payments_url": {
"type": "string"
},
"bank_account": {
"type": "string"
},
"current_month": {
"type": "string"
}
},
"additionalProperties": false,
"type": "object",
"required": [
"months",
"raw_months",
"results",
"totals",
"member_data",
"month_labels",
"raw_payments",
"credits",
"debts",
"unmatched",
"attendance_url",
"payments_url",
"bank_account",
"current_month"
]
},
"MemberOtherEntry": {
"properties": {
"amount": {
"type": "number"
},
"date": {
"type": "string"
},
"sender": {
"type": "string"
},
"message": {
"type": "string"
},
"purpose": {
"type": "string"
},
"confidence": {
"type": "string"
}
},
"additionalProperties": false,
"type": "object",
"required": [
"amount",
"date",
"sender",
"message",
"purpose",
"confidence"
]
},
"MemberRow": {
"properties": {
"name": {
"type": "string"
},
"months": {
"items": {
"$ref": "#/$defs/MonthCell"
},
"type": "array"
},
"balance": {
"type": "integer"
},
"unpaid_periods": {
"type": "string"
},
"raw_unpaid_periods": {
"type": "string"
},
"payable_amount": {
"type": "integer"
}
},
"additionalProperties": false,
"type": "object",
"required": [
"name",
"months",
"balance",
"unpaid_periods",
"raw_unpaid_periods",
"payable_amount"
]
},
"MemberTxEntry": {
"properties": {
"amount": {
"type": "number"
},
"date": {
"type": "string"
},
"sender": {
"type": "string"
},
"message": {
"type": "string"
},
"confidence": {
"type": "string"
}
},
"additionalProperties": false,
"type": "object",
"required": [
"amount",
"date",
"sender",
"message",
"confidence"
]
},
"MonthCell": {
"properties": {
"text": {
"type": "string"
},
"overridden": {
"type": "boolean"
},
"status": {
"type": "string"
},
"amount": {
"type": "integer"
},
"month": {
"type": "string"
},
"raw_month": {
"type": "string"
},
"tooltip": {
"type": "string"
}
},
"additionalProperties": false,
"type": "object",
"required": [
"text",
"overridden",
"status",
"amount",
"month",
"raw_month",
"tooltip"
]
},
"RawTransaction": {
"properties": {
"date": {
"type": "string"
},
"amount": {
"type": "number"
},
"manual_fix": {
"type": "string"
},
"person": {
"type": "string"
},
"purpose": {
"type": "string"
},
"inferred_amount": {
"type": "number"
},
"sender": {
"type": "string"
},
"vs": {
"type": "string"
},
"message": {
"type": "string"
},
"bank_id": {
"type": "string"
},
"sync_id": {
"type": "string"
}
},
"additionalProperties": false,
"type": "object",
"required": [
"date",
"amount",
"manual_fix",
"person",
"purpose",
"inferred_amount",
"sender",
"vs",
"message",
"bank_id",
"sync_id"
]
},
"TotalCell": {
"properties": {
"text": {
"type": "string"
},
"status": {
"type": "string"
}
},
"additionalProperties": false,
"type": "object",
"required": [
"text",
"status"
]
}
}
}

View File

@@ -0,0 +1,91 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$ref": "#/$defs/PaymentsResponse",
"$defs": {
"PaymentsResponse": {
"properties": {
"grouped_payments": {
"additionalProperties": {
"items": {
"$ref": "#/$defs/RawTransaction"
},
"type": "array"
},
"type": "object"
},
"sorted_people": {
"items": {
"type": "string"
},
"type": "array"
},
"attendance_url": {
"type": "string"
},
"payments_url": {
"type": "string"
}
},
"additionalProperties": false,
"type": "object",
"required": [
"grouped_payments",
"sorted_people",
"attendance_url",
"payments_url"
]
},
"RawTransaction": {
"properties": {
"date": {
"type": "string"
},
"amount": {
"type": "number"
},
"manual_fix": {
"type": "string"
},
"person": {
"type": "string"
},
"purpose": {
"type": "string"
},
"inferred_amount": {
"type": "number"
},
"sender": {
"type": "string"
},
"vs": {
"type": "string"
},
"message": {
"type": "string"
},
"bank_id": {
"type": "string"
},
"sync_id": {
"type": "string"
}
},
"additionalProperties": false,
"type": "object",
"required": [
"date",
"amount",
"manual_fix",
"person",
"purpose",
"inferred_amount",
"sender",
"vs",
"message",
"bank_id",
"sync_id"
]
}
}
}

View File

@@ -0,0 +1,26 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$ref": "#/$defs/VersionResponse",
"$defs": {
"VersionResponse": {
"properties": {
"tag": {
"type": "string"
},
"commit": {
"type": "string"
},
"build_date": {
"type": "string"
}
},
"additionalProperties": false,
"type": "object",
"required": [
"tag",
"commit",
"build_date"
]
}
}
}

View File

@@ -3,11 +3,12 @@
package build_name_variants_parity_test
import (
"fuj-management/go/internal/domain/matching"
"fuj-management/go/tests/parity"
"reflect"
"sort"
"testing"
"fuj-management/go/internal/domain/matching"
"fuj-management/go/tests/parity"
)
// Verify expected values against live Python:

View File

@@ -3,10 +3,11 @@
package calculate_fee_parity_test
import (
"fuj-management/go/internal/domain/fees"
"fuj-management/go/tests/parity"
"reflect"
"testing"
"fuj-management/go/internal/domain/fees"
"fuj-management/go/tests/parity"
)
// Verify expected values against live Python:

View File

@@ -3,10 +3,11 @@
package calculate_junior_fee_parity_test
import (
"fuj-management/go/internal/domain/fees"
"fuj-management/go/tests/parity"
"reflect"
"testing"
"fuj-management/go/internal/domain/fees"
"fuj-management/go/tests/parity"
)
// Verify expected values against live Python:

View File

@@ -3,10 +3,11 @@
package format_date_parity_test
import (
"fuj-management/go/internal/domain/matching"
"fuj-management/go/tests/parity"
"reflect"
"testing"
"fuj-management/go/internal/domain/matching"
"fuj-management/go/tests/parity"
)
// Verify expected values against live Python:

View File

@@ -3,10 +3,11 @@
package generate_sync_id_parity_test
import (
"fuj-management/go/internal/domain/synch"
"fuj-management/go/tests/parity"
"reflect"
"testing"
"fuj-management/go/internal/domain/synch"
"fuj-management/go/tests/parity"
)
// Verify expected values against live Python:

View File

@@ -3,10 +3,11 @@
package infer_transaction_details_parity_test
import (
"fuj-management/go/internal/domain/matching"
"fuj-management/go/tests/parity"
"reflect"
"testing"
"fuj-management/go/internal/domain/matching"
"fuj-management/go/tests/parity"
)
// Verify expected values against live Python:

View File

@@ -3,10 +3,11 @@
package match_members_parity_test
import (
"fuj-management/go/internal/domain/matching"
"fuj-management/go/tests/parity"
"reflect"
"testing"
"fuj-management/go/internal/domain/matching"
"fuj-management/go/tests/parity"
)
// Verify expected values against live Python:

View File

@@ -3,10 +3,11 @@
package normalize_parity_test
import (
"fuj-management/go/internal/domain/czech"
"fuj-management/go/tests/parity"
"reflect"
"testing"
"fuj-management/go/internal/domain/czech"
"fuj-management/go/tests/parity"
)
// Verify expected values against live Python:

View File

@@ -3,10 +3,11 @@
package parse_czk_amount_parity_test
import (
"fuj-management/go/internal/domain/money"
"fuj-management/go/tests/parity"
"math"
"testing"
"fuj-management/go/internal/domain/money"
"fuj-management/go/tests/parity"
)
// Verify expected values against live Python:

View File

@@ -3,11 +3,12 @@
package parse_month_references_parity_test
import (
"fuj-management/go/internal/domain/czech"
"fuj-management/go/tests/parity"
"reflect"
"sort"
"testing"
"fuj-management/go/internal/domain/czech"
"fuj-management/go/tests/parity"
)
// Verify expected values against live Python:

View File

@@ -15,12 +15,13 @@ package reconcile_parity_test
import (
"encoding/json"
"fmt"
"fuj-management/go/internal/domain/czech"
"fuj-management/go/internal/domain/reconcile"
"math"
"os"
"path/filepath"
"testing"
"fuj-management/go/internal/domain/czech"
"fuj-management/go/internal/domain/reconcile"
)
// ---------------------------------------------------------------------------

445
scripts/views.py Normal file
View File

@@ -0,0 +1,445 @@
import json
import re
from datetime import datetime
from attendance import ADULT_MERGED_MONTHS, JUNIOR_MERGED_MONTHS
from match_payments import canonical_member_key
def get_month_labels(sorted_months, merged_months):
labels = {}
for m in sorted_months:
dt = datetime.strptime(m, "%Y-%m")
merged_in = sorted([k for k, v in merged_months.items() if v == m])
if merged_in:
all_dts = [datetime.strptime(x, "%Y-%m") for x in sorted(merged_in + [m])]
years = {d.year for d in all_dts}
if len(years) > 1:
parts = [d.strftime("%b %Y") for d in all_dts]
labels[m] = "+".join(parts)
else:
parts = [d.strftime("%b") for d in all_dts]
labels[m] = f"{'+'.join(parts)} {dt.strftime('%Y')}"
else:
labels[m] = dt.strftime("%b %Y")
return labels
def group_payments_by_person(transactions, member_names=None):
canonical_by_key = (
{canonical_member_key(n): n for n in member_names} if member_names else {}
)
grouped = {}
for tx in transactions:
person = str(tx.get("person", "")).strip()
if not person:
continue
for p in person.split(","):
p = re.sub(r"\[\?\]\s*", "", p).strip()
if not p:
continue
key = canonical_by_key.get(canonical_member_key(p), p)
grouped.setdefault(key, []).append(tx)
for rows in grouped.values():
rows.sort(key=lambda t: str(t.get("date", "")), reverse=True)
return grouped
def adapt_junior_members(junior_members):
"""Convert 4-tuple junior fee data to (fee, total_count) for reconcile."""
adapted = []
for name, tier, fees_dict in junior_members:
adapted_fees = {}
for m, fee_data in fees_dict.items():
if len(fee_data) == 4:
fee, total_count, _, _ = fee_data
adapted_fees[m] = (fee, total_count)
else:
fee, count = fee_data
adapted_fees[m] = (fee, count)
adapted.append((name, tier, adapted_fees))
return adapted
def build_adults_view_model(
members,
sorted_months,
result,
transactions,
current_month,
*,
attendance_url,
payments_url,
bank_account,
):
month_labels = get_month_labels(sorted_months, ADULT_MERGED_MONTHS)
adult_names = sorted([name for name, tier, _ in members if tier == "A"])
monthly_totals = {m: {"expected": 0, "paid": 0} for m in sorted_months}
formatted_results = []
for name in adult_names:
data = result["members"][name]
row = {
"name": name,
"months": [],
"balance": data["total_balance"],
"unpaid_periods": "",
"raw_unpaid_periods": "",
}
unpaid_months = []
raw_unpaid_months = []
for m in sorted_months:
mdata = data["months"].get(m, {
"expected": 0, "original_expected": 0,
"attendance_count": 0, "paid": 0, "exception": None,
})
expected = mdata.get("expected", 0)
original_expected = mdata.get("original_expected", 0)
count = mdata.get("attendance_count", 0)
paid = int(mdata.get("paid", 0))
exception_info = mdata.get("exception", None)
monthly_totals[m]["expected"] += expected
monthly_totals[m]["paid"] += paid
override_amount = exception_info["amount"] if exception_info else None
if override_amount is not None and override_amount != original_expected:
is_overridden = True
fee_display = (
f"{override_amount} ({original_expected}) CZK ({count})"
if count > 0
else f"{override_amount} ({original_expected}) CZK"
)
else:
is_overridden = False
fee_display = (
f"{expected} CZK ({count})" if count > 0 else f"{expected} CZK"
)
status = "empty"
cell_text = "-"
amount_to_pay = 0
if expected > 0:
amount_to_pay = max(0, expected - paid)
if paid >= expected:
status = "ok"
cell_text = f"{paid}/{fee_display}"
elif paid > 0:
status = "partial"
cell_text = f"{paid}/{fee_display}"
if m < current_month:
unpaid_months.append(month_labels[m])
raw_unpaid_months.append(
datetime.strptime(m, "%Y-%m").strftime("%m/%Y")
)
else:
status = "unpaid"
cell_text = f"0/{fee_display}"
if m < current_month:
unpaid_months.append(month_labels[m])
raw_unpaid_months.append(
datetime.strptime(m, "%Y-%m").strftime("%m/%Y")
)
elif paid > 0:
status = "surplus"
cell_text = f"PAID {paid}"
else:
cell_text = "-"
amount_to_pay = 0
if expected > 0 or paid > 0:
tooltip = f"Received: {paid}, Expected: {expected}"
else:
tooltip = ""
row["months"].append({
"text": cell_text,
"overridden": is_overridden,
"status": status,
"amount": amount_to_pay,
"month": month_labels[m],
"raw_month": m,
"tooltip": tooltip,
})
settled_balance = 0
for m, mdata in data["months"].items():
if m >= current_month:
continue
exp = mdata.get("expected", 0)
if isinstance(exp, int):
settled_balance += int(mdata.get("paid", 0)) - exp
payable_amount = max(0, -settled_balance)
row["unpaid_periods"] = ", ".join(unpaid_months)
row["raw_unpaid_periods"] = "+".join(raw_unpaid_months)
row["balance"] = settled_balance
row["payable_amount"] = payable_amount
formatted_results.append(row)
formatted_totals = []
for m in sorted_months:
t = monthly_totals[m]
status = "empty"
if t["expected"] > 0 or t["paid"] > 0:
if t["paid"] == t["expected"]:
status = "ok"
elif t["paid"] < t["expected"]:
status = "unpaid"
else:
status = "surplus"
formatted_totals.append({
"text": f"{t['paid']} / {t['expected']} CZK",
"status": status,
})
def _settled_balance(name):
data = result["members"][name]
total = 0
for m, mdata in data["months"].items():
if m >= current_month:
continue
exp = mdata.get("expected", 0)
if isinstance(exp, int):
total += int(mdata.get("paid", 0)) - exp
return total
credits = sorted(
[{"name": n, "amount": _settled_balance(n)} for n in adult_names if _settled_balance(n) > 0],
key=lambda x: x["name"],
)
debts = sorted(
[{"name": n, "amount": abs(_settled_balance(n))} for n in adult_names if _settled_balance(n) < 0],
key=lambda x: x["name"],
)
raw_payments_by_person = group_payments_by_person(
transactions, [name for name, _, _ in members]
)
return dict(
months=[month_labels[m] for m in sorted_months],
raw_months=sorted_months,
results=formatted_results,
totals=formatted_totals,
member_data=json.dumps(result["members"]),
month_labels_json=json.dumps(month_labels),
raw_payments_json=json.dumps(raw_payments_by_person),
credits=credits,
debts=debts,
unmatched=result["unmatched"],
attendance_url=attendance_url,
payments_url=payments_url,
bank_account=bank_account,
current_month=current_month,
)
def build_juniors_view_model(
junior_members,
adapted_members,
sorted_months,
result,
transactions,
current_month,
*,
attendance_url,
payments_url,
bank_account,
):
month_labels = get_month_labels(sorted_months, JUNIOR_MERGED_MONTHS)
junior_names = sorted([name for name, tier, _ in adapted_members])
junior_members_dict = {name: fees_dict for name, _, fees_dict in junior_members}
monthly_totals = {m: {"expected": 0, "paid": 0} for m in sorted_months}
formatted_results = []
for name in junior_names:
data = result["members"][name]
row = {
"name": name,
"months": [],
"balance": data["total_balance"],
"unpaid_periods": "",
"raw_unpaid_periods": "",
}
unpaid_months = []
raw_unpaid_months = []
for m in sorted_months:
mdata = data["months"].get(m, {
"expected": 0, "original_expected": 0,
"attendance_count": 0, "paid": 0, "exception": None,
})
expected = mdata.get("expected", 0)
original_expected = mdata.get("original_expected", 0)
count = mdata.get("attendance_count", 0)
paid = int(mdata.get("paid", 0))
exception_info = mdata.get("exception", None)
if expected != "?" and isinstance(expected, int):
monthly_totals[m]["expected"] += expected
monthly_totals[m]["paid"] += paid
orig_fee_data = junior_members_dict.get(name, {}).get(m)
adult_count = 0
junior_count = 0
if orig_fee_data and len(orig_fee_data) == 4:
_, _, adult_count, junior_count = orig_fee_data
breakdown = ""
if adult_count > 0 and junior_count > 0:
breakdown = f":{junior_count}J,{adult_count}A"
elif junior_count > 0:
breakdown = f":{junior_count}J"
elif adult_count > 0:
breakdown = f":{adult_count}A"
count_str = f" ({count}{breakdown})" if count > 0 else ""
override_amount = exception_info["amount"] if exception_info else None
if override_amount is not None and override_amount != original_expected:
is_overridden = True
fee_display = f"{override_amount} ({original_expected}) CZK{count_str}"
else:
is_overridden = False
fee_display = f"{expected} CZK{count_str}"
status = "empty"
cell_text = "-"
amount_to_pay = 0
if expected == "?" or (isinstance(expected, int) and expected > 0):
if expected == "?":
status = "empty"
cell_text = f"?{count_str}"
elif paid >= expected:
status = "ok"
cell_text = f"{paid}/{fee_display}"
elif paid > 0:
status = "partial"
cell_text = f"{paid}/{fee_display}"
amount_to_pay = expected - paid
if m < current_month:
unpaid_months.append(month_labels[m])
raw_unpaid_months.append(
datetime.strptime(m, "%Y-%m").strftime("%m/%Y")
)
else:
status = "unpaid"
cell_text = f"0/{fee_display}"
amount_to_pay = expected
if m < current_month:
unpaid_months.append(month_labels[m])
raw_unpaid_months.append(
datetime.strptime(m, "%Y-%m").strftime("%m/%Y")
)
elif paid > 0:
status = "surplus"
cell_text = f"PAID {paid}"
if (isinstance(expected, int) and expected > 0) or paid > 0:
tooltip = f"Received: {paid}, Expected: {expected}"
else:
tooltip = ""
row["months"].append({
"text": cell_text,
"overridden": is_overridden,
"status": status,
"amount": amount_to_pay,
"month": month_labels[m],
"raw_month": m,
"tooltip": tooltip,
})
settled_balance = 0
for m, mdata in data["months"].items():
if m >= current_month:
continue
exp = mdata.get("expected", 0)
if isinstance(exp, int):
settled_balance += int(mdata.get("paid", 0)) - exp
payable_amount = max(0, -settled_balance)
row["unpaid_periods"] = ", ".join(unpaid_months)
row["raw_unpaid_periods"] = "+".join(raw_unpaid_months)
row["balance"] = settled_balance
row["payable_amount"] = payable_amount
formatted_results.append(row)
formatted_totals = []
for m in sorted_months:
t = monthly_totals[m]
status = "empty"
if t["expected"] > 0 or t["paid"] > 0:
if t["paid"] == t["expected"]:
status = "ok"
elif t["paid"] < t["expected"]:
status = "unpaid"
else:
status = "surplus"
formatted_totals.append({
"text": f"{t['paid']} / {t['expected']} CZK",
"status": status,
})
junior_all_names = [name for name, _, _ in adapted_members]
def _junior_settled_balance(name):
data = result["members"][name]
total = 0
for m, mdata in data["months"].items():
if m >= current_month:
continue
exp = mdata.get("expected", 0)
if isinstance(exp, int):
total += int(mdata.get("paid", 0)) - exp
return total
credits = sorted(
[{"name": n, "amount": _junior_settled_balance(n)} for n in junior_all_names if _junior_settled_balance(n) > 0],
key=lambda x: x["name"],
)
debts = sorted(
[{"name": n, "amount": abs(_junior_settled_balance(n))} for n in junior_all_names if _junior_settled_balance(n) < 0],
key=lambda x: x["name"],
)
raw_payments_by_person = group_payments_by_person(
transactions, [name for name, _, _ in adapted_members]
)
return dict(
months=[month_labels[m] for m in sorted_months],
raw_months=sorted_months,
results=formatted_results,
totals=formatted_totals,
member_data=json.dumps(result["members"]),
month_labels_json=json.dumps(month_labels),
raw_payments_json=json.dumps(raw_payments_by_person),
credits=credits,
debts=debts,
unmatched=result["unmatched"],
attendance_url=attendance_url,
payments_url=payments_url,
bank_account=bank_account,
current_month=current_month,
)
def build_payments_view_model(transactions, member_names, *, attendance_url, payments_url):
grouped = group_payments_by_person(transactions, member_names)
for tx in transactions:
if not str(tx.get("person", "")).strip():
grouped.setdefault("Unmatched / Unknown", []).append(tx)
for rows in grouped.values():
rows.sort(key=lambda t: str(t.get("date", "")), reverse=True)
sorted_people = sorted(grouped.keys())
return dict(
grouped_payments=grouped,
sorted_people=sorted_people,
attendance_url=attendance_url,
payments_url=payments_url,
)