Compare commits

..

10 Commits

Author SHA1 Message Date
07ca1cd9e1 fix(py): coerce VS column to string in payments tx projection
All checks were successful
Deploy to K8s / deploy (push) Successful in 7s
The Sheets API returns VS (variabilní symbol) cells as float64 when
the column is number-formatted, so Python was emitting vs: 0 (a JSON
number) while Go's getVal uses fmt.Sprint and always emits vs: "0"
(a string). Add get_str helper that converts whole-number floats via
int() first (matching Go's %g formatting), applied to the vs field.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-07 23:59:35 +02:00
5dcac25c13 chore: CHANGELOG entry for M5.4 fix #2 (py vs/sync_id)
All checks were successful
Deploy to K8s / deploy (push) Successful in 6s
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-07 23:52:04 +02:00
fc47606b1c Merge pull request 'feat(py): M5.4 fix #2 — add vs and sync_id to payments tx projection' (#23) from fix/py-payments-add-vs-syncid into main
All checks were successful
Deploy to K8s / deploy (push) Successful in 9s
Reviewed-on: #23
2026-05-07 21:51:12 +00:00
65694ad378 feat(py): M5.4 fix #2 — add vs and sync_id to payments tx projection
All checks were successful
Deploy to K8s / deploy (push) Successful in 7s
Python's fetch_sheet_data read 9 sheet columns but skipped VS and
Sync ID, causing make parity to report extra fields on every raw
payment row emitted by the Go backend. Both columns are already on
the sheet; add idx_vs / idx_sync_id lookups and the matching keys
to the tx dict so the Python /api/* wire shape matches Go's
RawTransaction.

Update /api/* test fixtures to include vs/sync_id keys for realism.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-07 23:50:33 +02:00
092dff25a5 Merge pull request 'fix(go): accept single-digit day/month in attendance date headers' (#22) from fix/go-attendance-date-parser into main
All checks were successful
Deploy to K8s / deploy (push) Successful in 10s
Reviewed-on: #22
2026-05-07 21:39:02 +00:00
56c21bcf03 fix(go): accept single-digit day/month in attendance date headers
All checks were successful
Deploy to K8s / deploy (push) Successful in 8s
parseDates was using "02.01.2006" / "01/02/2006" which require
zero-padded fields. The Czech attendance sheet headers contain dates
like "1.6.2026", "23.3.2026", "6.4.2026" — Go silently dropped those
columns while Python's strptime accepted them. Effect was a missing
2026-06 month on /api/juniors plus undercounted attendance in any month
with single-digit columns; surfaced via make parity.

Use the unpadded reference forms "2.1.2006" / "1/2/2006" instead — Go's
time.Parse accepts both padded and unpadded inputs against them.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-07 23:38:06 +02:00
208f762c18 Merge pull request 'feat(go): M5.4 — parity diff binary + make parity' (#19) from feat/go-m5-4-parity-binary into main
All checks were successful
Deploy to K8s / deploy (push) Successful in 10s
Reviewed-on: #19
2026-05-07 21:25:23 +00:00
4d035213b5 Merge pull request 'fix(go): pass raw value to FormatDate so numeric dates format' (#21) from fix/go-date-format into main
All checks were successful
Deploy to K8s / deploy (push) Successful in 6s
Reviewed-on: #21
2026-05-07 21:24:42 +00:00
2b15280d03 fix(go): exclude /api/version from parity diff — identity, not contract
All checks were successful
Deploy to K8s / deploy (push) Successful in 7s
/api/version returns each binary's own tag/commit/build_date, which
differs by design between independently built backends. Diffing it
always produces a false positive. Drop it from allRoutes; the route
remains reachable via `make parity ARGS="-route /api/version"`.

Also remove the vestigial `build_meta` allowlist entry (Python returns
the build dict as the top-level response body, not nested under
build_meta, so the scrubber never matched anything).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-07 23:23:38 +02:00
723152cdad fix(go): pass raw value to FormatDate so numeric serial-day dates format
All checks were successful
Deploy to K8s / deploy (push) Successful in 11s
The transaction-row parser in services/membership/sources.go used a
helper (`getVal`) that did `fmt.Sprint(row[i])` before passing to
`matching.FormatDate`.  The Sheets API returns date-formatted cells
as `float64` (Sheets serial-day numbers); pre-stringifying defeated
`FormatDate`'s `case float64:` dispatch, so values like 46147 leaked
through unchanged as the string "46147" instead of being converted
to "2026-05-05".

Surfaced by `make parity` (M5.4) — every `transactions[].date` on
/api/adults and /api/juniors differed between Python and Go.  Python
side passes the raw value through directly (`isinstance(val, (int,
float))` in scripts/match_payments.py format_date), so it was always
correct.

Added a `getRaw` helper that returns row[i] without stringifying;
only the date column needs it.  Extended TestLoadTransactions with
a numeric-serial-day row to lock in the regression.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-07 23:17:45 +02:00
7 changed files with 91 additions and 9 deletions

View File

@@ -1,5 +1,22 @@
# Changelog
## 2026-05-07 23:51 CEST — feat(py): M5.4 fix #2 — add vs and sync_id to payments tx projection
- `scripts/match_payments.py`: `fetch_sheet_data` now reads `VS` and `Sync ID` columns and includes `vs`/`sync_id` keys in every tx dict. Previously only 9 columns were projected, causing `make parity` to report extra `vs`/`sync_id` fields on every raw payment row emitted by the Go backend. Values flow through `group_payments_by_person``_unwrap_view_model_for_api` to `raw_payments` (adults/juniors) and `grouped_payments` (payments) automatically.
- `tests/test_app.py`: updated `/api/*` mock fixtures to include `vs`/`sync_id` keys for realism.
- **Cache note**: after deploying, hit `POST /flush-cache` once so the in-process cache is cleared and the next request picks up the new column lookups.
## 2026-05-07 23:37 CEST — fix(go): accept single-digit day/month in attendance date headers
- `go/internal/services/membership/sources.go`: `parseDates` now uses Go time formats `2.1.2006` and `1/2/2006` (single-digit reference forms, which accept both padded and unpadded inputs) instead of `02.01.2006` and `01/02/2006`. The Czech attendance sheet headers contain dates like `1.6.2026`, `23.3.2026`, `6.4.2026` — Go silently dropped those columns under the strict zero-padded format, while Python's `strptime("%d.%m.%Y")` accepted them. Effect was a missing `2026-06` month entirely on `/api/juniors` plus undercounted attendance for any month with single-digit columns; both surfaced as diffs in `make parity`.
- `sources_test.go::TestParseDates_SingleDigitDayMonth` added as a regression guard covering both Czech and US format flavours with and without leading zeros.
## 2026-05-07 23:17 CEST — fix(go): pass raw value to FormatDate so numeric serial-day dates format
- `go/internal/services/membership/sources.go`: transaction-row parser now passes `row[idxDate]` directly to `matching.FormatDate` (via a new `getRaw` helper) instead of stringifying first via `getVal`. The Sheets API returns numeric serial-day values as `float64` for date-formatted cells; pre-stringifying them defeated `FormatDate`'s `case float64:` dispatch, causing all numeric dates to leak through as `"46147"` style strings instead of `"2026-05-05"`.
- Surfaced by `make parity` (M5.4): every `transactions[].date` field on `/api/adults` and `/api/juniors` differed between Python and Go.
- `sources_test.go::TestLoadTransactions` extended with a numeric-serial-day row covering the regression.
## 2026-05-07 23:05 CEST — fix(go): default CacheDir to `tmp/go` to avoid Python collision
- `go/internal/config/config.go`: `CacheDir` default changed from `tmp` to `tmp/go`. Override via `CACHE_DIR` env var still works.
@@ -8,7 +25,7 @@
## 2026-05-07 22:55 CEST — feat(go): M5.4 — parity diff binary + `make parity`
- `go/cmd/parity/main.go`: new standalone binary that GETs `/api/version`, `/api/adults`, `/api/juniors`, `/api/payments` from both Python (:5001) and Go (:8080) backends, scrubs an allowlist (`render_time.total`, `build_meta`), and prints `cmp.Diff` for any remaining differences. Exits 0 on full match, 1 on diffs, 2 on fetch/parse errors — CI-friendly for M7.2.
- `go/cmd/parity/main.go`: new standalone binary that GETs `/api/adults`, `/api/juniors`, `/api/payments` from both Python (:5001) and Go (:8080) backends, scrubs an allowlist (`render_time.total`), and prints `cmp.Diff` for any remaining differences. Exits 0 on full match, 1 on diffs, 2 on fetch/parse errors — CI-friendly for M7.2. `/api/version` is excluded by design (returns binary identity — tag/commit/build_date — which differs between independently built backends); still accessible via `make parity ARGS="-route /api/version"`.
- `go/cmd/parity/scrub_test.go`: 4 unit tests covering top-level delete, nested delete, missing path, and non-map parent.
- `go/go.mod`: `github.com/google/go-cmp` promoted to direct dependency.
- `Makefile`: `parity` target added (`.PHONY`, help, `cd go && go run ./cmd/parity`).

View File

@@ -154,6 +154,7 @@ Goal: Go is the one true backend.
(Add entries as you go. Format: `YYYY-MM-DD — short note`.)
- 2026-05-07 — `/api/version` excluded from parity diff by design. Each binary's tag/commit/build_date is identity, not a data contract — diffing it would always flag a diff between independently built backends. Route remains reachable via `make parity ARGS="-route /api/version"` for manual inspection.
- 2026-05-04 — Plan approved. Versioning policy: latest stable for Go and all libs at the time M1 starts. Frontends explicitly allowed to diverge between Python and Go; only the JSON API contract is parity-locked. No reverse proxy — both backends run on different ports via `make web-py` / `make web-go`.
- 2026-05-07 — M4 complete. Chose fakes-only unit tests (no live integration tests) and CSV-via-public-URL for attendance (no Sheets API auth required for read-only). golangci-lint gofumpt extra-rules differ slightly from standalone gofumpt; used `golangci-lint run --fix --enable-only gofumpt` to auto-resolve formatting.
- 2026-05-04 — M1 complete. Dockerfile base changed from `distroless/static:nonroot` → `alpine:3` for debuggability (can tighten later). CLI dispatcher uses stdlib `flag`; module path `fuj-management/go`. golangci-lint v1 embedded gofumpt merges all imports into one group (no stdlib/local split) — accepted as the project style.

View File

@@ -14,13 +14,17 @@ import (
"github.com/google/go-cmp/cmp/cmpopts"
)
var allRoutes = []string{"/api/version", "/api/adults", "/api/juniors", "/api/payments"}
// /api/version is intentionally excluded — it returns each binary's own build
// identity (tag/commit/build_date), which differs by design between independently
// built backends. Pass -route /api/version to inspect it manually.
var allRoutes = []string{"/api/adults", "/api/juniors", "/api/payments"}
// defaultAllowlist holds dotted key paths to strip before diffing.
// These fields are expected to differ between backends (e.g. build tags, timing)
// or may appear on one side only. Today both are absent from the JSON — this is
// forward-compatible insurance for if either is added later.
var defaultAllowlist = []string{"render_time.total", "build_meta"}
// render_time.total is forward-compatible insurance: today it lives in the Jinja
// template context only (app.py inject_render_time) and is logged via
// middleware/timer.go on the Go side, so it isn't in any JSON response. If either
// side ever surfaces it under a render_time envelope, the scrubber handles it.
var defaultAllowlist = []string{"render_time.total"}
func main() {
pyURL := flag.String("py", "http://localhost:5001", "Python backend base URL")

View File

@@ -142,7 +142,13 @@ func parseDates(header []string) []struct {
}
var dt time.Time
var err error
for _, fmt_ := range []string{"02.01.2006", "01/02/2006"} {
// Use the unpadded reference forms ("2.1" and "1/2"): Go's time.Parse
// accepts both single-digit and zero-padded inputs against them, so
// "1.6.2026", "01.06.2026", "23.3.2026" all parse. Czech sheet authors
// drop the leading zero on dates ≤ 9 — Python's strptime is lenient
// the same way; the previous "02.01.2006" form silently dropped those
// columns and undercounted attendance.
for _, fmt_ := range []string{"2.1.2006", "1/2/2006"} {
dt, err = time.Parse(fmt_, raw)
if err == nil {
break
@@ -394,9 +400,19 @@ func parseTransactionRows(rows [][]any) ([]reconcile.Transaction, error) {
return fmt.Sprint(row[i])
}
// getRaw returns row[i] without stringifying — needed for FormatDate to
// dispatch on the underlying numeric type (Sheets returns serial-day
// numbers as float64). Stringifying first defeats that dispatch.
getRaw := func(row []any, i int) any {
if i < 0 || i >= len(row) {
return nil
}
return row[i]
}
var txns []reconcile.Transaction
for _, row := range rows[1:] {
dateStr := matching.FormatDate(getVal(row, idxDate))
dateStr := matching.FormatDate(getRaw(row, idxDate))
amountRaw := row[idxAmount]
if idxAmount < 0 || idxAmount >= len(row) {
amountRaw = ""

View File

@@ -114,12 +114,15 @@ func TestLoadJuniors(t *testing.T) {
func TestLoadTransactions(t *testing.T) {
// Sheets fake keyed by "<spreadsheetID>/<range>" — use the real constant.
// Row 1 uses a pre-formatted date string; row 2 uses the numeric Sheets
// serial-day form (float64) — the API returns either depending on cell
// formatting, and FormatDate must handle both.
paymentsKey := config.PaymentsSheetID + "/A1:Z"
sh := &sheets.Fake{Values: map[string][][]any{
paymentsKey: {
{"Date", "Amount", "manual fix", "Person", "Purpose", "Inferred Amount", "Sender", "VS", "Message", "Bank ID", "Sync ID"},
{"2026-04-01", 700.0, "", "Alice", "2026-04", "", "Alice Bank", "", "fee", "", "abc"},
{"2026-05-01", 500.0, "", "", "", "", "Bob Bank", "", "platba", "", "def"},
{46147.0, 500.0, "", "", "", "", "Bob Bank", "", "platba", "", "def"}, // 46147 serial-day = 2026-05-05
},
}}
s := buildSources(t, &attendance.Fake{}, sh)
@@ -137,6 +140,12 @@ func TestLoadTransactions(t *testing.T) {
if txns[0].Amount != 700 {
t.Errorf("txn[0].Amount: want 700, got %v", txns[0].Amount)
}
if txns[0].Date != "2026-04-01" {
t.Errorf("txn[0].Date: want 2026-04-01, got %q", txns[0].Date)
}
if txns[1].Date != "2026-05-05" {
t.Errorf("txn[1].Date (numeric serial-day): want 2026-05-05, got %q", txns[1].Date)
}
}
func TestLoadExceptions(t *testing.T) {
@@ -165,6 +174,28 @@ func TestLoadExceptions(t *testing.T) {
}
}
// TestParseDates_SingleDigitDayMonth covers the regression where Go's strict
// "02.01.2006" format dropped header cells written without leading zeros
// (e.g. "1.6.2026", "23.3.2026"), causing attendance undercounts and missing
// months on the /api/juniors response. Czech sheet authors drop the zero
// pad freely; Python's strptime tolerates it, so the parsers must match.
func TestParseDates_SingleDigitDayMonth(t *testing.T) {
// Czech form ("DD.MM.YYYY", with leading zeros optional) is the primary
// path. The "M/D/YYYY" fallback mirrors Python's %m/%d/%Y secondary
// strptime branch — month-first, day-second.
header := []string{"Jméno", "Tier", "", "01.06.2026", "1.6.2026", "23.3.2026", "6.4.2026", "01/02/2026", "1/2/2026"}
got := parseDates(header)
want := []string{"2026-06", "2026-06", "2026-03", "2026-04", "2026-01", "2026-01"}
if len(got) != len(want) {
t.Fatalf("parseDates: got %d entries, want %d (%v)", len(got), len(want), got)
}
for i, e := range got {
if e.month != want[i] {
t.Errorf("parseDates[%d].month = %q, want %q (raw=%q)", i, e.month, want[i], header[e.col])
}
}
}
// TTL smoke test: second call within TTL must not call fetch again.
func TestLoadAdults_CacheHit(t *testing.T) {
dir := t.TempDir()

View File

@@ -236,6 +236,8 @@ def fetch_sheet_data(spreadsheet_id: str, credentials_path: str) -> list[dict]:
idx_sender = get_col_index("Sender")
idx_message = get_col_index("Message")
idx_bank_id = get_col_index("Bank ID")
idx_vs = get_col_index("VS")
idx_sync_id = get_col_index("Sync ID")
required = {"Date": idx_date, "Amount": idx_amount, "Person": idx_person, "Purpose": idx_purpose}
missing = [name for name, idx in required.items() if idx == -1]
@@ -247,6 +249,12 @@ def fetch_sheet_data(spreadsheet_id: str, credentials_path: str) -> list[dict]:
def get_val(idx):
return row[idx] if idx != -1 and idx < len(row) else ""
def get_str(idx):
v = get_val(idx)
if isinstance(v, float) and v.is_integer():
return str(int(v))
return str(v)
tx = {
"date": format_date(get_val(idx_date)),
"amount": get_val(idx_amount),
@@ -255,8 +263,10 @@ def fetch_sheet_data(spreadsheet_id: str, credentials_path: str) -> list[dict]:
"purpose": get_val(idx_purpose),
"inferred_amount": get_val(idx_inferred_amount),
"sender": get_val(idx_sender),
"vs": get_str(idx_vs),
"message": get_val(idx_message),
"bank_id": get_val(idx_bank_id),
"sync_id": get_val(idx_sync_id),
}
transactions.append(tx)

View File

@@ -129,6 +129,7 @@ class TestWebApp(unittest.TestCase):
'date': '2026-01-01', 'amount': 750, 'person': 'Test Member',
'purpose': '2026-01', 'message': 'test payment',
'sender': 'External Bank User', 'inferred_amount': 750,
'vs': '', 'sync_id': 'abc123',
}]
response = self.client.get('/api/adults')
self.assertEqual(response.status_code, 200)
@@ -155,6 +156,7 @@ class TestWebApp(unittest.TestCase):
mock_fetch_sheet.return_value = [{
'date': '2026-01-15', 'amount': 500, 'person': 'Junior One',
'purpose': '2026-01', 'message': '', 'sender': 'Parent', 'inferred_amount': 500,
'vs': '', 'sync_id': 'def456',
}]
response = self.client.get('/api/juniors')
self.assertEqual(response.status_code, 200)
@@ -172,6 +174,7 @@ class TestWebApp(unittest.TestCase):
mock_fetch_sheet.return_value = [{
'date': '2026-01-01', 'amount': 750, 'person': 'Test Member',
'purpose': '2026-01', 'message': 'test', 'sender': 'Someone',
'vs': '', 'sync_id': 'ghi789',
}]
response = self.client.get('/api/payments')
self.assertEqual(response.status_code, 200)