Compare commits

...

5 Commits

Author SHA1 Message Date
56c21bcf03 fix(go): accept single-digit day/month in attendance date headers
All checks were successful
Deploy to K8s / deploy (push) Successful in 8s
parseDates was using "02.01.2006" / "01/02/2006" which require
zero-padded fields. The Czech attendance sheet headers contain dates
like "1.6.2026", "23.3.2026", "6.4.2026" — Go silently dropped those
columns while Python's strptime accepted them. Effect was a missing
2026-06 month on /api/juniors plus undercounted attendance in any month
with single-digit columns; surfaced via make parity.

Use the unpadded reference forms "2.1.2006" / "1/2/2006" instead — Go's
time.Parse accepts both padded and unpadded inputs against them.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-07 23:38:06 +02:00
208f762c18 Merge pull request 'feat(go): M5.4 — parity diff binary + make parity' (#19) from feat/go-m5-4-parity-binary into main
All checks were successful
Deploy to K8s / deploy (push) Successful in 10s
Reviewed-on: #19
2026-05-07 21:25:23 +00:00
4d035213b5 Merge pull request 'fix(go): pass raw value to FormatDate so numeric dates format' (#21) from fix/go-date-format into main
All checks were successful
Deploy to K8s / deploy (push) Successful in 6s
Reviewed-on: #21
2026-05-07 21:24:42 +00:00
2b15280d03 fix(go): exclude /api/version from parity diff — identity, not contract
All checks were successful
Deploy to K8s / deploy (push) Successful in 7s
/api/version returns each binary's own tag/commit/build_date, which
differs by design between independently built backends. Diffing it
always produces a false positive. Drop it from allRoutes; the route
remains reachable via `make parity ARGS="-route /api/version"`.

Also remove the vestigial `build_meta` allowlist entry (Python returns
the build dict as the top-level response body, not nested under
build_meta, so the scrubber never matched anything).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-07 23:23:38 +02:00
723152cdad fix(go): pass raw value to FormatDate so numeric serial-day dates format
All checks were successful
Deploy to K8s / deploy (push) Successful in 11s
The transaction-row parser in services/membership/sources.go used a
helper (`getVal`) that did `fmt.Sprint(row[i])` before passing to
`matching.FormatDate`.  The Sheets API returns date-formatted cells
as `float64` (Sheets serial-day numbers); pre-stringifying defeated
`FormatDate`'s `case float64:` dispatch, so values like 46147 leaked
through unchanged as the string "46147" instead of being converted
to "2026-05-05".

Surfaced by `make parity` (M5.4) — every `transactions[].date` on
/api/adults and /api/juniors differed between Python and Go.  Python
side passes the raw value through directly (`isinstance(val, (int,
float))` in scripts/match_payments.py format_date), so it was always
correct.

Added a `getRaw` helper that returns row[i] without stringifying;
only the date column needs it.  Extended TestLoadTransactions with
a numeric-serial-day row to lock in the regression.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-07 23:17:45 +02:00
5 changed files with 72 additions and 9 deletions

View File

@@ -1,5 +1,16 @@
# Changelog # Changelog
## 2026-05-07 23:37 CEST — fix(go): accept single-digit day/month in attendance date headers
- `go/internal/services/membership/sources.go`: `parseDates` now uses Go time formats `2.1.2006` and `1/2/2006` (single-digit reference forms, which accept both padded and unpadded inputs) instead of `02.01.2006` and `01/02/2006`. The Czech attendance sheet headers contain dates like `1.6.2026`, `23.3.2026`, `6.4.2026` — Go silently dropped those columns under the strict zero-padded format, while Python's `strptime("%d.%m.%Y")` accepted them. Effect was a missing `2026-06` month entirely on `/api/juniors` plus undercounted attendance for any month with single-digit columns; both surfaced as diffs in `make parity`.
- `sources_test.go::TestParseDates_SingleDigitDayMonth` added as a regression guard covering both Czech and US format flavours with and without leading zeros.
## 2026-05-07 23:17 CEST — fix(go): pass raw value to FormatDate so numeric serial-day dates format
- `go/internal/services/membership/sources.go`: transaction-row parser now passes `row[idxDate]` directly to `matching.FormatDate` (via a new `getRaw` helper) instead of stringifying first via `getVal`. The Sheets API returns numeric serial-day values as `float64` for date-formatted cells; pre-stringifying them defeated `FormatDate`'s `case float64:` dispatch, causing all numeric dates to leak through as `"46147"` style strings instead of `"2026-05-05"`.
- Surfaced by `make parity` (M5.4): every `transactions[].date` field on `/api/adults` and `/api/juniors` differed between Python and Go.
- `sources_test.go::TestLoadTransactions` extended with a numeric-serial-day row covering the regression.
## 2026-05-07 23:05 CEST — fix(go): default CacheDir to `tmp/go` to avoid Python collision ## 2026-05-07 23:05 CEST — fix(go): default CacheDir to `tmp/go` to avoid Python collision
- `go/internal/config/config.go`: `CacheDir` default changed from `tmp` to `tmp/go`. Override via `CACHE_DIR` env var still works. - `go/internal/config/config.go`: `CacheDir` default changed from `tmp` to `tmp/go`. Override via `CACHE_DIR` env var still works.
@@ -8,7 +19,7 @@
## 2026-05-07 22:55 CEST — feat(go): M5.4 — parity diff binary + `make parity` ## 2026-05-07 22:55 CEST — feat(go): M5.4 — parity diff binary + `make parity`
- `go/cmd/parity/main.go`: new standalone binary that GETs `/api/version`, `/api/adults`, `/api/juniors`, `/api/payments` from both Python (:5001) and Go (:8080) backends, scrubs an allowlist (`render_time.total`, `build_meta`), and prints `cmp.Diff` for any remaining differences. Exits 0 on full match, 1 on diffs, 2 on fetch/parse errors — CI-friendly for M7.2. - `go/cmd/parity/main.go`: new standalone binary that GETs `/api/adults`, `/api/juniors`, `/api/payments` from both Python (:5001) and Go (:8080) backends, scrubs an allowlist (`render_time.total`), and prints `cmp.Diff` for any remaining differences. Exits 0 on full match, 1 on diffs, 2 on fetch/parse errors — CI-friendly for M7.2. `/api/version` is excluded by design (returns binary identity — tag/commit/build_date — which differs between independently built backends); still accessible via `make parity ARGS="-route /api/version"`.
- `go/cmd/parity/scrub_test.go`: 4 unit tests covering top-level delete, nested delete, missing path, and non-map parent. - `go/cmd/parity/scrub_test.go`: 4 unit tests covering top-level delete, nested delete, missing path, and non-map parent.
- `go/go.mod`: `github.com/google/go-cmp` promoted to direct dependency. - `go/go.mod`: `github.com/google/go-cmp` promoted to direct dependency.
- `Makefile`: `parity` target added (`.PHONY`, help, `cd go && go run ./cmd/parity`). - `Makefile`: `parity` target added (`.PHONY`, help, `cd go && go run ./cmd/parity`).

View File

@@ -154,6 +154,7 @@ Goal: Go is the one true backend.
(Add entries as you go. Format: `YYYY-MM-DD — short note`.) (Add entries as you go. Format: `YYYY-MM-DD — short note`.)
- 2026-05-07 — `/api/version` excluded from parity diff by design. Each binary's tag/commit/build_date is identity, not a data contract — diffing it would always flag a diff between independently built backends. Route remains reachable via `make parity ARGS="-route /api/version"` for manual inspection.
- 2026-05-04 — Plan approved. Versioning policy: latest stable for Go and all libs at the time M1 starts. Frontends explicitly allowed to diverge between Python and Go; only the JSON API contract is parity-locked. No reverse proxy — both backends run on different ports via `make web-py` / `make web-go`. - 2026-05-04 — Plan approved. Versioning policy: latest stable for Go and all libs at the time M1 starts. Frontends explicitly allowed to diverge between Python and Go; only the JSON API contract is parity-locked. No reverse proxy — both backends run on different ports via `make web-py` / `make web-go`.
- 2026-05-07 — M4 complete. Chose fakes-only unit tests (no live integration tests) and CSV-via-public-URL for attendance (no Sheets API auth required for read-only). golangci-lint gofumpt extra-rules differ slightly from standalone gofumpt; used `golangci-lint run --fix --enable-only gofumpt` to auto-resolve formatting. - 2026-05-07 — M4 complete. Chose fakes-only unit tests (no live integration tests) and CSV-via-public-URL for attendance (no Sheets API auth required for read-only). golangci-lint gofumpt extra-rules differ slightly from standalone gofumpt; used `golangci-lint run --fix --enable-only gofumpt` to auto-resolve formatting.
- 2026-05-04 — M1 complete. Dockerfile base changed from `distroless/static:nonroot` → `alpine:3` for debuggability (can tighten later). CLI dispatcher uses stdlib `flag`; module path `fuj-management/go`. golangci-lint v1 embedded gofumpt merges all imports into one group (no stdlib/local split) — accepted as the project style. - 2026-05-04 — M1 complete. Dockerfile base changed from `distroless/static:nonroot` → `alpine:3` for debuggability (can tighten later). CLI dispatcher uses stdlib `flag`; module path `fuj-management/go`. golangci-lint v1 embedded gofumpt merges all imports into one group (no stdlib/local split) — accepted as the project style.

View File

@@ -14,13 +14,17 @@ import (
"github.com/google/go-cmp/cmp/cmpopts" "github.com/google/go-cmp/cmp/cmpopts"
) )
var allRoutes = []string{"/api/version", "/api/adults", "/api/juniors", "/api/payments"} // /api/version is intentionally excluded — it returns each binary's own build
// identity (tag/commit/build_date), which differs by design between independently
// built backends. Pass -route /api/version to inspect it manually.
var allRoutes = []string{"/api/adults", "/api/juniors", "/api/payments"}
// defaultAllowlist holds dotted key paths to strip before diffing. // defaultAllowlist holds dotted key paths to strip before diffing.
// These fields are expected to differ between backends (e.g. build tags, timing) // render_time.total is forward-compatible insurance: today it lives in the Jinja
// or may appear on one side only. Today both are absent from the JSON — this is // template context only (app.py inject_render_time) and is logged via
// forward-compatible insurance for if either is added later. // middleware/timer.go on the Go side, so it isn't in any JSON response. If either
var defaultAllowlist = []string{"render_time.total", "build_meta"} // side ever surfaces it under a render_time envelope, the scrubber handles it.
var defaultAllowlist = []string{"render_time.total"}
func main() { func main() {
pyURL := flag.String("py", "http://localhost:5001", "Python backend base URL") pyURL := flag.String("py", "http://localhost:5001", "Python backend base URL")

View File

@@ -142,7 +142,13 @@ func parseDates(header []string) []struct {
} }
var dt time.Time var dt time.Time
var err error var err error
for _, fmt_ := range []string{"02.01.2006", "01/02/2006"} { // Use the unpadded reference forms ("2.1" and "1/2"): Go's time.Parse
// accepts both single-digit and zero-padded inputs against them, so
// "1.6.2026", "01.06.2026", "23.3.2026" all parse. Czech sheet authors
// drop the leading zero on dates ≤ 9 — Python's strptime is lenient
// the same way; the previous "02.01.2006" form silently dropped those
// columns and undercounted attendance.
for _, fmt_ := range []string{"2.1.2006", "1/2/2006"} {
dt, err = time.Parse(fmt_, raw) dt, err = time.Parse(fmt_, raw)
if err == nil { if err == nil {
break break
@@ -394,9 +400,19 @@ func parseTransactionRows(rows [][]any) ([]reconcile.Transaction, error) {
return fmt.Sprint(row[i]) return fmt.Sprint(row[i])
} }
// getRaw returns row[i] without stringifying — needed for FormatDate to
// dispatch on the underlying numeric type (Sheets returns serial-day
// numbers as float64). Stringifying first defeats that dispatch.
getRaw := func(row []any, i int) any {
if i < 0 || i >= len(row) {
return nil
}
return row[i]
}
var txns []reconcile.Transaction var txns []reconcile.Transaction
for _, row := range rows[1:] { for _, row := range rows[1:] {
dateStr := matching.FormatDate(getVal(row, idxDate)) dateStr := matching.FormatDate(getRaw(row, idxDate))
amountRaw := row[idxAmount] amountRaw := row[idxAmount]
if idxAmount < 0 || idxAmount >= len(row) { if idxAmount < 0 || idxAmount >= len(row) {
amountRaw = "" amountRaw = ""

View File

@@ -114,12 +114,15 @@ func TestLoadJuniors(t *testing.T) {
func TestLoadTransactions(t *testing.T) { func TestLoadTransactions(t *testing.T) {
// Sheets fake keyed by "<spreadsheetID>/<range>" — use the real constant. // Sheets fake keyed by "<spreadsheetID>/<range>" — use the real constant.
// Row 1 uses a pre-formatted date string; row 2 uses the numeric Sheets
// serial-day form (float64) — the API returns either depending on cell
// formatting, and FormatDate must handle both.
paymentsKey := config.PaymentsSheetID + "/A1:Z" paymentsKey := config.PaymentsSheetID + "/A1:Z"
sh := &sheets.Fake{Values: map[string][][]any{ sh := &sheets.Fake{Values: map[string][][]any{
paymentsKey: { paymentsKey: {
{"Date", "Amount", "manual fix", "Person", "Purpose", "Inferred Amount", "Sender", "VS", "Message", "Bank ID", "Sync ID"}, {"Date", "Amount", "manual fix", "Person", "Purpose", "Inferred Amount", "Sender", "VS", "Message", "Bank ID", "Sync ID"},
{"2026-04-01", 700.0, "", "Alice", "2026-04", "", "Alice Bank", "", "fee", "", "abc"}, {"2026-04-01", 700.0, "", "Alice", "2026-04", "", "Alice Bank", "", "fee", "", "abc"},
{"2026-05-01", 500.0, "", "", "", "", "Bob Bank", "", "platba", "", "def"}, {46147.0, 500.0, "", "", "", "", "Bob Bank", "", "platba", "", "def"}, // 46147 serial-day = 2026-05-05
}, },
}} }}
s := buildSources(t, &attendance.Fake{}, sh) s := buildSources(t, &attendance.Fake{}, sh)
@@ -137,6 +140,12 @@ func TestLoadTransactions(t *testing.T) {
if txns[0].Amount != 700 { if txns[0].Amount != 700 {
t.Errorf("txn[0].Amount: want 700, got %v", txns[0].Amount) t.Errorf("txn[0].Amount: want 700, got %v", txns[0].Amount)
} }
if txns[0].Date != "2026-04-01" {
t.Errorf("txn[0].Date: want 2026-04-01, got %q", txns[0].Date)
}
if txns[1].Date != "2026-05-05" {
t.Errorf("txn[1].Date (numeric serial-day): want 2026-05-05, got %q", txns[1].Date)
}
} }
func TestLoadExceptions(t *testing.T) { func TestLoadExceptions(t *testing.T) {
@@ -165,6 +174,28 @@ func TestLoadExceptions(t *testing.T) {
} }
} }
// TestParseDates_SingleDigitDayMonth covers the regression where Go's strict
// "02.01.2006" format dropped header cells written without leading zeros
// (e.g. "1.6.2026", "23.3.2026"), causing attendance undercounts and missing
// months on the /api/juniors response. Czech sheet authors drop the zero
// pad freely; Python's strptime tolerates it, so the parsers must match.
func TestParseDates_SingleDigitDayMonth(t *testing.T) {
// Czech form ("DD.MM.YYYY", with leading zeros optional) is the primary
// path. The "M/D/YYYY" fallback mirrors Python's %m/%d/%Y secondary
// strptime branch — month-first, day-second.
header := []string{"Jméno", "Tier", "", "01.06.2026", "1.6.2026", "23.3.2026", "6.4.2026", "01/02/2026", "1/2/2026"}
got := parseDates(header)
want := []string{"2026-06", "2026-06", "2026-03", "2026-04", "2026-01", "2026-01"}
if len(got) != len(want) {
t.Fatalf("parseDates: got %d entries, want %d (%v)", len(got), len(want), got)
}
for i, e := range got {
if e.month != want[i] {
t.Errorf("parseDates[%d].month = %q, want %q (raw=%q)", i, e.month, want[i], header[e.col])
}
}
}
// TTL smoke test: second call within TTL must not call fetch again. // TTL smoke test: second call within TTL must not call fetch again.
func TestLoadAdults_CacheHit(t *testing.T) { func TestLoadAdults_CacheHit(t *testing.T) {
dir := t.TempDir() dir := t.TempDir()