Compare commits

..

7 Commits

Author SHA1 Message Date
f0de300292 chore: CHANGELOG for --print-fio-table, debug logging, and date parser fix
All checks were successful
Deploy to K8s / deploy (push) Successful in 9s
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-07 14:13:56 +02:00
2164e99866 Merge pull request 'feat(go): add --print-fio-table debug flag to fuj sync' (#14) from feat/fuj-sync-print-fio-table into main
All checks were successful
Deploy to K8s / deploy (push) Successful in 9s
Reviewed-on: #14
2026-05-07 12:13:19 +00:00
b41b8ef29c fix(go/fio): accept 2-digit year format in transparent date parser
All checks were successful
Deploy to K8s / deploy (push) Successful in 9s
Fio's transparent account page now serves dates as DD.MM.YY (e.g.
07.05.26) rather than the previously expected 4-digit-year format.
Extends parseCzechDate to try all eight layout variants: padded and
non-padded, dot and slash separators, 4-digit and 2-digit years.

Go maps 2-digit year 00-68 → 2000-2068, so 26 → 2026.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-07 14:12:34 +02:00
80db33945d chore: add make go-sync-debug target
All checks were successful
Deploy to K8s / deploy (push) Successful in 6s
Wraps `LOG_LEVEL=DEBUG ./bin/fuj sync -dry-run -print-fio-table -days N`
behind a single make target. Default DAYS=30, override with
`make go-sync-debug DAYS=90`. Builds the Go binary first via go-build.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-07 14:01:46 +02:00
f87adeff9f feat(go/fio): debug logging via slog at LOG_LEVEL=DEBUG
All checks were successful
Deploy to K8s / deploy (push) Successful in 8s
Wires slog.SetDefault to honour LOG_LEVEL in all CLI commands and adds
debug logs on the Fio fetch path so a silent "fetched 0 transaction(s)"
can be diagnosed without code changes:

- fio.New: which client variant (api/transparent) was selected
- apiClient: GET URL (token redacted as ****), HTTP status, body bytes,
  parsed transaction count
- transparentClient: GET URL, HTTP status, body bytes, plus parser
  stats (raw rows from second table, kept, dropped_bad_date,
  dropped_nonpositive_amount)

Also suppresses the --print-fio-table block when zero transactions were
fetched, so the bare header no longer prints under that condition.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-07 13:59:22 +02:00
a7cf45fc95 feat(go): add --print-fio-table flag to fuj sync --dry-run
All checks were successful
Deploy to K8s / deploy (push) Successful in 7s
Prints an aligned tabwriter table of every Fio transaction in the
look-back window, with a STATUS column showing NEW (would be appended)
or DUP (already in sheet). Only fires when --dry-run is also set, so
it can't affect real syncs. Refactors Sync ID computation into a single
pre-pass shared by both the table printer and the row builder.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-07 13:49:42 +02:00
f0a0f79475 Merge pull request 'feat(go): IO layer behind interfaces (M4)' (#13) from feat/m4-io-layer into main
All checks were successful
Deploy to K8s / deploy (push) Successful in 10s
Reviewed-on: #13
2026-05-07 08:48:54 +00:00
10 changed files with 141 additions and 16 deletions

View File

@@ -1,5 +1,12 @@
# Changelog
## 2026-05-07 14:13 CEST — feat(go): --print-fio-table + Fio debug logging + date parser fix
- Added `--print-fio-table` flag to `fuj sync --dry-run`: prints an aligned table of every Fio transaction in the window with `STATUS=NEW/DUP`, using `text/tabwriter`. Key files: `go/internal/services/banksync/fio_table.go`, `sync.go`, `cmd/fuj/main.go`.
- Added `LOG_LEVEL=DEBUG` debug logging on the Fio fetch path: client variant selected, full GET URL (token redacted on API path), HTTP status, body bytes, and per-parse drop-reason counters (`raw_rows`, `kept`, `dropped_bad_date`, `dropped_nonpositive_amount`). Key files: `go/internal/io/fio/{client,api,transparent}.go`.
- Fixed `parseCzechDate` to accept `DD.MM.YY` (2-digit year) in addition to the 4-digit variant — Fio's transparent page now serves this format. Key file: `go/internal/io/fio/transparent.go`.
- Added `make go-sync-debug [DAYS=N]` Makefile target (default 30 days).
## 2026-05-07 10:32 CEST — feat(go): --dry-run for fuj sync
- `SyncOpts.DryRun bool` added; when true, `SyncToSheets` prints planned writes (`would write header row`, `would append date=… amount=… sender=…`, `would sort by date`) and returns without calling `WriteHeader`, `AppendValues`, or `SortByDateColumn`.

View File

@@ -1,4 +1,4 @@
.PHONY: help fees match web web-py web-debug web-go go-build go-test go-test-all go-parity go-run go-lint capture-fixtures image run sync sync-2026 test test-v docs
.PHONY: help fees match web web-py web-debug web-go go-build go-test go-test-all go-parity go-run go-sync-debug go-lint capture-fixtures image run sync sync-2026 test test-v docs
export PYTHONPATH := scripts:$(PYTHONPATH)
VENV := .venv
@@ -27,6 +27,7 @@ help:
@echo " make go-parity - Run Go parity tests (requires -tags=parity fixture corpus)"
@echo " make go-test-all - Run both unit and parity tests"
@echo " make go-lint - Run golangci-lint on Go code"
@echo " make go-sync-debug [DAYS=N] - Dry-run Go sync with Fio debug logs and txn table (default DAYS=30)"
@echo " make capture-fixtures - Regenerate parity fixture corpus from live Python"
@echo " make image - Build Python OCI container image"
@echo " make run - Run the built Python Docker image locally"
@@ -91,6 +92,10 @@ capture-fixtures: $(PYTHON)
go-run: go-build
./$(GO_BIN) $(ARGS)
DAYS ?= 30
go-sync-debug: go-build
LOG_LEVEL=DEBUG ./$(GO_BIN) sync -dry-run -print-fio-table -days $(DAYS)
go-lint:
cd $(GO_SRC) && golangci-lint run ./...

View File

@@ -0,0 +1,29 @@
# Add `--print-fio-table` debug flag to `fuj sync`
## Context
The Go port of `fuj sync --dry-run` currently prints only the **new**
transactions — i.e. rows that will be appended to the payments sheet after
deduping against existing Sync IDs (see [sync.go:125-129](../../go/internal/services/banksync/sync.go#L125-L129)).
When debugging Fio sync issues ("why isn't transaction X showing up?",
"is the dedup working?"), there's no way to see what Fio actually
returned versus what got filtered as a duplicate.
This change adds a `--print-fio-table` flag that, **only when combined
with `--dry-run`**, prints an aligned table of every Fio transaction in
the window with each row marked `NEW` (would be appended) or `DUP`
(already in sheet, skipped). The flag is silently ignored without
`--dry-run`, so it can't accidentally fire during a real sync.
## Decisions
- Flag name: `--print-fio-table` (specific, not generic `--verbose`).
- Columns: `DATE | AMOUNT | SENDER | VS | MESSAGE | BANKID | STATUS`,
with MESSAGE truncated and STATUS = `NEW` / `DUP`.
- Scope: only effective when `--dry-run` is also set.
## Files modified
- [go/cmd/fuj/main.go](../../go/cmd/fuj/main.go) — new flag + SyncOpts field
- [go/internal/services/banksync/sync.go](../../go/internal/services/banksync/sync.go) — SyncOpts struct + refactored step 4
- [go/internal/services/banksync/debug.go](../../go/internal/services/banksync/debug.go) — printFioTable helper (new)

View File

@@ -11,6 +11,7 @@ import (
"fuj-management/go/internal/services/banksync"
"fuj-management/go/internal/services/membership"
"fuj-management/go/internal/web"
"log/slog"
"os"
"time"
)
@@ -28,6 +29,9 @@ func main() {
os.Exit(2)
}
// Honour LOG_LEVEL for slog calls in any package (e.g. internal/io/fio debug logs).
slog.SetDefault(logging.New(os.Getenv("LOG_LEVEL")))
cmd, args := os.Args[1], os.Args[2:]
switch cmd {
@@ -135,8 +139,9 @@ func syncCmd(args []string) {
toStr := fs.String("to", "", "end date YYYY-MM-DD")
sort := fs.Bool("sort", true, "sort sheet by date after appending")
dryRun := fs.Bool("dry-run", false, "print planned writes without modifying the sheet")
printFioTable := fs.Bool("print-fio-table", false, "with --dry-run: print aligned table of every Fio transaction with NEW/DUP status")
fs.Usage = func() {
fmt.Fprintln(os.Stderr, "usage: fuj sync [--days N] [--from YYYY-MM-DD --to YYYY-MM-DD] [--sort] [--dry-run]")
fmt.Fprintln(os.Stderr, "usage: fuj sync [--days N] [--from YYYY-MM-DD --to YYYY-MM-DD] [--sort] [--dry-run] [--print-fio-table]")
fs.PrintDefaults()
}
if err := fs.Parse(args); err != nil {
@@ -154,7 +159,7 @@ func syncCmd(args []string) {
}
fioCli := fio.New(cfg.FioAPIToken, config.IBANAccountNum(cfg.BankAccount), nil)
opts := banksync.SyncOpts{Days: *days, Sort: *sort, DryRun: *dryRun}
opts := banksync.SyncOpts{Days: *days, Sort: *sort, DryRun: *dryRun, PrintFioTable: *printFioTable}
if *fromStr != "" && *toStr != "" {
opts.From, err = time.Parse("2006-01-02", *fromStr)
if err != nil {

View File

@@ -5,7 +5,9 @@ import (
"encoding/json"
"fmt"
"io"
"log/slog"
"net/http"
"strings"
"time"
)
@@ -25,6 +27,9 @@ func (c *apiClient) FetchTransactions(ctx context.Context, from, to time.Time) (
const layout = "2006-01-02"
url := fmt.Sprintf("https://fioapi.fio.cz/v1/rest/periods/%s/%s/%s/transactions.json",
c.token, from.Format(layout), to.Format(layout))
slog.Debug("fio api: GET",
"url", strings.Replace(url, c.token, "****", 1),
"from", from.Format(layout), "to", to.Format(layout))
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
if err != nil {
@@ -35,6 +40,7 @@ func (c *apiClient) FetchTransactions(ctx context.Context, from, to time.Time) (
return nil, err
}
defer resp.Body.Close()
slog.Debug("fio api: response", "status", resp.StatusCode)
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("fio api: HTTP %d", resp.StatusCode)
}
@@ -42,7 +48,9 @@ func (c *apiClient) FetchTransactions(ctx context.Context, from, to time.Time) (
if err != nil {
return nil, err
}
return parseAPIResponse(body)
txns, err := parseAPIResponse(body)
slog.Debug("fio api: parsed", "body_bytes", len(body), "parsed_count", len(txns))
return txns, err
}
// fioAPIResponse is the top-level envelope from the Fio JSON API.

View File

@@ -4,6 +4,7 @@ package fio
import (
"context"
"log/slog"
"net/http"
"time"
)
@@ -36,7 +37,9 @@ func New(token, accountNum string, hc httpDoer) Client {
hc = http.DefaultClient
}
if token != "" {
slog.Debug("fio: client selected", "type", "api")
return &apiClient{token: token, hc: hc}
}
slog.Debug("fio: client selected", "type", "transparent", "account_num", accountNum)
return &transparentClient{accountNum: accountNum, hc: hc}
}

View File

@@ -97,6 +97,9 @@ func TestParseCzechDate(t *testing.T) {
{"10/04/2026", "2026-04-10"},
{"7.5.2026", "2026-05-07"}, // non-padded — real Fio transparent page format
{"3.12.2025", "2025-12-03"}, // non-padded single-digit day, double-digit month
{"07.05.26", "2026-05-07"}, // padded 2-digit year — current Fio transparent page format
{"7.5.26", "2026-05-07"}, // non-padded 2-digit year
{"07/05/26", "2026-05-07"}, // slash variant
{"", ""},
{"invalid", ""},
}

View File

@@ -4,6 +4,7 @@ import (
"context"
"fmt"
"io"
"log/slog"
"net/http"
"regexp"
"strings"
@@ -28,6 +29,10 @@ func (c *transparentClient) FetchTransactions(ctx context.Context, from, to time
from.Format("2.1.2006"),
to.Format("2.1.2006"),
)
slog.Debug("fio transparent: GET",
"url", url,
"from", from.Format("2006-01-02"), "to", to.Format("2006-01-02"))
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
if err != nil {
return nil, err
@@ -37,6 +42,7 @@ func (c *transparentClient) FetchTransactions(ctx context.Context, from, to time
return nil, err
}
defer resp.Body.Close()
slog.Debug("fio transparent: response", "status", resp.StatusCode)
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("fio transparent: HTTP %d", resp.StatusCode)
}
@@ -44,6 +50,7 @@ func (c *transparentClient) FetchTransactions(ctx context.Context, from, to time
if err != nil {
return nil, err
}
slog.Debug("fio transparent: body read", "body_bytes", len(body))
return parseTransparentHTML(body)
}
@@ -63,6 +70,7 @@ func parseTransparentHTML(body []byte) ([]Transaction, error) {
rows := extractSecondTableRows(body)
var txns []Transaction
var droppedBadDate, droppedNonpositive int
for _, row := range rows {
col := func(i int) string {
if i < len(row) {
@@ -72,7 +80,12 @@ func parseTransparentHTML(body []byte) ([]Transaction, error) {
}
dateStr := parseCzechDate(col(tColDate))
amount := parseCzechAmount(col(tColAmount))
if dateStr == "" || amount <= 0 {
if dateStr == "" {
droppedBadDate++
continue
}
if amount <= 0 {
droppedNonpositive++
continue
}
txns = append(txns, Transaction{
@@ -86,6 +99,11 @@ func parseTransparentHTML(body []byte) ([]Transaction, error) {
BankID: "", // not available on HTML path
})
}
slog.Debug("fio transparent: parsed",
"raw_rows", len(rows),
"kept", len(txns),
"dropped_bad_date", droppedBadDate,
"dropped_nonpositive_amount", droppedNonpositive)
return txns, nil
}
@@ -191,7 +209,10 @@ func hasClass(t ghtml.Token, cls string) bool {
// Returns "" on parse error.
func parseCzechDate(s string) string {
s = strings.TrimSpace(s)
for _, layout := range []string{"2.1.2006", "02.01.2006", "2/1/2006", "02/01/2006"} {
for _, layout := range []string{
"2.1.2006", "02.01.2006", "2/1/2006", "02/01/2006",
"2.1.06", "02.01.06", "2/1/06", "02/01/06",
} {
if t, err := time.Parse(layout, s); err == nil {
return t.Format("2006-01-02")
}

View File

@@ -0,0 +1,32 @@
package banksync
import (
"fmt"
"io"
"text/tabwriter"
"fuj-management/go/internal/io/fio"
)
func printFioTable(w io.Writer, txns []fio.Transaction, syncIDs []string, existing map[string]bool) {
tw := tabwriter.NewWriter(w, 0, 0, 2, ' ', 0)
fmt.Fprintln(tw, "DATE\tAMOUNT\tSENDER\tVS\tMESSAGE\tBANKID\tSTATUS")
for i, tx := range txns {
status := "NEW"
if existing[syncIDs[i]] {
status = "DUP"
}
fmt.Fprintf(tw, "%s\t%.2f\t%s\t%s\t%s\t%s\t%s\n",
tx.Date, tx.Amount, tx.Sender, tx.VS,
truncRunes(tx.Message, 40), tx.BankID, status)
}
_ = tw.Flush()
}
func truncRunes(s string, n int) string {
rs := []rune(s)
if len(rs) <= n {
return s
}
return string(rs[:n-1]) + "…"
}

View File

@@ -6,6 +6,7 @@ import (
"fmt"
"fuj-management/go/internal/domain/synch"
"fuj-management/go/internal/io/fio"
"os"
"strings"
"time"
)
@@ -27,10 +28,11 @@ type sheetsWriter interface {
// SyncOpts controls the date window and sort behaviour.
type SyncOpts struct {
Days int // look-back window when From/To are zero
From, To time.Time // explicit window (overrides Days)
Sort bool // sort the sheet by Date after appending
DryRun bool // print planned writes without modifying the sheet
Days int // look-back window when From/To are zero
From, To time.Time // explicit window (overrides Days)
Sort bool // sort the sheet by Date after appending
DryRun bool // print planned writes without modifying the sheet
PrintFioTable bool // with DryRun: print every fetched Fio txn with NEW/DUP status
}
// SyncToSheets fetches Fio transactions and appends new ones to the payments sheet.
@@ -92,14 +94,14 @@ func SyncToSheets(
from.Format("2006-01-02"), to.Format("2006-01-02"), len(txns))
}
// 4. Append new rows.
var newRows [][]any
for _, tx := range txns {
// 4a. Compute Sync IDs for every fetched txn (shared by table-print and row-build).
syncIDs := make([]string, len(txns))
for i, tx := range txns {
currency := tx.Currency
if currency == "" {
currency = "CZK"
}
id := synch.GenerateSyncID(synch.Transaction{
syncIDs[i] = synch.GenerateSyncID(synch.Transaction{
Date: tx.Date,
Amount: tx.Amount,
Currency: currency,
@@ -108,13 +110,23 @@ func SyncToSheets(
Message: tx.Message,
BankID: tx.BankID,
})
if existingIDs[id] {
}
// 4b. Optional debug table (dry-run only; suppress when nothing was fetched).
if opts.DryRun && opts.PrintFioTable && len(txns) > 0 {
printFioTable(os.Stdout, txns, syncIDs, existingIDs)
}
// 4c. Build new rows.
var newRows [][]any
for i, tx := range txns {
if existingIDs[syncIDs[i]] {
continue
}
newRows = append(newRows, []any{
tx.Date, tx.Amount,
"", "", "", "", // manual fix, Person, Purpose, Inferred Amount
tx.Sender, tx.VS, tx.Message, tx.BankID, id,
tx.Sender, tx.VS, tx.Message, tx.BankID, syncIDs[i],
})
}