diff --git a/CHANGELOG.md b/CHANGELOG.md index 50e3907..c246cc7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,14 @@ # Changelog +## 2026-10-08 12:16 CEST — feat(go): scheduled bank sync (Go only) + +- The Go server now runs the bank sync (sync → infer → flush cache) by itself at the times in `SYNC_SCHEDULE` (comma-separated `HH:MM`, default `04:00`, `off` disables), read in `SYNC_TZ` (default `Europe/Prague`). Nothing runs at startup; an invalid value stops the server at startup with an error. +- Manual `/sync-bank` and scheduled runs share one lock, so two syncs can no longer overlap; a second request gets "a sync is already running". +- The page footer shows the last sync since start (time, trigger, ok/FAILED) and the next scheduled run. +- The binary embeds the time zone database (`time/tzdata`), since the Alpine image has none. +- Deliberately not implemented in the legacy Python app, which is being phased out; recorded in `CLAUDE.md` under "Deliberately Go-only features". +- Key files: `go/internal/scheduler/`, `go/internal/services/banksync/runner.go`, `go/cmd/fuj/main.go`, `go/internal/config/config.go`, `go/internal/web/`. + ## 2026-10-08 12:04 CEST — fix(sync): dashboard sync covers the current year plus Nov–Dec of the previous one - `/sync-bank` now uses one shared default window in both implementations: 1 November of the previous year to 31 December of the current year. Payments dated around New Year are picked up in January; duplicates are still filtered by Sync ID. diff --git a/CLAUDE.md b/CLAUDE.md index 7c861b8..69892a7 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -20,6 +20,12 @@ Until the Python version is retired, **every new feature and every behaviour cha Go-only or Python-only changes are fine for things that exist on one side only (build, tooling, HTML/CSS/JS, refactors with no behaviour change). +### Deliberately Go-only features + +These exist only in the Go app on purpose. Do not port them to the legacy Python app, which is being phased out: + +- **Scheduled bank sync** — the Go server runs sync → infer → flush-cache by itself at the times in `SYNC_SCHEDULE`. Python's multi-worker gunicorn setup would need a cross-process lock for this, which is not worth building in code that is going away. In Python the sync still runs only from `/sync-bank` or the CLI. + ## Key Constraints - **PII separation**: Member data (names, emails, payment info) must never be committed to git. Enforce config/data separation from day one. @@ -104,10 +110,11 @@ Fio Bank API ──► bank sync ──► Google Sheets (payments - `go/cmd/fuj` — CLI entry point; subcommands `server`, `fees`, `reconcile`, `sync`, `infer`, `version`. - `go/cmd/parity` — diffs the JSON API of the Go and Python servers. -- `go/internal/config` — sheet IDs, bank accounts, env-driven settings (`CREDENTIALS_PATH`, `CACHE_DIR`, `CACHE_TTL_SECONDS`, `CACHE_API_CHECK_TTL_SECONDS`, `MONTHS_TO_SHOW`, `SERVER_ADDR`, `LOG_LEVEL`, `FIO_API_TOKEN_NEW` / `FIO_API_TOKEN_OLD`). +- `go/internal/config` — sheet IDs, bank accounts, env-driven settings (`CREDENTIALS_PATH`, `CACHE_DIR`, `CACHE_TTL_SECONDS`, `CACHE_API_CHECK_TTL_SECONDS`, `MONTHS_TO_SHOW`, `SERVER_ADDR`, `LOG_LEVEL`, `FIO_API_TOKEN_NEW` / `FIO_API_TOKEN_OLD`, `SYNC_SCHEDULE`, `SYNC_TZ`). - `go/internal/domain` — pure logic: `fees` (rates, `CalculateFee`, `CalculateJuniorFee`), `reconcile`, `matching` (name matching, inference), `czech` (normalize, month parsing), `money`, `synch` (sync IDs). - `go/internal/io` — external systems: `attendance` (CSV export), `sheets`, `drive`, `fio` (API + transparent-page scraper), `cache` (file cache). Each has a fake for tests. -- `go/internal/services` — `membership` (loading, merged months, fee and reconcile reports) and `banksync` (Fio sync + inference). +- `go/internal/services` — `membership` (loading, merged months, fee and reconcile reports) and `banksync` (Fio sync + inference; `Runner` serialises manual and scheduled runs). +- `go/internal/scheduler` — runs a job at fixed times of day; used by `fuj server` for the scheduled bank sync. - `go/internal/web` — routes for `/adults`, `/juniors`, `/payments`, `/sync-bank`, `/flush-cache`, `/qr`, `/version`, `/api/*`; `templates/` and `static/` are embedded. - `go/tests` — parity tests and the scrubbed fixture corpus captured from Python. @@ -144,6 +151,13 @@ Tiers are set in column B of the attendance sheet: Both implementations check the Drive API `modifiedTime` and serve a cached JSON file when the sheet hasn't changed, falling back to 5-minute TTL buckets when the Drive API is unavailable. Go: `go/internal/io/cache`, files in `tmp/go/` (`CACHE_DIR`). Python: `get_cached_data()` in `legacy/app.py`, files in `legacy/tmp/`, warmed up at startup (`warmup_cache()`). Flush via `/flush-cache` (POST). +### Bank sync + +- Manual: open `/sync-bank` (both apps) or run `fuj sync` + `fuj infer` / `make sync-year` + `make infer`. +- Scheduled (Go server only): at each time in `SYNC_SCHEDULE` (comma-separated `HH:MM`, default `04:00`; `off` disables) read in `SYNC_TZ` (default `Europe/Prague`). Nothing runs at startup. Manual and scheduled runs share one lock and never overlap. The page footer shows the last run since start and the next one. +- Both use the same window: 1 November of the previous year to 31 December of the current year; rows are de-duplicated by Sync ID. +- The schedule runs per process: keep the deployment at one replica, and note that a local `make web-go` with real credentials also syncs at the scheduled time unless `SYNC_SCHEDULE=off`. + ### Payments sheet columns `Date | Amount | manual fix | Person | Purpose | Inferred Amount | Sender | VS | Message | Bank ID | Sync ID` diff --git a/docs/plans/2026-10-08-1026-pre-feature-cleanup-tasks.md b/docs/plans/2026-10-08-1026-pre-feature-cleanup-tasks.md index e4a3f62..29604cf 100644 --- a/docs/plans/2026-10-08-1026-pre-feature-cleanup-tasks.md +++ b/docs/plans/2026-10-08-1026-pre-feature-cleanup-tasks.md @@ -136,7 +136,9 @@ before new feature work starts. Companion to the Go rewrite tracker, - `warmup_cache()` runs at import of `app.py`, so `make test` makes real Drive and Sheets read calls and needs the credentials file. - Moot if Python is retired. -- [ ] **T18** Set the time zone in the Go container. +- [ ] **T18** Set the time zone in the Go container. Partly addressed + 2026-10-08: the binary now embeds the zone database and the scheduled sync + uses `SYNC_TZ`; the "current month" logic below is still open. - The image has no time zone configured, so `time.Now()` is UTC and the "current month" flips one to two hours late at month boundaries. - The Python image sets `Europe/Prague`. @@ -161,6 +163,12 @@ Plan: [2026-10-08-1103-api-parity-vs-months-to-show.md](2026-10-08-1103-api-pari dropped leading zeros. New rows now carry a text marker; existing rows are left as they are and are read correctly either way. +## Features added since the audit + +- 2026-10-08 — Scheduled bank sync in the Go server, deliberately Go-only. + Plan: [2026-10-08-1216-scheduled-bank-sync.md](2026-10-08-1216-scheduled-bank-sync.md). + It also added a lock so two syncs cannot overlap (relevant to T8). + ## Notes & decisions (Add entries as you go. Format: `YYYY-MM-DD — short note`.) diff --git a/docs/plans/2026-10-08-1216-scheduled-bank-sync.md b/docs/plans/2026-10-08-1216-scheduled-bank-sync.md new file mode 100644 index 0000000..ada11f7 --- /dev/null +++ b/docs/plans/2026-10-08-1216-scheduled-bank-sync.md @@ -0,0 +1,134 @@ +# Scheduled bank sync in the Go server (Go only) + +## Context + +Bank synchronization only runs when someone opens `/sync-bank`; the payments +sheet is as fresh as the last click. The Go server should run the same +sync → infer → flush-cache sequence by itself on a configured schedule, +defaulting to once a day. + +Decisions already made: + +- **Go only.** Deliberately not implemented in the legacy Python app, which is + being phased out (gunicorn's multi-worker model would also need a + cross-process lock). This is recorded as an explicit exception to the dual + implementation rule. +- **Schedule format:** one or more times of day. On by default, once a day. + +## Behaviour + +- `SYNC_SCHEDULE` — comma-separated `HH:MM` times, default `04:00`. + `off` disables the scheduler. An invalid value makes `fuj server` exit at + startup with a clear error. +- `SYNC_TZ` — IANA zone the times are read in, default `Europe/Prague`. + The binary embeds the zone database (`import _ "time/tzdata"`), because the + Alpine image has none. +- At each scheduled time the server runs exactly what `/sync-bank` runs + (`banksync.DefaultWindow`, `SyncToSheets`, `InferPayments`, cache flush), + with a 10-minute timeout, and logs start, result and row counts via `slog`. +- Nothing runs at startup; a restart only recomputes the next run time. +- A scheduled run and a manual `/sync-bank` can no longer overlap: both go + through one mutex. If a sync is already running, the manual page shows + "a sync is already running" and a scheduled run is skipped with a log line. + (This also closes today's gap where two overlapping manual runs could append + the same rows.) +- The footer of every page shows the last sync since the server started: + time, trigger (scheduled / manual), ok or failed, and the next scheduled + run. Status is in memory only and resets on restart. + +## Implementation + +Branch `feat/scheduled-bank-sync` off `main`; MR via `tea`. + +1. **Config** — `go/internal/config/config.go` + - Add `SyncSchedule []TimeOfDay` and `SyncLocation *time.Location` to + `Config`. + - Add `ParseSyncSchedule(s string) ([]TimeOfDay, error)` (`"off"` → empty, + sorted, de-duplicated, rejects anything not `HH:MM`) and a + `LoadSyncSchedule() (…, error)` reading `SYNC_SCHEDULE` / `SYNC_TZ`. + `Load()` keeps its signature; `serverCmd` calls the new loader and exits + on error. + - New `config_test.go` for the parser (first tests in this package). + +2. **Scheduler** — new package `go/internal/scheduler` + - `Next(now time.Time, times []TimeOfDay, loc *time.Location) time.Time` — + pure; builds candidates with `time.Date` in `loc` for today and tomorrow + and returns the earliest one strictly after `now` (handles DST days). + - `Run(ctx, times, loc, logger, job func(context.Context))` — loop: + compute `Next`, wait on a timer or `ctx.Done()`, call `job`. + - Tests: `Next` table test (before/after today's time, multiple times, + midnight rollover, the two DST switch days in Europe/Prague); `Run` with + a cancelled context returns. + +3. **Sync runner with lock and status** — new + `go/internal/services/banksync/runner.go` + - `Runner` wraps the existing sync+infer function (moved out of the + closure in `go/cmd/fuj/main.go`) with a `sync.Mutex` (`TryLock`) and + records `Status{Started, Finished, Trigger, Err}`. + - `Run(ctx, trigger string, out io.Writer) error` returns `ErrBusy` when a + run is in progress. `Last() (Status, bool)` for display. + - Tests with a fake job: status recorded, error recorded, second + concurrent call gets `ErrBusy`. + +4. **Wiring** — `go/cmd/fuj/main.go` + - Add `import _ "time/tzdata"`. + - Build one `Runner`; `ActionHandlers.BankSync` calls + `runner.Run(ctx, "manual", out)` (signature unchanged, so + `TestServeSync` keeps working). + - If the schedule is non-empty, start `go scheduler.Run(...)` whose job + calls `runner.Run(ctx, "scheduled", &buf)` with a 10-minute timeout, then + `sources.FlushCache()`, and logs the outcome. Log the schedule and next + run at startup, or "scheduled sync disabled". + +5. **Display** — `go/internal/web` + - `ActionHandlers` gains optional `SyncInfo func() string` (last + next run + as one line). `PageData` gains `SyncInfo string`; add a small + `h.page(active)` helper in `html_handler.go` and use it where + `PageData{Active: …, Build: h.build}` is built today. + - `templates/partials/footer.tmpl`: append the line when non-empty. + - `templates/sync.tmpl`: fix the description, which still says "current + year" (stale since the window change). + - Extend `html_handler_test.go`: footer shows the info when provided, + nothing extra when not. + +6. **Docs and notes** + - `CLAUDE.md`: document `SYNC_SCHEDULE` / `SYNC_TZ` under the Go config + list and the commands/architecture text; under "Dual implementation + rule" add a "Deliberately Go-only" list with this feature and the reason. + - `legacy/app.py`: one comment above `sync_bank()` stating that scheduled + sync exists only in Go, on purpose. + - `CHANGELOG.md` entry; copy this plan to + `docs/plans/2026-10-08-HHMM-scheduled-bank-sync.md`. + - Add a note to the cleanup task list: T18 (time zone) is addressed for + the scheduler only; the "current month" logic still uses the process + clock. + +No new dependencies. No Dockerfile change. No change to parity fixtures or +the JSON API. + +## Deployment notes (for the MR description) + +- On by default at 04:00 Europe/Prague. Set `SYNC_SCHEDULE=off` to disable. +- A locally running `make web-go` with real credentials will also sync at + that time. +- With more than one replica each pod would run its own schedule; the + deployment must stay at one replica (or disable the schedule on extras). +- Without `FIO_API_TOKEN_*` the sync scrapes the transparent-account pages, + as today. + +## Verification + +1. `make go-test-all` and `make go-lint` equivalent (`go vet`, `gofmt -l`); + `make test` to confirm Python is untouched. +2. Start `fuj server` with `SYNC_SCHEDULE=off` → log says disabled, footer + shows no schedule. +3. Start with an invalid value (`SYNC_SCHEDULE=25:00`) → exits with an error. +4. Live check: start with `SYNC_SCHEDULE` set two minutes ahead and watch the + log for the scheduled run and the footer for the status line. **This + performs one real sync** against the payments sheet (currently one pending + transaction would be appended) — the same thing a click on `/sync-bank` + does. +5. While that run is in progress, or with a test job, confirm a second + trigger reports "already running". +6. Build the Go image and confirm the schedule log line shows the Prague + zone inside the container (proves the embedded zone data works). diff --git a/go/cmd/fuj/main.go b/go/cmd/fuj/main.go index 7d54c93..bbc4f65 100644 --- a/go/cmd/fuj/main.go +++ b/go/cmd/fuj/main.go @@ -1,20 +1,25 @@ package main import ( + "bytes" "context" + "errors" "flag" "fmt" "fuj-management/go/internal/config" "fuj-management/go/internal/io/fio" "fuj-management/go/internal/io/sheets" "fuj-management/go/internal/logging" + "fuj-management/go/internal/scheduler" "fuj-management/go/internal/services/banksync" "fuj-management/go/internal/services/membership" "fuj-management/go/internal/web" "io" "log/slog" "os" + "strings" "time" + _ "time/tzdata" // the container image has no zone database; SYNC_TZ needs one ) // Injected at build time via -ldflags "-X main.version=... -X main.commit=... -X main.buildDate=..." @@ -90,8 +95,16 @@ func serverCmd(args []string) { } fioClients := buildFioClients(cfg) - actions := web.ActionHandlers{ - BankSync: func(ctx context.Context, out io.Writer) error { + syncTimes, syncLoc, err := config.LoadSyncSchedule() + if err != nil { + fmt.Fprintf(os.Stderr, "fuj server: %v\n", err) + os.Exit(2) + } + + // One runner serves both the /sync-bank page and the scheduler, so the two + // can never run at the same time. + runner := banksync.NewRunner( + func(ctx context.Context, out io.Writer) error { from, to := banksync.DefaultWindow(time.Now()) fmt.Fprintln(out, "=== Sync Fio Transactions ===") @@ -110,6 +123,22 @@ func serverCmd(args []string) { fmt.Fprintf(out, "Inferred %d row(s).\n", n) return nil }, + ) + + actions := web.ActionHandlers{ + BankSync: func(ctx context.Context, out io.Writer) error { + return runner.Run(ctx, "manual", out) + }, + SyncInfo: func() string { return syncInfo(runner, syncTimes, syncLoc, time.Now()) }, + } + + if len(syncTimes) == 0 { + logger.Info("scheduled bank sync disabled (SYNC_SCHEDULE=off)") + } else { + logger.Info("scheduled bank sync enabled", "times", fmt.Sprint(syncTimes), "tz", syncLoc.String()) + go scheduler.Run(ctx, syncTimes, syncLoc, logger, func(ctx context.Context) { + scheduledSync(ctx, logger, runner, sources) + }) } build := web.BuildInfo{Version: version, Commit: commit, BuildDate: buildDate} @@ -120,6 +149,55 @@ func serverCmd(args []string) { } } +// scheduledSyncTimeout bounds one scheduled run (two bank fetches plus sheet writes). +const scheduledSyncTimeout = 10 * time.Minute + +// scheduledSync is the scheduler's job: the same sync → infer → flush-cache +// sequence as the /sync-bank page, with the outcome written to the log. +func scheduledSync(ctx context.Context, logger *slog.Logger, runner *banksync.Runner, sources membership.Sources) { + ctx, cancel := context.WithTimeout(ctx, scheduledSyncTimeout) + defer cancel() + + logger.Info("scheduled bank sync starting") + var buf bytes.Buffer + err := runner.Run(ctx, "scheduled", &buf) + if errors.Is(err, banksync.ErrBusy) { + logger.Warn("scheduled bank sync skipped: a sync is already running") + return + } + output := strings.Join(strings.Fields(buf.String()), " ") + if _, ferr := sources.FlushCache(); ferr != nil { + logger.Error("scheduled bank sync: cache flush failed", "err", ferr) + } + if err != nil { + logger.Error("scheduled bank sync failed", "err", err, "output", output) + return + } + logger.Info("scheduled bank sync finished", "output", output) +} + +// syncInfo renders the footer line: the last run since the server started +// and the next scheduled one. +func syncInfo(runner *banksync.Runner, times []scheduler.TimeOfDay, loc *time.Location, now time.Time) string { + const layout = "2006-01-02 15:04" + var parts []string + if st, ok := runner.Last(); ok { + result := "ok" + if st.Err != nil { + result = "FAILED" + } + parts = append(parts, fmt.Sprintf("last sync %s (%s) %s", st.Finished.In(loc).Format(layout), st.Trigger, result)) + } else { + parts = append(parts, "no sync since start") + } + if next := scheduler.Next(now, times, loc); !next.IsZero() { + parts = append(parts, "next "+next.Format(layout)) + } else { + parts = append(parts, "scheduled sync off") + } + return strings.Join(parts, " | ") +} + func feesCmd(args []string) { fs := flag.NewFlagSet("fees", flag.ExitOnError) fs.Usage = func() { diff --git a/go/cmd/fuj/main_test.go b/go/cmd/fuj/main_test.go new file mode 100644 index 0000000..c6732ac --- /dev/null +++ b/go/cmd/fuj/main_test.go @@ -0,0 +1,35 @@ +package main + +import ( + "context" + "errors" + "fuj-management/go/internal/scheduler" + "fuj-management/go/internal/services/banksync" + "io" + "strings" + "testing" + "time" +) + +func TestSyncInfo(t *testing.T) { + now := time.Date(2026, 10, 8, 12, 0, 0, 0, time.UTC) + four := []scheduler.TimeOfDay{{Hour: 4}} + + ok := banksync.NewRunner(func(context.Context, io.Writer) error { return nil }) + if got := syncInfo(ok, four, time.UTC, now); got != "no sync since start | next 2026-10-09 04:00" { + t.Errorf("before any run: %q", got) + } + if err := ok.Run(context.Background(), "scheduled", io.Discard); err != nil { + t.Fatal(err) + } + got := syncInfo(ok, nil, time.UTC, now) + if !strings.HasPrefix(got, "last sync ") || !strings.HasSuffix(got, "(scheduled) ok | scheduled sync off") { + t.Errorf("after ok run, schedule off: %q", got) + } + + bad := banksync.NewRunner(func(context.Context, io.Writer) error { return errors.New("boom") }) + _ = bad.Run(context.Background(), "manual", io.Discard) + if got := syncInfo(bad, four, time.UTC, now); !strings.Contains(got, "(manual) FAILED | next 2026-10-09 04:00") { + t.Errorf("after failed run: %q", got) + } +} diff --git a/go/internal/config/config.go b/go/internal/config/config.go index f2cba64..1468099 100644 --- a/go/internal/config/config.go +++ b/go/internal/config/config.go @@ -1,6 +1,8 @@ package config import ( + "fmt" + "fuj-management/go/internal/scheduler" "os" "strconv" "strings" @@ -92,6 +94,28 @@ func Load() Config { } } +// Scheduled bank sync defaults (Go server only; the legacy Python app has no scheduler). +const ( + DefaultSyncSchedule = "04:00" // SYNC_SCHEDULE: comma-separated HH:MM times, or "off" + DefaultSyncTimezone = "Europe/Prague" // SYNC_TZ: IANA zone the schedule is read in +) + +// LoadSyncSchedule reads SYNC_SCHEDULE and SYNC_TZ. An empty schedule means +// the scheduled sync is disabled. Resolving the zone needs a zone database: +// the server binary embeds one via time/tzdata. +func LoadSyncSchedule() ([]scheduler.TimeOfDay, *time.Location, error) { + times, err := scheduler.ParseTimes(env("SYNC_SCHEDULE", DefaultSyncSchedule)) + if err != nil { + return nil, nil, fmt.Errorf("SYNC_SCHEDULE: %w", err) + } + tz := env("SYNC_TZ", DefaultSyncTimezone) + loc, err := time.LoadLocation(tz) + if err != nil { + return nil, nil, fmt.Errorf("SYNC_TZ: unknown time zone %q", tz) + } + return times, loc, nil +} + // IBANAccountNum extracts the bare account number from a Czech IBAN. // "CZ8520100000002800359168" → "2800359168" // Structure: CZ(2 check)(4 bank code)(16 zero-padded account). diff --git a/go/internal/config/config_test.go b/go/internal/config/config_test.go new file mode 100644 index 0000000..aa11de0 --- /dev/null +++ b/go/internal/config/config_test.go @@ -0,0 +1,45 @@ +package config + +import ( + "testing" +) + +func TestLoadSyncSchedule(t *testing.T) { + cases := []struct { + name, schedule, tz string + wantTimes int + wantZone string + wantErr bool + }{ + {"defaults", "", "", 1, "Europe/Prague", false}, + {"two times", "06:00,18:00", "", 2, "Europe/Prague", false}, + {"off", "off", "", 0, "Europe/Prague", false}, + {"other zone", "04:00", "UTC", 1, "UTC", false}, + {"bad time", "25:00", "", 0, "", true}, + {"bad zone", "04:00", "Mars/Olympus", 0, "", true}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + t.Setenv("SYNC_SCHEDULE", tc.schedule) + t.Setenv("SYNC_TZ", tc.tz) + times, loc, err := LoadSyncSchedule() + if (err != nil) != tc.wantErr { + t.Fatalf("err = %v, wantErr %v", err, tc.wantErr) + } + if tc.wantErr { + return + } + if len(times) != tc.wantTimes { + t.Errorf("times = %v, want %d entries", times, tc.wantTimes) + } + if loc.String() != tc.wantZone { + t.Errorf("zone = %s, want %s", loc, tc.wantZone) + } + }) + } + t.Setenv("SYNC_SCHEDULE", "") + t.Setenv("SYNC_TZ", "") + if times, _, _ := LoadSyncSchedule(); len(times) != 1 || times[0].String() != DefaultSyncSchedule { + t.Errorf("default schedule = %v, want [%s]", times, DefaultSyncSchedule) + } +} diff --git a/go/internal/scheduler/scheduler.go b/go/internal/scheduler/scheduler.go new file mode 100644 index 0000000..56f9deb --- /dev/null +++ b/go/internal/scheduler/scheduler.go @@ -0,0 +1,95 @@ +// Package scheduler runs a job at fixed times of day. +package scheduler + +import ( + "context" + "fmt" + "log/slog" + "sort" + "strings" + "time" +) + +// TimeOfDay is a wall-clock time without a date. +type TimeOfDay struct { + Hour, Minute int +} + +func (t TimeOfDay) String() string { return fmt.Sprintf("%02d:%02d", t.Hour, t.Minute) } + +// ParseTimes parses a comma-separated list of HH:MM times, e.g. "04:00" or +// "06:00,18:00". "off" yields an empty schedule. The result is sorted and +// de-duplicated. +func ParseTimes(s string) ([]TimeOfDay, error) { + s = strings.TrimSpace(s) + if strings.EqualFold(s, "off") { + return nil, nil + } + seen := map[TimeOfDay]bool{} + var times []TimeOfDay + for _, part := range strings.Split(s, ",") { + part = strings.TrimSpace(part) + parsed, err := time.Parse("15:04", part) + if err != nil { + return nil, fmt.Errorf("invalid time %q: want HH:MM, a comma-separated list of them, or \"off\"", part) + } + t := TimeOfDay{Hour: parsed.Hour(), Minute: parsed.Minute()} + if !seen[t] { + seen[t] = true + times = append(times, t) + } + } + sort.Slice(times, func(i, j int) bool { + if times[i].Hour != times[j].Hour { + return times[i].Hour < times[j].Hour + } + return times[i].Minute < times[j].Minute + }) + return times, nil +} + +// Next returns the earliest scheduled instant strictly after now, with the +// times read in loc. It returns the zero time when times is empty. +func Next(now time.Time, times []TimeOfDay, loc *time.Location) time.Time { + local := now.In(loc) + var best time.Time + for dayOffset := 0; dayOffset <= 1; dayOffset++ { + for _, t := range times { + c := time.Date(local.Year(), local.Month(), local.Day()+dayOffset, t.Hour, t.Minute, 0, 0, loc) + if c.After(now) && (best.IsZero() || c.Before(best)) { + best = c + } + } + } + return best +} + +// pollInterval bounds each wait so the loop follows the wall clock across +// clock adjustments and host suspend instead of relying on one long timer. +const pollInterval = time.Minute + +// Run calls job at every scheduled time until ctx is cancelled. Nothing runs +// at startup. Runs do not overlap: the next time is computed after job returns. +func Run(ctx context.Context, times []TimeOfDay, loc *time.Location, logger *slog.Logger, job func(context.Context)) { + for { + next := Next(time.Now(), times, loc) + if next.IsZero() { + return + } + logger.Info("next scheduled run", "at", next.Format(time.RFC3339)) + for { + wait := time.Until(next) + if wait <= 0 { + break + } + timer := time.NewTimer(min(wait, pollInterval)) + select { + case <-ctx.Done(): + timer.Stop() + return + case <-timer.C: + } + } + job(ctx) + } +} diff --git a/go/internal/scheduler/scheduler_test.go b/go/internal/scheduler/scheduler_test.go new file mode 100644 index 0000000..c4ecb19 --- /dev/null +++ b/go/internal/scheduler/scheduler_test.go @@ -0,0 +1,115 @@ +package scheduler + +import ( + "context" + "io" + "log/slog" + "reflect" + "testing" + "time" +) + +func TestParseTimes(t *testing.T) { + cases := []struct { + in string + want []TimeOfDay + wantErr bool + }{ + {"04:00", []TimeOfDay{{4, 0}}, false}, + {" 18:30 , 06:00,06:00 ", []TimeOfDay{{6, 0}, {18, 30}}, false}, + {"off", nil, false}, + {"OFF", nil, false}, + {"", nil, true}, + {"25:00", nil, true}, + {"04:60", nil, true}, + {"daily", nil, true}, + {"04:00,", nil, true}, + } + for _, tc := range cases { + got, err := ParseTimes(tc.in) + if (err != nil) != tc.wantErr { + t.Errorf("ParseTimes(%q): err = %v, wantErr %v", tc.in, err, tc.wantErr) + continue + } + if !reflect.DeepEqual(got, tc.want) { + t.Errorf("ParseTimes(%q) = %v, want %v", tc.in, got, tc.want) + } + } +} + +func TestNext(t *testing.T) { + prague, err := time.LoadLocation("Europe/Prague") + if err != nil { + t.Fatal(err) + } + at := func(s string) time.Time { + v, err := time.ParseInLocation("2006-01-02 15:04:05", s, prague) + if err != nil { + t.Fatal(err) + } + return v + } + four := []TimeOfDay{{4, 0}} + cases := []struct { + name string + now time.Time + times []TimeOfDay + want time.Time + }{ + {"before today's time", at("2026-10-08 03:59:59"), four, at("2026-10-08 04:00:00")}, + {"exactly at the time moves to tomorrow", at("2026-10-08 04:00:00"), four, at("2026-10-09 04:00:00")}, + {"after today's time", at("2026-10-08 12:00:00"), four, at("2026-10-09 04:00:00")}, + {"picks the next of several", at("2026-10-08 12:00:00"), []TimeOfDay{{6, 0}, {18, 0}}, at("2026-10-08 18:00:00")}, + {"year rollover", at("2026-12-31 23:00:00"), four, at("2027-01-01 04:00:00")}, + {"now given in UTC", at("2026-10-08 03:00:00").UTC(), four, at("2026-10-08 04:00:00")}, + // Spring forward (2026-03-29 02:00 → 03:00): still 04:00 local, 23h after the previous run. + {"spring forward", at("2026-03-28 04:00:00"), four, at("2026-03-29 04:00:00")}, + // Fall back (2026-10-25 03:00 → 02:00): still 04:00 local, 25h after the previous run. + {"fall back", at("2026-10-24 04:00:00"), four, at("2026-10-25 04:00:00")}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + got := Next(tc.now, tc.times, prague) + if !got.Equal(tc.want) { + t.Errorf("Next = %s, want %s", got, tc.want) + } + }) + } + if got := Next(at("2026-10-08 12:00:00"), nil, prague); !got.IsZero() { + t.Errorf("Next with empty schedule = %s, want zero", got) + } + if d := at("2026-03-29 04:00:00").Sub(at("2026-03-28 04:00:00")); d != 23*time.Hour { + t.Errorf("spring-forward gap = %s, want 23h", d) + } +} + +func TestRunStopsOnCancelAndEmptySchedule(t *testing.T) { + logger := slog.New(slog.NewTextHandler(io.Discard, nil)) + called := false + job := func(context.Context) { called = true } + + done := make(chan struct{}) + go func() { Run(context.Background(), nil, time.UTC, logger, job); close(done) }() + select { + case <-done: + case <-time.After(2 * time.Second): + t.Fatal("Run with empty schedule did not return") + } + + ctx, cancel := context.WithCancel(context.Background()) + done = make(chan struct{}) + far := time.Now().UTC().Add(12 * time.Hour) + go func() { + Run(ctx, []TimeOfDay{{far.Hour(), far.Minute()}}, time.UTC, logger, job) + close(done) + }() + cancel() + select { + case <-done: + case <-time.After(2 * time.Second): + t.Fatal("Run did not return after cancel") + } + if called { + t.Error("job must not run before its scheduled time") + } +} diff --git a/go/internal/services/banksync/runner.go b/go/internal/services/banksync/runner.go new file mode 100644 index 0000000..61ee7cd --- /dev/null +++ b/go/internal/services/banksync/runner.go @@ -0,0 +1,63 @@ +package banksync + +import ( + "context" + "errors" + "io" + "sync" + "time" +) + +// ErrBusy is returned by Runner.Run when another run is still in progress. +var ErrBusy = errors.New("a sync is already running") + +// RunStatus describes one finished run of a Runner. +type RunStatus struct { + Started, Finished time.Time + Trigger string // "manual" or "scheduled" + Err error +} + +// Runner serialises bank sync runs so a scheduled run and a manual one never +// overlap (overlapping runs could append the same rows twice), and remembers +// the outcome of the last run for display. +type Runner struct { + job func(ctx context.Context, out io.Writer) error + + running sync.Mutex + + mu sync.Mutex + last RunStatus + hasLast bool +} + +// NewRunner wraps job, which performs one full sync and writes a +// human-readable log to out. +func NewRunner(job func(ctx context.Context, out io.Writer) error) *Runner { + return &Runner{job: job} +} + +// Run executes the job unless one is already running, in which case it +// returns ErrBusy without waiting. +func (r *Runner) Run(ctx context.Context, trigger string, out io.Writer) error { + if !r.running.TryLock() { + return ErrBusy + } + defer r.running.Unlock() + + st := RunStatus{Started: time.Now(), Trigger: trigger} + st.Err = r.job(ctx, out) + st.Finished = time.Now() + + r.mu.Lock() + r.last, r.hasLast = st, true + r.mu.Unlock() + return st.Err +} + +// Last returns the outcome of the most recent finished run, if any. +func (r *Runner) Last() (RunStatus, bool) { + r.mu.Lock() + defer r.mu.Unlock() + return r.last, r.hasLast +} diff --git a/go/internal/services/banksync/runner_test.go b/go/internal/services/banksync/runner_test.go new file mode 100644 index 0000000..0340c4c --- /dev/null +++ b/go/internal/services/banksync/runner_test.go @@ -0,0 +1,65 @@ +package banksync + +import ( + "context" + "errors" + "io" + "testing" +) + +func TestRunner_RecordsStatus(t *testing.T) { + r := NewRunner(func(context.Context, io.Writer) error { return nil }) + if _, ok := r.Last(); ok { + t.Fatal("Last before any run: want ok=false") + } + if err := r.Run(context.Background(), "scheduled", io.Discard); err != nil { + t.Fatal(err) + } + st, ok := r.Last() + if !ok || st.Trigger != "scheduled" || st.Err != nil || st.Finished.Before(st.Started) { + t.Errorf("unexpected status %+v ok=%v", st, ok) + } +} + +func TestRunner_RecordsError(t *testing.T) { + boom := errors.New("boom") + r := NewRunner(func(context.Context, io.Writer) error { return boom }) + if err := r.Run(context.Background(), "manual", io.Discard); !errors.Is(err, boom) { + t.Fatalf("Run err = %v, want boom", err) + } + if st, _ := r.Last(); !errors.Is(st.Err, boom) || st.Trigger != "manual" { + t.Errorf("unexpected status %+v", st) + } +} + +func TestRunner_BusyWhileRunning(t *testing.T) { + started := make(chan struct{}) + release := make(chan struct{}) + first := true + r := NewRunner(func(context.Context, io.Writer) error { + if first { // runs are serialised by Runner, so no extra locking is needed + first = false + close(started) + <-release + } + return nil + }) + done := make(chan error, 1) + go func() { done <- r.Run(context.Background(), "scheduled", io.Discard) }() + <-started + + if err := r.Run(context.Background(), "manual", io.Discard); !errors.Is(err, ErrBusy) { + t.Errorf("second Run err = %v, want ErrBusy", err) + } + close(release) + if err := <-done; err != nil { + t.Fatal(err) + } + // The skipped run must not overwrite the status of the real one. + if st, _ := r.Last(); st.Trigger != "scheduled" { + t.Errorf("Last trigger = %q, want scheduled", st.Trigger) + } + if err := r.Run(context.Background(), "manual", io.Discard); err != nil { + t.Errorf("Run after release: %v", err) + } +} diff --git a/go/internal/web/html_handler.go b/go/internal/web/html_handler.go index de36ab3..6c8704b 100644 --- a/go/internal/web/html_handler.go +++ b/go/internal/web/html_handler.go @@ -21,17 +21,26 @@ func NewHTMLHandler(r *Renderer, b BuildInfo, ah *api.Handler, actions ActionHan return &HTMLHandler{renderer: r, build: b, apiHandler: ah, actions: actions} } +// page builds the data shared by every page (nav highlight, build info, sync summary). +func (h *HTMLHandler) page(active string) PageData { + pd := PageData{Active: active, Build: h.build} + if h.actions.SyncInfo != nil { + pd.SyncInfo = h.actions.SyncInfo() + } + return pd +} + func (h *HTMLHandler) ServeAdults(w http.ResponseWriter, r *http.Request) { data, err := h.apiHandler.AssembleAdults(r.Context()) if err != nil { h.renderer.Render(w, "adults", AdultsPageData{ - PageData: PageData{Active: "adults", Build: h.build}, + PageData: h.page("adults"), Error: err.Error(), }) return } h.renderer.Render(w, "adults", AdultsPageData{ - PageData: PageData{Active: "adults", Build: h.build}, + PageData: h.page("adults"), Data: data, }) } @@ -40,13 +49,13 @@ func (h *HTMLHandler) ServeJuniors(w http.ResponseWriter, r *http.Request) { data, err := h.apiHandler.AssembleJuniors(r.Context()) if err != nil { h.renderer.Render(w, "juniors", JuniorsPageData{ - PageData: PageData{Active: "juniors", Build: h.build}, + PageData: h.page("juniors"), Error: err.Error(), }) return } h.renderer.Render(w, "juniors", JuniorsPageData{ - PageData: PageData{Active: "juniors", Build: h.build}, + PageData: h.page("juniors"), Data: data, }) } @@ -55,20 +64,20 @@ func (h *HTMLHandler) ServePayments(w http.ResponseWriter, r *http.Request) { data, err := h.apiHandler.AssemblePayments(r.Context()) if err != nil { h.renderer.Render(w, "payments", PaymentsPageData{ - PageData: PageData{Active: "payments", Build: h.build}, + PageData: h.page("payments"), Error: err.Error(), }) return } h.renderer.Render(w, "payments", PaymentsPageData{ - PageData: PageData{Active: "payments", Build: h.build}, + PageData: h.page("payments"), Data: data, }) } // ServeSync handles GET /sync-bank: runs sync+infer+flush then renders the result. func (h *HTMLHandler) ServeSync(w http.ResponseWriter, r *http.Request) { - pd := PageData{Active: "sync", Build: h.build} + pd := h.page("sync") if h.actions.BankSync == nil { h.renderer.Render(w, "sync", SyncPageData{ @@ -96,7 +105,7 @@ func (h *HTMLHandler) ServeSync(w http.ResponseWriter, r *http.Request) { } h.renderer.Render(w, "sync", SyncPageData{ - PageData: pd, + PageData: h.page("sync"), // rebuilt so the footer reflects this run Output: buf.String(), Success: success, }) @@ -105,7 +114,7 @@ func (h *HTMLHandler) ServeSync(w http.ResponseWriter, r *http.Request) { // ServeFlushCacheGET handles GET /flush-cache: renders the confirmation form. func (h *HTMLHandler) ServeFlushCacheGET(w http.ResponseWriter, r *http.Request) { h.renderer.Render(w, "flush_cache", FlushPageData{ - PageData: PageData{Active: "flush", Build: h.build}, + PageData: h.page("flush"), }) } @@ -113,7 +122,7 @@ func (h *HTMLHandler) ServeFlushCacheGET(w http.ResponseWriter, r *http.Request) func (h *HTMLHandler) ServeFlushCachePOST(w http.ResponseWriter, r *http.Request) { n, _ := h.apiHandler.FlushCache() h.renderer.Render(w, "flush_cache", FlushPageData{ - PageData: PageData{Active: "flush", Build: h.build}, + PageData: h.page("flush"), Flushed: true, Deleted: n, }) diff --git a/go/internal/web/html_handler_test.go b/go/internal/web/html_handler_test.go index e15c2c1..362caac 100644 --- a/go/internal/web/html_handler_test.go +++ b/go/internal/web/html_handler_test.go @@ -314,6 +314,28 @@ func TestServeSync(t *testing.T) { } } +func TestFooterSyncInfo(t *testing.T) { + renderer, err := web.NewRenderer() + if err != nil { + t.Fatalf("NewRenderer: %v", err) + } + b := web.BuildInfo{Version: "v0", Commit: "abc1234", BuildDate: "2026-01-01"} + get := func(actions web.ActionHandlers) string { + h := web.NewHTMLHandler(renderer, b, fixtureHandler(t), actions) + w := httptest.NewRecorder() + h.ServeFlushCacheGET(w, httptest.NewRequest(http.MethodGet, "/flush-cache", nil)) + return w.Body.String() + } + + const info = "last sync 2026-10-08 04:00 (scheduled) ok" + if body := get(web.ActionHandlers{SyncInfo: func() string { return info }}); !strings.Contains(body, "| "+info) { + t.Errorf("footer missing sync info %q", info) + } + if body := get(web.ActionHandlers{}); strings.Contains(body, "last sync") { + t.Errorf("footer shows sync info although none was provided") + } +} + func TestPaymentQRMarkup(t *testing.T) { renderer, err := web.NewRenderer() if err != nil { diff --git a/go/internal/web/render.go b/go/internal/web/render.go index 6990a1c..746d552 100644 --- a/go/internal/web/render.go +++ b/go/internal/web/render.go @@ -10,8 +10,9 @@ import ( // PageData is the view model passed to every HTML template. type PageData struct { - Active string - Build BuildInfo + Active string + Build BuildInfo + SyncInfo string // last/next bank sync summary shown in the footer; empty hides it } // AdultsPageData is the view model for the /adults HTML page. diff --git a/go/internal/web/server.go b/go/internal/web/server.go index 15f7998..679b66c 100644 --- a/go/internal/web/server.go +++ b/go/internal/web/server.go @@ -26,6 +26,9 @@ type ActionHandlers struct { // BankSync runs sync+infer and writes a human-readable log to out. // nil disables the /sync-bank action (renders an error instead). BankSync func(ctx context.Context, out io.Writer) error + // SyncInfo returns a one-line summary of the last and next bank sync for + // the page footer. nil or an empty result shows nothing. + SyncInfo func() string } // Run registers routes and starts the HTTP server on addr. diff --git a/go/internal/web/templates/partials/footer.tmpl b/go/internal/web/templates/partials/footer.tmpl index 7421db0..8ac7bfe 100644 --- a/go/internal/web/templates/partials/footer.tmpl +++ b/go/internal/web/templates/partials/footer.tmpl @@ -1,3 +1,3 @@ {{define "footer"}} -
+ {{end}} diff --git a/go/internal/web/templates/sync.tmpl b/go/internal/web/templates/sync.tmpl index 3f4f7ba..2de6dc2 100644 --- a/go/internal/web/templates/sync.tmpl +++ b/go/internal/web/templates/sync.tmpl @@ -9,7 +9,7 @@ {{end}}{{.Output}}
{{else}}
- Fetches Fio transactions for the current year, infers payment details, and flushes the cache.
+Fetches Fio transactions for the current year and the last two months of the previous one, infers payment details, and flushes the cache.
{{end}} {{end}} diff --git a/legacy/app.py b/legacy/app.py index 9134392..501cb39 100644 --- a/legacy/app.py +++ b/legacy/app.py @@ -121,6 +121,8 @@ def flush_cache_endpoint(): deleted = flush_cache() return render_template("flush-cache.html", flushed=True, deleted=deleted) +# Scheduled (daily) bank sync exists only in the Go server, on purpose: this +# app is legacy and being phased out. Here the sync runs only on request. @app.route("/sync-bank") def sync_bank(): import contextlib