Compare commits
2 Commits
feat/multi
...
152908fec6
| Author | SHA1 | Date | |
|---|---|---|---|
| 152908fec6 | |||
| fbc5a41d12 |
1
.gitignore
vendored
1
.gitignore
vendored
@@ -7,3 +7,4 @@ tmp/
|
||||
|
||||
# go build output
|
||||
bin/
|
||||
go/parity
|
||||
|
||||
10
CHANGELOG.md
10
CHANGELOG.md
@@ -1,15 +1,5 @@
|
||||
# Changelog
|
||||
|
||||
## 2026-05-24 21:42 CEST — feat: multi-account Fio sync + switch QR default to 2502035405/2010
|
||||
|
||||
- Added second bank account `2502035405/2010` (IBAN `CZ0820100000002502035405`) to sync.
|
||||
- Both accounts are fetched on every sync; dedup by existing sync_id keeps the payments sheet clean.
|
||||
- QR codes now default to the new account (`CZ0820100000002502035405`).
|
||||
- Go: `config.go` gains hardcoded `Accounts`/`LoadedAccount` slice; `Config.BankAccount` renamed to `Config.QRAccount`; `FioAPIToken` removed (tokens are per-account via `FIO_API_TOKEN_NEW` / `FIO_API_TOKEN_OLD`).
|
||||
- Go: `SyncToSheets` now accepts `[]fio.Client`; new `TestSyncToSheets_MultiAccount` test.
|
||||
- Python: `config.py` gains `ACCOUNTS` / `LOADED_ACCOUNTS`; `fio_utils.py` adds `fetch_transactions_for` and `fetch_transactions_all`; `sync_fio_to_sheets.py` uses `fetch_transactions_all`.
|
||||
- Key files: `go/internal/config/config.go`, `go/internal/services/banksync/sync.go`, `go/cmd/fuj/main.go`, `scripts/config.py`, `scripts/fio_utils.py`, `scripts/sync_fio_to_sheets.py`.
|
||||
|
||||
## 2026-05-11 23:58 CEST — fix(reconcile): fill earliest month deficit first in multi-month allocations
|
||||
|
||||
- Multi-month payment allocation now fills the earliest in-window deficit first and spills
|
||||
|
||||
@@ -1,269 +0,0 @@
|
||||
# Multi-account Fio sync + switch QR target to 2502035405/2010
|
||||
|
||||
## Context
|
||||
|
||||
The club is opening a second Fio transparent account, **`2502035405/2010`** (IBAN
|
||||
`CZ??...0000002502035405`). Going forward:
|
||||
|
||||
- **Sync** must pull transactions from **both** accounts into the existing
|
||||
payments sheet — the old `2800359168/2010` stays active so historical fees
|
||||
paid to it still reconcile.
|
||||
- **QR codes** generated by `/qr` should default to the **new** account
|
||||
(`2502035405/2010`), so new payers send money there.
|
||||
|
||||
The payments sheet schema and `sync_id` hash stay **unchanged** — no
|
||||
"Source Account" column, no migration, no risk of re-appending historical rows.
|
||||
Bank-ID uniqueness from Fio plus the existing hash inputs are good enough to
|
||||
dedupe across two transparent accounts in practice.
|
||||
|
||||
Implementation order: Go first, then port to Python. Both apps use the same
|
||||
`PaymentsSheetID`, so once Go writes from two accounts the Python code only
|
||||
needs to do the same when invoked.
|
||||
|
||||
### Decisions locked in with the user
|
||||
- **Config style:** hardcoded list of accounts in `config.go` / `config.py`
|
||||
(sheet IDs are already hardcoded; tokens stay env-driven).
|
||||
- **Source tagging:** no schema change, `sync_id` formula unchanged.
|
||||
- **Old account:** kept syncing in parallel; only QR target changes.
|
||||
|
||||
---
|
||||
|
||||
## Go implementation
|
||||
|
||||
### 1. Config — hardcoded `Accounts` list, derived `QRAccount`
|
||||
|
||||
[go/internal/config/config.go](../../go/internal/config/config.go)
|
||||
|
||||
- Replace scalar `BankAccount` + `FioAPIToken` with a hardcoded slice and a
|
||||
primary pointer. Tokens stay env-driven via per-account env names.
|
||||
|
||||
```go
|
||||
// Account describes one Fio bank account we sync from.
|
||||
type Account struct {
|
||||
IBAN string // e.g. "CZ0820100000002502035405"
|
||||
AcctNum string // bare account number, e.g. "2502035405"
|
||||
TokenEnv string // env var name holding the Fio API token (empty token => transparent scraper path)
|
||||
Primary bool // true for the QR-code default account
|
||||
}
|
||||
|
||||
var Accounts = []Account{
|
||||
{IBAN: "CZ0820100000002502035405", AcctNum: "2502035405", TokenEnv: "FIO_API_TOKEN_NEW", Primary: true},
|
||||
{IBAN: "CZ8520100000002800359168", AcctNum: "2800359168", TokenEnv: "FIO_API_TOKEN_OLD"},
|
||||
}
|
||||
```
|
||||
Compute the **real** IBAN check digit for `2502035405` once and bake it in
|
||||
(replace the `85` placeholder above). The same `IBANAccountNum` helper at
|
||||
[config.go:66](../../go/internal/config/config.go#L66) keeps producing
|
||||
`AcctNum`, but having it on the struct avoids re-parsing at every call site.
|
||||
|
||||
- `Config` struct:
|
||||
- Drop `BankAccount` and `FioAPIToken` fields.
|
||||
- Add `QRAccount string` (the IBAN of `Accounts[i].Primary == true`).
|
||||
- Add `LoadedAccounts []LoadedAccount` where each entry pairs an `Account`
|
||||
with the token resolved from `os.Getenv(account.TokenEnv)`.
|
||||
- `Load()` reads each `TokenEnv` and stores tokens with the account. Logs a
|
||||
warning (info-level) if no token is set — that's the transparent-scraper
|
||||
path, which is fine.
|
||||
|
||||
### 2. Sync loop — one `fio.Client` per account, single dedup pass
|
||||
|
||||
[go/internal/services/banksync/sync.go](../../go/internal/services/banksync/sync.go)
|
||||
|
||||
Change `SyncToSheets` to accept a slice of clients instead of one:
|
||||
|
||||
```go
|
||||
func SyncToSheets(
|
||||
ctx context.Context,
|
||||
spreadsheetID string,
|
||||
fioClients []fio.Client, // was: fioClient fio.Client
|
||||
sh sheetsWriter,
|
||||
opts SyncOpts,
|
||||
) (int, error)
|
||||
```
|
||||
|
||||
Inside:
|
||||
|
||||
- Keep steps 1 (read existing IDs) and 2 (date window) as-is.
|
||||
- Step 3 (fetch) becomes a loop: call `FetchTransactions` on each client; tag
|
||||
each `fio.Transaction` with the account it came from for logging only
|
||||
(don't write it to the sheet). Concatenate into one slice `txns`.
|
||||
- Steps 4a–4c (sync_id calc, debug table, row build, dedup) are unchanged —
|
||||
the combined `txns` slice flows through the same code path. Existing
|
||||
`sync_id` collisions across accounts are dropped silently, which is the
|
||||
desired dedup behaviour.
|
||||
- `printFioTable` (dry-run debug) should print an extra column showing the
|
||||
source account so per-account fetches are visible — small change in
|
||||
[fio_table.go](../../go/internal/services/banksync/fio_table.go).
|
||||
|
||||
Logger output: log fetched count per account and the total combined count.
|
||||
|
||||
### 3. Call sites — build clients from the config list
|
||||
|
||||
Both entry points construct `[]fio.Client`, one per `LoadedAccount`:
|
||||
|
||||
- [go/cmd/fuj/main.go:91](../../go/cmd/fuj/main.go#L91) — server `BankSync` action.
|
||||
- [go/cmd/fuj/main.go:200](../../go/cmd/fuj/main.go#L200) — `fuj sync` CLI.
|
||||
|
||||
Helper to add somewhere in `cmd/fuj/main.go` or a new
|
||||
`internal/services/banksync/clients.go`:
|
||||
|
||||
```go
|
||||
func clientsFromConfig(cfg config.Config) []fio.Client {
|
||||
out := make([]fio.Client, 0, len(cfg.LoadedAccounts))
|
||||
for _, a := range cfg.LoadedAccounts {
|
||||
out = append(out, fio.New(a.Token, a.AcctNum, nil))
|
||||
}
|
||||
return out
|
||||
}
|
||||
```
|
||||
|
||||
The existing factory at
|
||||
[go/internal/io/fio/client.go:35](../../go/internal/io/fio/client.go#L35)
|
||||
already routes per-account between the API path and the transparent scraper
|
||||
based on whether the token is empty — no change there.
|
||||
|
||||
### 4. QR endpoint — default to `QRAccount`
|
||||
|
||||
[go/internal/web/html_handler.go:129](../../go/internal/web/html_handler.go#L129)
|
||||
and [html_handler.go:135](../../go/internal/web/html_handler.go#L135):
|
||||
swap `h.apiHandler.Config.BankAccount` → `h.apiHandler.Config.QRAccount`.
|
||||
|
||||
API DTOs that surface the payment account in JSON:
|
||||
[build_adults.go:141](../../go/internal/web/api/build_adults.go#L141) and
|
||||
[build_juniors.go:137](../../go/internal/web/api/build_juniors.go#L137) — same
|
||||
swap.
|
||||
|
||||
The fields on the API structs at
|
||||
[adults.go:40](../../go/internal/web/api/adults.go#L40) and
|
||||
[juniors.go:39](../../go/internal/web/api/juniors.go#L39) keep their JSON
|
||||
names so the Python parity binary doesn't see a schema diff.
|
||||
|
||||
### 5. Tests
|
||||
|
||||
- **New** [go/internal/services/banksync/sync_test.go](../../go/internal/services/banksync/sync_test.go):
|
||||
add a test `TestSyncToSheets_MultiAccount` that wires up two fake
|
||||
`fio.Client`s (via the existing
|
||||
[fake.go](../../go/internal/io/fio/fake.go)), one returning a txn unique to
|
||||
it and one returning a txn that duplicates an existing sync_id. Assert:
|
||||
rows from both accounts get appended, the duplicate is dropped.
|
||||
- **Update** [go/internal/web/html_handler_test.go:207](../../go/internal/web/html_handler_test.go#L207)
|
||||
`TestServeQR` and [qr_test.go](../../go/internal/web/qr_test.go)
|
||||
`TestQRBuildSPD` — expected SPD strings now contain the new IBAN by default.
|
||||
- **Update** any config tests asserting `cfg.BankAccount` directly to use
|
||||
`cfg.QRAccount` and the new `LoadedAccounts` slice. (Grep for `BankAccount`
|
||||
under `go/` after the rename.)
|
||||
- Existing fio parser tests
|
||||
([fio_test.go](../../go/internal/io/fio/fio_test.go)) and sync_id test
|
||||
([synch_test.go](../../go/internal/domain/synch/synch_test.go)) need no
|
||||
changes — they don't know about the account list.
|
||||
|
||||
### 6. Verification (Go)
|
||||
|
||||
1. `cd go && go build ./... && go test ./...` — all green.
|
||||
2. `FIO_API_TOKEN_NEW="" FIO_API_TOKEN_OLD="" ./build/fuj sync --dry-run --print-fio-table --days 7`
|
||||
— see fetched transactions from **both** accounts, with NEW/DUP status,
|
||||
without writing.
|
||||
3. `./build/fuj server` then visit
|
||||
`http://localhost:8080/qr?amount=700&message=test` — QR payload SPD must
|
||||
contain the new IBAN `CZ??...2502035405`.
|
||||
4. Real sync once dry-run looks right: `./build/fuj sync --days 30` — confirm
|
||||
in the Google sheet that rows from the old account still appear and no
|
||||
duplicates are written.
|
||||
|
||||
---
|
||||
|
||||
## Python port (after Go is merged)
|
||||
|
||||
The Python app uses the same payments sheet and the same column schema, so
|
||||
the port mirrors the Go change one-for-one. Goal: one round-trip of
|
||||
`make sync-2026` pulls from both accounts.
|
||||
|
||||
### 1. `scripts/config.py`
|
||||
|
||||
- Replace the scalar `BANK_ACCOUNT` constant at
|
||||
[scripts/config.py:25](../../scripts/config.py#L25) with a hardcoded list
|
||||
plus a derived primary:
|
||||
|
||||
```python
|
||||
ACCOUNTS = [
|
||||
{"iban": "CZ0820100000002502035405", "acct_num": "2502035405", "token_env": "FIO_API_TOKEN_NEW", "primary": True},
|
||||
{"iban": "CZ8520100000002800359168", "acct_num": "2800359168", "token_env": "FIO_API_TOKEN_OLD", "primary": False},
|
||||
]
|
||||
BANK_ACCOUNT = next(a["iban"] for a in ACCOUNTS if a["primary"]) # QR / template default
|
||||
```
|
||||
Use the real check-digit IBAN for `2502035405` (the same one baked into Go).
|
||||
|
||||
- Resolve tokens at module load time with `os.environ.get(token_env, "")` and
|
||||
stash them on a parallel `LOADED_ACCOUNTS` list (or attach to the dicts) so
|
||||
other modules don't all re-read env.
|
||||
|
||||
### 2. `scripts/fio_utils.py`
|
||||
|
||||
- [fio_utils.py:106](../../scripts/fio_utils.py#L106) and
|
||||
[fio_utils.py:219](../../scripts/fio_utils.py#L219): drop the hardcoded
|
||||
`"2800359168"` default for `account_id`. Make it a required argument.
|
||||
- `fetch_transactions(date_from, date_to, *, account)` — change signature to
|
||||
take one account dict, return that account's transactions. Internally still
|
||||
routes via `FIO_API_TOKEN` if the account dict has a token, otherwise the
|
||||
transparent scraper at
|
||||
[fio_utils.py:105](../../scripts/fio_utils.py#L105).
|
||||
- Add `fetch_transactions_all(date_from, date_to, accounts=None)` that loops
|
||||
over `accounts` (default `config.LOADED_ACCOUNTS`), calls the per-account
|
||||
fetch, concatenates results. Logs per-account counts.
|
||||
|
||||
### 3. `scripts/sync_fio_to_sheets.py`
|
||||
|
||||
- Where it currently calls `fio_utils.fetch_transactions(...)`, switch to
|
||||
`fio_utils.fetch_transactions_all(...)`. The dedup at
|
||||
[sync_fio_to_sheets.py:62](../../scripts/sync_fio_to_sheets.py#L62) and
|
||||
append at
|
||||
[sync_fio_to_sheets.py:248](../../scripts/sync_fio_to_sheets.py#L248) need
|
||||
no changes — same as in Go, the combined list flows through unchanged.
|
||||
|
||||
### 4. `app.py` (QR + template defaults)
|
||||
|
||||
- `/qr` at [app.py:321](../../app.py#L321) already reads `BANK_ACCOUNT` from
|
||||
config; since the config default now points at the new IBAN, this is a
|
||||
zero-line change in `app.py`.
|
||||
- The `BANK_ACCOUNT` passed into templates at
|
||||
[app.py:180](../../app.py#L180), [app.py:204](../../app.py#L204),
|
||||
[app.py:255](../../app.py#L255), [app.py:291](../../app.py#L291) likewise
|
||||
picks up the new default automatically.
|
||||
|
||||
### 5. Tests
|
||||
|
||||
The Python suite has **no** existing coverage of `fio_utils`,
|
||||
`sync_fio_to_sheets`, or `/qr`. Don't add new tests — keep the port minimal
|
||||
and rely on the Go test suite + manual verification for correctness.
|
||||
|
||||
### 6. Verification (Python)
|
||||
|
||||
1. `make test` — existing suite stays green (the config refactor must not
|
||||
break [tests/test_app.py](../../tests/test_app.py)).
|
||||
2. `make sync-2026` against the real sheet, dry-run first if the script
|
||||
supports it, then real run — confirm rows from both accounts arrive.
|
||||
3. `make web-debug` → open `/qr?amount=700&message=test` → QR payload SPD
|
||||
contains the new IBAN.
|
||||
|
||||
---
|
||||
|
||||
## Branching
|
||||
|
||||
Per `CLAUDE.md`: this is a feature, so `feat/multi-account-bank-sync` off
|
||||
`main`. Two MRs:
|
||||
|
||||
1. `feat/multi-account-bank-sync-go` — Go change end-to-end.
|
||||
2. `feat/multi-account-bank-sync-py` — Python port, opened after (1) is
|
||||
merged.
|
||||
|
||||
Each MR opened via `tea pr create`. CHANGELOG.md entry per MR.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- No sheet schema migration.
|
||||
- No change to `sync_id` hash inputs at
|
||||
[domain/synch/synch.go:14](../../go/internal/domain/synch/synch.go#L14) or
|
||||
[sync_fio_to_sheets.py:67](../../scripts/sync_fio_to_sheets.py#L67).
|
||||
- No UI surface for switching the QR account per-request (the existing
|
||||
`?account=` query param on `/qr` keeps working as an override).
|
||||
- No backfill / dedup pass over historical rows.
|
||||
@@ -88,7 +88,7 @@ func serverCmd(args []string) {
|
||||
fmt.Fprintf(os.Stderr, "fuj server: sheets client for sync: %v\n", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
fioClients := buildFioClients(cfg)
|
||||
fioCli := fio.New(cfg.FioAPIToken, config.IBANAccountNum(cfg.BankAccount), nil)
|
||||
|
||||
actions := web.ActionHandlers{
|
||||
BankSync: func(ctx context.Context, out io.Writer) error {
|
||||
@@ -97,7 +97,7 @@ func serverCmd(args []string) {
|
||||
to := time.Date(yr, 12, 31, 23, 59, 59, 0, time.UTC)
|
||||
|
||||
fmt.Fprintln(out, "=== Sync Fio Transactions ===")
|
||||
n, err := banksync.SyncToSheets(ctx, config.PaymentsSheetID, fioClients, sheetsCli,
|
||||
n, err := banksync.SyncToSheets(ctx, config.PaymentsSheetID, fioCli, sheetsCli,
|
||||
banksync.SyncOpts{From: from, To: to, Sort: true})
|
||||
if err != nil {
|
||||
return fmt.Errorf("sync: %w", err)
|
||||
@@ -197,7 +197,7 @@ func syncCmd(args []string) {
|
||||
fmt.Fprintf(os.Stderr, "fuj sync: sheets client: %v\n", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
fioClients := buildFioClients(cfg)
|
||||
fioCli := fio.New(cfg.FioAPIToken, config.IBANAccountNum(cfg.BankAccount), nil)
|
||||
|
||||
opts := banksync.SyncOpts{Days: *days, Sort: *sort, DryRun: *dryRun, PrintFioTable: *printFioTable}
|
||||
if *fromStr != "" && *toStr != "" {
|
||||
@@ -213,7 +213,7 @@ func syncCmd(args []string) {
|
||||
}
|
||||
}
|
||||
|
||||
n, err := banksync.SyncToSheets(ctx, config.PaymentsSheetID, fioClients, sheetsCli, opts)
|
||||
n, err := banksync.SyncToSheets(ctx, config.PaymentsSheetID, fioCli, sheetsCli, opts)
|
||||
if err != nil {
|
||||
fmt.Fprintf(os.Stderr, "fuj sync: %v\n", err)
|
||||
os.Exit(1)
|
||||
@@ -274,13 +274,3 @@ Commands:
|
||||
sync Sync Fio transactions to payments sheet
|
||||
infer Infer payment details in payments sheet`)
|
||||
}
|
||||
|
||||
// buildFioClients constructs one fio.Client per configured account.
|
||||
// Each client uses the account's token if available, otherwise the transparent-scraper path.
|
||||
func buildFioClients(cfg config.Config) []fio.Client {
|
||||
clients := make([]fio.Client, 0, len(cfg.LoadedAccounts))
|
||||
for _, a := range cfg.LoadedAccounts {
|
||||
clients = append(clients, fio.New(a.Token, a.AcctNum, nil))
|
||||
}
|
||||
return clients
|
||||
}
|
||||
|
||||
@@ -7,28 +7,6 @@ import (
|
||||
"time"
|
||||
)
|
||||
|
||||
// Account describes a Fio bank account.
|
||||
type Account struct {
|
||||
IBAN string // e.g. "CZ0820100000002502035405"
|
||||
AcctNum string // bare account number, e.g. "2502035405"
|
||||
TokenEnv string // env var name holding the optional Fio API token
|
||||
Primary bool // true for the account QR codes default to
|
||||
}
|
||||
|
||||
// LoadedAccount is an Account with its token resolved from the environment.
|
||||
type LoadedAccount struct {
|
||||
Account
|
||||
Token string // value of os.Getenv(Account.TokenEnv); empty → transparent-scraper path
|
||||
}
|
||||
|
||||
// Accounts is the hardcoded list of Fio bank accounts to sync from.
|
||||
// The first entry with Primary=true is used for QR codes.
|
||||
// Tokens are loaded at runtime from each account's TokenEnv.
|
||||
var Accounts = []Account{
|
||||
{IBAN: "CZ0820100000002502035405", AcctNum: "2502035405", TokenEnv: "FIO_API_TOKEN_NEW", Primary: true},
|
||||
{IBAN: "CZ8520100000002800359168", AcctNum: "2800359168", TokenEnv: "FIO_API_TOKEN_OLD"},
|
||||
}
|
||||
|
||||
// Google Sheets IDs — change in code if sheets change (not from env).
|
||||
const (
|
||||
AttendanceSheetID = "1E2e_gT_K5AwSRCDLDTa2UetZTkHmBOcz0kFbBUNUNBA"
|
||||
@@ -56,36 +34,28 @@ var CacheSheetMap = map[string]string{
|
||||
// Mirrors scripts/config.py.
|
||||
type Config struct {
|
||||
CredentialsPath string
|
||||
QRAccount string // IBAN of the primary account used for QR codes
|
||||
LoadedAccounts []LoadedAccount // all accounts to sync, tokens resolved from env
|
||||
BankAccount string
|
||||
CacheDir string
|
||||
CacheTTL time.Duration
|
||||
CacheAPICheckTTL time.Duration
|
||||
DriveTimeout time.Duration
|
||||
LogLevel string
|
||||
FioAPIToken string
|
||||
ServerAddr string
|
||||
}
|
||||
|
||||
// Load reads configuration from the environment, applying defaults that
|
||||
// match the Python side.
|
||||
func Load() Config {
|
||||
loaded := make([]LoadedAccount, len(Accounts))
|
||||
var qrAccount string
|
||||
for i, a := range Accounts {
|
||||
loaded[i] = LoadedAccount{Account: a, Token: os.Getenv(a.TokenEnv)}
|
||||
if a.Primary {
|
||||
qrAccount = a.IBAN
|
||||
}
|
||||
}
|
||||
return Config{
|
||||
CredentialsPath: env("CREDENTIALS_PATH", ".secret/fuj-management-bot-credentials.json"),
|
||||
QRAccount: qrAccount,
|
||||
LoadedAccounts: loaded,
|
||||
BankAccount: env("BANK_ACCOUNT", "CZ8520100000002800359168"),
|
||||
CacheDir: env("CACHE_DIR", "tmp/go"),
|
||||
CacheTTL: envDuration("CACHE_TTL_SECONDS", 300),
|
||||
CacheAPICheckTTL: envDuration("CACHE_API_CHECK_TTL_SECONDS", 300),
|
||||
DriveTimeout: envDuration("DRIVE_TIMEOUT_SECONDS", 10),
|
||||
LogLevel: env("LOG_LEVEL", "INFO"),
|
||||
FioAPIToken: env("FIO_API_TOKEN", ""),
|
||||
ServerAddr: env("SERVER_ADDR", ":8080"),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -11,7 +11,7 @@ const (
|
||||
var AdultFeeMonthlyRate = map[string]int{
|
||||
"2025-09": 750, "2025-10": 750, "2025-11": 750, "2025-12": 750,
|
||||
"2026-01": 750, "2026-02": 750, "2026-03": 350,
|
||||
"2026-04": 700, "2026-05": 450, "2026-06": 600, "2026-07": 600, "2026-08": 600,
|
||||
"2026-04": 700, "2026-05": 700,
|
||||
}
|
||||
|
||||
// CalculateFee returns the adult fee in CZK for attendanceCount practices in
|
||||
|
||||
@@ -35,13 +35,13 @@ type SyncOpts struct {
|
||||
PrintFioTable bool // with DryRun: print every fetched Fio txn with NEW/DUP status
|
||||
}
|
||||
|
||||
// SyncToSheets fetches Fio transactions from all provided clients and appends
|
||||
// new ones to the payments sheet. Returns the number of rows appended.
|
||||
// SyncToSheets fetches Fio transactions and appends new ones to the payments sheet.
|
||||
// Returns the number of rows appended.
|
||||
// Ports scripts/sync_fio_to_sheets.py sync_to_sheets.
|
||||
func SyncToSheets(
|
||||
ctx context.Context,
|
||||
spreadsheetID string,
|
||||
fioClients []fio.Client,
|
||||
fioClient fio.Client,
|
||||
sh sheetsWriter,
|
||||
opts SyncOpts,
|
||||
) (int, error) {
|
||||
@@ -84,14 +84,10 @@ func SyncToSheets(
|
||||
from = to.AddDate(0, 0, -days)
|
||||
}
|
||||
|
||||
// 3. Fetch Fio transactions from each account and combine.
|
||||
var txns []fio.Transaction
|
||||
for _, client := range fioClients {
|
||||
got, err := client.FetchTransactions(ctx, from, to)
|
||||
if err != nil {
|
||||
return 0, fmt.Errorf("sync: fetch fio: %w", err)
|
||||
}
|
||||
txns = append(txns, got...)
|
||||
// 3. Fetch Fio transactions.
|
||||
txns, err := fioClient.FetchTransactions(ctx, from, to)
|
||||
if err != nil {
|
||||
return 0, fmt.Errorf("sync: fetch fio: %w", err)
|
||||
}
|
||||
if opts.DryRun {
|
||||
fmt.Printf("Dry run: window %s to %s, fetched %d transaction(s) from Fio\n",
|
||||
|
||||
@@ -20,7 +20,7 @@ func TestSyncToSheets_EmptySheet(t *testing.T) {
|
||||
}}
|
||||
fioFake := &fio.Fake{Transactions: testFioTxns}
|
||||
|
||||
n, err := SyncToSheets(context.Background(), "SHEETID", []fio.Client{fioFake}, sh, SyncOpts{Days: 30})
|
||||
n, err := SyncToSheets(context.Background(), "SHEETID", fioFake, sh, SyncOpts{Days: 30})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -51,7 +51,7 @@ func TestSyncToSheets_Dedup(t *testing.T) {
|
||||
}}
|
||||
fioFake := &fio.Fake{Transactions: testFioTxns}
|
||||
|
||||
n, err := SyncToSheets(context.Background(), "SHEETID", []fio.Client{fioFake}, sh, SyncOpts{Days: 30})
|
||||
n, err := SyncToSheets(context.Background(), "SHEETID", fioFake, sh, SyncOpts{Days: 30})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -72,7 +72,7 @@ func TestSyncToSheets_NoNewTxns(t *testing.T) {
|
||||
}}
|
||||
fioFake := &fio.Fake{Transactions: testFioTxns}
|
||||
|
||||
n, err := SyncToSheets(context.Background(), "SHEETID", []fio.Client{fioFake}, sh, SyncOpts{Days: 30})
|
||||
n, err := SyncToSheets(context.Background(), "SHEETID", fioFake, sh, SyncOpts{Days: 30})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -92,7 +92,7 @@ func TestSyncToSheets_MissingHeader(t *testing.T) {
|
||||
}}
|
||||
fioFake := &fio.Fake{Transactions: testFioTxns[:1]}
|
||||
|
||||
n, err := SyncToSheets(context.Background(), "SHEETID", []fio.Client{fioFake}, sh, SyncOpts{Days: 30})
|
||||
n, err := SyncToSheets(context.Background(), "SHEETID", fioFake, sh, SyncOpts{Days: 30})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -105,7 +105,7 @@ func TestSyncToSheets_Sort(t *testing.T) {
|
||||
sh := &sheets.Fake{Values: map[string][][]any{"SHEETID/A1:K": {}}}
|
||||
fioFake := &fio.Fake{Transactions: testFioTxns[:1]}
|
||||
|
||||
_, err := SyncToSheets(context.Background(), "SHEETID", []fio.Client{fioFake}, sh, SyncOpts{Days: 30, Sort: true})
|
||||
_, err := SyncToSheets(context.Background(), "SHEETID", fioFake, sh, SyncOpts{Days: 30, Sort: true})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -118,7 +118,7 @@ func TestSyncToSheets_ExplicitDateWindow(t *testing.T) {
|
||||
|
||||
from := time.Date(2026, 4, 1, 0, 0, 0, 0, time.UTC)
|
||||
to := time.Date(2026, 4, 30, 0, 0, 0, 0, time.UTC)
|
||||
n, err := SyncToSheets(context.Background(), "SHEETID", []fio.Client{fioFake}, sh, SyncOpts{From: from, To: to})
|
||||
n, err := SyncToSheets(context.Background(), "SHEETID", fioFake, sh, SyncOpts{From: from, To: to})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -131,7 +131,7 @@ func TestSyncToSheets_DryRun(t *testing.T) {
|
||||
sh := &sheets.Fake{Values: map[string][][]any{"SHEETID/A1:K": {}}}
|
||||
fioFake := &fio.Fake{Transactions: testFioTxns}
|
||||
|
||||
n, err := SyncToSheets(context.Background(), "SHEETID", []fio.Client{fioFake}, sh,
|
||||
n, err := SyncToSheets(context.Background(), "SHEETID", fioFake, sh,
|
||||
SyncOpts{Days: 30, Sort: true, DryRun: true})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
@@ -144,40 +144,6 @@ func TestSyncToSheets_DryRun(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestSyncToSheets_MultiAccount(t *testing.T) {
|
||||
txnsA := []fio.Transaction{
|
||||
{Date: "2026-04-10", Amount: 700, Sender: "Alice", Message: "april", VS: "1", BankID: "A1"},
|
||||
}
|
||||
txnsB := []fio.Transaction{
|
||||
{Date: "2026-04-11", Amount: 500, Sender: "Bob", Message: "duben", VS: "2", BankID: "B1"},
|
||||
}
|
||||
// One transaction that duplicates the first one from account A (same sync_id).
|
||||
dupID := syncIDFor(txnsA[0])
|
||||
sh := &sheets.Fake{Values: map[string][][]any{
|
||||
"SHEETID/A1:K": {
|
||||
{"Date", "Amount", "manual fix", "Person", "Purpose", "Inferred Amount", "Sender", "VS", "Message", "Bank ID", "Sync ID"},
|
||||
{"2026-04-10", 700.0, "", "", "", "", "Alice", "1", "april", "A1", dupID},
|
||||
},
|
||||
}}
|
||||
fakeA := &fio.Fake{Transactions: txnsA}
|
||||
fakeB := &fio.Fake{Transactions: txnsB}
|
||||
|
||||
n, err := SyncToSheets(context.Background(), "SHEETID", []fio.Client{fakeA, fakeB}, sh, SyncOpts{Days: 30})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if n != 1 {
|
||||
t.Errorf("want 1 new row (B1 from account B; A1 is duplicate), got %d", n)
|
||||
}
|
||||
if len(sh.Appended) != 1 || len(sh.Appended[0].Rows) != 1 {
|
||||
t.Fatalf("want exactly 1 row appended, got %v", sh.Appended)
|
||||
}
|
||||
row := sh.Appended[0].Rows[0]
|
||||
if row[6] != "Bob" {
|
||||
t.Errorf("expected Bob's row, got sender=%v", row[6])
|
||||
}
|
||||
}
|
||||
|
||||
// syncIDFor mirrors what SyncToSheets computes for a given fio.Transaction.
|
||||
func syncIDFor(tx fio.Transaction) string {
|
||||
currency := tx.Currency
|
||||
|
||||
@@ -138,7 +138,7 @@ func buildAdultsResponse(
|
||||
Unmatched: unmatched,
|
||||
AttendanceURL: "https://docs.google.com/spreadsheets/d/" + config.AttendanceSheetID + "/edit",
|
||||
PaymentsURL: "https://docs.google.com/spreadsheets/d/" + config.PaymentsSheetID + "/edit",
|
||||
BankAccount: cfg.QRAccount,
|
||||
BankAccount: cfg.BankAccount,
|
||||
CurrentMonth: currentMonth,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -134,7 +134,7 @@ func buildJuniorsResponse(
|
||||
Unmatched: unmatched,
|
||||
AttendanceURL: juniorURL,
|
||||
PaymentsURL: "https://docs.google.com/spreadsheets/d/" + config.PaymentsSheetID + "/edit",
|
||||
BankAccount: cfg.QRAccount,
|
||||
BankAccount: cfg.BankAccount,
|
||||
CurrentMonth: currentMonth,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -126,13 +126,13 @@ func (h *HTMLHandler) ServeQR(w http.ResponseWriter, r *http.Request) {
|
||||
amount := q.Get("amount")
|
||||
message := q.Get("message")
|
||||
if account == "" {
|
||||
account = h.apiHandler.Config.QRAccount
|
||||
account = h.apiHandler.Config.BankAccount
|
||||
}
|
||||
if amount == "" {
|
||||
amount = "0"
|
||||
}
|
||||
|
||||
payload := BuildSPD(account, amount, message, h.apiHandler.Config.QRAccount)
|
||||
payload := BuildSPD(account, amount, message, h.apiHandler.Config.BankAccount)
|
||||
png, err := RenderQRCode(payload)
|
||||
if err != nil {
|
||||
http.Error(w, "qr encode: "+err.Error(), http.StatusInternalServerError)
|
||||
|
||||
@@ -50,7 +50,7 @@ func fixtureHandler(t *testing.T) *api.Handler {
|
||||
t.Helper()
|
||||
return &api.Handler{
|
||||
Sources: fixtureSources{},
|
||||
Config: config.Config{QRAccount: "CZ0000000000000000000000"},
|
||||
Config: config.Config{BankAccount: "CZ0000000000000000000000"},
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -21,10 +21,7 @@ ADULT_FEE_MONTHLY_RATE = {
|
||||
"2026-02": 750,
|
||||
"2026-03": 350,
|
||||
"2026-04": 700,
|
||||
"2026-05": 450,
|
||||
"2026-06": 600,
|
||||
"2026-07": 600,
|
||||
"2026-08": 600,
|
||||
"2026-05": 700,
|
||||
}
|
||||
|
||||
JUNIOR_FEE_DEFAULT = 500 # CZK for 2+ practices
|
||||
|
||||
@@ -21,18 +21,8 @@ PAYMENTS_SHEET_ID = "1Om0YPoDVCH5cV8BrNz5LG5eR5MMU05ypQC7UMN1xn_Y"
|
||||
# Attendance sheet tab GIDs
|
||||
JUNIOR_SHEET_GID = "1213318614"
|
||||
|
||||
# Bank accounts — hardcoded list mirroring go/internal/config/config.go.
|
||||
# The entry with primary=True is used for QR codes and as the BANK_ACCOUNT default.
|
||||
ACCOUNTS = [
|
||||
{"iban": "CZ0820100000002502035405", "acct_num": "2502035405", "token_env": "FIO_API_TOKEN_NEW", "primary": True},
|
||||
{"iban": "CZ8520100000002800359168", "acct_num": "2800359168", "token_env": "FIO_API_TOKEN_OLD", "primary": False},
|
||||
]
|
||||
# Resolve API tokens from the environment once at import time.
|
||||
LOADED_ACCOUNTS = [
|
||||
{**a, "token": os.environ.get(a["token_env"], "")}
|
||||
for a in ACCOUNTS
|
||||
]
|
||||
BANK_ACCOUNT = next(a["iban"] for a in ACCOUNTS if a["primary"])
|
||||
# Bank
|
||||
BANK_ACCOUNT = os.environ.get("BANK_ACCOUNT", "CZ8520100000002800359168")
|
||||
|
||||
# Cache settings
|
||||
CACHE_DIR = PROJECT_ROOT / "tmp"
|
||||
|
||||
@@ -103,7 +103,7 @@ def parse_czech_date(s: str) -> str | None:
|
||||
|
||||
|
||||
def fetch_transactions_transparent(
|
||||
date_from: str, date_to: str, account_id: str
|
||||
date_from: str, date_to: str, account_id: str = "2800359168"
|
||||
) -> list[dict]:
|
||||
"""Fetch transactions from Fio transparent account HTML page.
|
||||
|
||||
@@ -206,47 +206,22 @@ def fetch_transactions_api(
|
||||
return transactions
|
||||
|
||||
|
||||
def fetch_transactions_for(account: dict, date_from: str, date_to: str) -> list[dict]:
|
||||
"""Fetch transactions for a single loaded account dict (from config.LOADED_ACCOUNTS).
|
||||
|
||||
Uses the API path if the account has a token, otherwise the transparent scraper.
|
||||
date_from/date_to: YYYY-MM-DD.
|
||||
"""
|
||||
token = (account.get("token") or "").strip()
|
||||
acct_num = account["acct_num"]
|
||||
def fetch_transactions(date_from: str, date_to: str) -> list[dict]:
|
||||
"""Fetch transactions, using API if token available, else transparent page."""
|
||||
token = os.environ.get("FIO_API_TOKEN", "").strip()
|
||||
if token:
|
||||
print(f"fio: account {acct_num}: using authenticated API, window {date_from}..{date_to}", file=sys.stderr)
|
||||
print(f"fio: using authenticated API, window {date_from}..{date_to}", file=sys.stderr)
|
||||
return fetch_transactions_api(token, date_from, date_to)
|
||||
|
||||
print(
|
||||
f"fio: account {acct_num}: using transparent page (no token — expect publishing lag), "
|
||||
f"window {date_from}..{date_to}",
|
||||
f"fio: using transparent page (FIO_API_TOKEN unset — expect publishing lag), "
|
||||
f"window {date_from}..{date_to}, account=2800359168",
|
||||
file=sys.stderr,
|
||||
)
|
||||
# Convert YYYY-MM-DD to DD.MM.YYYY for the transparent page URL
|
||||
from_dt = datetime.strptime(date_from, "%Y-%m-%d")
|
||||
to_dt = datetime.strptime(date_to, "%Y-%m-%d")
|
||||
return fetch_transactions_transparent(
|
||||
from_dt.strftime("%d.%m.%Y"),
|
||||
to_dt.strftime("%d.%m.%Y"),
|
||||
account_id=acct_num,
|
||||
)
|
||||
|
||||
|
||||
def fetch_transactions_all(
|
||||
date_from: str, date_to: str, accounts: list[dict] | None = None
|
||||
) -> list[dict]:
|
||||
"""Fetch and combine transactions from all configured accounts.
|
||||
|
||||
accounts: list of loaded account dicts (defaults to config.LOADED_ACCOUNTS).
|
||||
Returns a flat list of all transactions across all accounts.
|
||||
"""
|
||||
if accounts is None:
|
||||
from config import LOADED_ACCOUNTS
|
||||
accounts = LOADED_ACCOUNTS
|
||||
all_txns: list[dict] = []
|
||||
for account in accounts:
|
||||
txns = fetch_transactions_for(account, date_from, date_to)
|
||||
print(f"fio: account {account['acct_num']}: {len(txns)} transaction(s)", file=sys.stderr)
|
||||
all_txns.extend(txns)
|
||||
print(f"fio: total {len(all_txns)} transaction(s) across {len(accounts)} account(s)", file=sys.stderr)
|
||||
return all_txns
|
||||
|
||||
@@ -12,7 +12,7 @@ from google_auth_oauthlib.flow import InstalledAppFlow
|
||||
from google.oauth2 import service_account
|
||||
from googleapiclient.discovery import build
|
||||
|
||||
from fio_utils import fetch_transactions_all
|
||||
from fio_utils import fetch_transactions
|
||||
|
||||
from config import PAYMENTS_SHEET_ID as DEFAULT_SPREADSHEET_ID
|
||||
SCOPES = ["https://www.googleapis.com/auth/spreadsheets"]
|
||||
@@ -188,7 +188,7 @@ def sync_to_sheets(
|
||||
dt_str = date_to.strftime("%Y-%m-%d")
|
||||
|
||||
print(f"Fetching Fio transactions from {df_str} to {dt_str}...")
|
||||
transactions = fetch_transactions_all(df_str, dt_str)
|
||||
transactions = fetch_transactions(df_str, dt_str)
|
||||
print(f"Found {len(transactions)} transactions.")
|
||||
|
||||
if dry_run:
|
||||
|
||||
Reference in New Issue
Block a user