Compare commits
7 Commits
0.29
...
feat/m2-1-
| Author | SHA1 | Date | |
|---|---|---|---|
| d9a61b338c | |||
| 91ac3b37cf | |||
| 394da2e6b8 | |||
| 81b36878b3 | |||
| 97f568f49f | |||
| cf0f176d3f | |||
| 5a41cdae83 |
16
.claude/settings.json
Normal file
16
.claude/settings.json
Normal file
@@ -0,0 +1,16 @@
|
|||||||
|
{
|
||||||
|
"permissions": {
|
||||||
|
"allow": [
|
||||||
|
"Bash(git add:*)",
|
||||||
|
"Bash(go version *)",
|
||||||
|
"Bash(go mod *)",
|
||||||
|
"Bash(golangci-lint run *)",
|
||||||
|
"Bash(golangci-lint --version)",
|
||||||
|
"Bash(gofumpt *)",
|
||||||
|
"Bash(./bin/fuj help *)",
|
||||||
|
"Bash(./bin/fuj version *)",
|
||||||
|
"Bash(make go-test *)",
|
||||||
|
"Bash(make go-lint *)"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -37,3 +37,29 @@ jobs:
|
|||||||
--build-arg BUILD_DATE=$(date -u +%Y-%m-%dT%H:%M:%SZ) \
|
--build-arg BUILD_DATE=$(date -u +%Y-%m-%dT%H:%M:%SZ) \
|
||||||
-t $IMAGE .
|
-t $IMAGE .
|
||||||
docker push $IMAGE
|
docker push $IMAGE
|
||||||
|
|
||||||
|
build-go:
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
permissions:
|
||||||
|
contents: read
|
||||||
|
packages: write
|
||||||
|
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v4
|
||||||
|
|
||||||
|
- name: Login to Gitea registry
|
||||||
|
run: echo "${{ secrets.REGISTRY_TOKEN }}" | docker login -u ${{ github.actor }} --password-stdin gitea.home.hrajfrisbee.cz
|
||||||
|
|
||||||
|
- name: Build and push Go image
|
||||||
|
run: |
|
||||||
|
TAG=${{ github.ref_name }}
|
||||||
|
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
|
||||||
|
TAG=${{ inputs.tag }}
|
||||||
|
fi
|
||||||
|
IMAGE=gitea.home.hrajfrisbee.cz/${{ github.repository }}:$TAG-go
|
||||||
|
docker build -f go/build/Dockerfile \
|
||||||
|
--build-arg GIT_TAG=$TAG \
|
||||||
|
--build-arg GIT_COMMIT=${{ github.sha }} \
|
||||||
|
--build-arg BUILD_DATE=$(date -u +%Y-%m-%dT%H:%M:%SZ) \
|
||||||
|
-t $IMAGE go/
|
||||||
|
docker push $IMAGE
|
||||||
|
|||||||
3
.gitignore
vendored
3
.gitignore
vendored
@@ -4,3 +4,6 @@
|
|||||||
|
|
||||||
# local tmp folder
|
# local tmp folder
|
||||||
tmp/
|
tmp/
|
||||||
|
|
||||||
|
# go build output
|
||||||
|
bin/
|
||||||
|
|||||||
30
CHANGELOG.md
30
CHANGELOG.md
@@ -1,5 +1,35 @@
|
|||||||
# Changelog
|
# Changelog
|
||||||
|
|
||||||
|
## 2026-05-04 23:08 CEST — fix: payment inference exact-match short-circuit
|
||||||
|
|
||||||
|
- `match_members()` now short-circuits on whole-word full-name hits; nickname/partial checks only run when no full name is present.
|
||||||
|
- Replaced bare `in` substring checks with `_word_in()` word-boundary regex throughout, closing the class of bugs where a short nickname (e.g. `tov`) matches inside another member's surname (`ottova`).
|
||||||
|
- Added `tests/test_match_members.py` (6 cases). Affects `scripts/match_payments.py`.
|
||||||
|
|
||||||
|
## 2026-05-04 23:08 CEST — feat: lower adult monthly fee to 700 CZK from April 2026
|
||||||
|
|
||||||
|
- `ADULT_FEE_DEFAULT` reduced from 750 → 700 CZK.
|
||||||
|
- `ADULT_FEE_MONTHLY_RATE` now pins Sep 2025 – Feb 2026 at 750 to preserve historical billing; Mar 2026 stays 350; Apr–May 2026 at 700. Affects `scripts/attendance.py`.
|
||||||
|
|
||||||
|
## 2026-05-04 12:02 CEST — Go rewrite M1: skeleton + tooling
|
||||||
|
|
||||||
|
- Created `go/` tree with module `fuj-management/go` (Go 1.26).
|
||||||
|
- `cmd/fuj`: stdlib-flag subcommand dispatcher; `server` and `version` implemented, stubs for M2/M4 commands.
|
||||||
|
- `internal/config`: env loader mirroring `scripts/config.py` (same env var names and defaults).
|
||||||
|
- `internal/logging`: slog setup accepting log level from config.
|
||||||
|
- `internal/web`: `net/http` ServeMux on `:8080`; `middleware/timer.go` logs method/path/status/ms.
|
||||||
|
- `go/build/Dockerfile`: multi-stage (`golang:1.26` → `alpine:3`) producing a static binary image.
|
||||||
|
- Makefile: `web` → `web-py` alias; added `web-go`, `go-build`, `go-test`, `go-run`, `go-lint`.
|
||||||
|
- `.gitea/workflows/build.yaml`: parallel `build-go` job pushing `<tag>-go` image.
|
||||||
|
- Gate: `make go-build`, `make go-lint`, `make go-test`, `curl :8080` all pass.
|
||||||
|
|
||||||
|
## 2026-05-03 20:37 CEST — Fix Balance column to correctly reflect past-month debt
|
||||||
|
|
||||||
|
- Balance (and Pay-All) are now computed as `sum(paid − expected)` over past months only, iterating directly over the ledger entries from `reconcile()`.
|
||||||
|
- Previously the balance used `total_balance` (which includes current/future-month activity and out-of-window credits) plus a one-sided current-month debt adjustment. Current-month *surplus* leaked through, making the balance appear less negative than the actual past-month debt.
|
||||||
|
- Pay-All is now `max(0, −balance)` so the two values are derived from a single source and can never disagree.
|
||||||
|
- Affected: `adults_view()` and `juniors_view()` in `app.py`.
|
||||||
|
|
||||||
## 2026-05-03 19:26 CEST — Fee-aware allocation for multi-month payments
|
## 2026-05-03 19:26 CEST — Fee-aware allocation for multi-month payments
|
||||||
|
|
||||||
- `reconcile()` no longer splits a multi-month payment evenly. Allocation is now per-member with two phases: greedy (if amount ≥ total expected, each month gets exactly its expected fee and overflow → credit) and proportional (otherwise distribute by each month's expected). Fixes the case where e.g. 1250 CZK covering 3 months with mixed fees (750/350/150) marked two months red.
|
- `reconcile()` no longer splits a multi-month payment evenly. Allocation is now per-member with two phases: greedy (if amount ≥ total expected, each month gets exactly its expected fee and overflow → credit) and proportional (otherwise distribute by each month's expected). Fixes the case where e.g. 1250 CZK covering 3 months with mixed fees (750/350/150) marked two months red.
|
||||||
|
|||||||
38
CLAUDE.md
38
CLAUDE.md
@@ -92,6 +92,34 @@ Tiers are set in column B of the attendance sheet:
|
|||||||
|
|
||||||
`/qr?account=…&amount=…&message=…` generates a Czech QR Platba PNG (SPD format).
|
`/qr?account=…&amount=…&message=…` generates a Czech QR Platba PNG (SPD format).
|
||||||
|
|
||||||
|
## Branching & merge requests
|
||||||
|
|
||||||
|
The remote is Gitea (`gitea.home.hrajfrisbee.cz/kacerr/fuj-management`).
|
||||||
|
For **features**, do not commit to `main` directly. Use a branch + merge
|
||||||
|
request flow:
|
||||||
|
|
||||||
|
1. **Create a branch off `main`** before starting work:
|
||||||
|
- `feat/<slug>` for features (e.g. `feat/qr-code-overlay`)
|
||||||
|
- `fix/<slug>` for bug-fix branches the user explicitly asks for
|
||||||
|
- `<slug>` is short kebab-case
|
||||||
|
2. **Commit on the branch** following the existing commit conventions
|
||||||
|
(Co-Authored-By trailer, etc.).
|
||||||
|
3. **Push the branch** to `origin` with `-u` so it tracks.
|
||||||
|
4. **Print the Gitea compare URL** so the user can open the MR in the browser:
|
||||||
|
`https://gitea.home.hrajfrisbee.cz/kacerr/fuj-management/compare/main...<branch>`
|
||||||
|
Do **not** use `tea`, `gh`, or call the Gitea API — the user opens and
|
||||||
|
merges the MR themselves.
|
||||||
|
5. **Do not merge or delete the branch** from the CLI. The user does that
|
||||||
|
in Gitea.
|
||||||
|
|
||||||
|
**Exceptions — when committing straight to `main` is fine:**
|
||||||
|
- Small bug fixes / hotfixes the user describes as such.
|
||||||
|
- Typo / comment / formatting tweaks.
|
||||||
|
- Edits the user explicitly says to push to `main`.
|
||||||
|
|
||||||
|
When uncertain whether something is a feature or a small fix, ask before
|
||||||
|
committing.
|
||||||
|
|
||||||
## Git Commits
|
## Git Commits
|
||||||
|
|
||||||
When making git commits, always append yourself as co-author trailer to the end of the commit message to indicate AI assistance
|
When making git commits, always append yourself as co-author trailer to the end of the commit message to indicate AI assistance
|
||||||
@@ -108,3 +136,13 @@ Maintain a running changelog in `CHANGELOG.md` at the repo root. After every sig
|
|||||||
```
|
```
|
||||||
|
|
||||||
Get the timestamp with `date "+%Y-%m-%d %H:%M %Z"`. Skip trivial edits (typos, formatting, comment tweaks); only log changes a future reader would care about.
|
Get the timestamp with `date "+%Y-%m-%d %H:%M %Z"`. Skip trivial edits (typos, formatting, comment tweaks); only log changes a future reader would care about.
|
||||||
|
|
||||||
|
## Plans
|
||||||
|
|
||||||
|
When Claude Code's plan mode is used, save the plan file inside the repo at
|
||||||
|
`docs/plans/YYYY-MM-DD-HHMM-<slug>.md` instead of the default `~/.claude/plans/`
|
||||||
|
location. Get the timestamp with `date "+%Y-%m-%d-%H%M"` (matches the changelog
|
||||||
|
convention). The `<slug>` should be a short kebab-case summary of the plan's topic.
|
||||||
|
|
||||||
|
Create the `docs/plans/` directory on first use. Plan files are committed to the
|
||||||
|
repo so other contributors can review historical decisions.
|
||||||
|
|||||||
49
Makefile
49
Makefile
@@ -1,10 +1,13 @@
|
|||||||
.PHONY: help fees match web web-debug image run sync sync-2026 test test-v docs
|
.PHONY: help fees match web web-py web-debug web-go go-build go-test go-run go-lint image run sync sync-2026 test test-v docs
|
||||||
|
|
||||||
export PYTHONPATH := scripts:$(PYTHONPATH)
|
export PYTHONPATH := scripts:$(PYTHONPATH)
|
||||||
VENV := .venv
|
VENV := .venv
|
||||||
PYTHON := $(VENV)/bin/python3
|
PYTHON := $(VENV)/bin/python3
|
||||||
CREDENTIALS := .secret/fuj-management-bot-credentials.json
|
CREDENTIALS := .secret/fuj-management-bot-credentials.json
|
||||||
|
|
||||||
|
GO_SRC := go
|
||||||
|
GO_BIN := bin/fuj
|
||||||
|
|
||||||
$(PYTHON): .venv/.last_sync
|
$(PYTHON): .venv/.last_sync
|
||||||
|
|
||||||
.venv/.last_sync: pyproject.toml
|
.venv/.last_sync: pyproject.toml
|
||||||
@@ -13,20 +16,25 @@ $(PYTHON): .venv/.last_sync
|
|||||||
|
|
||||||
help:
|
help:
|
||||||
@echo "Available targets:"
|
@echo "Available targets:"
|
||||||
@echo " make fees - Calculate monthly fees from the attendance sheet"
|
@echo " make fees - Calculate monthly fees from the attendance sheet"
|
||||||
@echo " make match - Match Fio bank payments against expected attendance fees"
|
@echo " make match - Match Fio bank payments against expected attendance fees"
|
||||||
@echo " make web - Start a dynamic web dashboard locally"
|
@echo " make web - Start Python dashboard (alias for web-py, until M8)"
|
||||||
@echo " make web-debug - Start a dynamic web dashboard locally in debug mode"
|
@echo " make web-py - Start Python dashboard on :5001"
|
||||||
@echo " make image - Build an OCI container image"
|
@echo " make web-go - Build and start Go dashboard on :8080"
|
||||||
@echo " make run - Run the built Docker image locally"
|
@echo " make web-debug - Start Python dashboard in debug mode"
|
||||||
|
@echo " make go-build - Build Go binary to bin/fuj"
|
||||||
|
@echo " make go-test - Run Go tests"
|
||||||
|
@echo " make go-lint - Run golangci-lint on Go code"
|
||||||
|
@echo " make image - Build Python OCI container image"
|
||||||
|
@echo " make run - Run the built Python Docker image locally"
|
||||||
@echo " make sync - Sync Fio transactions to Google Sheets"
|
@echo " make sync - Sync Fio transactions to Google Sheets"
|
||||||
@echo " make sync-2025 - Sync Fio transactions for Q4 2025 (Oct-Dec)"
|
@echo " make sync-2025 - Sync Fio transactions for Q4 2025 (Oct-Dec)"
|
||||||
@echo " make sync-2026 - Sync Fio transactions for the whole year of 2026"
|
@echo " make sync-2026 - Sync Fio transactions for the whole year of 2026"
|
||||||
@echo " make infer - Infer payment details (Person, Purpose, Amount) in the sheet"
|
@echo " make infer - Infer payment details (Person, Purpose, Amount) in the sheet"
|
||||||
@echo " make reconcile - Show balance report using Google Sheets data"
|
@echo " make reconcile - Show balance report using Google Sheets data"
|
||||||
@echo " make venv - Sync virtual environment with pyproject.toml"
|
@echo " make venv - Sync virtual environment with pyproject.toml"
|
||||||
@echo " make test - Run web application infrastructure tests"
|
@echo " make test - Run Python web application infrastructure tests"
|
||||||
@echo " make test-v - Run tests with verbose output"
|
@echo " make test-v - Run Python tests with verbose output"
|
||||||
@echo " make docs - Serve documentation in a browser"
|
@echo " make docs - Serve documentation in a browser"
|
||||||
|
|
||||||
venv:
|
venv:
|
||||||
@@ -38,12 +46,33 @@ fees: $(PYTHON)
|
|||||||
match: $(PYTHON)
|
match: $(PYTHON)
|
||||||
$(PYTHON) scripts/match_payments.py
|
$(PYTHON) scripts/match_payments.py
|
||||||
|
|
||||||
web: $(PYTHON)
|
web: web-py
|
||||||
|
|
||||||
|
web-py: $(PYTHON)
|
||||||
$(PYTHON) app.py
|
$(PYTHON) app.py
|
||||||
|
|
||||||
web-debug: $(PYTHON)
|
web-debug: $(PYTHON)
|
||||||
FLASK_DEBUG=1 $(PYTHON) app.py
|
FLASK_DEBUG=1 $(PYTHON) app.py
|
||||||
|
|
||||||
|
go-build:
|
||||||
|
cd $(GO_SRC) && go build -trimpath \
|
||||||
|
-ldflags "-X main.version=$$(git describe --tags --always 2>/dev/null || echo dev) \
|
||||||
|
-X main.commit=$$(git rev-parse --short HEAD) \
|
||||||
|
-X main.buildDate=$$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
|
||||||
|
-o ../$(GO_BIN) ./cmd/fuj
|
||||||
|
|
||||||
|
go-test:
|
||||||
|
cd $(GO_SRC) && go test -race ./...
|
||||||
|
|
||||||
|
go-run: go-build
|
||||||
|
./$(GO_BIN) $(ARGS)
|
||||||
|
|
||||||
|
go-lint:
|
||||||
|
cd $(GO_SRC) && golangci-lint run ./...
|
||||||
|
|
||||||
|
web-go: go-build
|
||||||
|
./$(GO_BIN) server
|
||||||
|
|
||||||
image:
|
image:
|
||||||
docker build -t fuj-management:latest \
|
docker build -t fuj-management:latest \
|
||||||
--build-arg GIT_TAG=$$(git describe --tags --always 2>/dev/null || echo "untagged") \
|
--build-arg GIT_TAG=$$(git describe --tags --always 2>/dev/null || echo "untagged") \
|
||||||
|
|||||||
125
app.py
125
app.py
@@ -22,7 +22,7 @@ from config import (
|
|||||||
BANK_ACCOUNT, CREDENTIALS_PATH,
|
BANK_ACCOUNT, CREDENTIALS_PATH,
|
||||||
)
|
)
|
||||||
from attendance import get_members_with_fees, get_junior_members_with_fees, ADULT_MERGED_MONTHS, JUNIOR_MERGED_MONTHS
|
from attendance import get_members_with_fees, get_junior_members_with_fees, ADULT_MERGED_MONTHS, JUNIOR_MERGED_MONTHS
|
||||||
from match_payments import reconcile, fetch_sheet_data, fetch_exceptions, normalize
|
from match_payments import reconcile, fetch_sheet_data, fetch_exceptions, normalize, canonical_member_key
|
||||||
from cache_utils import get_sheet_modified_time, read_cache, write_cache, _LAST_CHECKED, flush_cache
|
from cache_utils import get_sheet_modified_time, read_cache, write_cache, _LAST_CHECKED, flush_cache
|
||||||
from sync_fio_to_sheets import sync_to_sheets
|
from sync_fio_to_sheets import sync_to_sheets
|
||||||
from infer_payments import infer_payments
|
from infer_payments import infer_payments
|
||||||
@@ -57,6 +57,25 @@ def get_month_labels(sorted_months, merged_months):
|
|||||||
labels[m] = dt.strftime("%b %Y")
|
labels[m] = dt.strftime("%b %Y")
|
||||||
return labels
|
return labels
|
||||||
|
|
||||||
|
def group_payments_by_person(transactions, member_names=None):
|
||||||
|
canonical_by_key = (
|
||||||
|
{canonical_member_key(n): n for n in member_names} if member_names else {}
|
||||||
|
)
|
||||||
|
grouped = {}
|
||||||
|
for tx in transactions:
|
||||||
|
person = str(tx.get("person", "")).strip()
|
||||||
|
if not person:
|
||||||
|
continue
|
||||||
|
for p in person.split(","):
|
||||||
|
p = re.sub(r"\[\?\]\s*", "", p).strip()
|
||||||
|
if not p:
|
||||||
|
continue
|
||||||
|
key = canonical_by_key.get(canonical_member_key(p), p)
|
||||||
|
grouped.setdefault(key, []).append(tx)
|
||||||
|
for rows in grouped.values():
|
||||||
|
rows.sort(key=lambda t: str(t.get("date", "")), reverse=True)
|
||||||
|
return grouped
|
||||||
|
|
||||||
def warmup_cache():
|
def warmup_cache():
|
||||||
"""Pre-fetch all cached data so first request is fast."""
|
"""Pre-fetch all cached data so first request is fast."""
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
@@ -192,7 +211,6 @@ def adults_view():
|
|||||||
row = {"name": name, "months": [], "balance": data["total_balance"], "unpaid_periods": "", "raw_unpaid_periods": ""}
|
row = {"name": name, "months": [], "balance": data["total_balance"], "unpaid_periods": "", "raw_unpaid_periods": ""}
|
||||||
unpaid_months = []
|
unpaid_months = []
|
||||||
raw_unpaid_months = []
|
raw_unpaid_months = []
|
||||||
payable_amount = 0
|
|
||||||
for m in sorted_months:
|
for m in sorted_months:
|
||||||
mdata = data["months"].get(m, {"expected": 0, "original_expected": 0, "attendance_count": 0, "paid": 0, "exception": None})
|
mdata = data["months"].get(m, {"expected": 0, "original_expected": 0, "attendance_count": 0, "paid": 0, "exception": None})
|
||||||
expected = mdata.get("expected", 0)
|
expected = mdata.get("expected", 0)
|
||||||
@@ -228,14 +246,12 @@ def adults_view():
|
|||||||
if m < current_month:
|
if m < current_month:
|
||||||
unpaid_months.append(month_labels[m])
|
unpaid_months.append(month_labels[m])
|
||||||
raw_unpaid_months.append(datetime.strptime(m, "%Y-%m").strftime("%m/%Y"))
|
raw_unpaid_months.append(datetime.strptime(m, "%Y-%m").strftime("%m/%Y"))
|
||||||
payable_amount += amount_to_pay
|
|
||||||
else:
|
else:
|
||||||
status = "unpaid"
|
status = "unpaid"
|
||||||
cell_text = f"0/{fee_display}"
|
cell_text = f"0/{fee_display}"
|
||||||
if m < current_month:
|
if m < current_month:
|
||||||
unpaid_months.append(month_labels[m])
|
unpaid_months.append(month_labels[m])
|
||||||
raw_unpaid_months.append(datetime.strptime(m, "%Y-%m").strftime("%m/%Y"))
|
raw_unpaid_months.append(datetime.strptime(m, "%Y-%m").strftime("%m/%Y"))
|
||||||
payable_amount += amount_to_pay
|
|
||||||
elif paid > 0:
|
elif paid > 0:
|
||||||
status = "surplus"
|
status = "surplus"
|
||||||
cell_text = f"PAID {paid}"
|
cell_text = f"PAID {paid}"
|
||||||
@@ -258,17 +274,17 @@ def adults_view():
|
|||||||
"tooltip": tooltip
|
"tooltip": tooltip
|
||||||
})
|
})
|
||||||
|
|
||||||
# Compute balance excluding current/future months
|
# Balance = sum of (paid - expected) for past months only; current/future months ignored.
|
||||||
current_month_debt = 0
|
settled_balance = 0
|
||||||
for m in sorted_months:
|
for m, mdata in data["months"].items():
|
||||||
if m >= current_month:
|
if m >= current_month:
|
||||||
mdata = data["months"].get(m, {"expected": 0, "paid": 0})
|
continue
|
||||||
exp = mdata.get("expected", 0)
|
exp = mdata.get("expected", 0)
|
||||||
pd = int(mdata.get("paid", 0))
|
if isinstance(exp, int):
|
||||||
current_month_debt += max(0, exp - pd)
|
settled_balance += int(mdata.get("paid", 0)) - exp
|
||||||
settled_balance = data["total_balance"] + current_month_debt
|
|
||||||
|
|
||||||
row["unpaid_periods"] = ", ".join(unpaid_months) if unpaid_months else ("Older debt" if settled_balance < 0 and payable_amount == 0 else "")
|
payable_amount = max(0, -settled_balance)
|
||||||
|
row["unpaid_periods"] = ", ".join(unpaid_months)
|
||||||
row["raw_unpaid_periods"] = "+".join(raw_unpaid_months)
|
row["raw_unpaid_periods"] = "+".join(raw_unpaid_months)
|
||||||
row["balance"] = settled_balance
|
row["balance"] = settled_balance
|
||||||
row["payable_amount"] = payable_amount
|
row["payable_amount"] = payable_amount
|
||||||
@@ -293,14 +309,21 @@ def adults_view():
|
|||||||
|
|
||||||
def settled_balance(name):
|
def settled_balance(name):
|
||||||
data = result["members"][name]
|
data = result["members"][name]
|
||||||
debt = sum(max(0, data["months"].get(m, {"expected": 0, "paid": 0}).get("expected", 0) - int(data["months"].get(m, {"expected": 0, "paid": 0}).get("paid", 0))) for m in sorted_months if m >= current_month)
|
total = 0
|
||||||
return data["total_balance"] + debt
|
for m, mdata in data["months"].items():
|
||||||
|
if m >= current_month:
|
||||||
|
continue
|
||||||
|
exp = mdata.get("expected", 0)
|
||||||
|
if isinstance(exp, int):
|
||||||
|
total += int(mdata.get("paid", 0)) - exp
|
||||||
|
return total
|
||||||
|
|
||||||
credits = sorted([{"name": n, "amount": settled_balance(n)} for n in adult_names if settled_balance(n) > 0], key=lambda x: x["name"])
|
credits = sorted([{"name": n, "amount": settled_balance(n)} for n in adult_names if settled_balance(n) > 0], key=lambda x: x["name"])
|
||||||
debts = sorted([{"name": n, "amount": abs(settled_balance(n))} for n in adult_names if settled_balance(n) < 0], key=lambda x: x["name"])
|
debts = sorted([{"name": n, "amount": abs(settled_balance(n))} for n in adult_names if settled_balance(n) < 0], key=lambda x: x["name"])
|
||||||
unmatched = result["unmatched"]
|
unmatched = result["unmatched"]
|
||||||
import json
|
import json
|
||||||
|
|
||||||
|
raw_payments_by_person = group_payments_by_person(transactions, [name for name, _, _ in members])
|
||||||
record_step("process_data")
|
record_step("process_data")
|
||||||
|
|
||||||
return render_template(
|
return render_template(
|
||||||
@@ -311,6 +334,7 @@ def adults_view():
|
|||||||
totals=formatted_totals,
|
totals=formatted_totals,
|
||||||
member_data=json.dumps(result["members"]),
|
member_data=json.dumps(result["members"]),
|
||||||
month_labels_json=json.dumps(month_labels),
|
month_labels_json=json.dumps(month_labels),
|
||||||
|
raw_payments_json=json.dumps(raw_payments_by_person),
|
||||||
credits=credits,
|
credits=credits,
|
||||||
debts=debts,
|
debts=debts,
|
||||||
unmatched=unmatched,
|
unmatched=unmatched,
|
||||||
@@ -373,7 +397,6 @@ def juniors_view():
|
|||||||
row = {"name": name, "months": [], "balance": data["total_balance"], "unpaid_periods": "", "raw_unpaid_periods": ""}
|
row = {"name": name, "months": [], "balance": data["total_balance"], "unpaid_periods": "", "raw_unpaid_periods": ""}
|
||||||
unpaid_months = []
|
unpaid_months = []
|
||||||
raw_unpaid_months = []
|
raw_unpaid_months = []
|
||||||
payable_amount = 0
|
|
||||||
for m in sorted_months:
|
for m in sorted_months:
|
||||||
mdata = data["months"].get(m, {"expected": 0, "original_expected": 0, "attendance_count": 0, "paid": 0, "exception": None})
|
mdata = data["months"].get(m, {"expected": 0, "original_expected": 0, "attendance_count": 0, "paid": 0, "exception": None})
|
||||||
expected = mdata.get("expected", 0)
|
expected = mdata.get("expected", 0)
|
||||||
@@ -429,7 +452,6 @@ def juniors_view():
|
|||||||
if m < current_month:
|
if m < current_month:
|
||||||
unpaid_months.append(month_labels[m])
|
unpaid_months.append(month_labels[m])
|
||||||
raw_unpaid_months.append(datetime.strptime(m, "%Y-%m").strftime("%m/%Y"))
|
raw_unpaid_months.append(datetime.strptime(m, "%Y-%m").strftime("%m/%Y"))
|
||||||
payable_amount += amount_to_pay
|
|
||||||
else:
|
else:
|
||||||
status = "unpaid"
|
status = "unpaid"
|
||||||
cell_text = f"0/{fee_display}"
|
cell_text = f"0/{fee_display}"
|
||||||
@@ -437,7 +459,6 @@ def juniors_view():
|
|||||||
if m < current_month:
|
if m < current_month:
|
||||||
unpaid_months.append(month_labels[m])
|
unpaid_months.append(month_labels[m])
|
||||||
raw_unpaid_months.append(datetime.strptime(m, "%Y-%m").strftime("%m/%Y"))
|
raw_unpaid_months.append(datetime.strptime(m, "%Y-%m").strftime("%m/%Y"))
|
||||||
payable_amount += amount_to_pay
|
|
||||||
elif paid > 0:
|
elif paid > 0:
|
||||||
status = "surplus"
|
status = "surplus"
|
||||||
cell_text = f"PAID {paid}"
|
cell_text = f"PAID {paid}"
|
||||||
@@ -457,18 +478,17 @@ def juniors_view():
|
|||||||
"tooltip": tooltip
|
"tooltip": tooltip
|
||||||
})
|
})
|
||||||
|
|
||||||
# Compute balance excluding current/future months
|
# Balance = sum of (paid - expected) for past months only; current/future months ignored.
|
||||||
current_month_debt = 0
|
settled_balance = 0
|
||||||
for m in sorted_months:
|
for m, mdata in data["months"].items():
|
||||||
if m >= current_month:
|
if m >= current_month:
|
||||||
mdata = data["months"].get(m, {"expected": 0, "paid": 0})
|
continue
|
||||||
exp = mdata.get("expected", 0)
|
exp = mdata.get("expected", 0)
|
||||||
if isinstance(exp, int):
|
if isinstance(exp, int):
|
||||||
pd = int(mdata.get("paid", 0))
|
settled_balance += int(mdata.get("paid", 0)) - exp
|
||||||
current_month_debt += max(0, exp - pd)
|
|
||||||
settled_balance = data["total_balance"] + current_month_debt
|
|
||||||
|
|
||||||
row["unpaid_periods"] = ", ".join(unpaid_months) if unpaid_months else ("Older debt" if settled_balance < 0 and payable_amount == 0 else "")
|
payable_amount = max(0, -settled_balance)
|
||||||
|
row["unpaid_periods"] = ", ".join(unpaid_months)
|
||||||
row["raw_unpaid_periods"] = "+".join(raw_unpaid_months)
|
row["raw_unpaid_periods"] = "+".join(raw_unpaid_months)
|
||||||
row["balance"] = settled_balance
|
row["balance"] = settled_balance
|
||||||
row["payable_amount"] = payable_amount
|
row["payable_amount"] = payable_amount
|
||||||
@@ -494,19 +514,20 @@ def juniors_view():
|
|||||||
# Format credits and debts
|
# Format credits and debts
|
||||||
def junior_settled_balance(name):
|
def junior_settled_balance(name):
|
||||||
data = result["members"][name]
|
data = result["members"][name]
|
||||||
debt = 0
|
total = 0
|
||||||
for m in sorted_months:
|
for m, mdata in data["months"].items():
|
||||||
if m >= current_month:
|
if m >= current_month:
|
||||||
mdata = data["months"].get(m, {"expected": 0, "paid": 0})
|
continue
|
||||||
exp = mdata.get("expected", 0)
|
exp = mdata.get("expected", 0)
|
||||||
if isinstance(exp, int):
|
if isinstance(exp, int):
|
||||||
debt += max(0, exp - int(mdata.get("paid", 0)))
|
total += int(mdata.get("paid", 0)) - exp
|
||||||
return data["total_balance"] + debt
|
return total
|
||||||
|
|
||||||
junior_all_names = [name for name, _, _ in adapted_members]
|
junior_all_names = [name for name, _, _ in adapted_members]
|
||||||
credits = sorted([{"name": n, "amount": junior_settled_balance(n)} for n in junior_all_names if junior_settled_balance(n) > 0], key=lambda x: x["name"])
|
credits = sorted([{"name": n, "amount": junior_settled_balance(n)} for n in junior_all_names if junior_settled_balance(n) > 0], key=lambda x: x["name"])
|
||||||
debts = sorted([{"name": n, "amount": abs(junior_settled_balance(n))} for n in junior_all_names if junior_settled_balance(n) < 0], key=lambda x: x["name"])
|
debts = sorted([{"name": n, "amount": abs(junior_settled_balance(n))} for n in junior_all_names if junior_settled_balance(n) < 0], key=lambda x: x["name"])
|
||||||
unmatched = result["unmatched"]
|
unmatched = result["unmatched"]
|
||||||
|
raw_payments_by_person = group_payments_by_person(transactions, [name for name, _, _ in adapted_members])
|
||||||
import json
|
import json
|
||||||
|
|
||||||
record_step("process_data")
|
record_step("process_data")
|
||||||
@@ -519,6 +540,7 @@ def juniors_view():
|
|||||||
totals=formatted_totals,
|
totals=formatted_totals,
|
||||||
member_data=json.dumps(result["members"]),
|
member_data=json.dumps(result["members"]),
|
||||||
month_labels_json=json.dumps(month_labels),
|
month_labels_json=json.dumps(month_labels),
|
||||||
|
raw_payments_json=json.dumps(raw_payments_by_person),
|
||||||
credits=credits,
|
credits=credits,
|
||||||
debts=debts,
|
debts=debts,
|
||||||
unmatched=unmatched,
|
unmatched=unmatched,
|
||||||
@@ -537,27 +559,22 @@ def payments():
|
|||||||
transactions = get_cached_data("payments_transactions", PAYMENTS_SHEET_ID, fetch_sheet_data, PAYMENTS_SHEET_ID, credentials_path)
|
transactions = get_cached_data("payments_transactions", PAYMENTS_SHEET_ID, fetch_sheet_data, PAYMENTS_SHEET_ID, credentials_path)
|
||||||
record_step("fetch_payments")
|
record_step("fetch_payments")
|
||||||
|
|
||||||
# Group transactions by person
|
adults_data = get_cached_data("attendance_regular", ATTENDANCE_SHEET_ID, get_members_with_fees)
|
||||||
grouped = {}
|
juniors_data = get_cached_data("attendance_juniors", ATTENDANCE_SHEET_ID, get_junior_members_with_fees)
|
||||||
|
member_names = []
|
||||||
|
if adults_data:
|
||||||
|
member_names.extend(name for name, _, _ in adults_data[0])
|
||||||
|
if juniors_data:
|
||||||
|
member_names.extend(name for name, _, _ in juniors_data[0])
|
||||||
|
|
||||||
|
grouped = group_payments_by_person(transactions, member_names)
|
||||||
|
# payments page also groups unmatched rows under a fallback key
|
||||||
for tx in transactions:
|
for tx in transactions:
|
||||||
person = str(tx.get("person", "")).strip()
|
if not str(tx.get("person", "")).strip():
|
||||||
if not person:
|
grouped.setdefault("Unmatched / Unknown", []).append(tx)
|
||||||
person = "Unmatched / Unknown"
|
for rows in grouped.values():
|
||||||
|
rows.sort(key=lambda t: str(t.get("date", "")), reverse=True)
|
||||||
# Handle multiple people (comma separated)
|
|
||||||
people = [p.strip() for p in person.split(",") if p.strip()]
|
|
||||||
for p in people:
|
|
||||||
# Strip markers
|
|
||||||
clean_p = re.sub(r"\[\?\]\s*", "", p)
|
|
||||||
if clean_p not in grouped:
|
|
||||||
grouped[clean_p] = []
|
|
||||||
grouped[clean_p].append(tx)
|
|
||||||
|
|
||||||
# Sort people and their transactions
|
|
||||||
sorted_people = sorted(grouped.keys())
|
sorted_people = sorted(grouped.keys())
|
||||||
for p in sorted_people:
|
|
||||||
# Sort by date descending
|
|
||||||
grouped[p].sort(key=lambda x: str(x.get("date", "")), reverse=True)
|
|
||||||
|
|
||||||
record_step("process_data")
|
record_step("process_data")
|
||||||
return render_template(
|
return render_template(
|
||||||
|
|||||||
@@ -0,0 +1,52 @@
|
|||||||
|
# Plan: Document plan-file location convention in `CLAUDE.md`
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
The user wants all plan files (created during Claude Code's plan mode) to live
|
||||||
|
inside the project at `docs/plans/`, with a creation timestamp in the filename.
|
||||||
|
This keeps planning artifacts version-controlled alongside the code, makes it
|
||||||
|
easy to see when each plan was drafted, and — critically — needs to be
|
||||||
|
discoverable by other contributors who use Claude Code on this repo. So the
|
||||||
|
convention belongs in `CLAUDE.md`, not in private agent memory.
|
||||||
|
|
||||||
|
## Approach
|
||||||
|
|
||||||
|
1. **Add a new section to `CLAUDE.md`** (placed near the existing "Changelog"
|
||||||
|
section, since both are about persisted artifacts that Claude maintains):
|
||||||
|
|
||||||
|
```markdown
|
||||||
|
## Plans
|
||||||
|
|
||||||
|
When Claude Code's plan mode is used, save the plan file inside the repo at
|
||||||
|
`docs/plans/YYYY-MM-DD-HHMM-<slug>.md` instead of the default
|
||||||
|
`~/.claude/plans/` location. Get the timestamp with
|
||||||
|
`date "+%Y-%m-%d-%H%M"` (matches the changelog convention). The `<slug>`
|
||||||
|
should be a short kebab-case summary of the plan's topic.
|
||||||
|
|
||||||
|
Create the `docs/plans/` directory on first use. Plan files are committed
|
||||||
|
to the repo so other contributors can review historical decisions.
|
||||||
|
```
|
||||||
|
|
||||||
|
2. **Create the `docs/plans/` directory** with a `.gitkeep` (or just let it
|
||||||
|
appear when the first plan is moved in) so the path exists.
|
||||||
|
|
||||||
|
3. **Move this current plan** into the new location once plan mode exits:
|
||||||
|
`docs/plans/2026-05-03-1200-document-plan-location-convention.md`
|
||||||
|
(timestamp will be re-generated with the actual `date` output).
|
||||||
|
|
||||||
|
4. **No memory entry needed** — the rule lives in `CLAUDE.md` and is loaded
|
||||||
|
automatically into every Claude Code session in this repo.
|
||||||
|
|
||||||
|
## Files touched
|
||||||
|
|
||||||
|
- [CLAUDE.md](CLAUDE.md) — add the new "## Plans" section.
|
||||||
|
- New directory: [docs/plans/](docs/plans/) — created on first use.
|
||||||
|
- Move this plan file from `~/.claude/plans/...` into `docs/plans/` with the
|
||||||
|
proper timestamped filename.
|
||||||
|
|
||||||
|
## Verification
|
||||||
|
|
||||||
|
- `grep -A 5 "## Plans" CLAUDE.md` shows the new section.
|
||||||
|
- `ls docs/plans/` lists this plan file with a `YYYY-MM-DD-HHMM-` prefix.
|
||||||
|
- Next time plan mode is entered in this repo, the new plan is written to
|
||||||
|
`docs/plans/` with a fresh timestamp (verify by re-entering plan mode).
|
||||||
158
docs/plans/2026-05-03-2349-go-backend-rewrite-progress.md
Normal file
158
docs/plans/2026-05-03-2349-go-backend-rewrite-progress.md
Normal file
@@ -0,0 +1,158 @@
|
|||||||
|
# Go Rewrite — Progress Tracker
|
||||||
|
|
||||||
|
Companion to [2026-05-03-2349-go-backend-rewrite.md](2026-05-03-2349-go-backend-rewrite.md).
|
||||||
|
|
||||||
|
**Current milestone:** M2 — Pure-domain helpers
|
||||||
|
**Started:** 2026-05-04
|
||||||
|
**Last updated:** 2026-05-04
|
||||||
|
|
||||||
|
## How to use
|
||||||
|
|
||||||
|
- Tick a checkbox when the task's PR/commit lands. Append the SHA in the same
|
||||||
|
line: `[x] **M1.1** ... — `abc1234``.
|
||||||
|
- One task = one focused commit or PR. If a task balloons, split it and add
|
||||||
|
sub-tasks below the parent.
|
||||||
|
- Note decisions, surprises, or blockers under "Notes & decisions" at the
|
||||||
|
bottom — that's where future-you (or a contributor) will look first.
|
||||||
|
- Don't reorder milestones. Within a milestone, tasks can be done in any
|
||||||
|
order unless explicitly noted.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## M1 — Skeleton + tooling
|
||||||
|
|
||||||
|
Goal: `make web-go` serves a hello page on :8080 in parallel with `make web-py` on :5001. Lint clean.
|
||||||
|
|
||||||
|
- [x] **M1.1** Create `go/` tree skeleton + `go.mod` initialized to latest stable Go
|
||||||
|
- [x] **M1.2** Add `cmd/fuj/main.go` with subcommand dispatcher — stdlib `flag` + `os.Args[1]` switch
|
||||||
|
- [x] **M1.3** Wire `fuj server` subcommand: `net/http` ServeMux on `:8080`, plaintext hello page
|
||||||
|
- [x] **M1.4** Add Makefile targets: `go-build`, `go-test`, `go-run`, `go-lint`
|
||||||
|
- [x] **M1.5** Rename existing `make web` → `make web-py`; added `make web-go`; kept `make web` as alias
|
||||||
|
- [x] **M1.6** Add `go/.golangci.yml` (govet, staticcheck, errcheck, gofumpt, unused) + `make go-lint` clean
|
||||||
|
- [x] **M1.7** Write `go/build/Dockerfile` (multi-stage `golang:1.26` → `alpine:3`); parallel `build-go` job in Gitea CI
|
||||||
|
- [x] **M1.8** Add `internal/config` package mirroring `scripts/config.py` (same env var names + defaults)
|
||||||
|
- [x] **M1.9** Add `internal/logging` (slog, level from config) + `middleware/timer.go` (method/path/status/ms)
|
||||||
|
- [x] **M1.10** Gate passed: `make go-build`, `make go-lint`, `make go-test`, `curl :8080` all green; CHANGELOG entry added
|
||||||
|
|
||||||
|
**Gate:** ✅ `make go-build` succeeds, `curl localhost:8080` returns hello page, `make go-lint` clean.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## M2 — Pure-domain helpers (port leaf-first)
|
||||||
|
|
||||||
|
Goal: every pure function from the Python backend exists in Go with a parity test against captured fixtures (M3 produces fixtures in parallel — order is M2.1 → M3.1/M3.2 → M3.3+ alongside M2.2+).
|
||||||
|
|
||||||
|
Each task: port the function, write Go unit tests for fresh cases, hook into the Tier-1 parity runner.
|
||||||
|
|
||||||
|
- [ ] **M2.1** `domain/czech.Normalize` — port [czech_utils.py](scripts/czech_utils.py) `normalize` (NFKD + combining-mark strip + lowercase)
|
||||||
|
- [ ] **M2.2** `domain/czech.ParseMonthReferences` — port `parse_month_references` (45 month declensions, range wrap, year inference)
|
||||||
|
- [ ] **M2.3** `domain/fees.CalculateFee` — port [attendance.py](scripts/attendance.py) `calculate_fee` (constants table)
|
||||||
|
- [ ] **M2.4** `domain/fees.CalculateJuniorFee` — port `calculate_junior_fee` with `Expected{Value int; Unknown bool}` for the `"?"` sentinel
|
||||||
|
- [ ] **M2.5** `domain/money.ParseCZK` — port [infer_payments.py](scripts/infer_payments.py) `parse_czk_amount` (Czech locale: comma decimal, dot/space thousand separators)
|
||||||
|
- [ ] **M2.6** `domain/synch.GenerateSyncID` — port [sync_fio_to_sheets.py](scripts/sync_fio_to_sheets.py) `generate_sync_id` (SHA-256, byte-stable hash; verify float string format against real sheet rows)
|
||||||
|
- [ ] **M2.7** `domain/matching.BuildNameVariants` + `MatchMembers` — port `_build_name_variants` and `match_members` from [match_payments.py](scripts/match_payments.py) (auto vs review confidence, common-surname filter)
|
||||||
|
- [ ] **M2.8** `domain/matching.InferTransactionDetails` — port `infer_transaction_details` (composes name + month parsing)
|
||||||
|
- [ ] **M2.9** `domain/matching.FormatDate` — port `format_date` (handles Google Sheets serial-day numbers since 1899-12-30)
|
||||||
|
- [ ] **M2.10** `domain/reconcile.Reconcile` — port `reconcile` (three-phase allocation: greedy / proportional with float-remainder absorption / even-split fallback). The single most load-bearing function; budget extra time.
|
||||||
|
- [ ] **M2.11** `fuj fees` subcommand wired up via `domain/fees` + (M4-stub) attendance loader — fail gracefully on missing IO until M4 lands
|
||||||
|
- [ ] **M2.12** `fuj reconcile` subcommand similarly stubbed
|
||||||
|
|
||||||
|
**Gate:** `cd go && go test -tags=parity ./tests/parity/pure/...` green for every fixture in `tests/fixtures/pure/`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## M3 — Fixture capture + characterization framework
|
||||||
|
|
||||||
|
Goal: deterministic, PII-free fixture corpus that drives parity tests. Runs in parallel with M2 (M3.1/M3.2 unblocks M2.1).
|
||||||
|
|
||||||
|
- [ ] **M3.1** `scripts/capture_fixtures.py` — pure-function output dumper. Reads inputs from stdin / argv, prints `{"input":..., "output":...}` JSON
|
||||||
|
- [ ] **M3.2** `scripts/scrub_fixtures.py` — replaces names with `Member_<8hex>` (deterministic per name); scrambles sender/account/VS/bank_id with stable bijection; preserves dates, amounts, exception keys
|
||||||
|
- [ ] **M3.3** Capture pure-fn fixtures for M2.1–M2.9 (run helper + scrubber, commit to `tests/fixtures/pure/<func>/<case>.json`)
|
||||||
|
- [ ] **M3.4** Capture ~10 reconcile fixtures spanning every code path: greedy, proportional (float remainder), even-split, out-of-window credit, exception override, `other:` purpose, junior `"?"`, multi-person comma-split, multi-month range, unmatched. Commit to `tests/fixtures/reconcile/`
|
||||||
|
- [ ] **M3.5** Hook fixtures into Tier-1 test runner with `-tags=parity` build constraint
|
||||||
|
- [ ] **M3.6** Document fixture-refresh workflow in `tests/fixtures/README.md` (what to do when sheet schema changes)
|
||||||
|
|
||||||
|
**Gate:** `tests/fixtures/` populated; M2 parity tests green; raw `tmp/*.json` confirmed gitignored.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## M4 — IO layer behind interfaces
|
||||||
|
|
||||||
|
Goal: every external IO (Sheets, Drive, Fio, file cache) accessed through a narrow Go interface with both a real and a fake implementation.
|
||||||
|
|
||||||
|
- [ ] **M4.1** Design IO interfaces (`SheetsClient`, `DriveClient`, `FioClient`, `FileCache`) + in-memory fakes seeded from M3 fixtures
|
||||||
|
- [ ] **M4.2** `internal/io/sheets` — Google client (read + append + batchUpdate); integration test against a separate test sheet (NOT prod)
|
||||||
|
- [ ] **M4.3** `internal/io/drive` — Drive `modifiedTime` client + integration test
|
||||||
|
- [ ] **M4.4** `internal/io/fio` — API JSON impl (token-based); parses by hardcoded `column0..column22` indices matching [fio_utils.py](scripts/fio_utils.py)
|
||||||
|
- [ ] **M4.5** `internal/io/fio` — transparent-page HTML scraper using `golang.org/x/net/html` token visitor; targets the **second** `<table class="table">`
|
||||||
|
- [ ] **M4.6** `internal/io/cache` — FileCache with `modifiedTime` gating + two TTL knobs + atomic writes (`os.Rename`)
|
||||||
|
- [ ] **M4.7** `services/banksync.SyncToSheets` + `fuj sync` subcommand
|
||||||
|
- [ ] **M4.8** `services/banksync.InferPayments` + `fuj infer [--dry-run]` subcommand
|
||||||
|
|
||||||
|
**Gate:** `go test -tags=integration ./internal/io/...` round-trips against test sheet; default-tag tests run on fakes.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## M5 — JSON-only `/api/...` routes
|
||||||
|
|
||||||
|
Goal: byte-equal JSON between Python and Go for every route. This is the parity contract.
|
||||||
|
|
||||||
|
- [ ] **M5.1** Hand-author Go structs for `/api/adults`, `/api/juniors`, `/api/payments`, `/api/version` with explicit `json:` tags matching Python keys; emit JSON Schemas via `github.com/invopop/jsonschema` to `tests/fixtures/api-schema/`
|
||||||
|
- [ ] **M5.2** Implement Go handlers for `/api/*` routes composing `services/*` results into the JSON structs
|
||||||
|
- [ ] **M5.3** Add Python `/api/X` shadow endpoints in [app.py](app.py): `jsonify(view_model_dict)` — no transformation
|
||||||
|
- [ ] **M5.4** Build `cmd/parity/main.go`: hits both backends' `/api/X`, normalizes allowlist (`render_time.total`, `build_meta`), prints `cmp.Diff`. Add `make parity` target
|
||||||
|
|
||||||
|
**Gate:** For each route, `make parity` reports zero non-allowlisted diffs across the M3 fixture corpus.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## M6 — Go-native HTML frontend
|
||||||
|
|
||||||
|
Goal: feature-equivalent UX on the Go side, designed cleanly. Not a Jinja port.
|
||||||
|
|
||||||
|
- [ ] **M6.1** Template skeleton: base layout, nav (Adults/Juniors/Payments/Sync/Flush), terminal-green-on-black theme; `embed.FS` for `templates/` + `static/`
|
||||||
|
- [ ] **M6.2** `/adults` page: table, name filter input, month range filter, totals row, credits/debts/unmatched sections, Pay buttons that link to `/qr`
|
||||||
|
- [ ] **M6.3** `/juniors` page: same structure + per-month J/A attendance breakdown + `"?"` sentinel rendering
|
||||||
|
- [ ] **M6.4** `/payments` page: grouped-by-person ledger view
|
||||||
|
- [ ] **M6.5** Modal JS module (`static/js/member-detail.js`): fetches `/api/adults` (or juniors), renders status/exceptions/transactions on row click; keyboard nav (Esc, ↑/↓)
|
||||||
|
- [ ] **M6.6** `/qr`, `/sync-bank`, `/flush-cache`, `/version` pages
|
||||||
|
- [ ] **M6.7** Wire `embed.FS` into handlers; verify single-binary deployment includes all assets
|
||||||
|
|
||||||
|
**Gate:** Browser smoke on :8080: all pages render, name+month filters work, modal opens with correct data, QR loads, sync/flush work end-to-end.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## M7 — Parallel-running watch period
|
||||||
|
|
||||||
|
Goal: prove parity over real time before flipping the default.
|
||||||
|
|
||||||
|
- [ ] **M7.1** Add Go service to `docker-compose.yml` on different port (alongside Python container)
|
||||||
|
- [ ] **M7.2** Set up `parity-nightly.yml` Gitea workflow: boot both, replay fixed transaction script, fail on diff
|
||||||
|
- [ ] **M7.3** Run `make parity` daily for 7–14 days, log any diffs; investigate and fix root cause (don't just allowlist)
|
||||||
|
- [ ] **M7.4** Manual feature parity check: walk through every UI feature on both sides, sign off in Notes section
|
||||||
|
|
||||||
|
**Gate:** Zero non-allowlisted JSON diffs over 7 consecutive days, including a sync-bank execution + flush + attendance update; user sign-off on UI feature parity.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## M8 — Cutover + Python retirement
|
||||||
|
|
||||||
|
Goal: Go is the one true backend.
|
||||||
|
|
||||||
|
- [ ] **M8.1** Update bookmarks, README, CLAUDE.md to point at Go (`make web` aliases to `make web-go`)
|
||||||
|
- [ ] **M8.2** Run Go-only for 2 weeks including a month-end settlement; keep Python container available but unrouted
|
||||||
|
- [ ] **M8.3** Manual reconciliation review: produce a balance report on `python-final` and on Go for the same period; sign off they match
|
||||||
|
- [ ] **M8.4** Tag final Python image as `python-final` in registry; remove Python service from `docker-compose.yml`
|
||||||
|
- [ ] **M8.5** Delete [app.py](app.py), [scripts/](scripts/), Python `Dockerfile`, [tests/](tests/), `pyproject.toml`, `uv.lock`
|
||||||
|
- [ ] **M8.6** Update [CLAUDE.md](CLAUDE.md) to reflect Go-only state (commands, architecture, key modules); CHANGELOG entry
|
||||||
|
|
||||||
|
**Gate:** Two consecutive months of Go-only operation with end-of-month settlement complete; zero rollbacks.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Notes & decisions
|
||||||
|
|
||||||
|
(Add entries as you go. Format: `YYYY-MM-DD — short note`.)
|
||||||
|
|
||||||
|
- 2026-05-04 — Plan approved. Versioning policy: latest stable for Go and all libs at the time M1 starts. Frontends explicitly allowed to diverge between Python and Go; only the JSON API contract is parity-locked. No reverse proxy — both backends run on different ports via `make web-py` / `make web-go`.
|
||||||
|
- 2026-05-04 — M1 complete. Dockerfile base changed from `distroless/static:nonroot` → `alpine:3` for debuggability (can tighten later). CLI dispatcher uses stdlib `flag`; module path `fuj-management/go`. golangci-lint v1 embedded gofumpt merges all imports into one group (no stdlib/local split) — accepted as the project style.
|
||||||
424
docs/plans/2026-05-03-2349-go-backend-rewrite.md
Normal file
424
docs/plans/2026-05-03-2349-go-backend-rewrite.md
Normal file
@@ -0,0 +1,424 @@
|
|||||||
|
# Plan: Full Go rewrite of the Python/Flask backend
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
The current Flask app ([app.py](app.py) + [scripts/](scripts/), ~2400 LOC of
|
||||||
|
Python) handles attendance-based fee calculation, Fio bank sync, payment
|
||||||
|
reconciliation, and a server-rendered dashboard. The user wants a full
|
||||||
|
rewrite in Go with two goals:
|
||||||
|
|
||||||
|
1. **Quality Go code** as the primary outcome — idiomatic stdlib-first
|
||||||
|
design, strong typing, proper layering. The Python codebase grew
|
||||||
|
organically and mixes domain logic, IO, and HTTP concerns.
|
||||||
|
2. **Feature-parity certainty** — no behavioural drift between the Python
|
||||||
|
and Go versions on anything that touches money. Reconciliation is real
|
||||||
|
money; silent divergence is unacceptable.
|
||||||
|
|
||||||
|
**Switchable runtime**: both backends run on different TCP ports, started
|
||||||
|
independently via Makefile targets (`make web-py` on :5001, `make web-go` on
|
||||||
|
:8080). The user opens whichever they want in a browser. No reverse proxy,
|
||||||
|
no traffic-splitting, no shared frontend constraint — just two services
|
||||||
|
that read the same Google Sheets and the same `tmp/` cache.
|
||||||
|
|
||||||
|
**Frontends are allowed to diverge.** The Go web layer is designed cleanly
|
||||||
|
in its own right rather than as a byte-compatible Jinja port. Both backends
|
||||||
|
expose a JSON API (`/api/...`) with an identical contract — that's what
|
||||||
|
parity testing locks down. Rendered HTML and inline JS can be different.
|
||||||
|
|
||||||
|
## Versioning policy
|
||||||
|
|
||||||
|
- **Go**: latest stable release at project start. Pin in `go.mod` via the
|
||||||
|
`go` directive (e.g. `go 1.X`) and use the matching `golang:1.X` builder
|
||||||
|
image. Bump on each new minor as it lands stable.
|
||||||
|
- **Go libraries**: latest stable for every dependency in `go.mod`; run
|
||||||
|
`go get -u ./... && go mod tidy` at the start and quarterly thereafter.
|
||||||
|
- **Python deps** (during the parallel-run period): keep
|
||||||
|
[pyproject.toml](pyproject.toml) on its current versions to avoid
|
||||||
|
destabilizing the parity baseline; bump only after Python retires.
|
||||||
|
- **Base images**: `golang:latest-stable` builder → `gcr.io/distroless/static:latest`
|
||||||
|
runtime, both pinned by digest in CI for reproducibility.
|
||||||
|
- **CI runners**: latest stable Linux image on Gitea Actions.
|
||||||
|
|
||||||
|
The plan does not hardcode specific version numbers below — implementation
|
||||||
|
picks current-stable at the time M1 starts.
|
||||||
|
|
||||||
|
## Approach summary
|
||||||
|
|
||||||
|
- **Three-layer Go architecture**: pure domain (no IO) → IO clients (behind
|
||||||
|
interfaces, easily faked) → HTTP/services (composition).
|
||||||
|
- **Capture-then-port**: dump current Python outputs as JSON fixtures, port
|
||||||
|
Go function-by-function, assert byte-equality with `cmp.Diff`.
|
||||||
|
- **JSON contract is the spec, not the templates.** Each Python route gets
|
||||||
|
an `/api/X` shadow that returns the dict already passed to the template.
|
||||||
|
Go defines typed structs matching that shape; both sides validate against
|
||||||
|
generated JSON Schema.
|
||||||
|
- **Money is integer CZK**: existing fees are integer CZK (750/200/500);
|
||||||
|
keep it that way to avoid float drift in reconcile allocation. Where
|
||||||
|
Sheets returns floats, parse and round at the boundary.
|
||||||
|
- **Frontend rewrite, not port**: Go uses `html/template` with cleanly
|
||||||
|
organized templates and JS extracted into static files served via
|
||||||
|
`embed.FS`. Same UX (filterable table, member-detail modal, QR launcher)
|
||||||
|
but designed natively, no Jinja-port baggage.
|
||||||
|
|
||||||
|
## Go project layout
|
||||||
|
|
||||||
|
`go/` lives at the repo root alongside `scripts/` and `templates/` so both
|
||||||
|
backends share the same git history during migration.
|
||||||
|
|
||||||
|
```
|
||||||
|
go/
|
||||||
|
cmd/
|
||||||
|
fuj/main.go # single binary, subcommands: server | fees | sync | infer | reconcile
|
||||||
|
parity/main.go # diff tool: hits both backends' /api/X, prints JSON diff
|
||||||
|
internal/
|
||||||
|
domain/ # pure, no IO, no net/*
|
||||||
|
czech/ # normalize, parse_month_references
|
||||||
|
fees/ # calculate_fee, calculate_junior_fee, "?" sentinel type
|
||||||
|
money/ # parse_czk_amount, format helpers
|
||||||
|
reconcile/ # reconcile() + Ledger, MemberResult types
|
||||||
|
matching/ # _build_name_variants, match_members, infer_transaction_details
|
||||||
|
synch/ # generate_sync_id (pure hash)
|
||||||
|
io/ # IO behind interfaces, all impls have an in-memory fake
|
||||||
|
sheets/ # SheetsClient + Google impl + fake
|
||||||
|
drive/ # DriveClient for modifiedTime
|
||||||
|
fio/ # FioClient: API JSON impl + transparent-page HTML scraper
|
||||||
|
cache/ # FileCache with modifiedTime gating + two TTL knobs
|
||||||
|
services/ # composition layer; pure + IO, no HTTP
|
||||||
|
attendance/ # GetMembersWithFees, GetJuniorMembersWithFees
|
||||||
|
payments/ # FetchTransactions, FetchExceptions, BuildView
|
||||||
|
banksync/ # SyncToSheets, InferPayments (write ops)
|
||||||
|
web/
|
||||||
|
handlers/ # one file per route family
|
||||||
|
view/ # HTML view-model structs (per route)
|
||||||
|
api/ # JSON view-model structs (the parity-locked contract)
|
||||||
|
templates/ # *.tmpl, embed.FS — designed natively, not a Jinja port
|
||||||
|
static/ # js/*.js, css/*.css served via embed.FS
|
||||||
|
middleware/ # request timer, recovery, slog
|
||||||
|
config/ # mirrors scripts/config.py (env loading)
|
||||||
|
qr/ # SPD string builder + PNG via go-qrcode
|
||||||
|
tests/
|
||||||
|
fixtures/ # JSON fixtures captured from Python (PII-scrubbed)
|
||||||
|
parity/ # Go-side characterization tests (replay fixtures)
|
||||||
|
build/Dockerfile # multi-stage: latest-stable golang builder → distroless static
|
||||||
|
go.mod
|
||||||
|
```
|
||||||
|
|
||||||
|
## Library choices
|
||||||
|
|
||||||
|
All on latest stable as per the versioning policy above.
|
||||||
|
|
||||||
|
| Concern | Pick | Rationale |
|
||||||
|
|---|---|---|
|
||||||
|
| HTTP routing | `net/http` ServeMux | 8 static routes; no need for chi/gin given modern stdlib pattern matching |
|
||||||
|
| Templates | `html/template` | Auto-escaping; native Go feel |
|
||||||
|
| Static assets | `embed.FS` | Single binary, no loose files |
|
||||||
|
| Sheets/Drive | `google.golang.org/api/{sheets/v4,drive/v3}` + `option` | Official client; service-account auth via `option.WithCredentialsFile` |
|
||||||
|
| OAuth | `golang.org/x/oauth2/google` (token only; drop installed-app flow + pickle) | Production already uses service accounts |
|
||||||
|
| QR PNG | `github.com/skip2/go-qrcode` | Mature, byte-stable PNG output |
|
||||||
|
| NFKD | `golang.org/x/text/unicode/norm` + `unicode.IsMark` | Direct equivalent of `unicodedata.normalize("NFKD", ...)` |
|
||||||
|
| HTML scrape | `golang.org/x/net/html` token visitor | Counts `<table class="table">` to target the second one |
|
||||||
|
| CSV | `encoding/csv` (stdlib) | Match for Python `csv.reader` |
|
||||||
|
| Logging | `log/slog` (stdlib) | Honors `LOG_LEVEL` env |
|
||||||
|
| Diff/testing | `testing` + `github.com/google/go-cmp/cmp` | Readable `cmp.Diff` for parity assertions |
|
||||||
|
| Lint | `golangci-lint` (govet, staticcheck, errcheck, gofumpt, unused) | Standard quality gate |
|
||||||
|
|
||||||
|
## Migration sequencing — eight milestones with hard gates
|
||||||
|
|
||||||
|
**M1 — Skeleton + tooling.** Create `go/` tree, `go.mod` (latest stable
|
||||||
|
Go), Makefile targets (`go-build`, `go-test`, `go-run`, `web-go`),
|
||||||
|
`golangci-lint` config. `cmd/fuj server` prints a hello + version and
|
||||||
|
listens on :8080.
|
||||||
|
*Gate:* `make go-build` succeeds; `make web-go` serves a "hello" page on
|
||||||
|
:8080 in parallel with `make web` on :5001; lint clean.
|
||||||
|
|
||||||
|
**M2 — Pure-domain helpers, port leaf-first.** Order:
|
||||||
|
[czech_utils.py](scripts/czech_utils.py) `normalize` → `parse_month_references` →
|
||||||
|
[attendance.py](scripts/attendance.py) `calculate_fee`/`calculate_junior_fee` →
|
||||||
|
[infer_payments.py](scripts/infer_payments.py) `parse_czk_amount` →
|
||||||
|
[sync_fio_to_sheets.py](scripts/sync_fio_to_sheets.py) `generate_sync_id` →
|
||||||
|
[match_payments.py](scripts/match_payments.py) helpers (`_build_name_variants`,
|
||||||
|
`match_members`, `infer_transaction_details`, `format_date`) → `reconcile`.
|
||||||
|
Each gets a Go unit test plus a parity test driven by JSON fixtures from M3.
|
||||||
|
Also: `fuj fees` and `fuj reconcile` subcommands wired up (pure-domain CLIs).
|
||||||
|
*Gate:* All ported helpers pass parity tests.
|
||||||
|
|
||||||
|
**M3 — Fixture capture + characterization framework.** Build
|
||||||
|
`scripts/capture_fixtures.py` (Python helper that prints function results as
|
||||||
|
JSON to stdout — user pipes to disk) and `scripts/scrub_fixtures.py`
|
||||||
|
(replaces member names with deterministic pseudonyms `Member_<8hex>`,
|
||||||
|
scrambles sender/account/VS/bank_id while preserving structural
|
||||||
|
relationships, dates, amounts, exception keys). Capture ~10 reconcile
|
||||||
|
fixtures spanning every code path: greedy, proportional with float
|
||||||
|
remainder, even-split fallback, out-of-window credit, exception override,
|
||||||
|
`other:` purpose, junior `"?"`, comma-separated multi-person, multi-month
|
||||||
|
range, unmatched.
|
||||||
|
*Gate:* `tests/fixtures/` populated and committed; M2 parity tests green.
|
||||||
|
|
||||||
|
**M4 — IO layer behind interfaces.** Implement Sheets/Drive/Fio clients
|
||||||
|
matching Python return shapes. Drop the OAuth+pickle path entirely (service
|
||||||
|
account only). All clients have in-memory fakes for tests. Wire `fuj sync`
|
||||||
|
and `fuj infer` subcommands.
|
||||||
|
*Gate:* `go test -tags=integration ./internal/io/...` round-trips against a
|
||||||
|
test sheet (separate from prod); default-tag tests use fakes.
|
||||||
|
|
||||||
|
**M5 — JSON-only `/api/...` routes.** Add 8 Go route handlers that return
|
||||||
|
JSON. Add symmetric `/api/X` shadow endpoints in [app.py](app.py) that
|
||||||
|
`jsonify` the existing view-model dict (no transformation).
|
||||||
|
*Gate:* For each route, `cmd/parity` asserts
|
||||||
|
`cmp.Diff(python.json, go.json) == ""` modulo allowlist
|
||||||
|
(`render_time.total`, `build_meta`).
|
||||||
|
|
||||||
|
**M6 — Go-native HTML frontend.** Design Go templates cleanly (not a Jinja
|
||||||
|
port). Extract JS from inline into `internal/web/static/js/*.js` served via
|
||||||
|
`embed.FS`. Vanilla JS, no framework — same UX as Python (sortable table,
|
||||||
|
member-detail modal, name filter, month range filter, QR launcher) but
|
||||||
|
organized as proper modules. Templates render the JSON API response into
|
||||||
|
HTML; frontend JS fetches additional data from `/api/X` for the modal
|
||||||
|
rather than embedding `member_data` in `<script>`.
|
||||||
|
*Gate:* Browser smoke test of all routes on :8080 covers: name filter,
|
||||||
|
month filter, modal opens with correct months/transactions/exceptions, QR
|
||||||
|
modal renders, navigation between adults/juniors/payments works.
|
||||||
|
|
||||||
|
**M7 — Parallel-running watch period.** Both `make web-py` and `make web-go`
|
||||||
|
running locally (and in production via two containers on different ports).
|
||||||
|
Daily/manual `cmd/parity` runs catch any JSON drift. The user verifies the
|
||||||
|
Go UI matches what they expect feature-by-feature against the Python UI.
|
||||||
|
Run 1–2 weeks.
|
||||||
|
*Gate:* Zero non-allowlisted JSON diffs over 7 consecutive days, including
|
||||||
|
a sync-bank execution, a flush, and an attendance update. User sign-off
|
||||||
|
that the Go UI is feature-complete.
|
||||||
|
|
||||||
|
**M8 — Cutover + Python retirement.** Switch the bookmarked URL / docs to
|
||||||
|
the Go port. Keep Python container running but unrouted (or stopped) for
|
||||||
|
1 week as rollback. Then delete [app.py](app.py), [scripts/](scripts/),
|
||||||
|
the Python `Dockerfile`, and the Python tests. Update
|
||||||
|
[CLAUDE.md](CLAUDE.md) to reflect the Go-only state.
|
||||||
|
*Gate:* Two consecutive months of Go-only operation including end-of-month
|
||||||
|
settlement.
|
||||||
|
|
||||||
|
## CLI port (decided: port as Go subcommands)
|
||||||
|
|
||||||
|
Single Go binary `fuj` with subcommands replacing the existing Makefile
|
||||||
|
targets. Each reuses the domain layer directly:
|
||||||
|
|
||||||
|
| Old | New | Backed by | Milestone |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `make fees` | `fuj fees` | `domain/fees` + `services/attendance` | M2 |
|
||||||
|
| `make reconcile` | `fuj reconcile` | `domain/reconcile` | M2 |
|
||||||
|
| `make sync-2026` | `fuj sync --year=2026` | `services/banksync.SyncToSheets` | M4 |
|
||||||
|
| `make infer` | `fuj infer [--dry-run]` | `services/banksync.InferPayments` | M4 |
|
||||||
|
| `make web` (py) | stays as Python `make web-py` until M8 | — | — |
|
||||||
|
| `make web-go` | `fuj server` | `web/handlers` | M1 |
|
||||||
|
|
||||||
|
Makefile targets get rewritten to invoke `./bin/fuj <subcommand>` once each
|
||||||
|
is ported. The Python `make` targets for already-ported commands stay as
|
||||||
|
`make X-py` aliases until M8, so you can run either side for cross-checks.
|
||||||
|
|
||||||
|
## JSON API contract strategy
|
||||||
|
|
||||||
|
**Go-defines, Python-conforms** with a 1-step bootstrap:
|
||||||
|
|
||||||
|
1. Run Python locally and dump `result["members"]`, `formatted_results`,
|
||||||
|
`monthly_totals`, etc., to JSON. This is the spec.
|
||||||
|
2. Hand-author Go structs with explicit `json:` tags matching exact Python
|
||||||
|
keys (`total_balance`, `original_expected`, `attendance_count` — no
|
||||||
|
reliance on default lowercasing).
|
||||||
|
3. Generate `tests/fixtures/api-schema/*.schema.json` from the Go structs
|
||||||
|
using `github.com/invopop/jsonschema`. Commit them.
|
||||||
|
4. Add a Python-side schema validator running in CI against the new
|
||||||
|
`/api/X` responses.
|
||||||
|
|
||||||
|
**Two known-tricky shapes:**
|
||||||
|
|
||||||
|
- Junior `expected: int | "?"` →
|
||||||
|
```go
|
||||||
|
type Expected struct{ Value int; Unknown bool }
|
||||||
|
// MarshalJSON emits 42 or "?"
|
||||||
|
```
|
||||||
|
Same for `original_expected`.
|
||||||
|
- Tuple dict keys `(normalize(name), normalize(period))` for exceptions —
|
||||||
|
internal only, never crosses JSON. Use
|
||||||
|
`map[ExceptionKey]Exception` with `ExceptionKey struct{ Name, Period string }`.
|
||||||
|
|
||||||
|
## Characterization test harness — two tiers
|
||||||
|
|
||||||
|
(HTML rendering parity dropped: frontends are intentionally different.)
|
||||||
|
|
||||||
|
**Tier 1 — Pure-function parity** (fast, every commit). Fixtures at
|
||||||
|
`tests/fixtures/pure/<func>/<case>.json` containing `{input, output}`,
|
||||||
|
captured once via `scripts/capture_fixtures.py`. Go test reads each, calls
|
||||||
|
the ported function, asserts deep equality with `cmp.Diff`. Functions in
|
||||||
|
scope: `normalize`, `parse_month_references`, `parse_czk_amount`,
|
||||||
|
`parse_czech_amount`, `parse_czech_date`, `format_date`,
|
||||||
|
`_build_name_variants`, `match_members`, `infer_transaction_details`,
|
||||||
|
`generate_sync_id`, `calculate_fee`, `calculate_junior_fee`, `reconcile`.
|
||||||
|
|
||||||
|
**Tier 2 — JSON API parity** (medium, on PR + nightly). `cmd/parity/main.go`
|
||||||
|
hits both `:5001/api/X` and `:8080/api/X` with a fixture-seeded `tmp/`
|
||||||
|
cache, normalizes volatile fields (`render_time`, build metadata), asserts
|
||||||
|
byte-equality. Cache freezing: pre-populate `tmp/*_cache.json` from
|
||||||
|
scrubbed snapshots so both backends read identical data.
|
||||||
|
|
||||||
|
**PII scrubbing** is mandatory ([CLAUDE.md](CLAUDE.md): "Member data must
|
||||||
|
never be committed"). `scripts/scrub_fixtures.py` produces deterministic
|
||||||
|
pseudonyms preserving uniqueness and structural relationships. Only
|
||||||
|
scrubbed fixtures land in `tests/fixtures/`; raw `tmp/*.json` stays
|
||||||
|
gitignored.
|
||||||
|
|
||||||
|
## Side-by-side runtime
|
||||||
|
|
||||||
|
Two services on different ports, started independently. No reverse proxy.
|
||||||
|
|
||||||
|
```
|
||||||
|
make web-py # Python on :5001 (existing target, perhaps renamed from `make web`)
|
||||||
|
make web-go # Go on :8080
|
||||||
|
```
|
||||||
|
|
||||||
|
Both read the same Google Sheets and write to the same `tmp/` cache
|
||||||
|
directory. The user opens `localhost:5001` or `localhost:8080` directly to
|
||||||
|
A/B compare.
|
||||||
|
|
||||||
|
**Cache directory coordination**: both backends use `tmp/`. Go writes via
|
||||||
|
`os.WriteFile` to `tmp/<key>_cache.json.tmp` then `os.Rename` (atomic on
|
||||||
|
Linux). Python's writes are pre-existing-non-atomic; accept until Python
|
||||||
|
retires.
|
||||||
|
|
||||||
|
**Sync coordination**: `/sync-bank` is non-idempotent under concurrency.
|
||||||
|
Both backends `flock` on `tmp/sync.lock`; Go uses `syscall.Flock`. (In
|
||||||
|
practice the user is unlikely to trigger sync from both UIs at once, but
|
||||||
|
the lock is cheap insurance.)
|
||||||
|
|
||||||
|
**Production deployment**: keep the existing Python container; add a Go
|
||||||
|
container in `docker-compose.yml` exposed on a different port. After M8,
|
||||||
|
remove the Python service.
|
||||||
|
|
||||||
|
## CI/CD
|
||||||
|
|
||||||
|
Currently zero test CI ([.gitea/workflows/build.yaml](.gitea/workflows/build.yaml)
|
||||||
|
only does `docker build`/`push`). Add `/.gitea/workflows/test.yml`:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
jobs:
|
||||||
|
python-tests: # fix M3 broken-test references first
|
||||||
|
- uv sync && pytest tests/
|
||||||
|
go-tests:
|
||||||
|
- cd go && go test -race ./...
|
||||||
|
- cd go && golangci-lint run
|
||||||
|
parity-pure: # Tier 1
|
||||||
|
- cd go && go test -tags=parity ./tests/parity/...
|
||||||
|
```
|
||||||
|
|
||||||
|
Branch protection: `python-tests`, `go-tests`, `parity-pure` block merge.
|
||||||
|
Tier-2 parity runs nightly via `parity-nightly.yml` (boots both servers
|
||||||
|
via docker-compose with seeded caches, replays a fixed transaction script,
|
||||||
|
fails on any non-allowlisted diff).
|
||||||
|
|
||||||
|
A new Go `build/Dockerfile` (multi-stage: latest-stable `golang` builder →
|
||||||
|
`gcr.io/distroless/static:latest`, both pinned by digest) mirrors the
|
||||||
|
existing Python build job and produces a single static binary image.
|
||||||
|
|
||||||
|
## Risk register (top 4)
|
||||||
|
|
||||||
|
(Template auto-escape divergence dropped: irrelevant when frontends differ.)
|
||||||
|
|
||||||
|
1. **Sync ID hash drift** — HIGH/HIGH. Python builds the SHA-256 input by
|
||||||
|
`str()`-ing each field then `.lower()`-ing the joined string;
|
||||||
|
`str(750.0) == "750.0"`, `str(750) == "750"`. If Sheets API returns
|
||||||
|
floats in Python but Go unmarshals as int, `750` vs `750.0` → different
|
||||||
|
hash → duplicate rows. *Mitigation:* dedicated parity test with ~50
|
||||||
|
real-row fixtures; if Go can't reproduce Python's float string format,
|
||||||
|
normalize at the boundary (round to 2 decimals, format with explicit
|
||||||
|
precision).
|
||||||
|
2. **Float allocation in `reconcile()` proportional phase** — HIGH/MEDIUM.
|
||||||
|
Python's "last month absorbs remainder" depends on dict iteration order;
|
||||||
|
Go map iteration is randomized. *Mitigation:* always iterate
|
||||||
|
`sorted_months` explicitly in Go, never the map. Lock the distribution
|
||||||
|
with a parity test on (300, 300, 150) months × 751-CZK payment.
|
||||||
|
3. **NFKD edge cases** — MEDIUM/MEDIUM. Python `unicodedata` and Go
|
||||||
|
`golang.org/x/text` use the same algorithm but can differ on niche
|
||||||
|
compatibility decompositions if `x/text` is older than CPython's tables.
|
||||||
|
*Mitigation:* parity test with every distinct character ever observed in
|
||||||
|
member names; pin `x/text` version explicitly.
|
||||||
|
4. **Czech month parser semantics** — MEDIUM/MEDIUM. Wrap-around year
|
||||||
|
inference (`if start_m > end_m and m >= start_m: year = default_year - 1`)
|
||||||
|
plus the "month >= 10 → previous year" heuristic are easy to mis-port.
|
||||||
|
*Mitigation:* port table and algorithm verbatim line-for-line; parity
|
||||||
|
test with ~30 real `message`-field fixture strings.
|
||||||
|
|
||||||
|
## Cutover plan
|
||||||
|
|
||||||
|
Simpler without a proxy in the middle:
|
||||||
|
|
||||||
|
1. After M7's 7-day clean window + user sign-off, treat Go as primary.
|
||||||
|
Update bookmarks, docs, `make web` to point at Go.
|
||||||
|
2. Keep `make web-py` available for 1-week rollback. Run both containers
|
||||||
|
in production but only point users at the Go one.
|
||||||
|
3. Watch 2 weeks including a month-end settlement on Go-only.
|
||||||
|
4. Decommission Python: remove from `docker-compose.yml`, delete
|
||||||
|
[app.py](app.py) and [scripts/](scripts/), update
|
||||||
|
[CLAUDE.md](CLAUDE.md). Keep image tagged `python-final` in registry as
|
||||||
|
a 6-month rollback option.
|
||||||
|
|
||||||
|
**Retirement criteria:** zero parity-diff incidents in last 30 days, zero
|
||||||
|
rollbacks, two month-end settlements completed Go-only, manual
|
||||||
|
reconciliation review against `python-final` signed off.
|
||||||
|
|
||||||
|
## Critical files
|
||||||
|
|
||||||
|
- [scripts/match_payments.py](scripts/match_payments.py) — `reconcile()` is
|
||||||
|
the single most load-bearing function (~200 lines of allocation logic)
|
||||||
|
that must port byte-equivalently.
|
||||||
|
- [scripts/czech_utils.py](scripts/czech_utils.py) — `normalize` and
|
||||||
|
`parse_month_references` underpin every member/month match across the
|
||||||
|
system. 45 Czech month declensions, range wrap-around, year inference.
|
||||||
|
- [app.py](app.py) — defines the 8-route HTTP surface and view-model
|
||||||
|
shapes. The spec for the Go web layer's JSON API.
|
||||||
|
- [scripts/sync_fio_to_sheets.py](scripts/sync_fio_to_sheets.py) —
|
||||||
|
`generate_sync_id` defines the dedup contract against existing rows in
|
||||||
|
the live sheet. Any drift creates duplicates.
|
||||||
|
- [scripts/attendance.py](scripts/attendance.py) — fee math + merged-month
|
||||||
|
logic + junior `"?"` sentinel.
|
||||||
|
- [scripts/cache_utils.py](scripts/cache_utils.py) — Drive `modifiedTime`
|
||||||
|
gating + two-TTL fallback that must be reproduced for shared-cache
|
||||||
|
safety.
|
||||||
|
- [templates/adults.html](templates/adults.html) — read for the JSON shape
|
||||||
|
the existing inline JS consumes (`member_data`); the Go frontend doesn't
|
||||||
|
have to mirror the template, but the JSON contract derived from this
|
||||||
|
page's data injection is the parity spec.
|
||||||
|
|
||||||
|
## Verification
|
||||||
|
|
||||||
|
End-to-end checks per milestone:
|
||||||
|
|
||||||
|
- **M1**: `make go-build && ./bin/fuj server --help` prints subcommand
|
||||||
|
list. `make web-go` serves :8080 in parallel with `make web-py` on :5001.
|
||||||
|
- **M2-M3**: `cd go && go test -tags=parity ./tests/parity/pure/...` green.
|
||||||
|
Spot-check: feed a known Czech-message string through both
|
||||||
|
`parse_month_references` implementations, diff outputs.
|
||||||
|
- **M4**: `go test -tags=integration ./internal/io/sheets/...` round-trips
|
||||||
|
against a test sheet (separate from prod).
|
||||||
|
- **M5**: `curl localhost:5001/api/adults | jq -S . > py.json && curl
|
||||||
|
localhost:8080/api/adults | jq -S . > go.json && diff py.json go.json` —
|
||||||
|
empty diff modulo allowlist.
|
||||||
|
- **M6**: Browser open `localhost:8080/adults`, click a member row, modal
|
||||||
|
opens with all months / transactions / exceptions correctly populated.
|
||||||
|
Same on `/juniors`. Click a Pay button → QR loads. Name filter and month
|
||||||
|
range filter work.
|
||||||
|
- **M7**: Run `cd go && ./bin/parity --base http://localhost:5001
|
||||||
|
--candidate http://localhost:8080 --routes adults,juniors,payments`
|
||||||
|
daily for 7 days, zero non-allowlisted diffs. User confirms Go UI is
|
||||||
|
feature-complete vs Python UI side-by-side.
|
||||||
|
- **M8**: `make web-py` removed from Makefile; `make web` points at Go;
|
||||||
|
manual end-of-month settlement on Go matches the prior month's
|
||||||
|
Python-produced report.
|
||||||
|
|
||||||
|
## Open questions / forks the user can override at review
|
||||||
|
|
||||||
|
- **Frontend JS organization in M6**: default is vanilla JS in separate
|
||||||
|
files via `embed.FS`. If the user wants HTMX, Alpine.js, or a small
|
||||||
|
framework, raise it before M6.
|
||||||
|
- **CI host**: Gitea Actions assumed (matches existing
|
||||||
|
[.gitea/workflows/build.yaml](.gitea/workflows/build.yaml)).
|
||||||
|
- **Test sheet for M4 integration tests**: would need provisioning.
|
||||||
|
Confirm whether to use a copy of the production sheet (PII!) or a
|
||||||
|
synthetic one seeded by the fixture-capture process.
|
||||||
233
docs/plans/2026-05-04-1115-go-rewrite-m1-kickoff.md
Normal file
233
docs/plans/2026-05-04-1115-go-rewrite-m1-kickoff.md
Normal file
@@ -0,0 +1,233 @@
|
|||||||
|
# Plan: Go rewrite — M1 kickoff (skeleton + tooling)
|
||||||
|
|
||||||
|
Companion to [2026-05-03-2349-go-backend-rewrite.md](2026-05-03-2349-go-backend-rewrite.md)
|
||||||
|
and the progress tracker
|
||||||
|
[2026-05-03-2349-go-backend-rewrite-progress.md](2026-05-03-2349-go-backend-rewrite-progress.md).
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
The master plan for a full Go rewrite of the Flask backend is approved
|
||||||
|
(2026-05-04). No Go code exists yet — this plan executes **M1** end-to-end:
|
||||||
|
a working `go/` skeleton, a `fuj` binary with a `server` subcommand serving
|
||||||
|
a hello page on `:8080`, lint config, Makefile + CI integration, and an
|
||||||
|
`internal/config` package mirroring [scripts/config.py](scripts/config.py).
|
||||||
|
|
||||||
|
After M1, both backends run side-by-side locally (`make web-py` on `:5001`,
|
||||||
|
`make web-go` on `:8080`) — that side-by-side capability is what unblocks
|
||||||
|
M2's parity testing and every later milestone.
|
||||||
|
|
||||||
|
## Locked-in decisions
|
||||||
|
|
||||||
|
| # | Decision | Choice |
|
||||||
|
|---|---|---|
|
||||||
|
| 1 | CLI dispatcher | stdlib `flag` + `os.Args[1]` switch (no cobra) |
|
||||||
|
| 2 | Go module path | `fuj-management/go` |
|
||||||
|
| 3 | Go version | `1.26` (latest stable; user toolchain is `go1.26.1`) |
|
||||||
|
| 4 | M1 scope | all 10 progress-tracker sub-tasks in one session |
|
||||||
|
| 5 | Lint | `golangci-lint` with govet, staticcheck, errcheck, gofumpt, unused |
|
||||||
|
| 6 | Logging | `log/slog` text handler, level from `LOG_LEVEL` env |
|
||||||
|
| 7 | HTTP | `net/http.ServeMux` (Go 1.22+ pattern matching) |
|
||||||
|
| 8 | Container base | `golang:1.26` builder → `gcr.io/distroless/static:nonroot` runtime |
|
||||||
|
| 9 | CI | extend [.gitea/workflows/build.yaml](.gitea/workflows/build.yaml) with a `go-build` job parallel to existing Python `build` job; tag suffix `-go` |
|
||||||
|
|
||||||
|
## Files to create
|
||||||
|
|
||||||
|
```
|
||||||
|
go/
|
||||||
|
go.mod # module fuj-management/go, go 1.26
|
||||||
|
go.sum # empty / generated
|
||||||
|
.golangci.yml # govet, staticcheck, errcheck, gofumpt, unused
|
||||||
|
cmd/fuj/main.go # subcommand dispatcher + version vars
|
||||||
|
internal/
|
||||||
|
config/config.go # env loader mirroring scripts/config.py
|
||||||
|
logging/logger.go # slog setup honoring LOG_LEVEL
|
||||||
|
web/
|
||||||
|
server.go # `fuj server` handler: ServeMux on :8080, hello page
|
||||||
|
middleware/timer.go # request-timer middleware (parity with Python `get_render_time`)
|
||||||
|
build/
|
||||||
|
Dockerfile # multi-stage golang:1.26 → distroless/static
|
||||||
|
```
|
||||||
|
|
||||||
|
No `embed.FS`, no templates, no static assets in M1 — the hello page is
|
||||||
|
inline HTML in `server.go`. Templates land in M6.
|
||||||
|
|
||||||
|
## Files to edit
|
||||||
|
|
||||||
|
- [Makefile](Makefile) — add Go targets, rename `web` → `web-py`, keep
|
||||||
|
`web` as transitional alias to `web-py` until M8.
|
||||||
|
- [.gitignore](.gitignore) — add `bin/` and `go/.cache/` (if any).
|
||||||
|
- [.gitea/workflows/build.yaml](.gitea/workflows/build.yaml) — add
|
||||||
|
`go-build` job that builds and pushes `<tag>-go` image.
|
||||||
|
- [CHANGELOG.md](CHANGELOG.md) — top-of-file entry per CLAUDE.md convention.
|
||||||
|
- [docs/plans/2026-05-03-2349-go-backend-rewrite-progress.md](docs/plans/2026-05-03-2349-go-backend-rewrite-progress.md)
|
||||||
|
— tick M1.1–M1.10 with commit SHAs as they land.
|
||||||
|
|
||||||
|
## Execution sequence
|
||||||
|
|
||||||
|
Order is tight: each step keeps the tree compilable and lint-clean.
|
||||||
|
|
||||||
|
1. **Skeleton (M1.1)** — `mkdir -p go/{cmd/fuj,internal/{config,logging,web/middleware},build}` and `cd go && go mod init fuj-management/go`. Pin `go 1.26` in `go.mod`.
|
||||||
|
|
||||||
|
2. **Config + logger (M1.8, M1.9)** — write `internal/config/config.go` mirroring [scripts/config.py](scripts/config.py): exported constants for `AttendanceSheetID`, `PaymentsSheetID`, `JuniorSheetGID`, env-driven `CredentialsPath`, `BankAccount`, `CacheTTL`, `CacheAPICheckTTL`, `LogLevel`, `FioAPIToken`. Write `internal/logging/logger.go` with a `New() *slog.Logger` honoring `LOG_LEVEL` (`DEBUG|INFO|WARN|ERROR`).
|
||||||
|
|
||||||
|
3. **Web middleware + handler (M1.3)** — `internal/web/middleware/timer.go` logs `method path status ms` for every request. `internal/web/server.go` exposes `Run(ctx, addr) error`: `http.ServeMux` with `GET /` returning a minimal HTML hello page that includes `version`, `commit`, and `buildDate` (linker-injected via `-X main.version=…`).
|
||||||
|
|
||||||
|
4. **Subcommand dispatcher (M1.2)** — `cmd/fuj/main.go`:
|
||||||
|
- Package-level `var version, commit, buildDate string` for `-ldflags -X` injection.
|
||||||
|
- `os.Args[1]` switch over `server | version | fees | reconcile | sync | infer | help`. M1 implements `server` and `version`; the rest print `<cmd>: not implemented yet (lands in M2/M4)` and exit 2.
|
||||||
|
- Each subcommand parses its own `flag.NewFlagSet`. `server` flags: `--addr` (default `:8080`).
|
||||||
|
|
||||||
|
5. **Lint config (M1.6)** — `go/.golangci.yml` enabling `govet`, `staticcheck`, `errcheck`, `gofumpt`, `unused`. Run `golangci-lint run ./...` to confirm clean.
|
||||||
|
|
||||||
|
6. **Makefile (M1.4, M1.5)** — add:
|
||||||
|
```make
|
||||||
|
GO_BIN := bin/fuj
|
||||||
|
GO_SRC := go
|
||||||
|
|
||||||
|
go-build:
|
||||||
|
cd $(GO_SRC) && go build -trimpath \
|
||||||
|
-ldflags "-X main.version=$$(git describe --tags --always 2>/dev/null || echo dev) \
|
||||||
|
-X main.commit=$$(git rev-parse --short HEAD) \
|
||||||
|
-X main.buildDate=$$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
|
||||||
|
-o ../$(GO_BIN) ./cmd/fuj
|
||||||
|
|
||||||
|
go-test:
|
||||||
|
cd $(GO_SRC) && go test -race ./...
|
||||||
|
|
||||||
|
go-run: go-build
|
||||||
|
./$(GO_BIN) $(ARGS)
|
||||||
|
|
||||||
|
go-lint:
|
||||||
|
cd $(GO_SRC) && golangci-lint run ./...
|
||||||
|
|
||||||
|
web-go: go-build
|
||||||
|
./$(GO_BIN) server --addr :8080
|
||||||
|
```
|
||||||
|
Rename existing `web:` target to `web-py:` and add `web: web-py` as alias.
|
||||||
|
|
||||||
|
7. **Dockerfile + CI (M1.7)** — `go/build/Dockerfile`:
|
||||||
|
```dockerfile
|
||||||
|
FROM golang:1.26 AS build
|
||||||
|
WORKDIR /src
|
||||||
|
COPY go/go.mod go/go.sum ./
|
||||||
|
RUN go mod download
|
||||||
|
COPY go/ ./
|
||||||
|
ARG GIT_TAG=unknown
|
||||||
|
ARG GIT_COMMIT=unknown
|
||||||
|
ARG BUILD_DATE=unknown
|
||||||
|
RUN CGO_ENABLED=0 go build -trimpath \
|
||||||
|
-ldflags "-s -w -X main.version=${GIT_TAG} -X main.commit=${GIT_COMMIT} -X main.buildDate=${BUILD_DATE}" \
|
||||||
|
-o /out/fuj ./cmd/fuj
|
||||||
|
|
||||||
|
FROM gcr.io/distroless/static:nonroot
|
||||||
|
COPY --from=build /out/fuj /usr/local/bin/fuj
|
||||||
|
EXPOSE 8080
|
||||||
|
USER nonroot:nonroot
|
||||||
|
ENTRYPOINT ["/usr/local/bin/fuj","server"]
|
||||||
|
```
|
||||||
|
In [.gitea/workflows/build.yaml](.gitea/workflows/build.yaml), add a parallel job:
|
||||||
|
```yaml
|
||||||
|
build-go:
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v4
|
||||||
|
- run: docker login ...
|
||||||
|
- run: |
|
||||||
|
docker build -f go/build/Dockerfile \
|
||||||
|
--build-arg GIT_TAG=$TAG \
|
||||||
|
--build-arg GIT_COMMIT=${{ github.sha }} \
|
||||||
|
--build-arg BUILD_DATE=$(date -u +%Y-%m-%dT%H:%M:%SZ) \
|
||||||
|
-t gitea.home.hrajfrisbee.cz/${{ github.repository }}:$TAG-go .
|
||||||
|
docker push gitea.home.hrajfrisbee.cz/${{ github.repository }}:$TAG-go
|
||||||
|
```
|
||||||
|
|
||||||
|
8. **Smoke verify (M1.10)** — see Verification section below; then append a CHANGELOG entry and tick M1 boxes in the progress tracker with commit SHAs.
|
||||||
|
|
||||||
|
## Reuse / parity with Python side
|
||||||
|
|
||||||
|
- `internal/config` mirrors [scripts/config.py](scripts/config.py) **exactly** — same env var names, same defaults. No new env knobs in M1.
|
||||||
|
- Request-timer middleware records elapsed milliseconds; this is the Go-side
|
||||||
|
equivalent of the Python `get_render_time` helper that supplies
|
||||||
|
`render_time.total` to templates. Allowlisted as volatile in the future
|
||||||
|
parity diff (M5).
|
||||||
|
- Constants `AttendanceSheetID`, `PaymentsSheetID`, `JuniorSheetGID` are
|
||||||
|
copied verbatim from [scripts/config.py](scripts/config.py); they don't
|
||||||
|
get used until M4 but live in `internal/config` from day one.
|
||||||
|
|
||||||
|
## Verification
|
||||||
|
|
||||||
|
Run from repo root after all changes are in place:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 1. Builds clean
|
||||||
|
make go-build && test -x bin/fuj
|
||||||
|
|
||||||
|
# 2. Lint clean
|
||||||
|
make go-lint
|
||||||
|
|
||||||
|
# 3. Subcommand dispatcher works
|
||||||
|
./bin/fuj help
|
||||||
|
./bin/fuj version # prints version/commit/buildDate
|
||||||
|
./bin/fuj fees # prints "not implemented yet" and exits 2
|
||||||
|
|
||||||
|
# 4. Server runs and hello page is served
|
||||||
|
make web-go &
|
||||||
|
GO_PID=$!
|
||||||
|
sleep 1
|
||||||
|
curl -sf http://localhost:8080/ | grep -q "fuj"
|
||||||
|
kill $GO_PID
|
||||||
|
|
||||||
|
# 5. Side-by-side: both backends up
|
||||||
|
make web-py & # :5001
|
||||||
|
PY_PID=$!
|
||||||
|
make web-go & # :8080
|
||||||
|
GO_PID=$!
|
||||||
|
sleep 2
|
||||||
|
curl -sf http://localhost:5001/ >/dev/null && echo "py OK"
|
||||||
|
curl -sf http://localhost:8080/ >/dev/null && echo "go OK"
|
||||||
|
kill $PY_PID $GO_PID
|
||||||
|
|
||||||
|
# 6. Race-free unit tests pass (none yet beyond a smoke test, but harness works)
|
||||||
|
make go-test
|
||||||
|
|
||||||
|
# 7. Docker image builds locally
|
||||||
|
docker build -f go/build/Dockerfile -t fuj-go:dev .
|
||||||
|
docker run --rm -p 8080:8080 fuj-go:dev &
|
||||||
|
sleep 1
|
||||||
|
curl -sf http://localhost:8080/ >/dev/null && echo "container OK"
|
||||||
|
docker stop $(docker ps -lq)
|
||||||
|
```
|
||||||
|
|
||||||
|
All seven steps must succeed. Then update the progress tracker and
|
||||||
|
CHANGELOG.
|
||||||
|
|
||||||
|
## Out of scope for M1 (deferred to later milestones)
|
||||||
|
|
||||||
|
- Domain logic — `czech.Normalize`, fees, reconcile, etc. → **M2**.
|
||||||
|
- Fixture capture and parity tests → **M3**.
|
||||||
|
- Sheets/Drive/Fio clients and `internal/io/*` → **M4**.
|
||||||
|
- `/api/*` JSON routes and `cmd/parity` → **M5**.
|
||||||
|
- HTML templates, static assets, `embed.FS` → **M6**.
|
||||||
|
- Removing the Python backend → **M8**.
|
||||||
|
|
||||||
|
## Open items / forks the user can override at review
|
||||||
|
|
||||||
|
- **CI tag suffix**: `<tag>-go` proposed. Alternative: separate image
|
||||||
|
repository (`fuj-management-go:<tag>`). The suffix keeps things in one
|
||||||
|
registry path; speak up if separate repos are preferred.
|
||||||
|
- **Distroless variant**: `nonroot` chosen for least privilege. If the
|
||||||
|
existing Python container runs as root and the user expects parity,
|
||||||
|
switch to `gcr.io/distroless/static` (root). Doesn't affect M1
|
||||||
|
functionality.
|
||||||
|
- **Hello page content**: minimal HTML mentioning `fuj`, version, commit,
|
||||||
|
build date, link list to future routes. Speak up if you want a different
|
||||||
|
shape — it gets thrown away in M6 anyway.
|
||||||
|
|
||||||
|
## Critical files
|
||||||
|
|
||||||
|
- [docs/plans/2026-05-03-2349-go-backend-rewrite.md](docs/plans/2026-05-03-2349-go-backend-rewrite.md) — master plan (approved 2026-05-04)
|
||||||
|
- [docs/plans/2026-05-03-2349-go-backend-rewrite-progress.md](docs/plans/2026-05-03-2349-go-backend-rewrite-progress.md) — task tracker; tick M1.1–M1.10 here
|
||||||
|
- [Makefile](Makefile) — current target structure (renaming `web` → `web-py`)
|
||||||
|
- [scripts/config.py](scripts/config.py) — source of truth for env vars / IDs that `internal/config` mirrors
|
||||||
|
- [build/Dockerfile](build/Dockerfile) — Python container (unchanged); the new Go Dockerfile lives at `go/build/Dockerfile`
|
||||||
|
- [.gitea/workflows/build.yaml](.gitea/workflows/build.yaml) — extended with parallel `build-go` job
|
||||||
81
docs/plans/2026-05-04-2249-payment-name-match-exact.md
Normal file
81
docs/plans/2026-05-04-2249-payment-name-match-exact.md
Normal file
@@ -0,0 +1,81 @@
|
|||||||
|
# Exact full-name match for payment inference
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
A bank payment with the message `Henrietta Ottová (Heny): 04/2026` is being inferred to **two** members: the correct `Henrietta Ottová` *and* the unrelated `Tomáš Němeček (Tov)`. As a result, `reconcile()` splits the amount 50/50 between them, producing wrong balances.
|
||||||
|
|
||||||
|
**Root cause** (`scripts/match_payments.py:51-115`): `match_members` runs four substring checks via raw Python `in`, with no word boundaries. Tomáš's nickname `Tov` normalizes to `tov`, which is literally a substring of `ottova`. Check #3 (`match_payments.py:79-85`) treats bare nickname presence as an `auto`-confidence match, so Tomáš is appended even though no part of his name is actually in the message. There is also no short-circuit when a member's full canonical name appears verbatim — every other member is still scored against the same haystack.
|
||||||
|
|
||||||
|
**Goal:** when a member's full canonical name (diacritics-insensitive) appears in the message as whole words, return only the full-name hit(s) and skip nickname/partial scoring entirely. Additionally, harden the remaining checks with word boundaries so future substring collisions (any nickname or short name part that happens to live inside another member's surname) can't reproduce this class of bug.
|
||||||
|
|
||||||
|
## Approach
|
||||||
|
|
||||||
|
Single-file change in [scripts/match_payments.py](scripts/match_payments.py). Two coordinated edits to `match_members` (`match_payments.py:51-115`):
|
||||||
|
|
||||||
|
### 1. Add an exact-canonical-name short-circuit (new, before the existing loop)
|
||||||
|
|
||||||
|
After computing `normalized_text`, do a first pass that collects every member whose `normalized_base` (the full name minus the parenthesized nickname, normalized) appears in the haystack as **whole words**. If at least one is found, return *only* those as `auto` matches and skip the rest of the function.
|
||||||
|
|
||||||
|
Implementation sketch (inserted between [match_payments.py:58](scripts/match_payments.py#L58) and [match_payments.py:61](scripts/match_payments.py#L61)):
|
||||||
|
|
||||||
|
```python
|
||||||
|
exact_matches = []
|
||||||
|
for name in member_names:
|
||||||
|
variants = _build_name_variants(name)
|
||||||
|
full_name = variants[0] if variants else ""
|
||||||
|
if full_name and re.search(rf"\b{re.escape(full_name)}\b", normalized_text):
|
||||||
|
exact_matches.append((name, "auto"))
|
||||||
|
if exact_matches:
|
||||||
|
return exact_matches
|
||||||
|
```
|
||||||
|
|
||||||
|
This satisfies the user's primary ask: when the message literally contains the canonical name, that wins outright. Multi-member messages still work — every full-name occurrence is collected.
|
||||||
|
|
||||||
|
### 2. Replace remaining `in normalized_text` checks with `\b…\b` regex
|
||||||
|
|
||||||
|
For the three checks that survive the short-circuit (and the `review`-tier partials), swap raw `in` for whole-word regex so `tov` cannot match inside `ottova`, `dan` cannot match inside `bohdan`, etc. Affected lines:
|
||||||
|
|
||||||
|
- [match_payments.py:73](scripts/match_payments.py#L73) — first+last name both present
|
||||||
|
- [match_payments.py:82](scripts/match_payments.py#L82) — nickname presence
|
||||||
|
- [match_payments.py:94](scripts/match_payments.py#L94) — last-name partial (`review`)
|
||||||
|
- [match_payments.py:99](scripts/match_payments.py#L99) — first-name partial (`review`)
|
||||||
|
- [match_payments.py:104](scripts/match_payments.py#L104) — single-name member partial
|
||||||
|
|
||||||
|
Helper to keep the call sites tidy:
|
||||||
|
|
||||||
|
```python
|
||||||
|
def _word_in(needle: str, haystack: str) -> bool:
|
||||||
|
return bool(re.search(rf"\b{re.escape(needle)}\b", haystack))
|
||||||
|
```
|
||||||
|
|
||||||
|
Check #1 (line 67) becomes redundant once the short-circuit is in place, but leave it untouched as a defensive fallback in case `_build_name_variants` ever returns a `full_name` shorter than the 3-char filter would allow. (No code change there.)
|
||||||
|
|
||||||
|
### 3. Why this is sufficient
|
||||||
|
|
||||||
|
- The reported message `Henrietta Ottová (Heny): 04/2026` hits the new short-circuit on `henrietta ottova`, returns `[("Henrietta Ottová", "auto")]`, and never even evaluates Tomáš.
|
||||||
|
- Bare-nickname messages (e.g. `Heny 04/2026`) skip the short-circuit (no full name present) and fall into the existing nickname check — now word-bounded, so `tov` no longer collides with `ottova` even there.
|
||||||
|
- Combined-payment messages listing two full names continue to work: both are collected by the short-circuit.
|
||||||
|
|
||||||
|
### Files to modify
|
||||||
|
|
||||||
|
- [scripts/match_payments.py](scripts/match_payments.py) — only `match_members` (lines 51-115). Add `_word_in` helper just above it.
|
||||||
|
|
||||||
|
### Files to read for confidence (no edits)
|
||||||
|
|
||||||
|
- [scripts/czech_utils.py](scripts/czech_utils.py) — confirm `normalize()` semantics (NFKD strip + lowercase). Already understood; relevant because `re.escape` on already-normalized lowercase ASCII is safe.
|
||||||
|
- [scripts/infer_payments.py](scripts/infer_payments.py) — confirm it just consumes the `match_members` output verbatim and writes comma-joined names. No change needed; the upstream fix propagates.
|
||||||
|
- [scripts/match_payments.py:336-362](scripts/match_payments.py#L336-L362) — `reconcile()` only re-runs inference when `Person` is empty, so existing wrong rows in the sheet must be cleared by hand or via the `manual fix`/blank-cell workflow before re-running `make infer`.
|
||||||
|
|
||||||
|
## Verification
|
||||||
|
|
||||||
|
1. **Unit test** — add `tests/test_match_members.py` (new file, mirroring `tests/test_reconcile_exceptions.py` style). Cases:
|
||||||
|
- `match_members("Henrietta Ottová (Heny): 04/2026", ["Henrietta Ottová", "Tomáš Němeček (Tov)"])` → `[("Henrietta Ottová", "auto")]` only.
|
||||||
|
- `match_members("Heny 04/2026", ["Tomáš Němeček (Tov)", "Henrietta Ottová"])` → no match for Tomáš (the substring trap is closed); whatever the legitimate behavior for "Heny" is, document it.
|
||||||
|
- Combined payment: `match_members("Henrietta Ottová a Tomáš Němeček 04/2026", ["Henrietta Ottová", "Tomáš Němeček (Tov)"])` → both as `auto`.
|
||||||
|
- Sanity: `match_members("VS 1234 Tomáš Němeček", [...])` still returns Tomáš.
|
||||||
|
|
||||||
|
2. **Run the suite**: `make test`.
|
||||||
|
|
||||||
|
3. **End-to-end**: clear the buggy row's `Person`/`Purpose` cells in the payments sheet, then `make infer`, then `make reconcile`. Confirm the payment now allocates fully to Henrietta and balance reflects it.
|
||||||
|
|
||||||
|
4. **Changelog**: per [CLAUDE.md](CLAUDE.md), append an entry to [CHANGELOG.md](CHANGELOG.md) once the user confirms the fix works in production. Format: `## 2026-05-04 HH:MM TZ — fix: payment inference exact-match short-circuit`.
|
||||||
@@ -0,0 +1,99 @@
|
|||||||
|
# Member modal — raw payments debug list
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
When a payer's bank message doesn't follow our convention, [`infer_payments.py`](scripts/infer_payments.py) may map the transfer to the wrong period (or none), and today the member detail modal hides this — it only shows the post-allocation, per-month splits produced by [`reconcile()`](scripts/match_payments.py:295). To diagnose these cases the user needs to see the **original sheet rows** that were attributed to a member: full `Amount`, `Inferred Amount`, `Person`, `Purpose`, `Sender`, `Message`, `Bank ID`, `manual fix`. The list should be hidden by default and revealed by a small toggle, since it is only relevant during debugging.
|
||||||
|
|
||||||
|
## Approach
|
||||||
|
|
||||||
|
Reuse the grouping logic that already exists in the [`/payments` route](app.py:540-553): group raw `tx` dicts by parsed `Person`, expose that mapping to the modal, and render it on demand under a new collapsible section.
|
||||||
|
|
||||||
|
### 1. Backend — group raw txs by member
|
||||||
|
|
||||||
|
In [`app.py`](app.py):
|
||||||
|
|
||||||
|
- Factor the existing per-person grouping in [`payments()`](app.py:530-568) into a small helper near the top of the file:
|
||||||
|
```python
|
||||||
|
def group_payments_by_person(transactions):
|
||||||
|
grouped = {}
|
||||||
|
for tx in transactions:
|
||||||
|
person = str(tx.get("person", "")).strip()
|
||||||
|
if not person:
|
||||||
|
continue # unmatched rows are not tied to a member
|
||||||
|
for p in person.split(","):
|
||||||
|
p = re.sub(r"\[\?\]\s*", "", p).strip()
|
||||||
|
if not p:
|
||||||
|
continue
|
||||||
|
grouped.setdefault(p, []).append(tx)
|
||||||
|
for rows in grouped.values():
|
||||||
|
rows.sort(key=lambda t: str(t.get("date", "")), reverse=True)
|
||||||
|
return grouped
|
||||||
|
```
|
||||||
|
Call it from [`payments()`](app.py:530), [`adults_view()`](app.py:160) and [`juniors_view()`](app.py:326) — the existing `payments()` body collapses to one line.
|
||||||
|
|
||||||
|
- In `adults_view()` and `juniors_view()`, after `transactions = get_cached_data(...)`, build `raw_payments_by_person = group_payments_by_person(transactions)` and pass it to `render_template` as `raw_payments_json=json.dumps(raw_payments_by_person)`.
|
||||||
|
|
||||||
|
- Note: rows where `Person` is empty are skipped on purpose — those have no member to attach to and are already shown by the dashboard's `Unmatched` block.
|
||||||
|
|
||||||
|
### 2. Templates — add a collapsible raw section to the modal
|
||||||
|
|
||||||
|
In [`templates/adults.html`](templates/adults.html) and [`templates/juniors.html`](templates/juniors.html), make the same structural and JS changes (the modal markup is mirrored in both files — `adults.html:677-682` and `juniors.html:658-663`).
|
||||||
|
|
||||||
|
- Inject the new dataset alongside the existing `memberData`:
|
||||||
|
```html
|
||||||
|
const rawPaymentsByPerson = {{ raw_payments_json| safe }};
|
||||||
|
```
|
||||||
|
(next to [`adults.html:696`](templates/adults.html#L696)).
|
||||||
|
|
||||||
|
- Add a new section directly **after** the Payment History block:
|
||||||
|
```html
|
||||||
|
<div class="modal-section">
|
||||||
|
<div class="modal-section-title">
|
||||||
|
Raw Payments
|
||||||
|
<a href="#" id="rawPaymentsToggle" class="raw-toggle"
|
||||||
|
onclick="toggleRawPayments(event)">[show]</a>
|
||||||
|
</div>
|
||||||
|
<div id="modalRawList" class="tx-list" style="display: none;">
|
||||||
|
<!-- Filled by JS -->
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
```
|
||||||
|
Add a small CSS rule for `.raw-toggle` (muted color, smaller font, `margin-left: 8px`) — a few lines next to the existing `.modal-section-title` style. Don't restyle the whole modal.
|
||||||
|
|
||||||
|
- In `showMemberDetails(name)`:
|
||||||
|
- Reset the toggle to `[show]` and the `#modalRawList` to `display: none` on every open (so the state doesn't leak between members).
|
||||||
|
- Populate `#modalRawList` from `rawPaymentsByPerson[name] || []`. For each row render: `Date | Purpose` on the meta line, `Amount CZK` (with `Inferred: X CZK` annotation when `inferred_amount` differs from `amount`), `Sender`, `Person` (full string — useful when split between multiple people), `Message`, and a small footer with `Bank ID` and a `[manual fix]` marker if `manual_fix` is truthy. Reuse the existing `tx-item` / `tx-meta` / `tx-main` / `tx-msg` styles to match the rest of the modal.
|
||||||
|
- When the list is empty, render `<div style="color: #444; font-style: italic; padding: 10px 0;">No raw payments tied to this member.</div>` (same idiom used at [`adults.html:813`](templates/adults.html#L813)).
|
||||||
|
|
||||||
|
- Add the toggle handler near `closeModal`:
|
||||||
|
```js
|
||||||
|
function toggleRawPayments(ev) {
|
||||||
|
ev.preventDefault();
|
||||||
|
const list = document.getElementById('modalRawList');
|
||||||
|
const link = document.getElementById('rawPaymentsToggle');
|
||||||
|
const hidden = list.style.display === 'none';
|
||||||
|
list.style.display = hidden ? 'block' : 'none';
|
||||||
|
link.textContent = hidden ? '[hide]' : '[show]';
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3. Why not extend `reconcile()` instead
|
||||||
|
|
||||||
|
`reconcile()` already collapses each row into per-month allocated shares and drops `purpose`, `inferred_amount`, `bank_id`, `manual_fix`, and the gross `amount` ([trace](scripts/match_payments.py:436-469)). Carrying the raw `tx` through `reconcile()` would inflate the contract for every consumer when only the modal needs it. Grouping the already-fetched `transactions` list at the route level is one extra dict per request and reuses the cached payments data — no new sheet reads.
|
||||||
|
|
||||||
|
## Critical files
|
||||||
|
|
||||||
|
- [app.py](app.py) — add `group_payments_by_person()` helper; call it in `adults_view()`, `juniors_view()`, and `payments()`; pass `raw_payments_json` to the two dashboard templates.
|
||||||
|
- [templates/adults.html](templates/adults.html) — modal section + JS + tiny CSS for the toggle link.
|
||||||
|
- [templates/juniors.html](templates/juniors.html) — same changes as adults.html.
|
||||||
|
|
||||||
|
## Verification
|
||||||
|
|
||||||
|
1. `make web-debug` and open `http://localhost:5001/adults`.
|
||||||
|
2. Pick a member known to have multiple payments (use the existing `/payments` page as a cross-reference).
|
||||||
|
3. Click `[i]` → modal opens, raw list is hidden, link shows `[show]`. Click the link → list appears with the raw rows; click again → hides, link returns to `[show]`.
|
||||||
|
4. Switch to another member via keyboard (ArrowDown) — the toggle resets to hidden and the list updates to the new member's rows (no leaking).
|
||||||
|
5. Compare the raw rows in the modal against the `/payments` page grouping for the same person — same set of rows, same `Date`/`Amount`/`Message`.
|
||||||
|
6. Pick a row with a non-conformant message (e.g. one where `Person` was inferred to multiple people) — confirm `Person` shows the full comma-separated string and `Inferred Amount` is visible when it differs from `Amount`.
|
||||||
|
7. Repeat the click-through on `/juniors` to confirm parity.
|
||||||
|
8. `make test` — no backend behavior change is expected, but run to catch template/route smoke breakage.
|
||||||
@@ -0,0 +1,135 @@
|
|||||||
|
# Tolerate diacritic / case / whitespace mismatches between `Person` column and member names
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
For "Mária Maco" there is a payment row in the payments sheet with `Purpose = 2026-04`, but the modal for that member shows neither a paid 2026-04 cell **nor** a row in payment history. Both symptoms collapse to a single root cause in [`reconcile()`](scripts/match_payments.py#L295), confirmed by reading the code:
|
||||||
|
|
||||||
|
- [`scripts/match_payments.py:404`](scripts/match_payments.py#L404) — `if member_name not in ledger:` is a **byte-exact** comparison. `member_name` is the `Person` cell from the payments sheet with only `.strip()` and `[?]` markers removed ([:349-353](scripts/match_payments.py#L349-L353)). `ledger` keys are the canonical names from the attendance sheet. There is no diacritic, case, or whitespace normalization on this path. (`czech_utils.normalize` is imported and used for the `exceptions` lookup at [:282-283 / :321-322](scripts/match_payments.py#L282-L322), but **not** for member-name matching.)
|
||||||
|
- When a row falls through that check, it is appended to `unmatched` and never reaches `ledger[member_name][m]['paid']` or `['transactions']`. The dashboard's per-month "paid" cell stays unpaid, and because the modal's payment history is built from `data.months[m].transactions` ([`templates/adults.html:772-776`](templates/adults.html#L772-L776)), the row also disappears from the modal's history list.
|
||||||
|
- The new "Raw Payments" debug section ([`templates/adults.html:861`](templates/adults.html#L861)) uses `rawPaymentsByPerson[name]`. Its keys come from [`group_payments_by_person()` in `app.py:60-73`](app.py#L60-L73), which also stores the **literal** `Person` string (only `.strip()` and `[?]` stripped). So if the attendance-sheet name and the `Person` cell differ at the byte level, that section also returns an empty list — which is why the user does not see the row anywhere in the modal.
|
||||||
|
|
||||||
|
The most likely cause for "Mária Maco" specifically: the `Person` cell was typed (or pasted) without the `á` diacritic — `Maria Maco` vs `Mária Maco`. Other plausible variants the current code silently drops: case differences (`mária maco`), trailing/embedded extra whitespace, and NBSP characters.
|
||||||
|
|
||||||
|
The fix is to make the matching tolerant via the existing [`czech_utils.normalize()`](scripts/czech_utils.py#L22-L25) helper (NFKD + lowercase), with a small whitespace-collapse on top, and apply the same canonicalization in `group_payments_by_person()` so the modal's raw-payments lookup uses the canonical attendance-sheet name as the key.
|
||||||
|
|
||||||
|
## Approach
|
||||||
|
|
||||||
|
### 1. `scripts/match_payments.py` — tolerant `Person` → `ledger` resolution in `reconcile()`
|
||||||
|
|
||||||
|
- Add a small private helper at module scope:
|
||||||
|
|
||||||
|
```python
|
||||||
|
def _canonical_key(name: str) -> str:
|
||||||
|
return re.sub(r"\s+", " ", normalize(name)).strip()
|
||||||
|
```
|
||||||
|
|
||||||
|
Uses the existing `normalize()` from `czech_utils` ([:22-25](scripts/czech_utils.py#L22-L25)) and additionally collapses whitespace runs to a single space so `"Mária Maco"` and `"Mária Maco"` both reduce to `"maria maco"`.
|
||||||
|
|
||||||
|
- Inside [`reconcile()`](scripts/match_payments.py#L295), right after `member_names` is computed ([:308](scripts/match_payments.py#L308)), build a lookup dict once:
|
||||||
|
|
||||||
|
```python
|
||||||
|
canonical_by_key: dict[str, str] = {}
|
||||||
|
for name in member_names:
|
||||||
|
key = _canonical_key(name)
|
||||||
|
canonical_by_key.setdefault(key, name) # first wins; ambiguity handled below
|
||||||
|
```
|
||||||
|
|
||||||
|
- Replace the byte-exact check at [:404](scripts/match_payments.py#L404). Resolve each `member_name` from `matched_members` to the canonical attendance-sheet name before any ledger / credits access:
|
||||||
|
|
||||||
|
```python
|
||||||
|
for raw_member_name, confidence in matched_members:
|
||||||
|
member_name = canonical_by_key.get(_canonical_key(raw_member_name))
|
||||||
|
if member_name is None:
|
||||||
|
logger.warning(
|
||||||
|
"Payment matched to unknown member %r (tx: %s, %s) — adding to unmatched",
|
||||||
|
raw_member_name, tx.get("date", "?"), tx.get("message", "?"),
|
||||||
|
)
|
||||||
|
unmatched.append(tx)
|
||||||
|
continue
|
||||||
|
if member_name != raw_member_name:
|
||||||
|
logger.info(
|
||||||
|
"Person cell %r resolved to canonical member %r — consider fixing the sheet",
|
||||||
|
raw_member_name, member_name,
|
||||||
|
)
|
||||||
|
# ... rest of the loop body unchanged: ledger[member_name], credits[member_name], …
|
||||||
|
```
|
||||||
|
|
||||||
|
The `logger.info` line lets the user see (in `make web-debug` logs) which sheet rows have a non-canonical `Person` value, so they can clean them up at their own pace — without breaking allocation in the meantime.
|
||||||
|
|
||||||
|
- Leave the rest of the function untouched. Once `member_name` is the canonical name, every downstream key (`ledger[member_name]`, `credits[member_name]`, `other_ledger[member_name]`, the `tx["person"]` echo into `transactions`) is already correct.
|
||||||
|
|
||||||
|
### 2. `app.py` — canonicalize the raw-payments grouping key
|
||||||
|
|
||||||
|
- The current [`group_payments_by_person()`](app.py#L60-L73) cannot canonicalize on its own because it does not know the attendance-sheet member list. Extend its signature to accept the member list and reuse `_canonical_key`:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from match_payments import _canonical_key # or re-export via a tiny public name
|
||||||
|
|
||||||
|
def group_payments_by_person(transactions, member_names=None):
|
||||||
|
canonical_by_key = (
|
||||||
|
{_canonical_key(n): n for n in member_names} if member_names else {}
|
||||||
|
)
|
||||||
|
grouped = {}
|
||||||
|
for tx in transactions:
|
||||||
|
person = str(tx.get("person", "")).strip()
|
||||||
|
if not person:
|
||||||
|
continue
|
||||||
|
for p in person.split(","):
|
||||||
|
p = re.sub(r"\[\?\]\s*", "", p).strip()
|
||||||
|
if not p:
|
||||||
|
continue
|
||||||
|
key = canonical_by_key.get(_canonical_key(p), p) # fallback: keep raw
|
||||||
|
grouped.setdefault(key, []).append(tx)
|
||||||
|
for rows in grouped.values():
|
||||||
|
rows.sort(key=lambda t: str(t.get("date", "")), reverse=True)
|
||||||
|
return grouped
|
||||||
|
```
|
||||||
|
|
||||||
|
- Update the three call sites to pass `member_names`:
|
||||||
|
- `adults_view()` around [`app.py:333`](app.py#L333) — `members` is already in scope; pass `[name for name, _, _ in members]`.
|
||||||
|
- `juniors_view()` around [`app.py:539`](app.py#L539) — same.
|
||||||
|
- `payments()` around [`app.py:549`](app.py#L549) — same; needs the adult+junior member names so the `/payments` per-person grouping is consistent.
|
||||||
|
|
||||||
|
- Naming: `_canonical_key` starts with an underscore inside `match_payments.py`. To avoid leaking a private symbol, expose it as `canonical_member_key` (no underscore) in `match_payments.py` and import that name from `app.py`.
|
||||||
|
|
||||||
|
### 3. Why not also touch `infer_payments.py`
|
||||||
|
|
||||||
|
`infer_payments.py` already writes canonical attendance-sheet names into the `Person` column (it picks from `member_names`). The bug only manifests when the cell was filled in **manually** by a human (typed without diacritics, different case) or was written by an older inference that has since drifted from a renamed attendance row. Making `reconcile()` tolerant fixes the symptom for both cases without changing inference. The `logger.info` line is sufficient signal for the user to clean up the sheet on their own schedule.
|
||||||
|
|
||||||
|
### 4. Tests
|
||||||
|
|
||||||
|
**4a. Delete obsolete route tests in [tests/test_app.py](tests/test_app.py).** Four tests target Flask routes that no longer exist (the old fee/reconcile pages were merged into `/adults` and `/juniors`); they currently fail with 404. Their coverage is already provided by `test_adults_route`, `test_juniors_route`, and `test_payments_route`. Delete:
|
||||||
|
|
||||||
|
- `test_fees_route` ([tests/test_app.py:22-35](tests/test_app.py#L22-L35)) — hits `/fees`
|
||||||
|
- `test_fees_juniors_route` ([tests/test_app.py:37-55](tests/test_app.py#L37-L55)) — hits `/fees-juniors`
|
||||||
|
- `test_reconcile_route` ([tests/test_app.py:57-81](tests/test_app.py#L57-L81)) — hits `/reconcile`; also asserts a literal `OK` string the merged dashboard no longer renders
|
||||||
|
- `test_reconcile_juniors_route` ([tests/test_app.py:101-131](tests/test_app.py#L101-L131)) — hits `/reconcile-juniors`; same `OK` assertion mismatch
|
||||||
|
|
||||||
|
The two tests that reference junior-only formatting (`? / 1 (J)` and `500 CZK / 4 (1A+3J)`) are testing a retired template, not the live `/juniors` page — no need to migrate those assertions; the live `/juniors` format is already covered by `test_juniors_route`.
|
||||||
|
|
||||||
|
**4b. Add `tests/test_match_payments.py`** (new file) covering the resolution helper and `reconcile()` end-to-end for the canonicalization fix:
|
||||||
|
|
||||||
|
- `_canonical_key("Mária Maco") == _canonical_key("maria maco")`
|
||||||
|
- `reconcile()` with member `"Mária Maco"` and a tx `{person: "Maria Maco", purpose: "2026-04", amount: 750, ...}` produces:
|
||||||
|
- `result['members']['Mária Maco']['months']['2026-04']['paid'] == 750`
|
||||||
|
- the tx appears in `result['members']['Mária Maco']['months']['2026-04']['transactions']`
|
||||||
|
- `result['unmatched']` is empty
|
||||||
|
- `reconcile()` with `Person = "Někdo Neznámý"` (no match in members) still routes to `unmatched`.
|
||||||
|
|
||||||
|
## Critical files
|
||||||
|
|
||||||
|
- [scripts/match_payments.py](scripts/match_payments.py) — add `canonical_member_key()` helper; build `canonical_by_key` once in `reconcile()`; resolve `raw_member_name` → `member_name` before ledger access at [:404](scripts/match_payments.py#L404).
|
||||||
|
- [app.py](app.py) — extend `group_payments_by_person()` to accept `member_names` and key the grouped dict by canonical attendance-sheet name; update three call sites.
|
||||||
|
- [tests/test_app.py](tests/test_app.py) — delete the four obsolete route tests listed in §4a.
|
||||||
|
- [tests/test_match_payments.py](tests/test_match_payments.py) — add the cases above (create the file if missing).
|
||||||
|
- [docs/plans/](docs/plans/) — per project [CLAUDE.md](CLAUDE.md), move this plan file to `docs/plans/2026-05-05-1640-payment-person-name-canonicalization.md` once execution starts (the plan-mode harness writes to `~/.claude/plans/` by default).
|
||||||
|
|
||||||
|
## Verification
|
||||||
|
|
||||||
|
1. **Reproduce first.** Before touching code, open `/adults`, click `[i]` next to "Mária Maco", and confirm both: 2026-04 is unpaid and the payment is missing from history. Inspect the actual `Person` cell value in the payments sheet for the 2026-04 row — confirm it differs from `"Mária Maco"` (likely missing the `á`). Record the exact string for the test case.
|
||||||
|
2. `make test` — new tests pass; existing tests still green.
|
||||||
|
3. `make web-debug` and reload `/adults`. The 2026-04 cell for "Mária Maco" turns green (`cell-ok`); the modal's payment history shows the row; the "Raw Payments" section also shows the row. Server log emits `Person cell 'Maria Maco' resolved to canonical member 'Mária Maco' — consider fixing the sheet`.
|
||||||
|
4. Cross-check `/payments` — the row appears under the `Mária Maco` group (canonical key), not under a separate `Maria Maco` group.
|
||||||
|
5. Spot-check one member with the conventionally-correct `Person` value (e.g. one of the recent payers visible on the dashboard) — paid cells and history are unchanged, no spurious resolution log line.
|
||||||
|
6. Confirm a payment with a genuinely unknown `Person` (typo of a non-member) still ends up in the dashboard's `Unmatched` block and emits the existing `Payment matched to unknown member …` warning.
|
||||||
|
7. Append a `CHANGELOG.md` entry per [CLAUDE.md](CLAUDE.md) once the user confirms the fix works.
|
||||||
83
docs/plans/2026-05-05-2144-branch-per-feature-workflow.md
Normal file
83
docs/plans/2026-05-05-2144-branch-per-feature-workflow.md
Normal file
@@ -0,0 +1,83 @@
|
|||||||
|
# Branch-per-feature + Gitea MR workflow
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
Until now, Claude has been committing feature work directly to `main`
|
||||||
|
(see recent history: `feat: Lower adult monthly fee…`, `feat: Go rewrite M1…`,
|
||||||
|
all on `main`). The user wants to switch to a branch-per-feature flow with
|
||||||
|
review via a Gitea merge request, so that:
|
||||||
|
|
||||||
|
- Feature work is reviewable as a self-contained diff before it lands.
|
||||||
|
- `main` stays releasable.
|
||||||
|
- The change history shows reviewed merges, not unsupervised pushes.
|
||||||
|
|
||||||
|
The remote is Gitea (`https://gitea.home.hrajfrisbee.cz/kacerr/fuj-management.git`),
|
||||||
|
which supports the standard pull/merge-request flow.
|
||||||
|
|
||||||
|
This plan only modifies `CLAUDE.md`. No code changes.
|
||||||
|
|
||||||
|
## Scope clarification (from user)
|
||||||
|
|
||||||
|
- **MR creation method:** Claude pushes the branch and prints the Gitea
|
||||||
|
"compare" URL. The user opens / merges the MR in the browser. No `tea` CLI,
|
||||||
|
no API calls.
|
||||||
|
- **When the flow applies:** Features only. Small bug fixes and hotfixes can
|
||||||
|
still be committed straight to `main`. Claude decides feature-vs-fix based
|
||||||
|
on scope; when uncertain, ask.
|
||||||
|
- **Branch naming:** `feat/<slug>` for features, `fix/<slug>` for the
|
||||||
|
occasional bug-fix branch the user explicitly requests. `<slug>` is
|
||||||
|
kebab-case, short, descriptive.
|
||||||
|
|
||||||
|
## Change
|
||||||
|
|
||||||
|
Add a new top-level section to `CLAUDE.md` titled **"Branching & merge requests"**,
|
||||||
|
placed immediately before the existing `## Git Commits` section so the workflow
|
||||||
|
context appears before the commit-message convention.
|
||||||
|
|
||||||
|
### Proposed section content
|
||||||
|
|
||||||
|
```markdown
|
||||||
|
## Branching & merge requests
|
||||||
|
|
||||||
|
The remote is Gitea (`gitea.home.hrajfrisbee.cz/kacerr/fuj-management`).
|
||||||
|
For **features**, do not commit to `main` directly. Use a branch + merge
|
||||||
|
request flow:
|
||||||
|
|
||||||
|
1. **Create a branch off `main`** before starting work:
|
||||||
|
- `feat/<slug>` for features (e.g. `feat/qr-code-overlay`)
|
||||||
|
- `fix/<slug>` for bug-fix branches the user explicitly asks for
|
||||||
|
- `<slug>` is short kebab-case
|
||||||
|
2. **Commit on the branch** following the existing commit conventions
|
||||||
|
(Co-Authored-By trailer, etc.).
|
||||||
|
3. **Push the branch** to `origin` with `-u` so it tracks.
|
||||||
|
4. **Print the Gitea compare URL** so the user can open the MR in the
|
||||||
|
browser:
|
||||||
|
`https://gitea.home.hrajfrisbee.cz/kacerr/fuj-management/compare/main...<branch>`
|
||||||
|
Do **not** use `tea`, `gh`, or call the Gitea API — the user opens and
|
||||||
|
merges the MR themselves.
|
||||||
|
5. **Do not merge or delete the branch** from the CLI. The user does that
|
||||||
|
in Gitea.
|
||||||
|
|
||||||
|
**Exceptions — when committing straight to `main` is fine:**
|
||||||
|
- Small bug fixes / hotfixes the user describes as such.
|
||||||
|
- Typo / comment / formatting tweaks.
|
||||||
|
- Edits the user explicitly says to push to `main`.
|
||||||
|
|
||||||
|
When uncertain whether something is "feature" or "small fix", ask before
|
||||||
|
committing.
|
||||||
|
```
|
||||||
|
|
||||||
|
## Files to modify
|
||||||
|
|
||||||
|
- [CLAUDE.md](CLAUDE.md) — insert the new `## Branching & merge requests`
|
||||||
|
section just above the existing `## Git Commits` section (around line 95).
|
||||||
|
|
||||||
|
## Verification
|
||||||
|
|
||||||
|
- Re-read `CLAUDE.md` and confirm the new section is well-placed and the
|
||||||
|
existing structure (`## Git Commits`, `## Changelog`, `## Plans`) is intact.
|
||||||
|
- `git diff CLAUDE.md` should show only an additive change.
|
||||||
|
- No code, tests, or runtime behavior changes — nothing else to test.
|
||||||
|
- Behavior verification happens on the **next** feature request: Claude
|
||||||
|
should create a `feat/<slug>` branch, commit there, push, and print the
|
||||||
|
compare URL instead of committing on `main`.
|
||||||
154
docs/plans/2026-05-05-2204-go-rewrite-m2-1-czech-normalize.md
Normal file
154
docs/plans/2026-05-05-2204-go-rewrite-m2-1-czech-normalize.md
Normal file
@@ -0,0 +1,154 @@
|
|||||||
|
# Plan: Go rewrite — M2.1 `domain/czech.Normalize`
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
The Go rewrite finished M1 (skeleton, tooling, hello server) in commit
|
||||||
|
`cf0f176` on 2026-05-04. The next milestone, **M2 — Pure-domain helpers**,
|
||||||
|
is current per [progress tracker](2026-05-03-2349-go-backend-rewrite-progress.md)
|
||||||
|
but has no work landed yet (all 12 sub-tasks unchecked).
|
||||||
|
|
||||||
|
This plan covers only the **first** M2 task: porting Python's
|
||||||
|
`normalize` from [scripts/czech_utils.py](../../scripts/czech_utils.py)
|
||||||
|
to Go as `internal/domain/czech.Normalize`. It is the lowest-level helper
|
||||||
|
in the domain — `parse_month_references`, `_build_name_variants`,
|
||||||
|
`match_members`, exception keys, and `reconcile` all transitively depend
|
||||||
|
on it. Getting it byte-equivalent first removes a class of "why does my
|
||||||
|
match not fire" failures from every later M2 task.
|
||||||
|
|
||||||
|
**Decision (confirmed in plan-mode Q):** start with hand-written Go unit
|
||||||
|
tests for fresh Czech edge cases. Defer parity-fixture wiring until
|
||||||
|
M3.1/M3.2 land (separate task); add the parity test for `Normalize`
|
||||||
|
retroactively at that point.
|
||||||
|
|
||||||
|
## Scope
|
||||||
|
|
||||||
|
- New package `go/internal/domain/czech/` with `Normalize` and unit tests.
|
||||||
|
- Add `golang.org/x/text` dependency to `go/go.mod` (currently zero deps).
|
||||||
|
- **Out of scope:** `ParseMonthReferences` (M2.2), fixture tooling
|
||||||
|
(M3.1/M3.2), CLI subcommand wiring (M2.11/M2.12), parity test runner.
|
||||||
|
|
||||||
|
## Recommended approach
|
||||||
|
|
||||||
|
### Python contract to match
|
||||||
|
|
||||||
|
```python
|
||||||
|
def normalize(text: str) -> str:
|
||||||
|
nfkd = unicodedata.normalize("NFKD", text)
|
||||||
|
return "".join(c for c in nfkd if not unicodedata.combining(c)).lower()
|
||||||
|
```
|
||||||
|
|
||||||
|
Three semantic operations:
|
||||||
|
1. NFKD decompose
|
||||||
|
2. Drop characters where `unicodedata.combining(c)` is non-zero
|
||||||
|
3. Lowercase
|
||||||
|
|
||||||
|
### Go implementation
|
||||||
|
|
||||||
|
`go/internal/domain/czech/normalize.go`:
|
||||||
|
|
||||||
|
```go
|
||||||
|
package czech
|
||||||
|
|
||||||
|
import (
|
||||||
|
"strings"
|
||||||
|
"unicode"
|
||||||
|
"golang.org/x/text/unicode/norm"
|
||||||
|
)
|
||||||
|
|
||||||
|
func Normalize(s string) string {
|
||||||
|
decomposed := norm.NFKD.String(s)
|
||||||
|
var b strings.Builder
|
||||||
|
b.Grow(len(decomposed))
|
||||||
|
for _, r := range decomposed {
|
||||||
|
if unicode.In(r, unicode.Mn) {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
b.WriteRune(r)
|
||||||
|
}
|
||||||
|
return strings.ToLower(b.String())
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Two precision points worth flagging:**
|
||||||
|
|
||||||
|
1. **`unicode.Mn` not `unicode.IsMark`.** The plan's library-choices
|
||||||
|
table mentions `unicode.IsMark`, but that covers Mn + Mc + Me. Python
|
||||||
|
`unicodedata.combining()` returns 0 for Mc/Me (their canonical
|
||||||
|
combining class is 0), so it effectively filters only Mn. Use
|
||||||
|
`unicode.In(r, unicode.Mn)` for byte-equivalence with Python. Cite
|
||||||
|
this in a one-line code comment; it's the kind of thing a future
|
||||||
|
reader will second-guess.
|
||||||
|
2. **`strings.ToLower` vs Go's locale-aware tools.** Python's `.lower()`
|
||||||
|
on already-decomposed Latin is straight ASCII lowercase for Czech.
|
||||||
|
Stdlib `strings.ToLower` matches; do not pull in `golang.org/x/text/cases`.
|
||||||
|
|
||||||
|
### Tests
|
||||||
|
|
||||||
|
`go/internal/domain/czech/normalize_test.go` — table-driven, covers:
|
||||||
|
|
||||||
|
- ASCII passthrough: `"Honza" → "honza"`
|
||||||
|
- Czech lowercase diacritics: `"žluťoučký" → "zlutoucky"`
|
||||||
|
- Mixed case + diacritics: `"Příliš" → "prilis"`
|
||||||
|
- Czech caron + ring: `"Dvořák" → "dvorak"`, `"Růžena" → "ruzena"`
|
||||||
|
- Hard letters: `"Čeněk" → "cenek"`, `"Kačer" → "kacer"`
|
||||||
|
- Empty string: `"" → ""`
|
||||||
|
- Already-normalized: `"prilis" → "prilis"` (idempotence)
|
||||||
|
- Pre-composed vs decomposed input both produce the same output (NFC
|
||||||
|
`"é"` and `"é"` both → `"e"`)
|
||||||
|
- Whitespace preserved: `"Jan Novák" → "jan novak"`
|
||||||
|
|
||||||
|
Run a one-shot cross-check against the live Python implementation for
|
||||||
|
each test input before locking the table:
|
||||||
|
```
|
||||||
|
PYTHONPATH=scripts:. python -c \
|
||||||
|
'from czech_utils import normalize; print(repr(normalize("Dvořák")))'
|
||||||
|
```
|
||||||
|
This is the manual stand-in for the M3 parity fixtures.
|
||||||
|
|
||||||
|
### Wire-up
|
||||||
|
|
||||||
|
- `go get golang.org/x/text@latest` (run from `go/`); `go mod tidy`.
|
||||||
|
- No CLI changes — `cmd/fuj` already stubs `fees`/`reconcile` with
|
||||||
|
exit code 2; no need to touch dispatcher for this task. `Normalize`
|
||||||
|
is consumed by other domain code, not by users directly.
|
||||||
|
|
||||||
|
## Critical files
|
||||||
|
|
||||||
|
- New: [go/internal/domain/czech/normalize.go](../../go/internal/domain/czech/normalize.go)
|
||||||
|
- New: [go/internal/domain/czech/normalize_test.go](../../go/internal/domain/czech/normalize_test.go)
|
||||||
|
- Modified: [go/go.mod](../../go/go.mod), `go/go.sum` (new)
|
||||||
|
- Reference (read-only): [scripts/czech_utils.py](../../scripts/czech_utils.py) — the porting source
|
||||||
|
- Reference (read-only): [docs/plans/2026-05-03-2349-go-backend-rewrite.md](2026-05-03-2349-go-backend-rewrite.md) — risk #3 (NFKD edge cases)
|
||||||
|
|
||||||
|
## Verification
|
||||||
|
|
||||||
|
End-to-end checks before marking M2.1 done:
|
||||||
|
|
||||||
|
1. `cd go && go build ./...` — clean compile.
|
||||||
|
2. `cd go && go test ./internal/domain/czech/...` — all table cases green.
|
||||||
|
3. `cd go && go test -race ./...` — race-clean.
|
||||||
|
4. `cd go && golangci-lint run` (or `make go-lint` from repo root) — clean.
|
||||||
|
5. **Spot parity** (manual, will be automated in M3): for each Go test
|
||||||
|
input, run the Python `normalize` via `PYTHONPATH=scripts:. python -c
|
||||||
|
'...'` and confirm bytes match. Capture the diff in the commit
|
||||||
|
message if anything surprises.
|
||||||
|
6. `make go-build && make go-test && make go-lint` from repo root — proves
|
||||||
|
the existing M1 gate still passes.
|
||||||
|
|
||||||
|
## Branching & follow-up
|
||||||
|
|
||||||
|
Per [CLAUDE.md](../../CLAUDE.md), this is feature work → branch + Gitea MR:
|
||||||
|
|
||||||
|
- Branch: `feat/m2-1-czech-normalize` off `main`.
|
||||||
|
- Single commit, Co-Authored-By trailer.
|
||||||
|
- Push with `-u`, print compare URL
|
||||||
|
`https://gitea.home.hrajfrisbee.cz/kacerr/fuj-management/compare/main...feat/m2-1-czech-normalize`
|
||||||
|
- User opens/merges the MR.
|
||||||
|
- After merge: tick `M2.1` in the progress tracker with the commit SHA;
|
||||||
|
add a one-line CHANGELOG entry; record any porting surprise in the
|
||||||
|
tracker's "Notes & decisions" section (e.g. the `Mn`-vs-`IsMark`
|
||||||
|
precision point if it bears noting).
|
||||||
|
|
||||||
|
Next task after this lands is **M2.2 `ParseMonthReferences`** — the
|
||||||
|
larger, edge-case-heavier sibling. Whether to start it before or after
|
||||||
|
M3.1/M3.2 is a separate decision the user can make then.
|
||||||
11
go/.golangci.yml
Normal file
11
go/.golangci.yml
Normal file
@@ -0,0 +1,11 @@
|
|||||||
|
linters:
|
||||||
|
enable:
|
||||||
|
- govet
|
||||||
|
- staticcheck
|
||||||
|
- errcheck
|
||||||
|
- gofumpt
|
||||||
|
- unused
|
||||||
|
|
||||||
|
linters-settings:
|
||||||
|
gofumpt:
|
||||||
|
extra-rules: true
|
||||||
30
go/build/Dockerfile
Normal file
30
go/build/Dockerfile
Normal file
@@ -0,0 +1,30 @@
|
|||||||
|
FROM golang:1.26 AS build
|
||||||
|
|
||||||
|
WORKDIR /src
|
||||||
|
|
||||||
|
COPY go.mod go.sum ./
|
||||||
|
RUN go mod download
|
||||||
|
|
||||||
|
COPY . .
|
||||||
|
|
||||||
|
ARG GIT_TAG=unknown
|
||||||
|
ARG GIT_COMMIT=unknown
|
||||||
|
ARG BUILD_DATE=unknown
|
||||||
|
|
||||||
|
RUN CGO_ENABLED=0 go build -trimpath \
|
||||||
|
-ldflags "-s -w \
|
||||||
|
-X main.version=${GIT_TAG} \
|
||||||
|
-X main.commit=${GIT_COMMIT} \
|
||||||
|
-X main.buildDate=${BUILD_DATE}" \
|
||||||
|
-o /out/fuj ./cmd/fuj
|
||||||
|
|
||||||
|
FROM alpine:3
|
||||||
|
|
||||||
|
RUN addgroup -S fuj && adduser -S fuj -G fuj
|
||||||
|
|
||||||
|
COPY --from=build /out/fuj /usr/local/bin/fuj
|
||||||
|
|
||||||
|
EXPOSE 8080
|
||||||
|
USER fuj
|
||||||
|
|
||||||
|
ENTRYPOINT ["/usr/local/bin/fuj", "server"]
|
||||||
84
go/cmd/fuj/main.go
Normal file
84
go/cmd/fuj/main.go
Normal file
@@ -0,0 +1,84 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"flag"
|
||||||
|
"fmt"
|
||||||
|
"fuj-management/go/internal/config"
|
||||||
|
"fuj-management/go/internal/logging"
|
||||||
|
"fuj-management/go/internal/web"
|
||||||
|
"os"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Injected at build time via -ldflags "-X main.version=... -X main.commit=... -X main.buildDate=..."
|
||||||
|
var (
|
||||||
|
version = "dev"
|
||||||
|
commit = "unknown"
|
||||||
|
buildDate = "unknown"
|
||||||
|
)
|
||||||
|
|
||||||
|
func main() {
|
||||||
|
if len(os.Args) < 2 {
|
||||||
|
usage()
|
||||||
|
os.Exit(2)
|
||||||
|
}
|
||||||
|
|
||||||
|
cmd, args := os.Args[1], os.Args[2:]
|
||||||
|
|
||||||
|
switch cmd {
|
||||||
|
case "server":
|
||||||
|
serverCmd(args)
|
||||||
|
case "version":
|
||||||
|
versionCmd()
|
||||||
|
case "fees", "reconcile", "sync", "infer":
|
||||||
|
fmt.Fprintf(os.Stderr, "fuj %s: not implemented yet (lands in M2/M4)\n", cmd)
|
||||||
|
os.Exit(2)
|
||||||
|
case "-h", "--help", "help":
|
||||||
|
usage()
|
||||||
|
default:
|
||||||
|
fmt.Fprintf(os.Stderr, "fuj: unknown command %q\n\n", cmd)
|
||||||
|
usage()
|
||||||
|
os.Exit(2)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func serverCmd(args []string) {
|
||||||
|
fs := flag.NewFlagSet("server", flag.ExitOnError)
|
||||||
|
addr := fs.String("addr", "", "listen address (default from SERVER_ADDR env or :8080)")
|
||||||
|
fs.Usage = func() {
|
||||||
|
fmt.Fprintln(os.Stderr, "usage: fuj server [--addr :8080]")
|
||||||
|
fs.PrintDefaults()
|
||||||
|
}
|
||||||
|
if err := fs.Parse(args); err != nil {
|
||||||
|
fmt.Fprintln(os.Stderr, err)
|
||||||
|
os.Exit(2)
|
||||||
|
}
|
||||||
|
|
||||||
|
cfg := config.Load()
|
||||||
|
if *addr != "" {
|
||||||
|
cfg.ServerAddr = *addr
|
||||||
|
}
|
||||||
|
|
||||||
|
logger := logging.New(cfg.LogLevel)
|
||||||
|
build := web.BuildInfo{Version: version, Commit: commit, BuildDate: buildDate}
|
||||||
|
|
||||||
|
if err := web.Run(logger, cfg.ServerAddr, build); err != nil {
|
||||||
|
fmt.Fprintln(os.Stderr, err)
|
||||||
|
os.Exit(1)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func versionCmd() {
|
||||||
|
fmt.Printf("fuj %s (%s) built %s\n", version, commit, buildDate)
|
||||||
|
}
|
||||||
|
|
||||||
|
func usage() {
|
||||||
|
fmt.Fprintln(os.Stderr, `usage: fuj <command> [flags]
|
||||||
|
|
||||||
|
Commands:
|
||||||
|
server Start HTTP server (default :8080)
|
||||||
|
version Print version information
|
||||||
|
fees Calculate monthly fees [M2]
|
||||||
|
reconcile Show balance report [M2]
|
||||||
|
sync Sync Fio transactions [M4]
|
||||||
|
infer Infer payment details [M4]`)
|
||||||
|
}
|
||||||
5
go/go.mod
Normal file
5
go/go.mod
Normal file
@@ -0,0 +1,5 @@
|
|||||||
|
module fuj-management/go
|
||||||
|
|
||||||
|
go 1.26.1
|
||||||
|
|
||||||
|
require golang.org/x/text v0.36.0
|
||||||
2
go/go.sum
Normal file
2
go/go.sum
Normal file
@@ -0,0 +1,2 @@
|
|||||||
|
golang.org/x/text v0.36.0 h1:JfKh3XmcRPqZPKevfXVpI1wXPTqbkE5f7JA92a55Yxg=
|
||||||
|
golang.org/x/text v0.36.0/go.mod h1:NIdBknypM8iqVmPiuco0Dh6P5Jcdk8lJL0CUebqK164=
|
||||||
56
go/internal/config/config.go
Normal file
56
go/internal/config/config.go
Normal file
@@ -0,0 +1,56 @@
|
|||||||
|
package config
|
||||||
|
|
||||||
|
import (
|
||||||
|
"os"
|
||||||
|
"strconv"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Google Sheets IDs — change in code if sheets change (not from env).
|
||||||
|
const (
|
||||||
|
AttendanceSheetID = "1E2e_gT_K5AwSRCDLDTa2UetZTkHmBOcz0kFbBUNUNBA"
|
||||||
|
PaymentsSheetID = "1Om0YPoDVCH5cV8BrNz5LG5eR5MMU05ypQC7UMN1xn_Y"
|
||||||
|
JuniorSheetGID = "1213318614"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Config holds all runtime configuration loaded from environment variables.
|
||||||
|
// Mirrors scripts/config.py.
|
||||||
|
type Config struct {
|
||||||
|
CredentialsPath string
|
||||||
|
BankAccount string
|
||||||
|
CacheTTL time.Duration
|
||||||
|
CacheAPICheckTTL time.Duration
|
||||||
|
LogLevel string
|
||||||
|
FioAPIToken string
|
||||||
|
ServerAddr string
|
||||||
|
}
|
||||||
|
|
||||||
|
// Load reads configuration from the environment, applying defaults that
|
||||||
|
// match the Python side.
|
||||||
|
func Load() Config {
|
||||||
|
return Config{
|
||||||
|
CredentialsPath: env("CREDENTIALS_PATH", ".secret/fuj-management-bot-credentials.json"),
|
||||||
|
BankAccount: env("BANK_ACCOUNT", "CZ8520100000002800359168"),
|
||||||
|
CacheTTL: envDuration("CACHE_TTL_SECONDS", 300),
|
||||||
|
CacheAPICheckTTL: envDuration("CACHE_API_CHECK_TTL_SECONDS", 300),
|
||||||
|
LogLevel: env("LOG_LEVEL", "INFO"),
|
||||||
|
FioAPIToken: env("FIO_API_TOKEN", ""),
|
||||||
|
ServerAddr: env("SERVER_ADDR", ":8080"),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func env(key, fallback string) string {
|
||||||
|
if v := os.Getenv(key); v != "" {
|
||||||
|
return v
|
||||||
|
}
|
||||||
|
return fallback
|
||||||
|
}
|
||||||
|
|
||||||
|
func envDuration(key string, defaultSeconds int) time.Duration {
|
||||||
|
if v := os.Getenv(key); v != "" {
|
||||||
|
if n, err := strconv.Atoi(v); err == nil && n > 0 {
|
||||||
|
return time.Duration(n) * time.Second
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return time.Duration(defaultSeconds) * time.Second
|
||||||
|
}
|
||||||
26
go/internal/domain/czech/normalize.go
Normal file
26
go/internal/domain/czech/normalize.go
Normal file
@@ -0,0 +1,26 @@
|
|||||||
|
package czech
|
||||||
|
|
||||||
|
import (
|
||||||
|
"strings"
|
||||||
|
"unicode"
|
||||||
|
|
||||||
|
"golang.org/x/text/unicode/norm"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Normalize strips diacritics and lowercases s.
|
||||||
|
//
|
||||||
|
// Matches Python: unicodedata.normalize("NFKD", s) then filter out
|
||||||
|
// combining characters (unicode.Mn only — not Mc/Me, which have
|
||||||
|
// combining class 0 in Python's unicodedata.combining()).
|
||||||
|
func Normalize(s string) string {
|
||||||
|
decomposed := norm.NFKD.String(s)
|
||||||
|
var b strings.Builder
|
||||||
|
b.Grow(len(decomposed))
|
||||||
|
for _, r := range decomposed {
|
||||||
|
if unicode.In(r, unicode.Mn) {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
b.WriteRune(r)
|
||||||
|
}
|
||||||
|
return strings.ToLower(b.String())
|
||||||
|
}
|
||||||
31
go/internal/domain/czech/normalize_test.go
Normal file
31
go/internal/domain/czech/normalize_test.go
Normal file
@@ -0,0 +1,31 @@
|
|||||||
|
package czech
|
||||||
|
|
||||||
|
import "testing"
|
||||||
|
|
||||||
|
func TestNormalize(t *testing.T) {
|
||||||
|
cases := []struct {
|
||||||
|
in string
|
||||||
|
want string
|
||||||
|
}{
|
||||||
|
{"Honza", "honza"},
|
||||||
|
{"žluťoučký", "zlutoucky"},
|
||||||
|
{"Příliš", "prilis"},
|
||||||
|
{"Dvořák", "dvorak"},
|
||||||
|
{"Růžena", "ruzena"},
|
||||||
|
{"Čeněk", "cenek"},
|
||||||
|
{"Kačer", "kacer"},
|
||||||
|
{"", ""},
|
||||||
|
{"prilis", "prilis"}, // idempotent
|
||||||
|
{"Jan Novák", "jan novak"}, // whitespace preserved
|
||||||
|
{"é", "e"}, // precomposed é (NFC)
|
||||||
|
{"é", "e"}, // decomposed e + combining acute
|
||||||
|
{"Ondřej Procházka", "ondrej prochazka"}, // realistic full name
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tc := range cases {
|
||||||
|
got := Normalize(tc.in)
|
||||||
|
if got != tc.want {
|
||||||
|
t.Errorf("Normalize(%q) = %q, want %q", tc.in, got, tc.want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
24
go/internal/logging/logger.go
Normal file
24
go/internal/logging/logger.go
Normal file
@@ -0,0 +1,24 @@
|
|||||||
|
package logging
|
||||||
|
|
||||||
|
import (
|
||||||
|
"log/slog"
|
||||||
|
"os"
|
||||||
|
"strings"
|
||||||
|
)
|
||||||
|
|
||||||
|
// New returns a slog.Logger at the given level (DEBUG|INFO|WARN|ERROR).
|
||||||
|
// Pass config.Config.LogLevel as the argument. Defaults to INFO on unrecognised input.
|
||||||
|
func New(level string) *slog.Logger {
|
||||||
|
var l slog.Level
|
||||||
|
switch strings.ToUpper(level) {
|
||||||
|
case "DEBUG":
|
||||||
|
l = slog.LevelDebug
|
||||||
|
case "WARN", "WARNING":
|
||||||
|
l = slog.LevelWarn
|
||||||
|
case "ERROR":
|
||||||
|
l = slog.LevelError
|
||||||
|
default:
|
||||||
|
l = slog.LevelInfo
|
||||||
|
}
|
||||||
|
return slog.New(slog.NewTextHandler(os.Stderr, &slog.HandlerOptions{Level: l}))
|
||||||
|
}
|
||||||
34
go/internal/web/middleware/timer.go
Normal file
34
go/internal/web/middleware/timer.go
Normal file
@@ -0,0 +1,34 @@
|
|||||||
|
package middleware
|
||||||
|
|
||||||
|
import (
|
||||||
|
"log/slog"
|
||||||
|
"net/http"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
type statusWriter struct {
|
||||||
|
http.ResponseWriter
|
||||||
|
status int
|
||||||
|
}
|
||||||
|
|
||||||
|
func (sw *statusWriter) WriteHeader(code int) {
|
||||||
|
sw.status = code
|
||||||
|
sw.ResponseWriter.WriteHeader(code)
|
||||||
|
}
|
||||||
|
|
||||||
|
// RequestTimer logs method, path, status, and elapsed milliseconds for every
|
||||||
|
// request. Parity with Python's get_render_time — the elapsed value maps to
|
||||||
|
// render_time.total in the M5 JSON allowlist.
|
||||||
|
func RequestTimer(logger *slog.Logger, next http.Handler) http.Handler {
|
||||||
|
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
start := time.Now()
|
||||||
|
sw := &statusWriter{ResponseWriter: w, status: http.StatusOK}
|
||||||
|
next.ServeHTTP(sw, r)
|
||||||
|
logger.Info("req",
|
||||||
|
"method", r.Method,
|
||||||
|
"path", r.URL.Path,
|
||||||
|
"status", sw.status,
|
||||||
|
"ms", time.Since(start).Milliseconds(),
|
||||||
|
)
|
||||||
|
})
|
||||||
|
}
|
||||||
32
go/internal/web/server.go
Normal file
32
go/internal/web/server.go
Normal file
@@ -0,0 +1,32 @@
|
|||||||
|
package web
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"fuj-management/go/internal/web/middleware"
|
||||||
|
"log/slog"
|
||||||
|
"net/http"
|
||||||
|
)
|
||||||
|
|
||||||
|
// BuildInfo carries the linker-injected build metadata.
|
||||||
|
type BuildInfo struct {
|
||||||
|
Version string
|
||||||
|
Commit string
|
||||||
|
BuildDate string
|
||||||
|
}
|
||||||
|
|
||||||
|
// Run registers routes and starts the HTTP server on addr.
|
||||||
|
func Run(logger *slog.Logger, addr string, build BuildInfo) error {
|
||||||
|
mux := http.NewServeMux()
|
||||||
|
mux.HandleFunc("GET /{$}", helloHandler(build))
|
||||||
|
|
||||||
|
logger.Info("starting server", "addr", addr)
|
||||||
|
return http.ListenAndServe(addr, middleware.RequestTimer(logger, mux))
|
||||||
|
}
|
||||||
|
|
||||||
|
func helloHandler(build BuildInfo) http.HandlerFunc {
|
||||||
|
return func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
w.Header().Set("Content-Type", "text/plain; charset=utf-8")
|
||||||
|
fmt.Fprintf(w, "fuj-go ok\nversion: %s\ncommit: %s\nbuilt: %s\n",
|
||||||
|
build.Version, build.Commit, build.BuildDate)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -10,10 +10,18 @@ from config import ATTENDANCE_SHEET_ID as SHEET_ID, JUNIOR_SHEET_GID
|
|||||||
EXPORT_URL = f"https://docs.google.com/spreadsheets/d/{SHEET_ID}/export?format=csv&gid=0"
|
EXPORT_URL = f"https://docs.google.com/spreadsheets/d/{SHEET_ID}/export?format=csv&gid=0"
|
||||||
JUNIOR_EXPORT_URL = f"https://docs.google.com/spreadsheets/d/{SHEET_ID}/export?format=csv&gid={JUNIOR_SHEET_GID}"
|
JUNIOR_EXPORT_URL = f"https://docs.google.com/spreadsheets/d/{SHEET_ID}/export?format=csv&gid={JUNIOR_SHEET_GID}"
|
||||||
|
|
||||||
ADULT_FEE_DEFAULT = 750 # CZK, for 2+ practices in a month
|
ADULT_FEE_DEFAULT = 700 # CZK, for 2+ practices in a month
|
||||||
ADULT_FEE_SINGLE = 200 # CZK, for exactly 1 practice in a month
|
ADULT_FEE_SINGLE = 200 # CZK, for exactly 1 practice in a month
|
||||||
ADULT_FEE_MONTHLY_RATE = {
|
ADULT_FEE_MONTHLY_RATE = {
|
||||||
"2026-03": 350
|
"2025-09": 750,
|
||||||
|
"2025-10": 750,
|
||||||
|
"2025-11": 750,
|
||||||
|
"2025-12": 750,
|
||||||
|
"2026-01": 750,
|
||||||
|
"2026-02": 750,
|
||||||
|
"2026-03": 350,
|
||||||
|
"2026-04": 700,
|
||||||
|
"2026-05": 700,
|
||||||
}
|
}
|
||||||
|
|
||||||
JUNIOR_FEE_DEFAULT = 500 # CZK for 2+ practices
|
JUNIOR_FEE_DEFAULT = 500 # CZK for 2+ practices
|
||||||
|
|||||||
@@ -17,6 +17,15 @@ from czech_utils import normalize, parse_month_references
|
|||||||
from sync_fio_to_sheets import get_sheets_service, DEFAULT_SPREADSHEET_ID
|
from sync_fio_to_sheets import get_sheets_service, DEFAULT_SPREADSHEET_ID
|
||||||
|
|
||||||
|
|
||||||
|
def canonical_member_key(name: str) -> str:
|
||||||
|
"""Diacritic-, case-, and whitespace-insensitive key for member-name matching.
|
||||||
|
|
||||||
|
Used to resolve `Person`-column values from the payments sheet to canonical
|
||||||
|
attendance-sheet names, tolerating cells like "Maria Maco" vs "Mária Maco".
|
||||||
|
"""
|
||||||
|
return re.sub(r"\s+", " ", normalize(name)).strip()
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# Name matching
|
# Name matching
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
@@ -48,6 +57,11 @@ def _build_name_variants(name: str) -> list[str]:
|
|||||||
return [v for v in variants if len(v) >= 3]
|
return [v for v in variants if len(v) >= 3]
|
||||||
|
|
||||||
|
|
||||||
|
def _word_in(needle: str, haystack: str) -> bool:
|
||||||
|
"""Return True if needle appears as a whole word in haystack."""
|
||||||
|
return bool(re.search(rf"\b{re.escape(needle)}\b", haystack))
|
||||||
|
|
||||||
|
|
||||||
def match_members(
|
def match_members(
|
||||||
text: str, member_names: list[str]
|
text: str, member_names: list[str]
|
||||||
) -> list[tuple[str, str]]:
|
) -> list[tuple[str, str]]:
|
||||||
@@ -56,6 +70,19 @@ def match_members(
|
|||||||
Returns list of (member_name, confidence) where confidence is 'auto' or 'review'.
|
Returns list of (member_name, confidence) where confidence is 'auto' or 'review'.
|
||||||
"""
|
"""
|
||||||
normalized_text = normalize(text)
|
normalized_text = normalize(text)
|
||||||
|
|
||||||
|
# Short-circuit: if any member's full canonical name appears verbatim (whole words),
|
||||||
|
# return only those matches and skip all fuzzy/nickname checks. This prevents a
|
||||||
|
# nickname that is a substring of another member's surname from producing false hits.
|
||||||
|
exact_matches = []
|
||||||
|
for name in member_names:
|
||||||
|
variants = _build_name_variants(name)
|
||||||
|
full_name = variants[0] if variants else ""
|
||||||
|
if full_name and _word_in(full_name, normalized_text):
|
||||||
|
exact_matches.append((name, "auto"))
|
||||||
|
if exact_matches:
|
||||||
|
return exact_matches
|
||||||
|
|
||||||
matches = []
|
matches = []
|
||||||
|
|
||||||
for name in member_names:
|
for name in member_names:
|
||||||
@@ -70,17 +97,16 @@ def match_members(
|
|||||||
|
|
||||||
# 2. Both first and last name present (any order) = high confidence
|
# 2. Both first and last name present (any order) = high confidence
|
||||||
if len(parts) >= 2:
|
if len(parts) >= 2:
|
||||||
if parts[0] in normalized_text and parts[-1] in normalized_text:
|
if _word_in(parts[0], normalized_text) and _word_in(parts[-1], normalized_text):
|
||||||
matches.append((name, "auto"))
|
matches.append((name, "auto"))
|
||||||
continue
|
continue
|
||||||
|
|
||||||
# 3. Nickname + one part of the name = high confidence
|
# 3. Nickname present = high confidence
|
||||||
nickname = ""
|
nickname = ""
|
||||||
nickname_match = re.search(r"\(([^)]+)\)", name)
|
nickname_match = re.search(r"\(([^)]+)\)", name)
|
||||||
if nickname_match:
|
if nickname_match:
|
||||||
nickname = normalize(nickname_match.group(1))
|
nickname = normalize(nickname_match.group(1))
|
||||||
if nickname and nickname in normalized_text:
|
if nickname and _word_in(nickname, normalized_text):
|
||||||
# Nickname alone is often enough, but let's check if it's combined with a name part
|
|
||||||
matches.append((name, "auto"))
|
matches.append((name, "auto"))
|
||||||
continue
|
continue
|
||||||
|
|
||||||
@@ -90,18 +116,15 @@ def match_members(
|
|||||||
last_name = parts[-1]
|
last_name = parts[-1]
|
||||||
_COMMON_SURNAMES = {"novak", "novakova", "prach"}
|
_COMMON_SURNAMES = {"novak", "novakova", "prach"}
|
||||||
|
|
||||||
# Match last name
|
if len(last_name) >= 4 and last_name not in _COMMON_SURNAMES and _word_in(last_name, normalized_text):
|
||||||
if len(last_name) >= 4 and last_name not in _COMMON_SURNAMES and last_name in normalized_text:
|
|
||||||
matches.append((name, "review"))
|
matches.append((name, "review"))
|
||||||
continue
|
continue
|
||||||
|
|
||||||
# Match first name (if not too short)
|
if len(first_name) >= 3 and _word_in(first_name, normalized_text):
|
||||||
if len(first_name) >= 3 and first_name in normalized_text:
|
|
||||||
matches.append((name, "review"))
|
matches.append((name, "review"))
|
||||||
continue
|
continue
|
||||||
elif len(parts) == 1:
|
elif len(parts) == 1:
|
||||||
# Single name member
|
if len(parts[0]) >= 4 and _word_in(parts[0], normalized_text):
|
||||||
if len(parts[0]) >= 4 and parts[0] in normalized_text:
|
|
||||||
matches.append((name, "review"))
|
matches.append((name, "review"))
|
||||||
continue
|
continue
|
||||||
|
|
||||||
@@ -109,7 +132,6 @@ def match_members(
|
|||||||
# If we have any "auto" matches, discard all "review" matches
|
# If we have any "auto" matches, discard all "review" matches
|
||||||
auto_matches = [m for m in matches if m[1] == "auto"]
|
auto_matches = [m for m in matches if m[1] == "auto"]
|
||||||
if auto_matches:
|
if auto_matches:
|
||||||
# If multiple auto matches, keep them (ambiguous but high priority)
|
|
||||||
return auto_matches
|
return auto_matches
|
||||||
|
|
||||||
return matches
|
return matches
|
||||||
@@ -296,6 +318,12 @@ def reconcile(
|
|||||||
member_tiers = {name: tier for name, tier, _ in members}
|
member_tiers = {name: tier for name, tier, _ in members}
|
||||||
member_fees = {name: fees for name, _, fees in members}
|
member_fees = {name: fees for name, _, fees in members}
|
||||||
|
|
||||||
|
# Map canonical key → first attendance-sheet name with that key, so a
|
||||||
|
# `Person` cell that drifts in diacritics/case/whitespace still resolves.
|
||||||
|
canonical_by_key: dict[str, str] = {}
|
||||||
|
for name in member_names:
|
||||||
|
canonical_by_key.setdefault(canonical_member_key(name), name)
|
||||||
|
|
||||||
# Initialize ledger
|
# Initialize ledger
|
||||||
ledger: dict[str, dict[str, dict]] = {}
|
ledger: dict[str, dict[str, dict]] = {}
|
||||||
other_ledger: dict[str, list] = {}
|
other_ledger: dict[str, list] = {}
|
||||||
@@ -373,8 +401,9 @@ def reconcile(
|
|||||||
if is_other:
|
if is_other:
|
||||||
num_allocations = len(matched_members)
|
num_allocations = len(matched_members)
|
||||||
per_allocation = amount / num_allocations if num_allocations > 0 else 0
|
per_allocation = amount / num_allocations if num_allocations > 0 else 0
|
||||||
for member_name, confidence in matched_members:
|
for raw_member_name, confidence in matched_members:
|
||||||
if member_name in other_ledger:
|
member_name = canonical_by_key.get(canonical_member_key(raw_member_name))
|
||||||
|
if member_name is not None:
|
||||||
other_ledger[member_name].append({
|
other_ledger[member_name].append({
|
||||||
"amount": per_allocation,
|
"amount": per_allocation,
|
||||||
"date": tx["date"],
|
"date": tx["date"],
|
||||||
@@ -387,14 +416,20 @@ def reconcile(
|
|||||||
|
|
||||||
member_share = amount / len(matched_members) if matched_members else 0
|
member_share = amount / len(matched_members) if matched_members else 0
|
||||||
|
|
||||||
for member_name, confidence in matched_members:
|
for raw_member_name, confidence in matched_members:
|
||||||
if member_name not in ledger:
|
member_name = canonical_by_key.get(canonical_member_key(raw_member_name))
|
||||||
|
if member_name is None:
|
||||||
logger.warning(
|
logger.warning(
|
||||||
"Payment matched to unknown member %r (tx: %s, %s) — adding to unmatched",
|
"Payment matched to unknown member %r (tx: %s, %s) — adding to unmatched",
|
||||||
member_name, tx.get("date", "?"), tx.get("message", "?"),
|
raw_member_name, tx.get("date", "?"), tx.get("message", "?"),
|
||||||
)
|
)
|
||||||
unmatched.append(tx)
|
unmatched.append(tx)
|
||||||
continue
|
continue
|
||||||
|
if member_name != raw_member_name:
|
||||||
|
logger.info(
|
||||||
|
"Person cell %r resolved to canonical member %r — consider fixing the sheet",
|
||||||
|
raw_member_name, member_name,
|
||||||
|
)
|
||||||
|
|
||||||
in_window = [(m, ledger[member_name][m]["expected"]) for m in matched_months if m in ledger[member_name]]
|
in_window = [(m, ledger[member_name][m]["expected"]) for m in matched_months if m in ledger[member_name]]
|
||||||
out_of_window = [m for m in matched_months if m not in ledger[member_name]]
|
out_of_window = [m for m in matched_months if m not in ledger[member_name]]
|
||||||
|
|||||||
@@ -365,6 +365,19 @@
|
|||||||
border-bottom: 1px dashed #222;
|
border-bottom: 1px dashed #222;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
.raw-toggle {
|
||||||
|
color: #333;
|
||||||
|
font-size: 9px;
|
||||||
|
text-transform: lowercase;
|
||||||
|
margin-left: 8px;
|
||||||
|
text-decoration: none;
|
||||||
|
letter-spacing: 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
.raw-toggle:hover {
|
||||||
|
color: #666;
|
||||||
|
}
|
||||||
|
|
||||||
.modal-table {
|
.modal-table {
|
||||||
width: 100%;
|
width: 100%;
|
||||||
border-collapse: collapse;
|
border-collapse: collapse;
|
||||||
@@ -680,6 +693,16 @@
|
|||||||
<!-- Filled by JS -->
|
<!-- Filled by JS -->
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
|
<div class="modal-section">
|
||||||
|
<div class="modal-section-title">
|
||||||
|
Raw Payments
|
||||||
|
<a href="#" id="rawPaymentsToggle" class="raw-toggle" onclick="toggleRawPayments(event)">[show]</a>
|
||||||
|
</div>
|
||||||
|
<div id="modalRawList" class="tx-list" style="display: none;">
|
||||||
|
<!-- Filled by JS -->
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
@@ -696,6 +719,7 @@
|
|||||||
const memberData = {{ member_data| safe }};
|
const memberData = {{ member_data| safe }};
|
||||||
const sortedMonths = {{ raw_months| tojson }};
|
const sortedMonths = {{ raw_months| tojson }};
|
||||||
const monthLabels = {{ month_labels_json| safe }};
|
const monthLabels = {{ month_labels_json| safe }};
|
||||||
|
const rawPaymentsByPerson = {{ raw_payments_json| safe }};
|
||||||
let currentMemberName = null;
|
let currentMemberName = null;
|
||||||
|
|
||||||
function showMemberDetails(name) {
|
function showMemberDetails(name) {
|
||||||
@@ -828,9 +852,49 @@
|
|||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Raw payments (debug) — hidden by default, reset toggle on each open
|
||||||
|
const rawList = document.getElementById('modalRawList');
|
||||||
|
const rawToggle = document.getElementById('rawPaymentsToggle');
|
||||||
|
rawList.style.display = 'none';
|
||||||
|
rawToggle.textContent = '[show]';
|
||||||
|
rawList.innerHTML = '';
|
||||||
|
const rawRows = rawPaymentsByPerson[name] || [];
|
||||||
|
if (rawRows.length === 0) {
|
||||||
|
rawList.innerHTML = '<div style="color: #444; font-style: italic; padding: 10px 0;">No raw payments tied to this member.</div>';
|
||||||
|
} else {
|
||||||
|
rawRows.forEach(tx => {
|
||||||
|
const inferredNote = tx.inferred_amount && tx.inferred_amount !== '' && tx.inferred_amount != tx.amount
|
||||||
|
? ` <span style="color:#888;">(inferred: ${tx.inferred_amount})</span>`
|
||||||
|
: '';
|
||||||
|
const manualNote = tx.manual_fix ? ' <span style="color:#ffaa00;">[manual fix]</span>' : '';
|
||||||
|
const bankIdNote = tx.bank_id ? `<span style="color:#444;"> · bank_id: ${tx.bank_id}</span>` : '';
|
||||||
|
const item = document.createElement('div');
|
||||||
|
item.className = 'tx-item';
|
||||||
|
item.innerHTML = `
|
||||||
|
<div class="tx-meta">${tx.date} | purpose: ${tx.purpose || '—'}${manualNote}</div>
|
||||||
|
<div class="tx-main">
|
||||||
|
<span class="tx-amount">${tx.amount} CZK${inferredNote}</span>
|
||||||
|
<span class="tx-sender">${tx.sender || ''}</span>
|
||||||
|
</div>
|
||||||
|
<div class="tx-msg">${tx.message || ''}</div>
|
||||||
|
<div class="tx-meta">${tx.person || ''}${bankIdNote}</div>
|
||||||
|
`;
|
||||||
|
rawList.appendChild(item);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
document.getElementById('memberModal').classList.add('active');
|
document.getElementById('memberModal').classList.add('active');
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function toggleRawPayments(ev) {
|
||||||
|
ev.preventDefault();
|
||||||
|
const list = document.getElementById('modalRawList');
|
||||||
|
const link = document.getElementById('rawPaymentsToggle');
|
||||||
|
const hidden = list.style.display === 'none';
|
||||||
|
list.style.display = hidden ? 'block' : 'none';
|
||||||
|
link.textContent = hidden ? '[hide]' : '[show]';
|
||||||
|
}
|
||||||
|
|
||||||
function closeModal(id) {
|
function closeModal(id) {
|
||||||
if (id) {
|
if (id) {
|
||||||
document.getElementById(id).style.display = 'none';
|
document.getElementById(id).style.display = 'none';
|
||||||
|
|||||||
@@ -365,6 +365,19 @@
|
|||||||
border-bottom: 1px dashed #222;
|
border-bottom: 1px dashed #222;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
.raw-toggle {
|
||||||
|
color: #333;
|
||||||
|
font-size: 9px;
|
||||||
|
text-transform: lowercase;
|
||||||
|
margin-left: 8px;
|
||||||
|
text-decoration: none;
|
||||||
|
letter-spacing: 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
.raw-toggle:hover {
|
||||||
|
color: #666;
|
||||||
|
}
|
||||||
|
|
||||||
.modal-table {
|
.modal-table {
|
||||||
width: 100%;
|
width: 100%;
|
||||||
border-collapse: collapse;
|
border-collapse: collapse;
|
||||||
@@ -661,6 +674,16 @@
|
|||||||
<!-- Filled by JS -->
|
<!-- Filled by JS -->
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
|
<div class="modal-section">
|
||||||
|
<div class="modal-section-title">
|
||||||
|
Raw Payments
|
||||||
|
<a href="#" id="rawPaymentsToggle" class="raw-toggle" onclick="toggleRawPayments(event)">[show]</a>
|
||||||
|
</div>
|
||||||
|
<div id="modalRawList" class="tx-list" style="display: none;">
|
||||||
|
<!-- Filled by JS -->
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
@@ -677,6 +700,7 @@
|
|||||||
const memberData = {{ member_data| safe }};
|
const memberData = {{ member_data| safe }};
|
||||||
const sortedMonths = {{ raw_months| tojson }};
|
const sortedMonths = {{ raw_months| tojson }};
|
||||||
const monthLabels = {{ month_labels_json| safe }};
|
const monthLabels = {{ month_labels_json| safe }};
|
||||||
|
const rawPaymentsByPerson = {{ raw_payments_json| safe }};
|
||||||
let currentMemberName = null;
|
let currentMemberName = null;
|
||||||
|
|
||||||
function showMemberDetails(name) {
|
function showMemberDetails(name) {
|
||||||
@@ -809,9 +833,49 @@
|
|||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Raw payments (debug) — hidden by default, reset toggle on each open
|
||||||
|
const rawList = document.getElementById('modalRawList');
|
||||||
|
const rawToggle = document.getElementById('rawPaymentsToggle');
|
||||||
|
rawList.style.display = 'none';
|
||||||
|
rawToggle.textContent = '[show]';
|
||||||
|
rawList.innerHTML = '';
|
||||||
|
const rawRows = rawPaymentsByPerson[name] || [];
|
||||||
|
if (rawRows.length === 0) {
|
||||||
|
rawList.innerHTML = '<div style="color: #444; font-style: italic; padding: 10px 0;">No raw payments tied to this member.</div>';
|
||||||
|
} else {
|
||||||
|
rawRows.forEach(tx => {
|
||||||
|
const inferredNote = tx.inferred_amount && tx.inferred_amount !== '' && tx.inferred_amount != tx.amount
|
||||||
|
? ` <span style="color:#888;">(inferred: ${tx.inferred_amount})</span>`
|
||||||
|
: '';
|
||||||
|
const manualNote = tx.manual_fix ? ' <span style="color:#ffaa00;">[manual fix]</span>' : '';
|
||||||
|
const bankIdNote = tx.bank_id ? `<span style="color:#444;"> · bank_id: ${tx.bank_id}</span>` : '';
|
||||||
|
const item = document.createElement('div');
|
||||||
|
item.className = 'tx-item';
|
||||||
|
item.innerHTML = `
|
||||||
|
<div class="tx-meta">${tx.date} | purpose: ${tx.purpose || '—'}${manualNote}</div>
|
||||||
|
<div class="tx-main">
|
||||||
|
<span class="tx-amount">${tx.amount} CZK${inferredNote}</span>
|
||||||
|
<span class="tx-sender">${tx.sender || ''}</span>
|
||||||
|
</div>
|
||||||
|
<div class="tx-msg">${tx.message || ''}</div>
|
||||||
|
<div class="tx-meta">${tx.person || ''}${bankIdNote}</div>
|
||||||
|
`;
|
||||||
|
rawList.appendChild(item);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
document.getElementById('memberModal').classList.add('active');
|
document.getElementById('memberModal').classList.add('active');
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function toggleRawPayments(ev) {
|
||||||
|
ev.preventDefault();
|
||||||
|
const list = document.getElementById('modalRawList');
|
||||||
|
const link = document.getElementById('rawPaymentsToggle');
|
||||||
|
const hidden = list.style.display === 'none';
|
||||||
|
list.style.display = hidden ? 'block' : 'none';
|
||||||
|
link.textContent = hidden ? '[hide]' : '[show]';
|
||||||
|
}
|
||||||
|
|
||||||
function closeModal(id) {
|
function closeModal(id) {
|
||||||
if (id) {
|
if (id) {
|
||||||
document.getElementById(id).style.display = 'none';
|
document.getElementById(id).style.display = 'none';
|
||||||
|
|||||||
@@ -19,67 +19,6 @@ class TestWebApp(unittest.TestCase):
|
|||||||
self.assertEqual(response.status_code, 200)
|
self.assertEqual(response.status_code, 200)
|
||||||
self.assertIn(b'url=/adults', response.data)
|
self.assertIn(b'url=/adults', response.data)
|
||||||
|
|
||||||
@patch('app.get_cached_data', side_effect=_bypass_cache)
|
|
||||||
@patch('app.get_members_with_fees')
|
|
||||||
@patch('app.fetch_exceptions', return_value={})
|
|
||||||
def test_fees_route(self, mock_exceptions, mock_get_members, mock_cache):
|
|
||||||
"""Test that /fees returns 200 and renders the dashboard"""
|
|
||||||
mock_get_members.return_value = (
|
|
||||||
[('Test Member', 'A', {'2026-01': (750, 4)})],
|
|
||||||
['2026-01']
|
|
||||||
)
|
|
||||||
|
|
||||||
response = self.client.get('/fees')
|
|
||||||
self.assertEqual(response.status_code, 200)
|
|
||||||
self.assertIn(b'FUJ Fees Dashboard', response.data)
|
|
||||||
self.assertIn(b'Test Member', response.data)
|
|
||||||
|
|
||||||
@patch('app.get_cached_data', side_effect=_bypass_cache)
|
|
||||||
@patch('app.get_junior_members_with_fees')
|
|
||||||
@patch('app.fetch_exceptions', return_value={})
|
|
||||||
def test_fees_juniors_route(self, mock_exceptions, mock_get_junior_members, mock_cache):
|
|
||||||
"""Test that /fees-juniors returns 200 and renders the junior dashboard"""
|
|
||||||
mock_get_junior_members.return_value = (
|
|
||||||
[
|
|
||||||
('Test Junior 1', 'J', {'2026-01': ('?', 1, 0, 1)}),
|
|
||||||
('Test Junior 2', 'J', {'2026-01': (500, 4, 1, 3)})
|
|
||||||
],
|
|
||||||
['2026-01']
|
|
||||||
)
|
|
||||||
|
|
||||||
response = self.client.get('/fees-juniors')
|
|
||||||
self.assertEqual(response.status_code, 200)
|
|
||||||
self.assertIn(b'FUJ Junior Fees Dashboard', response.data)
|
|
||||||
self.assertIn(b'Test Junior 1', response.data)
|
|
||||||
self.assertIn(b'? / 1 (J)', response.data)
|
|
||||||
self.assertIn(b'500 CZK / 4 (1A+3J)', response.data)
|
|
||||||
|
|
||||||
@patch('app.get_cached_data', side_effect=_bypass_cache)
|
|
||||||
@patch('app.fetch_sheet_data')
|
|
||||||
@patch('app.fetch_exceptions', return_value={})
|
|
||||||
@patch('app.get_members_with_fees')
|
|
||||||
def test_reconcile_route(self, mock_get_members, mock_exceptions, mock_fetch_sheet, mock_cache):
|
|
||||||
"""Test that /reconcile returns 200 and shows matches"""
|
|
||||||
mock_get_members.return_value = (
|
|
||||||
[('Test Member', 'A', {'2026-01': (750, 4)})],
|
|
||||||
['2026-01']
|
|
||||||
)
|
|
||||||
mock_fetch_sheet.return_value = [{
|
|
||||||
'date': '2026-01-01',
|
|
||||||
'amount': 750,
|
|
||||||
'person': 'Test Member',
|
|
||||||
'purpose': '2026-01',
|
|
||||||
'message': 'test payment',
|
|
||||||
'sender': 'External Bank User',
|
|
||||||
'inferred_amount': 750
|
|
||||||
}]
|
|
||||||
|
|
||||||
response = self.client.get('/reconcile')
|
|
||||||
self.assertEqual(response.status_code, 200)
|
|
||||||
self.assertIn(b'Payment Reconciliation', response.data)
|
|
||||||
self.assertIn(b'Test Member', response.data)
|
|
||||||
self.assertIn(b'OK', response.data)
|
|
||||||
|
|
||||||
@patch('app.get_cached_data', side_effect=_bypass_cache)
|
@patch('app.get_cached_data', side_effect=_bypass_cache)
|
||||||
@patch('app.fetch_sheet_data')
|
@patch('app.fetch_sheet_data')
|
||||||
def test_payments_route(self, mock_fetch_sheet, mock_cache):
|
def test_payments_route(self, mock_fetch_sheet, mock_cache):
|
||||||
@@ -98,38 +37,6 @@ class TestWebApp(unittest.TestCase):
|
|||||||
self.assertIn(b'Test Member', response.data)
|
self.assertIn(b'Test Member', response.data)
|
||||||
self.assertIn(b'Direct Member Payment', response.data)
|
self.assertIn(b'Direct Member Payment', response.data)
|
||||||
|
|
||||||
@patch('app.get_cached_data', side_effect=_bypass_cache)
|
|
||||||
@patch('app.fetch_sheet_data')
|
|
||||||
@patch('app.fetch_exceptions')
|
|
||||||
@patch('app.get_junior_members_with_fees')
|
|
||||||
def test_reconcile_juniors_route(self, mock_get_junior, mock_exceptions, mock_transactions, mock_cache):
|
|
||||||
"""Test that /reconcile-juniors correctly computes balances for juniors."""
|
|
||||||
mock_get_junior.return_value = (
|
|
||||||
[
|
|
||||||
('Junior One', 'J', {'2026-01': (500, 4, 2, 2)}),
|
|
||||||
('Junior Two', 'X', {'2026-01': ('?', 1, 0, 1)})
|
|
||||||
],
|
|
||||||
['2026-01']
|
|
||||||
)
|
|
||||||
mock_exceptions.return_value = {}
|
|
||||||
mock_transactions.return_value = [{
|
|
||||||
'date': '2026-01-15',
|
|
||||||
'amount': 500,
|
|
||||||
'person': 'Junior One',
|
|
||||||
'purpose': '2026-01',
|
|
||||||
'message': '',
|
|
||||||
'sender': 'Parent',
|
|
||||||
'inferred_amount': 500
|
|
||||||
}]
|
|
||||||
|
|
||||||
response = self.client.get('/reconcile-juniors')
|
|
||||||
self.assertEqual(response.status_code, 200)
|
|
||||||
self.assertIn(b'Junior Payment Reconciliation', response.data)
|
|
||||||
self.assertIn(b'Junior One', response.data)
|
|
||||||
self.assertIn(b'Junior Two', response.data)
|
|
||||||
self.assertIn(b'OK', response.data)
|
|
||||||
self.assertIn(b'?', response.data)
|
|
||||||
|
|
||||||
@patch('app.get_cached_data', side_effect=_bypass_cache)
|
@patch('app.get_cached_data', side_effect=_bypass_cache)
|
||||||
@patch('app.fetch_sheet_data')
|
@patch('app.fetch_sheet_data')
|
||||||
@patch('app.fetch_exceptions', return_value={})
|
@patch('app.fetch_exceptions', return_value={})
|
||||||
|
|||||||
53
tests/test_match_members.py
Normal file
53
tests/test_match_members.py
Normal file
@@ -0,0 +1,53 @@
|
|||||||
|
import unittest
|
||||||
|
from scripts.match_payments import match_members
|
||||||
|
|
||||||
|
|
||||||
|
MEMBERS = [
|
||||||
|
"Henrietta Ottová",
|
||||||
|
"Tomáš Němeček (Tov)",
|
||||||
|
"František Vrbík (Štrúdl)",
|
||||||
|
"Jana Nováková",
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
class TestMatchMembersExact(unittest.TestCase):
|
||||||
|
def test_full_name_in_message_returns_only_that_member(self):
|
||||||
|
# "tov" is a substring of "ottova" — the old code returned both members
|
||||||
|
result = match_members("Henrietta Ottová (Heny): 04/2026", MEMBERS)
|
||||||
|
names = [r[0] for r in result]
|
||||||
|
self.assertEqual(names, ["Henrietta Ottová"])
|
||||||
|
self.assertTrue(all(conf == "auto" for _, conf in result))
|
||||||
|
|
||||||
|
def test_nickname_tov_not_matched_inside_ottova(self):
|
||||||
|
# Bare nickname message should NOT match Tomáš via "tov" inside "ottova"
|
||||||
|
result = match_members("platba ottova 04/2026", MEMBERS)
|
||||||
|
names = [r[0] for r in result]
|
||||||
|
self.assertNotIn("Tomáš Němeček (Tov)", names)
|
||||||
|
|
||||||
|
def test_combined_payment_two_full_names(self):
|
||||||
|
result = match_members("Henrietta Ottová a Tomáš Němeček 04/2026", MEMBERS)
|
||||||
|
names = [r[0] for r in result]
|
||||||
|
self.assertIn("Henrietta Ottová", names)
|
||||||
|
self.assertIn("Tomáš Němeček (Tov)", names)
|
||||||
|
self.assertTrue(all(conf == "auto" for _, conf in result))
|
||||||
|
|
||||||
|
def test_nickname_alone_still_matches_correctly(self):
|
||||||
|
# "Tov" alone should still match Tomáš (as long as "ottova" is not in the text)
|
||||||
|
result = match_members("Tov platba 04/2026", MEMBERS)
|
||||||
|
names = [r[0] for r in result]
|
||||||
|
self.assertIn("Tomáš Němeček (Tov)", names)
|
||||||
|
|
||||||
|
def test_full_name_no_diacritics_still_matches(self):
|
||||||
|
result = match_members("Henrietta Ottova 04/2026", MEMBERS)
|
||||||
|
names = [r[0] for r in result]
|
||||||
|
self.assertIn("Henrietta Ottová", names)
|
||||||
|
self.assertNotIn("Tomáš Němeček (Tov)", names)
|
||||||
|
|
||||||
|
def test_first_last_name_present_any_order(self):
|
||||||
|
result = match_members("Platba od Nemeček Tomas 04/2026", MEMBERS)
|
||||||
|
names = [r[0] for r in result]
|
||||||
|
self.assertIn("Tomáš Němeček (Tov)", names)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
69
tests/test_match_payments.py
Normal file
69
tests/test_match_payments.py
Normal file
@@ -0,0 +1,69 @@
|
|||||||
|
import unittest
|
||||||
|
|
||||||
|
from scripts.match_payments import canonical_member_key, reconcile
|
||||||
|
|
||||||
|
|
||||||
|
class TestCanonicalMemberKey(unittest.TestCase):
|
||||||
|
def test_diacritics_and_case_collapse(self):
|
||||||
|
self.assertEqual(canonical_member_key("Mária Maco"), "maria maco")
|
||||||
|
self.assertEqual(canonical_member_key("MARIA MACO"), "maria maco")
|
||||||
|
self.assertEqual(canonical_member_key("maria maco"), "maria maco")
|
||||||
|
|
||||||
|
def test_whitespace_runs_collapse(self):
|
||||||
|
self.assertEqual(canonical_member_key("Mária Maco"), "maria maco")
|
||||||
|
self.assertEqual(canonical_member_key(" Mária Maco "), "maria maco")
|
||||||
|
|
||||||
|
def test_unknown_name_passes_through_normalized(self):
|
||||||
|
# Two genuinely different names must not collide.
|
||||||
|
self.assertNotEqual(
|
||||||
|
canonical_member_key("Mária Maco"),
|
||||||
|
canonical_member_key("Marek Maco"),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class TestReconcileTolerantPersonMatching(unittest.TestCase):
|
||||||
|
def _members(self):
|
||||||
|
return [("Mária Maco", "A", {"2026-04": (750, 4)})]
|
||||||
|
|
||||||
|
def _tx(self, person):
|
||||||
|
return {
|
||||||
|
"date": "2026-04-15",
|
||||||
|
"amount": 750,
|
||||||
|
"person": person,
|
||||||
|
"purpose": "2026-04",
|
||||||
|
"inferred_amount": 750,
|
||||||
|
"sender": "Maco Family",
|
||||||
|
"message": "fee",
|
||||||
|
}
|
||||||
|
|
||||||
|
def test_person_without_diacritics_matches(self):
|
||||||
|
result = reconcile(self._members(), ["2026-04"], [self._tx("Maria Maco")], {})
|
||||||
|
|
||||||
|
member = result["members"]["Mária Maco"]
|
||||||
|
self.assertEqual(member["months"]["2026-04"]["paid"], 750)
|
||||||
|
self.assertEqual(len(member["months"]["2026-04"]["transactions"]), 1)
|
||||||
|
self.assertEqual(result["unmatched"], [])
|
||||||
|
|
||||||
|
def test_person_with_extra_whitespace_matches(self):
|
||||||
|
result = reconcile(self._members(), ["2026-04"], [self._tx("Mária Maco")], {})
|
||||||
|
|
||||||
|
self.assertEqual(result["members"]["Mária Maco"]["months"]["2026-04"]["paid"], 750)
|
||||||
|
self.assertEqual(result["unmatched"], [])
|
||||||
|
|
||||||
|
def test_person_lowercase_matches(self):
|
||||||
|
result = reconcile(self._members(), ["2026-04"], [self._tx("mária maco")], {})
|
||||||
|
|
||||||
|
self.assertEqual(result["members"]["Mária Maco"]["months"]["2026-04"]["paid"], 750)
|
||||||
|
self.assertEqual(result["unmatched"], [])
|
||||||
|
|
||||||
|
def test_truly_unknown_person_still_unmatched(self):
|
||||||
|
result = reconcile(
|
||||||
|
self._members(), ["2026-04"], [self._tx("Někdo Neznámý")], {}
|
||||||
|
)
|
||||||
|
|
||||||
|
self.assertEqual(result["members"]["Mária Maco"]["months"]["2026-04"]["paid"], 0)
|
||||||
|
self.assertEqual(len(result["unmatched"]), 1)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
Reference in New Issue
Block a user