fuj-management/docs/plans/2026-05-07-2114-go-rewrite-m5-3-python-api-shadow.md

M5.3 — Python /api/X shadow endpoints

Companion to:

• 2026-05-03-2349-go-backend-rewrite.md (master design)
• 2026-05-03-2349-go-backend-rewrite-progress.md (M5.3 row)
• 2026-05-07-1431-m5-json-api-parity.md (Python view-model extraction prep)
• 2026-05-07-1650-go-rewrite-m5-1-api-structs-schemas.md (Go wire types)

Context

M5.1 (Go wire types + JSON Schemas) and M5.2 (Go HTTP handlers for /api/adults /api/juniors /api/payments /api/version) have merged. M5.3 mirrors the same four endpoints on the Python Flask side so M5.4's cmd/parity tool can hit both backends and diff the JSON. After M5.3, every byte the Go side emits has a Python counterpart to compare against.

The Python view-model builders (scripts/views.py) already produce dicts very close to the wire shape — except three template-only fields (member_data, month_labels_json, raw_payments_json) are pre-json.dumps'd for inline <script> blocks. M5.1's plan called this out explicitly: M5.3's /api/X is jsonify(unwrap_json_strings(view_model_dict)) — a 4-line shim, not real transformation logic.

Approach

Add four shadow routes to app.py and one private unwrap helper. Builders and templates are untouched.

Decisions

1. Unwrap shim lives in app.py, not scripts/views.py. It's 4 lines, only the API routes use it, and it's parity-only scaffolding that M8 will delete. Keeping views.py free of HTTP-layer concerns means cleaner deletion later.
2. No data-loading helper extraction. Each shadow route duplicates ~8 lines of cache loads from its sibling HTML route. A helper would have to thread attendance_url / payments_url / bank_account and the adults-vs-juniors-vs-payments branching back out — net negative for code that M8 will erase wholesale.
3. Drop record_step calls in API routes. record_step only feeds inject_render_time (a Jinja context_processor); JSON responses don't go through templates, so timing breakdown has no consumer.
4. /api/version is a one-liner. BUILD_META already has the keys (tag, commit, build_date) Go emits. Just jsonify(BUILD_META). Existing /version route stays as-is — the new endpoint sits alongside.
5. Tests assert key sets and unwrap, not values. Hard-code EXPECTED_ADULTS_KEYS etc. as module constants in tests/test_app.py. Catches drift in unit tests rather than waiting for M5.4 parity diffs. Type-check the unwrapped fields (isinstance(member_data, dict) etc.) to prove the shim ran.

The shim

def _unwrap_view_model_for_api(vm: dict) -> dict:
    out = dict(vm)
    out["member_data"] = _json.loads(out.pop("member_data"))
    out["month_labels"] = _json.loads(out.pop("month_labels_json"))
    out["raw_payments"] = _json.loads(out.pop("raw_payments_json"))
    return out

Note the rename: month_labels_json → month_labels, raw_payments_json → raw_payments (matches Go contract per adults.go JSON tags).

Route skeleton (/api/adults)

@app.route("/api/adults")
def api_adults():
    attendance_url = f"https://docs.google.com/spreadsheets/d/{ATTENDANCE_SHEET_ID}/edit"
    payments_url = f"https://docs.google.com/spreadsheets/d/{PAYMENTS_SHEET_ID}/edit"
    members_data = get_cached_data("attendance_regular", ATTENDANCE_SHEET_ID, get_members_with_fees)
    if not members_data:
        return jsonify({"error": "no data"}), 503
    members, sorted_months = members_data
    transactions = get_cached_data("payments_transactions", PAYMENTS_SHEET_ID, fetch_sheet_data, PAYMENTS_SHEET_ID, CREDENTIALS_PATH)
    exceptions = get_cached_data(
        "exceptions_dict", PAYMENTS_SHEET_ID, fetch_exceptions,
        PAYMENTS_SHEET_ID, CREDENTIALS_PATH,
        serialize=lambda d: [[list(k), v] for k, v in d.items()],
        deserialize=lambda c: {tuple(k): v for k, v in c},
    )
    result = reconcile(members, sorted_months, transactions, exceptions)
    vm = build_adults_view_model(
        members, sorted_months, result, transactions,
        datetime.now().strftime("%Y-%m"),
        attendance_url=attendance_url, payments_url=payments_url, bank_account=BANK_ACCOUNT,
    )
    return jsonify(_unwrap_view_model_for_api(vm))

/api/juniors mirrors with adapt_junior_members + JUNIOR_SHEET_GID + build_juniors_view_model. /api/payments skips the unwrap (its builder has no JSON-string fields): return jsonify(vm). /api/version: return jsonify(BUILD_META).

Files to modify

• app.py
  • L10: add jsonify to from flask import ....
  • Add _unwrap_view_model_for_api near BUILD_META (it already imports json as _json).
  • Add four routes: /api/version, /api/adults, /api/juniors, /api/payments. Place them after /version for grouping.
• tests/test_app.py
  • Add EXPECTED_ADULTS_KEYS, EXPECTED_JUNIORS_KEYS, EXPECTED_PAYMENTS_KEYS, EXPECTED_VERSION_KEYS module constants (sourced from adults.go / juniors.go / payments.go / version.go JSON tags).
  • Four new test functions: test_api_adults, test_api_juniors, test_api_payments, test_api_version. Reuse the existing _bypass_cache patcher and the fetch_sheet_data / fetch_exceptions / get_members_with_fees / get_junior_members_with_fees mocks already in the file.
  • Each adults/juniors test asserts: 200, response.is_json, set(json.keys()) == EXPECTED_*_KEYS, isinstance(json["member_data"], dict), isinstance(json["month_labels"], dict), isinstance(json["raw_payments"], dict).
• CHANGELOG.md: post-merge entry per CLAUDE.md format.
• docs/plans/2026-05-03-2349-go-backend-rewrite-progress.md:102: tick M5.3 with merge SHA.

Reusable existing code

• build_adults_view_model, build_juniors_view_model, build_payments_view_model — call as-is, no changes.
• reconcile and adapt_junior_members — same.
• get_cached_data, fetch_sheet_data, fetch_exceptions, get_members_with_fees, get_junior_members_with_fees — same call sites as the HTML routes.
• BUILD_META — already shaped to match Go's VersionResponse.

Verification

1. make test — all existing tests still pass; four new test_api_* tests pass.
2. python -c "import app" — catches the missing jsonify import.
3. make web-py, then:
   • curl -s localhost:5001/api/version | jq . → {tag, commit, build_date}.
   • curl -s localhost:5001/api/adults | jq 'keys' → 14 keys, no _json suffix anywhere.
   • curl -s localhost:5001/api/adults | jq '.member_data | type, .month_labels | type, .raw_payments | type' → all "object" (proves unwrap).
   • Same checks on /api/juniors and /api/payments.
4. Visit /adults, /juniors, /payments in browser — HTML still renders identically (regression check on builders).
5. Optional pre-M5.4 peek: make web-go on :8080 + make web-py on :5001, then diff <(curl -s :5001/api/adults | jq -S .) <(curl -s :8080/api/adults | jq -S .). Expect non-zero diff (raw transaction key shape, see below) — that is fine; M5.4 surfaces and resolves these.

Out of scope (M5.4 will surface and resolve)

These are known parity friction points; don't fix in M5.3 — the whole point of M5.4 is to enumerate them:

• raw_payments[name][i] row shape: Python emits raw Google Sheets row dicts with column-header keys (e.g. "VS", "Sync ID"); Go's RawTransaction uses snake_case (vs, sync_id). Keys and types will diverge.
• unmatched[]: same divergence (same raw row dicts).
• null vs missing keys: Python omits keys never set; Go zero-value structs may emit null/"" depending on omitempty.
• Decimal / float precision in grouped_payments amounts.
• Field insertion order: jsonify preserves insertion order (Python 3.7+); Go marshals struct fields in declaration order. Likely fine, parity tool will tell.

Branch + MR

Per CLAUDE.md: branch feat/go-m5-3-python-api-shadow, push with -u, open MR via tea pr create --title ... --description ... --base main --head feat/go-m5-3-python-api-shadow. Do not merge from CLI.