Compare commits
3 Commits
7d51f9ca77
...
0.17
| Author | SHA1 | Date | |
|---|---|---|---|
| dca0c6c933 | |||
| 9b99f6d33b | |||
| e83d6af1f5 |
17
app.py
17
app.py
@@ -55,7 +55,24 @@ def get_month_labels(sorted_months, merged_months):
|
|||||||
labels[m] = dt.strftime("%b %Y")
|
labels[m] = dt.strftime("%b %Y")
|
||||||
return labels
|
return labels
|
||||||
|
|
||||||
|
def warmup_cache():
|
||||||
|
"""Pre-fetch all cached data so first request is fast."""
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
logger.info("Warming up cache...")
|
||||||
|
credentials_path = CREDENTIALS_PATH
|
||||||
|
|
||||||
|
get_cached_data("attendance_regular", ATTENDANCE_SHEET_ID, get_members_with_fees)
|
||||||
|
get_cached_data("attendance_juniors", ATTENDANCE_SHEET_ID, get_junior_members_with_fees)
|
||||||
|
get_cached_data("payments_transactions", PAYMENTS_SHEET_ID, fetch_sheet_data, PAYMENTS_SHEET_ID, credentials_path)
|
||||||
|
get_cached_data("exceptions_dict", PAYMENTS_SHEET_ID, fetch_exceptions,
|
||||||
|
PAYMENTS_SHEET_ID, credentials_path,
|
||||||
|
serialize=lambda d: [[list(k), v] for k, v in d.items()],
|
||||||
|
deserialize=lambda c: {tuple(k): v for k, v in c},
|
||||||
|
)
|
||||||
|
logger.info("Cache warmup complete.")
|
||||||
|
|
||||||
app = Flask(__name__)
|
app = Flask(__name__)
|
||||||
|
warmup_cache()
|
||||||
|
|
||||||
@app.before_request
|
@app.before_request
|
||||||
def start_timer():
|
def start_timer():
|
||||||
|
|||||||
15
docs/README.md
Normal file
15
docs/README.md
Normal file
@@ -0,0 +1,15 @@
|
|||||||
|
# FUJ Management Documentation
|
||||||
|
|
||||||
|
Welcome to the documentation for the FUJ Management application.
|
||||||
|
|
||||||
|
This project automates financial and operational management for the FUJ (Frisbee Ultimate Jablonec) club.
|
||||||
|
|
||||||
|
## Navigation
|
||||||
|
|
||||||
|
Use the sidebar to explore the documentation:
|
||||||
|
|
||||||
|
* **[Project Notes](project-notes.md)**: Main brainstorming and domain model.
|
||||||
|
* **[Scripts](scripts.md)**: Details about available CLI tools.
|
||||||
|
* **[Fee Specification](fee-calculation-spec.md)**: Rules for fee calculation.
|
||||||
|
|
||||||
|
For more technical details, check out the guides by Claude and Gemini in the sidebar.
|
||||||
25
docs/_sidebar.md
Normal file
25
docs/_sidebar.md
Normal file
@@ -0,0 +1,25 @@
|
|||||||
|
* [Home](README.md)
|
||||||
|
* [Project Notes](project-notes.md)
|
||||||
|
* [Scripts](scripts.md)
|
||||||
|
* [Fee Spec](fee-calculation-spec.md)
|
||||||
|
|
||||||
|
* **By Claude Opus**
|
||||||
|
* [README](by-claude-opus/README.md)
|
||||||
|
* [User Guide](by-claude-opus/user-guide.md)
|
||||||
|
* [Web App](by-claude-opus/web-app.md)
|
||||||
|
* [Deployment](by-claude-opus/deployment.md)
|
||||||
|
* [Architecture](by-claude-opus/architecture.md)
|
||||||
|
* [Data Model](by-claude-opus/data-model.md)
|
||||||
|
* [Development](by-claude-opus/development.md)
|
||||||
|
* [Scripts](by-claude-opus/scripts.md)
|
||||||
|
* [Testing](by-claude-opus/testing.md)
|
||||||
|
|
||||||
|
* **By Gemini**
|
||||||
|
* [README](by-gemini/README.md)
|
||||||
|
* [User Guide](by-gemini/user-guide.md)
|
||||||
|
* [Architecture](by-gemini/architecture.md)
|
||||||
|
* [Deployment](by-gemini/deployment.md)
|
||||||
|
* [Scripts](by-gemini/scripts.md)
|
||||||
|
|
||||||
|
* **Specs**
|
||||||
|
* [Fio Sync](spec/fio_to_sheets_sync.md)
|
||||||
214
docs/by-claude-opus/README.md
Normal file
214
docs/by-claude-opus/README.md
Normal file
@@ -0,0 +1,214 @@
|
|||||||
|
# FUJ Management — Comprehensive Documentation
|
||||||
|
|
||||||
|
> **FUJ = Frisbee Ultimate Jablonec** — a small sports club in the Czech Republic.
|
||||||
|
|
||||||
|
## What Is This Project?
|
||||||
|
|
||||||
|
FUJ Management is a purpose-built financial management system for a small ultimate frisbee club. It automates the tedious process of tracking **who attended practice**, **how much they owe**, **who has paid**, and **who still owes money** — a workflow that would otherwise require manual cross-referencing between attendance spreadsheets and bank statements.
|
||||||
|
|
||||||
|
The system is built around two Google Sheets (one for attendance, one for payments) and a Fio bank transparent account. A set of Python scripts sync and process the data, while a Flask-based web dashboard provides real-time visibility into fees, payments, and reconciliation status.
|
||||||
|
|
||||||
|
### The Problem It Solves
|
||||||
|
|
||||||
|
Before this system, the club treasurer had to:
|
||||||
|
|
||||||
|
1. **Manually count** attendance marks for each member each month
|
||||||
|
2. **Calculate** whether each person owes 0, 200, or 750 CZK based on how many times they showed up
|
||||||
|
3. **Cross-reference** bank statements to figure out who paid and for which month
|
||||||
|
4. **Chase** members who hadn't paid, often losing track of partial payments and advance payments
|
||||||
|
5. **Handle edge cases** like members paying for multiple months at once, using nicknames in payment messages, or paying via a family member's account
|
||||||
|
|
||||||
|
This system automates steps 1–4 entirely, and provides tooling for step 5.
|
||||||
|
|
||||||
|
## System Overview
|
||||||
|
|
||||||
|
```
|
||||||
|
┌──────────────────────────┐ ┌──────────────────────────┐
|
||||||
|
│ Attendance Sheet │ │ Fio Bank Account │
|
||||||
|
│ (Google Sheets) │ │ (transparent account) │
|
||||||
|
│ │ │ │
|
||||||
|
│ Members × Dates × ✓/✗ │ │ Incoming payments with │
|
||||||
|
│ Tier (A/J/X) │ │ sender, amount, message │
|
||||||
|
└──────────┬───────────────┘ └──────────┬───────────────┘
|
||||||
|
│ │
|
||||||
|
│ CSV export │ API / HTML scraping
|
||||||
|
│ │
|
||||||
|
▼ ▼
|
||||||
|
┌─────────────────┐ ┌───────────────────────┐
|
||||||
|
│ attendance.py │ │ sync_fio_to_sheets.py │
|
||||||
|
│ │ │ │
|
||||||
|
│ Fetches sheet, │ │ Syncs bank txns to │
|
||||||
|
│ computes fees │ │ Payments Google Sheet │
|
||||||
|
└────────┬────────┘ └───────────┬────────────┘
|
||||||
|
│ │
|
||||||
|
│ ▼
|
||||||
|
│ ┌───────────────────────┐
|
||||||
|
│ │ Payments Sheet │
|
||||||
|
│ │ (Google Sheets) │
|
||||||
|
│ │ │
|
||||||
|
│ │ Date|Amount|Person| │
|
||||||
|
│ │ Purpose|Sender|etc. │
|
||||||
|
│ └───────────┬────────────┘
|
||||||
|
│ │
|
||||||
|
│ ┌─────────────────────────┤
|
||||||
|
│ │ │
|
||||||
|
│ ▼ ▼
|
||||||
|
│ ┌──────────────┐ ┌──────────────────┐
|
||||||
|
│ │infer_payments│ │ match_payments.py │
|
||||||
|
│ │ .py │ │ │
|
||||||
|
│ │ │ │ Reconciliation │
|
||||||
|
│ │ Auto-fills │ │ engine: matches │
|
||||||
|
│ │ Person, │ │ payments against │
|
||||||
|
│ │ Purpose, │ │ expected fees │
|
||||||
|
│ │ Amount │ └────────┬──────────┘
|
||||||
|
│ └──────────────┘ │
|
||||||
|
│ │
|
||||||
|
└────────────────┬───────────────┘
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
┌──────────────────────┐
|
||||||
|
│ Flask Web App │
|
||||||
|
│ (app.py) │
|
||||||
|
│ │
|
||||||
|
│ /fees – fee │
|
||||||
|
│ table │
|
||||||
|
│ /reconcile – balance │
|
||||||
|
│ matrix │
|
||||||
|
│ /payments – ledger │
|
||||||
|
│ /qr – QR code │
|
||||||
|
└───────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
## Quick Start
|
||||||
|
|
||||||
|
### Prerequisites
|
||||||
|
|
||||||
|
- **Python 3.13+**
|
||||||
|
- **[uv](https://docs.astral.sh/uv/)** — fast Python package manager
|
||||||
|
- **Google Sheets API credentials** — a service account JSON file placed at `.secret/fuj-management-bot-credentials.json`
|
||||||
|
- *Optional*: `FIO_API_TOKEN` environment variable for Fio REST API access (falls back to transparent page scraping)
|
||||||
|
|
||||||
|
### Setup
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Clone and install dependencies
|
||||||
|
git clone <repo-url>
|
||||||
|
cd fuj-management
|
||||||
|
uv sync # Installs all dependencies from pyproject.toml
|
||||||
|
|
||||||
|
# Place your Google API credentials
|
||||||
|
mkdir -p .secret
|
||||||
|
cp /path/to/your/credentials.json .secret/fuj-management-bot-credentials.json
|
||||||
|
```
|
||||||
|
|
||||||
|
### Common Operations
|
||||||
|
|
||||||
|
| Command | Purpose |
|
||||||
|
|---------|---------|
|
||||||
|
| `make web` | Start the web dashboard at `http://localhost:5001` |
|
||||||
|
| `make sync` | Pull new bank transactions into the Google Sheet |
|
||||||
|
| `make infer` | Auto-fill Person/Purpose/Amount for new transactions |
|
||||||
|
| `make reconcile` | Print a CLI balance report |
|
||||||
|
| `make fees` | Print fee calculation table from attendance |
|
||||||
|
| `make test` | Run the test suite |
|
||||||
|
| `make image` | Build the Docker container image |
|
||||||
|
|
||||||
|
### Typical Workflow
|
||||||
|
|
||||||
|
```
|
||||||
|
make sync → make infer → (manual review in Google Sheets) → make web
|
||||||
|
↓ ↓ ↓ ↓
|
||||||
|
Pull new bank Auto-match Fix any [?] View live
|
||||||
|
transactions payments to flagged rows dashboard
|
||||||
|
into sheet members/months in the sheet
|
||||||
|
```
|
||||||
|
|
||||||
|
## Documentation Index
|
||||||
|
|
||||||
|
| Document | Contents |
|
||||||
|
|----------|----------|
|
||||||
|
| [Architecture](architecture.md) | System design, data flow diagrams, module dependency graph |
|
||||||
|
| [Web Application](web-app.md) | Flask app architecture, routes, templates, interactive features |
|
||||||
|
| [User Guide](user-guide.md) | End-user guide for the web dashboard — what each page shows |
|
||||||
|
| [Scripts Reference](scripts.md) | Detailed reference for all CLI scripts and shared modules |
|
||||||
|
| [Data Model](data-model.md) | Google Sheets schemas, fee calculation rules, bank integration |
|
||||||
|
| [Deployment](deployment.md) | Docker containerization, Gitea CI/CD, Kubernetes deployment |
|
||||||
|
| [Testing](testing.md) | Test infrastructure, coverage, how to write new tests |
|
||||||
|
| [Development Guide](development.md) | Local setup, coding conventions, tooling, project history |
|
||||||
|
|
||||||
|
## Technology Stack
|
||||||
|
|
||||||
|
| Layer | Technology |
|
||||||
|
|-------|-----------|
|
||||||
|
| Language | Python 3.13+ |
|
||||||
|
| Web framework | Flask 3.1 |
|
||||||
|
| Package management | uv + pyproject.toml |
|
||||||
|
| Data sources | Google Sheets API, Fio Bank API / HTML scraping |
|
||||||
|
| QR codes | `qrcode` library (PIL backend) |
|
||||||
|
| Containerization | Docker (Alpine-based) |
|
||||||
|
| CI/CD | Gitea Actions |
|
||||||
|
| Deployment target | Self-hosted Kubernetes |
|
||||||
|
| Frontend | Server-rendered HTML/CSS/JS (terminal-aesthetic theme) |
|
||||||
|
|
||||||
|
## Project Structure
|
||||||
|
|
||||||
|
```
|
||||||
|
fuj-management/
|
||||||
|
├── app.py # Flask web application (4 routes)
|
||||||
|
├── Makefile # Build automation (13 targets)
|
||||||
|
├── pyproject.toml # Python dependencies and metadata
|
||||||
|
│
|
||||||
|
├── scripts/
|
||||||
|
│ ├── attendance.py # Shared: attendance data + fee calculation
|
||||||
|
│ ├── calculate_fees.py # CLI: print fee table
|
||||||
|
│ ├── match_payments.py # Core: reconciliation engine + CLI report
|
||||||
|
│ ├── infer_payments.py # Auto-fill Person/Purpose in Google Sheet
|
||||||
|
│ ├── sync_fio_to_sheets.py # Sync Fio bank → Google Sheet
|
||||||
|
│ ├── fio_utils.py # Shared: Fio bank data fetching
|
||||||
|
│ └── czech_utils.py # Shared: diacritics normalization + Czech month parsing
|
||||||
|
│
|
||||||
|
├── templates/
|
||||||
|
│ ├── fees.html # Attendance/fees dashboard
|
||||||
|
│ ├── reconcile.html # Payment reconciliation with modals + QR
|
||||||
|
│ └── payments.html # Payments ledger grouped by member
|
||||||
|
│
|
||||||
|
├── tests/
|
||||||
|
│ ├── test_app.py # Flask route tests (mocked data)
|
||||||
|
│ └── test_reconcile_exceptions.py # Reconciliation with fee exceptions
|
||||||
|
│
|
||||||
|
├── build/
|
||||||
|
│ ├── Dockerfile # Alpine-based container image
|
||||||
|
│ └── entrypoint.sh # Container entry point
|
||||||
|
│
|
||||||
|
├── .gitea/workflows/
|
||||||
|
│ ├── build.yaml # CI: build + push Docker image
|
||||||
|
│ └── kubernetes-deploy.yaml # CD: deploy to K8s cluster
|
||||||
|
│
|
||||||
|
├── .secret/ # (gitignored) API credentials
|
||||||
|
├── docs/ # Project documentation
|
||||||
|
│ ├── project-notes.md # Original brainstorming and design notes
|
||||||
|
│ ├── fee-calculation-spec.md # Fee rules and payment matching spec
|
||||||
|
│ ├── scripts.md # Legacy scripts documentation
|
||||||
|
│ └── spec/
|
||||||
|
│ └── fio_to_sheets_sync.md # Fio-to-Sheets sync specification
|
||||||
|
│
|
||||||
|
└── CLAUDE.md # AI assistant context file
|
||||||
|
```
|
||||||
|
|
||||||
|
## Key Design Decisions
|
||||||
|
|
||||||
|
1. **No database** — Google Sheets serves as both the data store and the manual editing interface. This keeps the system simple and accessible to non-technical club members who can review and edit data directly in the spreadsheet.
|
||||||
|
|
||||||
|
2. **PII separation** — No member names or personal data are stored in the git repository. All data is fetched at runtime from Google Sheets and the bank account.
|
||||||
|
|
||||||
|
3. **Idempotent sync** — The Fio-to-Sheets sync uses SHA-256 hashes as deduplication keys, making re-runs safe and append-only.
|
||||||
|
|
||||||
|
4. **Graceful fallbacks** — Bank data can be fetched via the REST API (if a token is available) or by scraping the public transparent account page. The system doesn't break if the API token is missing.
|
||||||
|
|
||||||
|
5. **Czech language support** — Payment messages are in Czech and use diacritics. The system normalizes text (strips diacritics) and understands Czech month names in all grammatical declensions.
|
||||||
|
|
||||||
|
6. **Terminal aesthetic** — The web dashboard uses a monospace, dark-themed, terminal-inspired design that matches the project's pragmatic, CLI-first philosophy.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
*This documentation was generated on 2026-03-03 by Claude Opus, based on a comprehensive analysis of the complete codebase.*
|
||||||
268
docs/by-claude-opus/architecture.md
Normal file
268
docs/by-claude-opus/architecture.md
Normal file
@@ -0,0 +1,268 @@
|
|||||||
|
# System Architecture
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
FUJ Management follows a **pipeline architecture** where data flows from external sources (Google Sheets, Fio Bank) through processing scripts into a web dashboard. There is no central database — Google Sheets serves as the persistent data store, and the Flask app renders views by fetching and processing data on every request.
|
||||||
|
|
||||||
|
## Component Architecture
|
||||||
|
|
||||||
|
```
|
||||||
|
┌─────────────────────────────────────────────────┐
|
||||||
|
│ EXTERNAL DATA SOURCES │
|
||||||
|
│ │
|
||||||
|
│ ┌──────────────────┐ ┌──────────────────────┐ │
|
||||||
|
│ │ Attendance Sheet │ │ Fio Bank Account │ │
|
||||||
|
│ │ (Google Sheets) │ │ │ │
|
||||||
|
│ │ │ │ ┌────────────────┐ │ │
|
||||||
|
│ │ ID: 1E2e_gT... │ │ │ REST API │ │ │
|
||||||
|
│ │ │ │ │ (JSON, w/token)│ │ │
|
||||||
|
│ │ CSV export (pub) │ │ ├────────────────┤ │ │
|
||||||
|
│ │ │ │ │ Transparent │ │ │
|
||||||
|
│ └────────┬─────────┘ │ │ page (HTML) │ │ │
|
||||||
|
│ │ │ └───────┬────────┘ │ │
|
||||||
|
│ │ └──────────┼──────────┘ │
|
||||||
|
└───────────┼───────────────────────┼────────────┘
|
||||||
|
│ │
|
||||||
|
─ ─ ─ ─ ─ ─ ┼ ─ ─ DATA INGESTION ─ ┼ ─ ─ ─ ─ ─
|
||||||
|
│ │
|
||||||
|
┌───────────▼──────┐ ┌───────────▼──────────┐
|
||||||
|
│ attendance.py │ │ fio_utils.py │
|
||||||
|
│ │ │ │
|
||||||
|
│ fetch_csv() │ │ fetch_transactions() │
|
||||||
|
│ parse_dates() │ │ FioTableParser │
|
||||||
|
│ group_by_month() │ │ parse_czech_amount() │
|
||||||
|
│ calculate_fee() │ │ parse_czech_date() │
|
||||||
|
│ get_members() │ │ │
|
||||||
|
│ get_members_ │ │ API + HTML fallback │
|
||||||
|
│ with_fees() │ │ │
|
||||||
|
└───────────┬──────┘ └───────────┬──────────┘
|
||||||
|
│ │
|
||||||
|
─ ─ ─ ─ ─ ─ ┼ ─ ─ PROCESSING ─ ─ ─ ┼ ─ ─ ─ ─ ─
|
||||||
|
│ │
|
||||||
|
│ ┌─────────────▼──────────┐
|
||||||
|
│ │ sync_fio_to_sheets.py │ ──▶ Payments Sheet
|
||||||
|
│ │ │ (Google Sheets)
|
||||||
|
│ │ generate_sync_id() │
|
||||||
|
│ │ sort_sheet_by_date() │
|
||||||
|
│ │ get_sheets_service() │
|
||||||
|
│ └────────────────────────┘
|
||||||
|
│ │
|
||||||
|
│ ┌─────────────▼──────────┐
|
||||||
|
│ │ infer_payments.py │ ──▶ Writes back to
|
||||||
|
│ │ │ Payments Sheet
|
||||||
|
│ │ infer Person/Purpose/ │
|
||||||
|
│ │ Amount for empty rows │
|
||||||
|
│ └────────────────────────┘
|
||||||
|
│ │
|
||||||
|
│ ┌──────────────────▼──────────┐
|
||||||
|
│ │ czech_utils.py │
|
||||||
|
│ │ │
|
||||||
|
│ │ normalize() — strip │
|
||||||
|
│ │ diacritics │
|
||||||
|
│ │ parse_month_references() │
|
||||||
|
│ │ CZECH_MONTHS dict │
|
||||||
|
│ └─────────────────────────────┘
|
||||||
|
│ │
|
||||||
|
─ ─ ─ ─ ─ ─ ┼ ─ RECONCILIATION ─ ─┼ ─ ─ ─ ─ ─
|
||||||
|
│ │
|
||||||
|
┌─────────▼───────────────────────▼───────────┐
|
||||||
|
│ match_payments.py │
|
||||||
|
│ │
|
||||||
|
│ _build_name_variants() — name matching │
|
||||||
|
│ match_members() — fuzzy match │
|
||||||
|
│ infer_transaction_details() │
|
||||||
|
│ fetch_sheet_data() — read payments │
|
||||||
|
│ fetch_exceptions() — fee overrides │
|
||||||
|
│ reconcile() — CORE ENGINE │
|
||||||
|
│ print_report() — CLI output │
|
||||||
|
└──────────────────────┬──────────────────────┘
|
||||||
|
│
|
||||||
|
─ ─ ─ ─ ─ ─ ─ PRESENTATION ┼ ─ ─ ─ ─ ─ ─ ─ ─ ─
|
||||||
|
│
|
||||||
|
┌──────────────────────▼──────────────────────┐
|
||||||
|
│ app.py (Flask) │
|
||||||
|
│ │
|
||||||
|
│ GET / → redirect to /fees │
|
||||||
|
│ GET /fees → fees.html │
|
||||||
|
│ GET /reconcile → reconcile.html │
|
||||||
|
│ GET /payments → payments.html │
|
||||||
|
│ GET /qr → PNG QR code (SPD format) │
|
||||||
|
└─────────────────────────────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
## Module Dependency Graph
|
||||||
|
|
||||||
|
```
|
||||||
|
app.py
|
||||||
|
├── attendance.py
|
||||||
|
│ └── (stdlib: csv, urllib, datetime)
|
||||||
|
└── match_payments.py
|
||||||
|
├── attendance.py
|
||||||
|
├── czech_utils.py
|
||||||
|
│ └── (stdlib: re, unicodedata)
|
||||||
|
└── sync_fio_to_sheets.py (for get_sheets_service, DEFAULT_SPREADSHEET_ID)
|
||||||
|
└── fio_utils.py
|
||||||
|
└── (stdlib: json, urllib, html.parser, datetime)
|
||||||
|
|
||||||
|
infer_payments.py
|
||||||
|
├── sync_fio_to_sheets.py
|
||||||
|
├── match_payments.py
|
||||||
|
└── attendance.py
|
||||||
|
|
||||||
|
calculate_fees.py
|
||||||
|
└── attendance.py
|
||||||
|
```
|
||||||
|
|
||||||
|
### Import Relationships
|
||||||
|
|
||||||
|
| Module | Imports from |
|
||||||
|
|--------|-------------|
|
||||||
|
| `app.py` | `attendance` (`get_members_with_fees`, `SHEET_ID`), `match_payments` (`reconcile`, `fetch_sheet_data`, `fetch_exceptions`, `normalize`, `DEFAULT_SPREADSHEET_ID`) |
|
||||||
|
| `match_payments.py` | `attendance` (`get_members_with_fees`), `czech_utils` (`normalize`, `parse_month_references`), `sync_fio_to_sheets` (`get_sheets_service`, `DEFAULT_SPREADSHEET_ID`) |
|
||||||
|
| `infer_payments.py` | `sync_fio_to_sheets` (`get_sheets_service`, `DEFAULT_SPREADSHEET_ID`), `match_payments` (`infer_transaction_details`), `attendance` (`get_members_with_fees`) |
|
||||||
|
| `sync_fio_to_sheets.py` | `fio_utils` (`fetch_transactions`) |
|
||||||
|
| `calculate_fees.py` | `attendance` (`get_members_with_fees`) |
|
||||||
|
|
||||||
|
## Data Flow Patterns
|
||||||
|
|
||||||
|
### Pattern 1: Sync & Enrich (Batch Pipeline)
|
||||||
|
|
||||||
|
This is the primary workflow for keeping the payments ledger up to date:
|
||||||
|
|
||||||
|
```
|
||||||
|
1. make sync 2. make infer
|
||||||
|
┌──────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐
|
||||||
|
│ Fio │───▶│ Payments │ │ Payments │───▶│ Payments │
|
||||||
|
│ Bank │ │ Sheet │ │ Sheet │ │ Sheet │
|
||||||
|
└──────┘ │ (append) │ │ (read) │ │ (update) │
|
||||||
|
└──────────┘ └──────────┘ └──────────┘
|
||||||
|
|
||||||
|
- Fetches last 30 days - Reads empty Person/Purpose rows
|
||||||
|
- SHA-256 dedup prevents - Uses name matching + Czech month
|
||||||
|
duplicate entries parsing to auto-fill
|
||||||
|
- Marks uncertain matches with [?]
|
||||||
|
```
|
||||||
|
|
||||||
|
### Pattern 2: Real-Time Rendering (Web Dashboard)
|
||||||
|
|
||||||
|
Every web request triggers a fresh data fetch — no caching layer exists:
|
||||||
|
|
||||||
|
```
|
||||||
|
Browser Request → Flask Route → Fetch (Google Sheets API/CSV) → Process → Render HTML
|
||||||
|
│ │
|
||||||
|
│ attendance.py │ reconcile()
|
||||||
|
│ fetch_sheet_data() │ or direct
|
||||||
|
│ fetch_exceptions() │ formatting
|
||||||
|
▼ ▼
|
||||||
|
~1-3 seconds Template with
|
||||||
|
(network I/O) inline CSS + JS
|
||||||
|
```
|
||||||
|
|
||||||
|
### Pattern 3: QR Code Generation (On-Demand)
|
||||||
|
|
||||||
|
```
|
||||||
|
Browser clicks "Pay" → GET /qr?account=...&amount=...&message=... → SPD QR PNG
|
||||||
|
│
|
||||||
|
qrcode lib
|
||||||
|
generates
|
||||||
|
in-memory PNG
|
||||||
|
```
|
||||||
|
|
||||||
|
## Key Design Patterns
|
||||||
|
|
||||||
|
### 1. Google Sheets as Database
|
||||||
|
|
||||||
|
Instead of a traditional database, the system uses two Google Sheets:
|
||||||
|
|
||||||
|
| Sheet | Purpose | Access Method |
|
||||||
|
|-------|---------|---------------|
|
||||||
|
| Attendance Sheet (`1E2e_gT...`) | Member names, tiers, practice dates, attendance marks | Public CSV export (no auth needed) |
|
||||||
|
| Payments Sheet (`1Om0YPo...`) | Bank transactions with Person/Purpose annotations | Google Sheets API (service account auth) |
|
||||||
|
|
||||||
|
**Trade-offs**:
|
||||||
|
- ✅ Non-technical users can view and edit data directly
|
||||||
|
- ✅ No database setup or maintenance
|
||||||
|
- ✅ Built-in audit trail (Google Sheets version history)
|
||||||
|
- ❌ Every page load incurs 1-3s of API latency
|
||||||
|
- ❌ No complex queries or indexing
|
||||||
|
- ❌ Rate limits on Google Sheets API
|
||||||
|
|
||||||
|
### 2. Dual-Mode Bank Access
|
||||||
|
|
||||||
|
`fio_utils.py` implements a transparent fallback pattern:
|
||||||
|
|
||||||
|
```python
|
||||||
|
def fetch_transactions(date_from, date_to):
|
||||||
|
token = os.environ.get("FIO_API_TOKEN", "").strip()
|
||||||
|
if token:
|
||||||
|
return fetch_transactions_api(token, date_from, date_to) # Structured JSON
|
||||||
|
return fetch_transactions_transparent(...) # HTML scraping
|
||||||
|
```
|
||||||
|
|
||||||
|
The API provides richer data (sender account numbers, stable bank IDs) but requires a token. The transparent page is always available but lacks some fields.
|
||||||
|
|
||||||
|
### 3. Name Matching with Confidence Levels
|
||||||
|
|
||||||
|
The reconciliation engine uses a multi-tier matching strategy:
|
||||||
|
|
||||||
|
| Priority | Method | Confidence | Example |
|
||||||
|
|----------|--------|-----------|---------|
|
||||||
|
| 1 | Full name match | `auto` | "František Vrbík" in message |
|
||||||
|
| 2 | Both first + last name (any order) | `auto` | "Vrbík František" |
|
||||||
|
| 3 | Nickname match | `auto` | "(Štrúdl)" from member list |
|
||||||
|
| 4 | Last name only (≥4 chars, not common) | `review` | "Vrbík" alone |
|
||||||
|
| 5 | First name only (≥3 chars) | `review` | "František" alone |
|
||||||
|
|
||||||
|
When both `auto` and `review` matches exist, `review` matches are discarded. This prevents false positives from generic first names.
|
||||||
|
|
||||||
|
### 4. Exception System
|
||||||
|
|
||||||
|
Fee overrides are managed through an `exceptions` sheet tab in the Payments Google Sheet:
|
||||||
|
|
||||||
|
| Column | Content |
|
||||||
|
|--------|---------|
|
||||||
|
| Name | Member name |
|
||||||
|
| Period | Month (YYYY-MM) |
|
||||||
|
| Amount | Overridden fee in CZK |
|
||||||
|
| Note | Reason for the exception |
|
||||||
|
|
||||||
|
Exceptions are applied during reconciliation, replacing the attendance-calculated fee with the manually specified amount.
|
||||||
|
|
||||||
|
### 5. Render-Time Performance Tracking
|
||||||
|
|
||||||
|
Every page includes a performance breakdown:
|
||||||
|
|
||||||
|
```python
|
||||||
|
@app.before_request
|
||||||
|
def start_timer():
|
||||||
|
g.start_time = time.perf_counter()
|
||||||
|
g.steps = []
|
||||||
|
|
||||||
|
def record_step(name):
|
||||||
|
g.steps.append((name, time.perf_counter()))
|
||||||
|
```
|
||||||
|
|
||||||
|
The footer displays total render time and, on click, reveals a detailed breakdown (e.g., `fetch_members:0.892s | fetch_payments:1.205s | reconcile:0.003s | render:0.015s`).
|
||||||
|
|
||||||
|
## Security Considerations
|
||||||
|
|
||||||
|
| Concern | Mitigation |
|
||||||
|
|---------|-----------|
|
||||||
|
| PII in git | `.secret/` is gitignored; all data fetched at runtime |
|
||||||
|
| Google API credentials | Service account JSON stored in `.secret/`, mounted as Docker secret |
|
||||||
|
| Bank API token | Passed via `FIO_API_TOKEN` environment variable, never committed |
|
||||||
|
| Web app authentication | **None currently** — the app has no auth layer |
|
||||||
|
| CSRF protection | **None currently** — Flask default (no POST routes exist) |
|
||||||
|
|
||||||
|
## Scalability Notes
|
||||||
|
|
||||||
|
This system is purpose-built for a small club (~20-40 members). It makes deliberate trade-offs favoring simplicity over scale:
|
||||||
|
|
||||||
|
- **No caching**: Every page load fetches live data from Google Sheets (1-3s latency). For a single-user admin dashboard, this is acceptable.
|
||||||
|
- **No background workers**: Sync and inference are manual `make` commands, not scheduled jobs.
|
||||||
|
- **No database**: Google Sheets handles 10s of members and 100s of transactions with ease.
|
||||||
|
- **Single-process Flask**: The built-in development server runs directly in production (via Docker). For this use case, this is intentional — it's a personal tool, not a public service.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
*Architecture documentation generated from comprehensive code analysis on 2026-03-03.*
|
||||||
201
docs/by-claude-opus/data-model.md
Normal file
201
docs/by-claude-opus/data-model.md
Normal file
@@ -0,0 +1,201 @@
|
|||||||
|
# Data Model
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
FUJ Management operates on two Google Sheets and an external bank account. There is no local database — all persistent data lives in Google Sheets, and all member data is fetched at runtime (never committed to git).
|
||||||
|
|
||||||
|
## External Data Sources
|
||||||
|
|
||||||
|
### 1. Attendance Google Sheet
|
||||||
|
|
||||||
|
| Property | Value |
|
||||||
|
|----------|-------|
|
||||||
|
| **Sheet ID** | `1E2e_gT_K5AwSRCDLDTa2UetZTkHmBOcz0kFbBUNUNBA` |
|
||||||
|
| **Access** | Public CSV export (no authentication required) |
|
||||||
|
| **Purpose** | Member roster, weekly practice attendance marks |
|
||||||
|
| **Scope** | Tuesday practices (20:30–22:00) |
|
||||||
|
|
||||||
|
#### Schema
|
||||||
|
|
||||||
|
```
|
||||||
|
Row 1: [Title] [blank] [blank] [10/1/2025] [10/8/2025] [10/15/2025] ...
|
||||||
|
Row 2: Venue per date (ignored by the system)
|
||||||
|
Row 3: Subtotals per date (ignored by the system)
|
||||||
|
Row 4+: [Name] [Tier] [Total] [TRUE/FALSE] [TRUE/FALSE] ...
|
||||||
|
...
|
||||||
|
Row N: # last line (sentinel — stops parsing)
|
||||||
|
```
|
||||||
|
|
||||||
|
| Column | Index | Content | Example |
|
||||||
|
|--------|-------|---------|---------|
|
||||||
|
| A | 0 | Member name | `Jan Novák` |
|
||||||
|
| B | 1 | Tier code | `A`, `J`, or `X` |
|
||||||
|
| C | 2 | Total attendance (auto-calculated, ignored by the system) | `12` |
|
||||||
|
| D+ | 3+ | Attendance per date | `TRUE` or `FALSE` |
|
||||||
|
|
||||||
|
#### Tier Codes
|
||||||
|
|
||||||
|
| Code | Meaning | Pays fees? |
|
||||||
|
|------|---------|-----------|
|
||||||
|
| `A` | Adult | Yes — calculated from this sheet |
|
||||||
|
| `J` | Junior | No — managed via a separate sheet |
|
||||||
|
| `X` | Exempt | No |
|
||||||
|
|
||||||
|
#### Sentinel Row
|
||||||
|
|
||||||
|
The system stops parsing member rows when it encounters a row whose first column contains `# last line` (case-insensitive). Rows starting with `#` are also skipped as comments.
|
||||||
|
|
||||||
|
### 2. Payments Google Sheet
|
||||||
|
|
||||||
|
| Property | Value |
|
||||||
|
|----------|-------|
|
||||||
|
| **Sheet ID** | `1Om0YPoDVCH5cV8BrNz5LG5eR5MMU05ypQC7UMN1xn_Y` |
|
||||||
|
| **Access** | Google Sheets API (service account authentication) |
|
||||||
|
| **Purpose** | Intermediary ledger for bank transactions + manual annotations |
|
||||||
|
| **Managed by** | `sync_fio_to_sheets.py` (append), `infer_payments.py` (update) |
|
||||||
|
|
||||||
|
#### Main Sheet Schema (Columns A–K)
|
||||||
|
|
||||||
|
| Column | Label | Populated by | Description |
|
||||||
|
|--------|-------|-------------|-------------|
|
||||||
|
| A | Date | `sync` | Transaction date (`YYYY-MM-DD`) |
|
||||||
|
| B | Amount | `sync` | Bank transaction amount in CZK |
|
||||||
|
| C | manual fix | Human | If non-empty, `infer` will skip this row |
|
||||||
|
| D | Person | `infer` or human | Member name(s), comma-separated for multi-person payments |
|
||||||
|
| E | Purpose | `infer` or human | Month(s) covered, e.g. `2026-01` or `2026-01, 2026-02` |
|
||||||
|
| F | Inferred Amount | `infer` or human | Amount to use for reconciliation (may differ from bank amount) |
|
||||||
|
| G | Sender | `sync` | Bank sender name/account |
|
||||||
|
| H | VS | `sync` | Variable symbol |
|
||||||
|
| I | Message | `sync` | Payment message for recipient |
|
||||||
|
| J | Bank ID | `sync` | Fio transaction ID (API only) |
|
||||||
|
| K | Sync ID | `sync` | SHA-256 deduplication hash |
|
||||||
|
|
||||||
|
#### Exceptions Sheet Tab
|
||||||
|
|
||||||
|
A separate tab named `exceptions` in the same spreadsheet, used for manual fee overrides:
|
||||||
|
|
||||||
|
| Column | Label | Content |
|
||||||
|
|--------|-------|---------|
|
||||||
|
| A | Name | Member name (plain text) |
|
||||||
|
| B | Period | Month (`YYYY-MM`) |
|
||||||
|
| C | Amount | Overridden fee in CZK |
|
||||||
|
| D | Note | Reason for override (optional) |
|
||||||
|
|
||||||
|
The first row is assumed to be a header and is skipped. Name and period values are normalized (diacritics stripped, lowercased) for matching.
|
||||||
|
|
||||||
|
### 3. Fio Bank Account
|
||||||
|
|
||||||
|
| Property | Value |
|
||||||
|
|----------|-------|
|
||||||
|
| **Account number** | `2800359168/2010` |
|
||||||
|
| **IBAN** | `CZ8520100000002800359168` |
|
||||||
|
| **Type** | Transparent account |
|
||||||
|
| **Owner** | Nathan Heilmann |
|
||||||
|
| **Public URL** | `https://ib.fio.cz/ib/transparent?a=2800359168` |
|
||||||
|
|
||||||
|
#### Access Methods
|
||||||
|
|
||||||
|
| Method | Trigger | Data richness |
|
||||||
|
|--------|---------|--------------|
|
||||||
|
| REST API | `FIO_API_TOKEN` env var set | Full data: sender account, bank ID, user identification, currency |
|
||||||
|
| HTML scraping | `FIO_API_TOKEN` not set | Partial: date, amount, sender name, message, VS/KS/SS |
|
||||||
|
|
||||||
|
#### API Rate Limit
|
||||||
|
|
||||||
|
The Fio REST API allows 1 request per 30 seconds per token.
|
||||||
|
|
||||||
|
## Fee Calculation Rules
|
||||||
|
|
||||||
|
Fees apply only to **tier A (Adult)** members. They are calculated per calendar month based on Tuesday practice attendance:
|
||||||
|
|
||||||
|
| Practices attended | Monthly fee |
|
||||||
|
|-------------------|-------------|
|
||||||
|
| 0 | 0 CZK |
|
||||||
|
| 1 | 200 CZK |
|
||||||
|
| 2+ | 750 CZK |
|
||||||
|
|
||||||
|
### Exception Overrides
|
||||||
|
|
||||||
|
The fee can be manually overridden per member per month via the `exceptions` tab. When an exception exists:
|
||||||
|
- The `expected` amount in reconciliation uses the exception amount
|
||||||
|
- The `original_expected` amount preserves the attendance-based calculation
|
||||||
|
- The override is displayed in amber/orange in the web UI
|
||||||
|
|
||||||
|
### Advance Payments
|
||||||
|
|
||||||
|
If a payment references a month not yet covered by attendance data:
|
||||||
|
- It is tracked as **credit** on the member's account
|
||||||
|
- Credits are added to the total balance
|
||||||
|
- When attendance data becomes available for that month, the credit effectively offsets the expected fee
|
||||||
|
|
||||||
|
## Reconciliation Data Model
|
||||||
|
|
||||||
|
The `reconcile()` function returns this structure:
|
||||||
|
|
||||||
|
```python
|
||||||
|
{
|
||||||
|
"members": {
|
||||||
|
"Jan Novák": {
|
||||||
|
"tier": "A",
|
||||||
|
"months": {
|
||||||
|
"2026-01": {
|
||||||
|
"expected": 750, # Fee after exception application
|
||||||
|
"original_expected": 750, # Attendance-based fee
|
||||||
|
"attendance_count": 4, # How many times they came
|
||||||
|
"exception": None, # or {"amount": 400, "note": "..."}
|
||||||
|
"paid": 750.0, # Total matched payments
|
||||||
|
"transactions": [ # Individual payment records
|
||||||
|
{
|
||||||
|
"amount": 750.0,
|
||||||
|
"date": "2026-01-15",
|
||||||
|
"sender": "Jan Novák",
|
||||||
|
"message": "leden",
|
||||||
|
"confidence": "auto"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"total_balance": 0 # sum(paid - expected) across all months + off-window credits
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"unmatched": [ # Transactions that couldn't be assigned
|
||||||
|
{
|
||||||
|
"date": "2026-01-20",
|
||||||
|
"amount": 500,
|
||||||
|
"sender": "Unknown",
|
||||||
|
"message": "dar"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"credits": { # Alias for positive total_balance entries
|
||||||
|
"Jan Novák": 200
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Sync ID Generation
|
||||||
|
|
||||||
|
The deduplication key for bank transactions is a SHA-256 hash of:
|
||||||
|
|
||||||
|
```
|
||||||
|
sha256("date|amount|currency|sender|vs|message|bank_id")
|
||||||
|
```
|
||||||
|
|
||||||
|
All values are lowercased before hashing. This ensures:
|
||||||
|
- Same transaction fetched twice produces the same ID
|
||||||
|
- Two payments on the same day with different amounts/senders produce different IDs
|
||||||
|
- The hash is stable across API and HTML scraping modes (shared fields)
|
||||||
|
|
||||||
|
## Date Handling
|
||||||
|
|
||||||
|
| Source | Format | Normalization |
|
||||||
|
|--------|--------|--------------|
|
||||||
|
| Attendance Sheet header | `M/D/YYYY` (US format) | `datetime.strptime(raw, "%m/%d/%Y")` |
|
||||||
|
| Fio API | `YYYY-MM-DD+HHMM` | Take first 10 characters |
|
||||||
|
| Fio transparent page | `DD.MM.YYYY` | `datetime.strptime(raw, "%d.%m.%Y")` |
|
||||||
|
| Google Sheets (unformatted) | Serial number (days since 1899-12-30) | `datetime(1899, 12, 30) + timedelta(days=val)` |
|
||||||
|
|
||||||
|
All internal date representation uses `YYYY-MM-DD` format. Month keys use `YYYY-MM`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
*Data model documentation generated from comprehensive code analysis on 2026-03-03.*
|
||||||
198
docs/by-claude-opus/deployment.md
Normal file
198
docs/by-claude-opus/deployment.md
Normal file
@@ -0,0 +1,198 @@
|
|||||||
|
# Deployment Guide
|
||||||
|
|
||||||
|
## Local Development
|
||||||
|
|
||||||
|
### Prerequisites
|
||||||
|
|
||||||
|
- **Python 3.13+** (required by `pyproject.toml`)
|
||||||
|
- **[uv](https://docs.astral.sh/uv/)** — Fast Python package manager
|
||||||
|
- Google Sheets API credentials (service account JSON)
|
||||||
|
|
||||||
|
### Setup
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git clone <repo-url>
|
||||||
|
cd fuj-management
|
||||||
|
|
||||||
|
# Install dependencies
|
||||||
|
uv sync
|
||||||
|
|
||||||
|
# Configure credentials
|
||||||
|
mkdir -p .secret
|
||||||
|
cp /path/to/credentials.json .secret/fuj-management-bot-credentials.json
|
||||||
|
|
||||||
|
# Optional: Set Fio API token for richer bank data
|
||||||
|
export FIO_API_TOKEN=your_token_here
|
||||||
|
|
||||||
|
# Start the web dashboard
|
||||||
|
make web
|
||||||
|
# → Flask server at http://localhost:5001
|
||||||
|
```
|
||||||
|
|
||||||
|
### Makefile Targets
|
||||||
|
|
||||||
|
| Target | Command | Description |
|
||||||
|
|--------|---------|-------------|
|
||||||
|
| `help` | `make help` | List all available targets |
|
||||||
|
| `venv` | `make venv` | Sync virtual environment with pyproject.toml |
|
||||||
|
| `fees` | `make fees` | Print fee calculation table |
|
||||||
|
| `match` | `make match` | (Legacy) Direct bank matching |
|
||||||
|
| `web` | `make web` | Start Flask dashboard on port 5001 |
|
||||||
|
| `sync` | `make sync` | Sync last 30 days of bank transactions |
|
||||||
|
| `sync-2026` | `make sync-2026` | Sync full year 2026 transactions |
|
||||||
|
| `infer` | `make infer` | Auto-fill Person/Purpose in the sheet |
|
||||||
|
| `reconcile` | `make reconcile` | Print CLI balance report |
|
||||||
|
| `test` | `make test` | Run test suite |
|
||||||
|
| `test-v` | `make test-v` | Run tests with verbose output |
|
||||||
|
| `image` | `make image` | Build Docker image |
|
||||||
|
| `run` | `make run` | Run Docker container locally |
|
||||||
|
|
||||||
|
The Makefile includes **automatic venv management**: targets that need Python depend on `.venv/.last_sync`, which triggers `uv sync` when `pyproject.toml` changes.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Docker Container
|
||||||
|
|
||||||
|
### Building
|
||||||
|
|
||||||
|
```bash
|
||||||
|
make image
|
||||||
|
# → docker build -t fuj-management:latest -f build/Dockerfile .
|
||||||
|
```
|
||||||
|
|
||||||
|
### Dockerfile Details
|
||||||
|
|
||||||
|
**Base image**: `python:3.13-alpine`
|
||||||
|
|
||||||
|
**Build stages**:
|
||||||
|
1. Install system packages (`bash`, `tzdata`)
|
||||||
|
2. Set timezone to `Europe/Prague`
|
||||||
|
3. Install Python dependencies via pip
|
||||||
|
4. Copy application files (`app.py`, `scripts/`, `templates/`, `Makefile`)
|
||||||
|
5. Copy entrypoint script
|
||||||
|
|
||||||
|
**Exposed port**: 5001
|
||||||
|
|
||||||
|
**Health check**: `wget -q -O /dev/null http://localhost:5001/` every 60s
|
||||||
|
|
||||||
|
### Running Locally via Docker
|
||||||
|
|
||||||
|
```bash
|
||||||
|
make run
|
||||||
|
# → docker run -it --rm -p 5001:5001 fuj-management:latest
|
||||||
|
|
||||||
|
# With credentials and environment:
|
||||||
|
docker run -it --rm \
|
||||||
|
-p 5001:5001 \
|
||||||
|
-v $(pwd)/.secret:/app/.secret:ro \
|
||||||
|
-e FIO_API_TOKEN=your_token \
|
||||||
|
-e BANK_ACCOUNT=CZ8520100000002800359168 \
|
||||||
|
fuj-management:latest
|
||||||
|
```
|
||||||
|
|
||||||
|
### Entrypoint
|
||||||
|
|
||||||
|
The `build/entrypoint.sh` script simply runs:
|
||||||
|
```bash
|
||||||
|
exec python3 /app/app.py
|
||||||
|
```
|
||||||
|
|
||||||
|
This uses Flask's built-in server directly. For a production deployment, consider adding gunicorn or waitress (noted as a TODO in the entrypoint).
|
||||||
|
|
||||||
|
### Environment Variables
|
||||||
|
|
||||||
|
| Variable | Default | Description |
|
||||||
|
|----------|---------|-------------|
|
||||||
|
| `BANK_ACCOUNT` | `CZ8520100000002800359168` | IBAN for QR code generation |
|
||||||
|
| `FIO_API_TOKEN` | *(none)* | Fio REST API token |
|
||||||
|
| `PYTHONUNBUFFERED` | `1` (set in Dockerfile) | Ensures real-time log output |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## CI/CD Pipeline
|
||||||
|
|
||||||
|
### Gitea Actions
|
||||||
|
|
||||||
|
The project uses two Gitea Actions workflows:
|
||||||
|
|
||||||
|
#### 1. Build and Push (`build.yaml`)
|
||||||
|
|
||||||
|
**Triggers**:
|
||||||
|
- Push of any tag
|
||||||
|
- Manual dispatch (with custom tag input)
|
||||||
|
|
||||||
|
**Steps**:
|
||||||
|
1. Checkout code
|
||||||
|
2. Login to Gitea container registry (`gitea.home.hrajfrisbee.cz`)
|
||||||
|
3. Build Docker image using `build/Dockerfile`
|
||||||
|
4. Push to `gitea.home.hrajfrisbee.cz/<owner>/<repo>:<tag>`
|
||||||
|
|
||||||
|
**Tag resolution**: Uses the git tag name. For manual dispatch, uses the provided input.
|
||||||
|
|
||||||
|
#### 2. Deploy to Kubernetes (`kubernetes-deploy.yaml`)
|
||||||
|
|
||||||
|
**Triggers**:
|
||||||
|
- Push to any branch
|
||||||
|
- Manual dispatch
|
||||||
|
|
||||||
|
**Steps**:
|
||||||
|
1. Checkout code
|
||||||
|
2. Install kubectl
|
||||||
|
3. Retrieve Kanidm token from HashiCorp Vault:
|
||||||
|
- Authenticate to Vault via AppRole (`VAULT_ROLE_ID` / `VAULT_SECRET_ID`)
|
||||||
|
- Fetch API token from `secret/data/gitea/gitea-ci`
|
||||||
|
4. Exchange API token for K8s OIDC token via Kanidm:
|
||||||
|
- POST to `https://idm.home.hrajfrisbee.cz/oauth2/token`
|
||||||
|
- Token exchange using `urn:ietf:params:oauth:grant-type:token-exchange`
|
||||||
|
5. Configure kubectl with the OIDC token
|
||||||
|
6. Run `kubectl auth whoami` and `kubectl get ns` (deploy commands are commented out — WIP)
|
||||||
|
|
||||||
|
**Required secrets**:
|
||||||
|
|
||||||
|
| Secret | Purpose |
|
||||||
|
|--------|---------|
|
||||||
|
| `REGISTRY_TOKEN` | Docker registry authentication |
|
||||||
|
| `VAULT_ROLE_ID` | HashiCorp Vault AppRole role ID |
|
||||||
|
| `VAULT_SECRET_ID` | HashiCorp Vault AppRole secret ID |
|
||||||
|
| `K8S_CA_CERT` | Kubernetes cluster CA certificate |
|
||||||
|
|
||||||
|
### Infrastructure Topology
|
||||||
|
|
||||||
|
```
|
||||||
|
Gitea (git push / tag)
|
||||||
|
│
|
||||||
|
├── build.yaml → Docker Build → Gitea Container Registry
|
||||||
|
│ (gitea.home.hrajfrisbee.cz)
|
||||||
|
│
|
||||||
|
└── kubernetes-deploy.yaml → Vault → Kanidm → K8s Cluster
|
||||||
|
(192.168.0.31:6443)
|
||||||
|
```
|
||||||
|
|
||||||
|
This is a self-hosted infrastructure stack:
|
||||||
|
- **Gitea** for git hosting and CI/CD
|
||||||
|
- **HashiCorp Vault** for secret management
|
||||||
|
- **Kanidm** for identity/OIDC
|
||||||
|
- **Kubernetes** for container orchestration
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Credentials Management
|
||||||
|
|
||||||
|
### Google Sheets API
|
||||||
|
|
||||||
|
The system uses a **Google Cloud service account** for accessing the Payments Google Sheet. The credentials file must be:
|
||||||
|
- Stored at `.secret/fuj-management-bot-credentials.json`
|
||||||
|
- In Google Cloud service account JSON format
|
||||||
|
- The service account must be shared (as editor) on the target Google Sheet
|
||||||
|
|
||||||
|
For local development with OAuth2 (personal Google account), the system also supports the OAuth2 installed app flow — it will generate a `token.pickle` file on first use.
|
||||||
|
|
||||||
|
### Fio Bank API
|
||||||
|
|
||||||
|
Optional. Set the `FIO_API_TOKEN` environment variable. The token is generated in Fio internetbanking under Settings → API.
|
||||||
|
|
||||||
|
**Rate limit**: 1 request per 30 seconds per token.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
*Deployment documentation generated from comprehensive code analysis on 2026-03-03.*
|
||||||
228
docs/by-claude-opus/development.md
Normal file
228
docs/by-claude-opus/development.md
Normal file
@@ -0,0 +1,228 @@
|
|||||||
|
# Development Guide
|
||||||
|
|
||||||
|
## Development Environment
|
||||||
|
|
||||||
|
### Required Tools
|
||||||
|
|
||||||
|
| Tool | Version | Purpose |
|
||||||
|
|------|---------|---------|
|
||||||
|
| Python | 3.13+ | Runtime |
|
||||||
|
| uv | Latest | Dependency management |
|
||||||
|
| Docker | Latest | Container builds |
|
||||||
|
| Git | Any | Version control |
|
||||||
|
| Make | Any | Build automation |
|
||||||
|
|
||||||
|
### Initial Setup
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 1. Clone the repository
|
||||||
|
git clone <repo-url>
|
||||||
|
cd fuj-management
|
||||||
|
|
||||||
|
# 2. Install dependencies (creates .venv automatically)
|
||||||
|
uv sync
|
||||||
|
|
||||||
|
# 3. Activate the virtual environment
|
||||||
|
source .venv/bin/activate
|
||||||
|
|
||||||
|
# 4. Set up credentials
|
||||||
|
mkdir -p .secret
|
||||||
|
# Copy your Google service account JSON here:
|
||||||
|
cp ~/Downloads/fuj-management-bot-credentials.json .secret/
|
||||||
|
|
||||||
|
# 5. (Optional) Set Fio API token
|
||||||
|
export FIO_API_TOKEN=your_token_here
|
||||||
|
```
|
||||||
|
|
||||||
|
### IDE Configuration
|
||||||
|
|
||||||
|
The `.vscode/` directory contains workspace settings. If using VS Code, the Python interpreter should automatically detect the `.venv` directory.
|
||||||
|
|
||||||
|
**PYTHONPATH note**: When running scripts from the project root, the Makefile sets `PYTHONPATH=scripts:$PYTHONPATH`. If your IDE doesn't do this, you may see import errors in `match_payments.py` and other scripts that import sibling modules.
|
||||||
|
|
||||||
|
## Project Dependencies
|
||||||
|
|
||||||
|
Defined in `pyproject.toml`:
|
||||||
|
|
||||||
|
| Dependency | Version | Purpose |
|
||||||
|
|------------|---------|---------|
|
||||||
|
| `flask` | ≥3.1.3 | Web framework |
|
||||||
|
| `google-api-python-client` | ≥2.162.0 | Google Sheets API |
|
||||||
|
| `google-auth-httplib2` | ≥0.2.0 | Google auth transport |
|
||||||
|
| `google-auth-oauthlib` | ≥1.2.1 | OAuth2 support |
|
||||||
|
| `qrcode[pil]` | ≥8.0 | QR code generation (with PIL/Pillow backend) |
|
||||||
|
|
||||||
|
The project uses `uv` with `package = false` in `[tool.uv]`, meaning it's not an installable package — dependencies are synced directly to the virtual environment.
|
||||||
|
|
||||||
|
## Coding Conventions
|
||||||
|
|
||||||
|
### Python Style
|
||||||
|
|
||||||
|
- No linter or formatter is configured — the codebase uses a pragmatic, readable style
|
||||||
|
- Type hints are used for function signatures but not exhaustively
|
||||||
|
- Docstrings follow Google-style format on key functions
|
||||||
|
- Scripts use `if __name__ == "__main__": main()` pattern
|
||||||
|
|
||||||
|
### Import Pattern
|
||||||
|
|
||||||
|
Scripts in the `scripts/` directory import from each other as top-level modules:
|
||||||
|
|
||||||
|
```python
|
||||||
|
# In match_payments.py:
|
||||||
|
from attendance import get_members_with_fees
|
||||||
|
from czech_utils import normalize, parse_month_references
|
||||||
|
from sync_fio_to_sheets import get_sheets_service, DEFAULT_SPREADSHEET_ID
|
||||||
|
```
|
||||||
|
|
||||||
|
This works because `scripts/` is added to `sys.path` at runtime (by `app.py` on startup, by Makefile via `PYTHONPATH`, or by scripts adding their own directory to `sys.path`).
|
||||||
|
|
||||||
|
### Template Style
|
||||||
|
|
||||||
|
- All CSS is inline (no external stylesheets)
|
||||||
|
- No CSS preprocessors or frameworks
|
||||||
|
- No JavaScript frameworks — plain DOM manipulation
|
||||||
|
- Terminal-inspired aesthetic: monospace fonts, green-on-black, dashed borders
|
||||||
|
|
||||||
|
### Commit Conventions
|
||||||
|
|
||||||
|
The project uses [Conventional Commits](https://www.conventionalcommits.org/):
|
||||||
|
```
|
||||||
|
feat: add keyboard navigation to member popup
|
||||||
|
fix: correct diacritic-insensitive search filter
|
||||||
|
chore: update dependencies
|
||||||
|
```
|
||||||
|
|
||||||
|
AI commits include a co-author trailer:
|
||||||
|
```
|
||||||
|
Co-authored-by: Antigravity <antigravity@google.com>
|
||||||
|
```
|
||||||
|
|
||||||
|
## Architecture Decisions
|
||||||
|
|
||||||
|
### Why No Database?
|
||||||
|
|
||||||
|
Google Sheets serves as the database because:
|
||||||
|
1. Club members can view and correct data without special tools
|
||||||
|
2. No database server to manage or back up
|
||||||
|
3. Built-in version history and collaborative editing
|
||||||
|
4. Good enough for ~40 members and ~hundreds of transactions
|
||||||
|
|
||||||
|
### Why No Template Inheritance?
|
||||||
|
|
||||||
|
Each HTML template is self-contained. While this means CSS duplication, it keeps each page fully independent and easy to understand. For a 3-page app, the duplication cost is minimal.
|
||||||
|
|
||||||
|
### Why Flask Development Server in Production?
|
||||||
|
|
||||||
|
The Docker container runs Flask's built-in server (`python3 app.py`) rather than gunicorn or waitress. This is intentional — the dashboard is an internal tool accessed by one person at a time. The simplicity outweighs the performance cost.
|
||||||
|
|
||||||
|
### Why Scrape HTML When There's an API?
|
||||||
|
|
||||||
|
The Fio transparent page scraping exists as a **zero-configuration fallback**. Not everyone has an API token, and the transparent page is always publicly accessible. The API is preferred when available (richer data, stable IDs).
|
||||||
|
|
||||||
|
## Common Development Tasks
|
||||||
|
|
||||||
|
### Adding a New Web Route
|
||||||
|
|
||||||
|
1. Add the route function in `app.py`:
|
||||||
|
```python
|
||||||
|
@app.route("/new-page")
|
||||||
|
def new_page():
|
||||||
|
# Fetch data
|
||||||
|
record_step("fetch_data")
|
||||||
|
# Process
|
||||||
|
record_step("process_data")
|
||||||
|
return render_template("new_page.html", ...)
|
||||||
|
```
|
||||||
|
|
||||||
|
2. Create `templates/new_page.html` (copy structure from `fees.html`)
|
||||||
|
|
||||||
|
3. Add a link in the nav bar across all templates:
|
||||||
|
```html
|
||||||
|
<a href="/new-page">[New Page]</a>
|
||||||
|
```
|
||||||
|
|
||||||
|
4. Add a test in `tests/test_app.py`
|
||||||
|
|
||||||
|
### Adding a New Script
|
||||||
|
|
||||||
|
1. Create `scripts/new_script.py`
|
||||||
|
2. Add a Makefile target:
|
||||||
|
```makefile
|
||||||
|
new-target: $(PYTHON)
|
||||||
|
$(PYTHON) scripts/new_script.py
|
||||||
|
```
|
||||||
|
3. Update `make help` output
|
||||||
|
4. Add the `.PHONY` declaration
|
||||||
|
|
||||||
|
### Modifying Fee Rules
|
||||||
|
|
||||||
|
Fee rules are defined as constants in `scripts/attendance.py`:
|
||||||
|
```python
|
||||||
|
FEE_FULL = 750 # 2+ practices
|
||||||
|
FEE_SINGLE = 200 # 1 practice
|
||||||
|
```
|
||||||
|
|
||||||
|
The calculation logic is in `calculate_fee()`:
|
||||||
|
```python
|
||||||
|
def calculate_fee(attendance_count: int) -> int:
|
||||||
|
if attendance_count == 0: return 0
|
||||||
|
if attendance_count == 1: return FEE_SINGLE
|
||||||
|
return FEE_FULL
|
||||||
|
```
|
||||||
|
|
||||||
|
### Adding a New Czech Month Form
|
||||||
|
|
||||||
|
If you encounter a Czech month declension not yet supported, add it to `CZECH_MONTHS` in `scripts/czech_utils.py`:
|
||||||
|
|
||||||
|
```python
|
||||||
|
CZECH_MONTHS = {
|
||||||
|
"leden": 1, "ledna": 1, "lednu": 1,
|
||||||
|
"lednem": 1, # New instrumental case
|
||||||
|
...
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Project History
|
||||||
|
|
||||||
|
The project evolved through distinct phases:
|
||||||
|
|
||||||
|
1. **Design phase** — Initial brainstorming captured in `docs/project-notes.md`
|
||||||
|
2. **CLI tools** — `calculate_fees.py` and `match_payments.py` for command-line workflows
|
||||||
|
3. **Bank integration** — `fio_utils.py` for transparent page scraping, later API support
|
||||||
|
4. **Google Sheets sync** — `sync_fio_to_sheets.py` + `infer_payments.py` for the ledger pipeline
|
||||||
|
5. **Web dashboard** — `app.py` with the `/fees`, `/reconcile`, and `/payments` pages
|
||||||
|
6. **Interactive features** — Modal popups, QR payments, keyboard navigation, search filter
|
||||||
|
7. **Fee exceptions** — Manual override system via the `exceptions` sheet tab
|
||||||
|
8. **CI/CD** — Gitea Actions for Docker builds and Kubernetes deployment
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
|
||||||
|
### "No data." on the web dashboard
|
||||||
|
|
||||||
|
The attendance Google Sheet couldn't be fetched, or it returned empty data. Check:
|
||||||
|
- Internet connectivity
|
||||||
|
- The sheet ID in `attendance.py` is still valid
|
||||||
|
- The sheet's public sharing settings haven't changed
|
||||||
|
|
||||||
|
### Slow page loads
|
||||||
|
|
||||||
|
Each page fetches data from Google Sheets on every request (no caching). Typical load times are 1-3 seconds. If significantly slower:
|
||||||
|
- Check the performance breakdown (click the render time in the footer)
|
||||||
|
- Google Sheets API rate limiting may be the cause
|
||||||
|
|
||||||
|
### Import errors in scripts
|
||||||
|
|
||||||
|
Ensure `PYTHONPATH` includes the `scripts/` directory:
|
||||||
|
```bash
|
||||||
|
export PYTHONPATH=scripts:$PYTHONPATH
|
||||||
|
```
|
||||||
|
|
||||||
|
Or use the Makefile, which sets this automatically.
|
||||||
|
|
||||||
|
### "Could not fetch exceptions" warning
|
||||||
|
|
||||||
|
The `exceptions` tab doesn't exist in the Payments Google Sheet. This is non-fatal — reconciliation proceeds without fee overrides.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
*Development guide generated from comprehensive code analysis on 2026-03-03.*
|
||||||
325
docs/by-claude-opus/scripts.md
Normal file
325
docs/by-claude-opus/scripts.md
Normal file
@@ -0,0 +1,325 @@
|
|||||||
|
# Scripts Reference
|
||||||
|
|
||||||
|
All scripts live in the `scripts/` directory and are invoked via `make` targets or directly with Python.
|
||||||
|
|
||||||
|
## Pipeline Scripts
|
||||||
|
|
||||||
|
These scripts form the core data processing pipeline. They are typically run in sequence:
|
||||||
|
|
||||||
|
### `sync_fio_to_sheets.py` — Bank → Google Sheet
|
||||||
|
|
||||||
|
Syncs incoming Fio bank transactions to the Payments Google Sheet. Implements an append-only, deduplicated sync — re-running is always safe.
|
||||||
|
|
||||||
|
**Usage**:
|
||||||
|
```bash
|
||||||
|
make sync # Last 30 days
|
||||||
|
make sync-2026 # Full year 2026 (Jan 1 – Dec 31, sorted)
|
||||||
|
|
||||||
|
# Direct invocation with options:
|
||||||
|
python scripts/sync_fio_to_sheets.py \
|
||||||
|
--credentials .secret/fuj-management-bot-credentials.json \
|
||||||
|
--from 2026-01-01 --to 2026-03-01 \
|
||||||
|
--sort-by-date
|
||||||
|
```
|
||||||
|
|
||||||
|
**Arguments**:
|
||||||
|
|
||||||
|
| Argument | Default | Description |
|
||||||
|
|----------|---------|-------------|
|
||||||
|
| `--days` | `30` | Days to look back (ignored if `--from`/`--to` set) |
|
||||||
|
| `--sheet-id` | Built-in ID | Target Google Sheet |
|
||||||
|
| `--credentials` | `credentials.json` | Path to Google API credentials |
|
||||||
|
| `--from` | *(auto)* | Start date (YYYY-MM-DD) |
|
||||||
|
| `--to` | *(auto)* | End date (YYYY-MM-DD) |
|
||||||
|
| `--sort-by-date` | `false` | Sort the entire sheet by date after sync |
|
||||||
|
|
||||||
|
**How it works**:
|
||||||
|
|
||||||
|
1. Reads existing Sync IDs (column K) from the Google Sheet
|
||||||
|
2. Fetches transactions from Fio bank (API or transparent page scraping)
|
||||||
|
3. For each transaction, generates a SHA-256 hash: `sha256(date|amount|currency|sender|vs|message|bank_id)`
|
||||||
|
4. Appends only transactions whose hash doesn't exist in the sheet
|
||||||
|
5. Optionally sorts the sheet by date
|
||||||
|
|
||||||
|
**Key functions**:
|
||||||
|
|
||||||
|
| Function | Signature | Description |
|
||||||
|
|----------|-----------|-------------|
|
||||||
|
| `get_sheets_service` | `(credentials_path: str) → Resource` | Authenticates with Google Sheets API. Supports both service accounts and OAuth2 flows. |
|
||||||
|
| `generate_sync_id` | `(tx: dict) → str` | Creates the SHA-256 deduplication hash for a transaction. |
|
||||||
|
| `sort_sheet_by_date` | `(service, spreadsheet_id)` | Sorts all rows (excluding header) by the Date column. |
|
||||||
|
| `sync_to_sheets` | `(spreadsheet_id, credentials_path, ...)` | Main sync logic — read existing, fetch new, deduplicate, append. |
|
||||||
|
|
||||||
|
**Output example**:
|
||||||
|
```
|
||||||
|
Connecting to Google Sheets using .secret/fuj-management-bot-credentials.json...
|
||||||
|
Reading existing sync IDs from sheet...
|
||||||
|
Fetching Fio transactions from 2026-02-01 to 2026-03-03...
|
||||||
|
Found 15 transactions.
|
||||||
|
Appending 3 new transactions to the sheet...
|
||||||
|
Sync completed successfully.
|
||||||
|
Sheet sorted by date.
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### `infer_payments.py` — Auto-Fill Person/Purpose
|
||||||
|
|
||||||
|
Scans the Payments Google Sheet for rows with empty Person/Purpose columns and uses name matching and Czech month parsing to fill them automatically.
|
||||||
|
|
||||||
|
**Usage**:
|
||||||
|
```bash
|
||||||
|
make infer
|
||||||
|
|
||||||
|
# Dry run (preview without writing):
|
||||||
|
python scripts/infer_payments.py \
|
||||||
|
--credentials .secret/fuj-management-bot-credentials.json \
|
||||||
|
--dry-run
|
||||||
|
```
|
||||||
|
|
||||||
|
**Arguments**:
|
||||||
|
|
||||||
|
| Argument | Default | Description |
|
||||||
|
|----------|---------|-------------|
|
||||||
|
| `--sheet-id` | Built-in ID | Target Google Sheet |
|
||||||
|
| `--credentials` | `credentials.json` | Path to Google API credentials |
|
||||||
|
| `--dry-run` | `false` | Print inferences without writing to the sheet |
|
||||||
|
|
||||||
|
**How it works**:
|
||||||
|
|
||||||
|
1. Reads all rows from the Payments Google Sheet
|
||||||
|
2. Fetches the member list from the Attendance Sheet
|
||||||
|
3. For each row where Person AND Purpose are empty AND there's no "manual fix":
|
||||||
|
- Combines sender name + message text
|
||||||
|
- Attempts to match against member names (using name variants and diacritics normalization)
|
||||||
|
- Parses Czech month references from the message
|
||||||
|
- Writes inferred Person, Purpose, and Amount back to the sheet
|
||||||
|
4. Low-confidence matches are prefixed with `[?]` for manual review
|
||||||
|
|
||||||
|
**Skipping rules**:
|
||||||
|
- If `manual fix` column has any value → skip
|
||||||
|
- If `Person` column already has a value → skip
|
||||||
|
- If `Purpose` column already has a value → skip
|
||||||
|
|
||||||
|
**Output example**:
|
||||||
|
```
|
||||||
|
Connecting to Google Sheets...
|
||||||
|
Reading sheet data...
|
||||||
|
Fetching member list for matching...
|
||||||
|
Inffering details for empty rows...
|
||||||
|
Row 45: Inferred Jan Novák for 2026-02 (750 CZK)
|
||||||
|
Row 46: Inferred [?] František Vrbík for 2026-01, 2026-02 (1500 CZK)
|
||||||
|
Applying 2 updates to the sheet...
|
||||||
|
Update completed successfully.
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### `match_payments.py` — Reconciliation Engine + CLI Report
|
||||||
|
|
||||||
|
The core reconciliation engine. Matches payment transactions against expected fees and generates a detailed report. Also used as a library by `app.py` and `infer_payments.py`.
|
||||||
|
|
||||||
|
**Usage**:
|
||||||
|
```bash
|
||||||
|
make reconcile
|
||||||
|
|
||||||
|
# Direct invocation:
|
||||||
|
python scripts/match_payments.py \
|
||||||
|
--credentials .secret/fuj-management-bot-credentials.json \
|
||||||
|
--sheet-id YOUR_SHEET_ID
|
||||||
|
```
|
||||||
|
|
||||||
|
**Arguments**:
|
||||||
|
|
||||||
|
| Argument | Default | Description |
|
||||||
|
|----------|---------|-------------|
|
||||||
|
| `--sheet-id` | Built-in ID | Payments Google Sheet |
|
||||||
|
| `--credentials` | `.secret/fuj-management-bot-credentials.json` | Google API credentials |
|
||||||
|
| `--bank` | `false` | Fetch directly from Fio bank instead of the Google Sheet |
|
||||||
|
|
||||||
|
**Key functions**:
|
||||||
|
|
||||||
|
| Function | Description |
|
||||||
|
|----------|-------------|
|
||||||
|
| `_build_name_variants(name)` | Generates searchable name variants from a member name. E.g., "František Vrbík (Štrúdl)" → `["frantisek vrbik", "strudl", "vrbik", "frantisek"]` |
|
||||||
|
| `match_members(text, member_names)` | Finds members mentioned in text. Returns `(name, confidence)` tuples where confidence is `auto` or `review`. |
|
||||||
|
| `infer_transaction_details(tx, member_names)` | Infers member(s) and month(s) for a single transaction. |
|
||||||
|
| `format_date(val)` | Normalizes dates from Google Sheets (handles serial numbers and strings). |
|
||||||
|
| `fetch_sheet_data(spreadsheet_id, credentials_path)` | Reads all rows from the Payments sheet as a list of dicts. |
|
||||||
|
| `fetch_exceptions(spreadsheet_id, credentials_path)` | Reads fee overrides from the `exceptions` sheet tab. |
|
||||||
|
| `reconcile(members, sorted_months, transactions, exceptions)` | **Core engine**: matches transactions to members/months, calculates balances. |
|
||||||
|
| `print_report(result, sorted_months)` | Prints the CLI reconciliation report. |
|
||||||
|
|
||||||
|
**Name matching strategy**:
|
||||||
|
|
||||||
|
The matching algorithm uses multiple tiers, in order of confidence:
|
||||||
|
|
||||||
|
| Priority | What it checks | Confidence |
|
||||||
|
|----------|---------------|-----------|
|
||||||
|
| 1 | Full name (normalized) found in text | `auto` |
|
||||||
|
| 2 | Both first and last name present (any order) | `auto` |
|
||||||
|
| 3 | Nickname from parentheses matches | `auto` |
|
||||||
|
| 4 | Last name only (≥4 chars, not in common surname list) | `review` |
|
||||||
|
| 5 | First name only (≥3 chars) | `review` |
|
||||||
|
|
||||||
|
**Common surnames excluded from last-name-only matching**: `novak`, `novakova`, `prach`
|
||||||
|
|
||||||
|
If any `auto`-confidence match exists, all `review` matches are discarded.
|
||||||
|
|
||||||
|
**Payment allocation**:
|
||||||
|
|
||||||
|
When a transaction matches multiple members and/or multiple months, the amount is split **evenly** across all allocations:
|
||||||
|
```
|
||||||
|
per_allocation = amount / (num_members × num_months)
|
||||||
|
```
|
||||||
|
|
||||||
|
**CLI report sections**:
|
||||||
|
|
||||||
|
1. **Summary table** — Per-member, per-month grid: `OK`, `UNPAID {amount}`, `{paid}/{expected}`, balance
|
||||||
|
2. **Credits** — Members with positive total balance
|
||||||
|
3. **Debts** — Members with negative total balance
|
||||||
|
4. **Unmatched transactions** — Payments that couldn't be assigned
|
||||||
|
5. **Matched transaction details** — Full breakdown with `[REVIEW]` flags
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### `calculate_fees.py` — Fee Calculation
|
||||||
|
|
||||||
|
Calculates and prints monthly fees in a simple table format.
|
||||||
|
|
||||||
|
**Usage**:
|
||||||
|
```bash
|
||||||
|
make fees
|
||||||
|
```
|
||||||
|
|
||||||
|
**Output example**:
|
||||||
|
```
|
||||||
|
Member | Jan 2026 | Feb 2026
|
||||||
|
-------------------------------------------------------
|
||||||
|
Jan Novák | 750 CZK (4) | 200 CZK (1)
|
||||||
|
Alice Testová | - | 750 CZK (3)
|
||||||
|
-------------------------------------------------------
|
||||||
|
TOTAL | 750 CZK | 950 CZK
|
||||||
|
```
|
||||||
|
|
||||||
|
This is a simpler CLI version of the `/fees` web page. It only shows adults (tier A).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Shared Modules
|
||||||
|
|
||||||
|
### `attendance.py` — Attendance Data & Fee Logic
|
||||||
|
|
||||||
|
Shared module that fetches attendance data from the Google Sheet and computes fees.
|
||||||
|
|
||||||
|
**Constants**:
|
||||||
|
|
||||||
|
| Constant | Value | Description |
|
||||||
|
|----------|-------|-------------|
|
||||||
|
| `SHEET_ID` | `1E2e_gT_K5AwSRCDLDTa2UetZTkHmBOcz0kFbBUNUNBA` | Attendance Google Sheet ID |
|
||||||
|
| `FEE_FULL` | `750` | Monthly fee for 2+ practices |
|
||||||
|
| `FEE_SINGLE` | `200` | Monthly fee for exactly 1 practice |
|
||||||
|
| `COL_NAME` | `0` | Column index for member name |
|
||||||
|
| `COL_TIER` | `1` | Column index for member tier |
|
||||||
|
| `FIRST_DATE_COL` | `3` | First column with date headers |
|
||||||
|
|
||||||
|
**Functions**:
|
||||||
|
|
||||||
|
| Function | Signature | Description |
|
||||||
|
|----------|-----------|-------------|
|
||||||
|
| `fetch_csv` | `() → list[list[str]]` | Downloads the attendance sheet as CSV via its public export URL. No authentication needed. |
|
||||||
|
| `parse_dates` | `(header_row) → list[tuple[int, datetime]]` | Parses `M/D/YYYY` dates from the header row and returns `(column_index, date)` pairs. |
|
||||||
|
| `group_by_month` | `(dates) → dict[str, list[int]]` | Groups column indices by `YYYY-MM` month key. |
|
||||||
|
| `calculate_fee` | `(count: int) → int` | Applies fee rules: 0→0, 1→200, 2+→750 CZK. |
|
||||||
|
| `get_members` | `(rows) → list[tuple[str, str, list[str]]]` | Parses member rows. Stops at `# last line` sentinel. Skips comment rows (starting with `#`). |
|
||||||
|
| `get_members_with_fees` | `() → tuple[list, list[str]]` | Full pipeline: fetch → parse → compute. Returns `(members, sorted_months)` where each member is `(name, tier, {month: (fee, count)})`. |
|
||||||
|
|
||||||
|
**Member tier codes**:
|
||||||
|
|
||||||
|
| Tier | Meaning | Fees? |
|
||||||
|
|------|---------|-------|
|
||||||
|
| `A` | Adult | Yes (200 or 750 CZK) |
|
||||||
|
| `J` | Junior | No (separate sheet) |
|
||||||
|
| `X` | Exempt | No |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### `fio_utils.py` — Fio Bank Integration
|
||||||
|
|
||||||
|
Handles fetching transactions from Fio bank, supporting both API and HTML scraping modes.
|
||||||
|
|
||||||
|
**Functions**:
|
||||||
|
|
||||||
|
| Function | Description |
|
||||||
|
|----------|-------------|
|
||||||
|
| `fetch_transactions(date_from, date_to)` | Main entry point. Uses API if `FIO_API_TOKEN` is set, falls back to transparent page scraping. |
|
||||||
|
| `fetch_transactions_api(token, date_from, date_to)` | Fetches via Fio REST API (JSON). Returns richer data including sender account and stable bank IDs. |
|
||||||
|
| `fetch_transactions_transparent(date_from, date_to, account_id)` | Scrapes the public Fio transparent account HTML page. |
|
||||||
|
| `parse_czech_amount(s)` | Parses Czech currency strings like `"1 500,00 CZK"` to float. |
|
||||||
|
| `parse_czech_date(s)` | Parses `DD.MM.YYYY` or `DD/MM/YYYY` to `YYYY-MM-DD`. |
|
||||||
|
|
||||||
|
**FioTableParser** — A custom `HTMLParser` subclass that extracts transaction rows from the second `<table class="table">` on the Fio transparent page. Column mapping:
|
||||||
|
|
||||||
|
| Index | Column |
|
||||||
|
|-------|--------|
|
||||||
|
| 0 | Date (Datum) |
|
||||||
|
| 1 | Amount (Částka) |
|
||||||
|
| 2 | Type (Typ) |
|
||||||
|
| 3 | Sender name (Název protiúčtu) |
|
||||||
|
| 4 | Message (Zpráva pro příjemce) |
|
||||||
|
| 5 | KS (constant symbol) |
|
||||||
|
| 6 | VS (variable symbol) |
|
||||||
|
| 7 | SS (specific symbol) |
|
||||||
|
| 8 | Note (Poznámka) |
|
||||||
|
|
||||||
|
**Transaction dict format** (returned by all fetch functions):
|
||||||
|
|
||||||
|
```python
|
||||||
|
{
|
||||||
|
"date": "2026-01-15", # YYYY-MM-DD
|
||||||
|
"amount": 750.0, # Float, always positive (outgoing filtered)
|
||||||
|
"sender": "Jan Novák", # Sender name
|
||||||
|
"message": "příspěvek", # Message for recipient
|
||||||
|
"vs": "12345", # Variable symbol
|
||||||
|
"ks": "", # Constant symbol
|
||||||
|
"ss": "", # Specific symbol
|
||||||
|
"bank_id": "abc123", # Bank operation ID (API only)
|
||||||
|
"user_id": "...", # User identification (API only)
|
||||||
|
"sender_account": "...", # Sender account number (API only)
|
||||||
|
"currency": "CZK" # Currency (API only)
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### `czech_utils.py` — Czech Language Utilities
|
||||||
|
|
||||||
|
Text processing utilities for Czech language content, critical for matching payment messages.
|
||||||
|
|
||||||
|
**`normalize(text: str) → str`**
|
||||||
|
|
||||||
|
Strips diacritics and lowercases text using Unicode NFKD normalization:
|
||||||
|
- `"Štrúdl"` → `"strudl"`
|
||||||
|
- `"František Vrbík"` → `"frantisek vrbik"`
|
||||||
|
- `"LEDEN 2026"` → `"leden 2026"`
|
||||||
|
|
||||||
|
**`parse_month_references(text: str, default_year=2026) → list[str]`**
|
||||||
|
|
||||||
|
Extracts YYYY-MM month references from Czech free text. Handles a remarkable variety of formats:
|
||||||
|
|
||||||
|
| Input | Output | Pattern |
|
||||||
|
|-------|--------|---------|
|
||||||
|
| `"leden"` | `["2026-01"]` | Czech month name |
|
||||||
|
| `"ledna"` | `["2026-01"]` | Czech month declension |
|
||||||
|
| `"01/26"` | `["2026-01"]` | Numeric short year |
|
||||||
|
| `"1/2026"` | `["2026-01"]` | Numeric full year |
|
||||||
|
| `"11+12/2025"` | `["2025-11", "2025-12"]` | Multiple slash-separated |
|
||||||
|
| `"12.2025"` | `["2025-12"]` | Dot notation |
|
||||||
|
| `"listopad-leden"` | `["2025-11", "2025-12", "2026-01"]` | Range with year wrap |
|
||||||
|
| `"říjen"` | `["2025-10"]` | Months ≥ October assumed previous year |
|
||||||
|
|
||||||
|
**`CZECH_MONTHS`** — Dictionary mapping all Czech month name forms (nominative, genitive, locative) to month numbers. 35 entries covering all 12 months in multiple declensions.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
*Scripts reference generated from comprehensive code analysis on 2026-03-03.*
|
||||||
145
docs/by-claude-opus/testing.md
Normal file
145
docs/by-claude-opus/testing.md
Normal file
@@ -0,0 +1,145 @@
|
|||||||
|
# Testing Guide
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
The project uses Python's built-in `unittest` framework with `unittest.mock` for mocking external dependencies (Google Sheets API, attendance data). Tests live in the `tests/` directory.
|
||||||
|
|
||||||
|
## Running Tests
|
||||||
|
|
||||||
|
```bash
|
||||||
|
make test # Run all tests
|
||||||
|
make test-v # Run with verbose output
|
||||||
|
```
|
||||||
|
|
||||||
|
Under the hood:
|
||||||
|
```bash
|
||||||
|
PYTHONPATH=scripts:. python3 -m unittest discover tests
|
||||||
|
```
|
||||||
|
|
||||||
|
The `PYTHONPATH` includes both `scripts/` and the project root so that test files can import from both `app.py` and `scripts/*.py`.
|
||||||
|
|
||||||
|
## Test Files
|
||||||
|
|
||||||
|
### `test_app.py` — Flask Route Tests
|
||||||
|
|
||||||
|
Tests the Flask web application routes using Flask's built-in test client. All external data fetching is mocked.
|
||||||
|
|
||||||
|
| Test | What it verifies |
|
||||||
|
|------|-----------------|
|
||||||
|
| `test_index_page` | `GET /` returns 200 and contains a redirect to `/fees` |
|
||||||
|
| `test_fees_route` | `GET /fees` renders the fees dashboard with correct member names |
|
||||||
|
| `test_reconcile_route` | `GET /reconcile` renders the reconciliation page with payment matching |
|
||||||
|
| `test_payments_route` | `GET /payments` renders the ledger with grouped transactions |
|
||||||
|
|
||||||
|
**Mocking strategy**:
|
||||||
|
|
||||||
|
```python
|
||||||
|
@patch('app.get_members_with_fees')
|
||||||
|
def test_fees_route(self, mock_get_members):
|
||||||
|
mock_get_members.return_value = (
|
||||||
|
[('Test Member', 'A', {'2026-01': (750, 4)})],
|
||||||
|
['2026-01']
|
||||||
|
)
|
||||||
|
response = self.client.get('/fees')
|
||||||
|
self.assertEqual(response.status_code, 200)
|
||||||
|
self.assertIn(b'Test Member', response.data)
|
||||||
|
```
|
||||||
|
|
||||||
|
Each test patches the data-fetching functions (`get_members_with_fees`, `fetch_sheet_data`) to return controlled test data, avoiding any network calls.
|
||||||
|
|
||||||
|
**Notable**: The reconcile route test also mocks `fetch_sheet_data` and verifies that the reconciliation engine correctly matches a payment against an expected fee (checking for "OK" in the response).
|
||||||
|
|
||||||
|
### `test_reconcile_exceptions.py` — Reconciliation Logic Tests
|
||||||
|
|
||||||
|
Tests the `reconcile()` function directly (unit tests for the core business logic):
|
||||||
|
|
||||||
|
| Test | What it verifies |
|
||||||
|
|------|-----------------|
|
||||||
|
| `test_reconcile_applies_exceptions` | When a fee exception exists (400 CZK instead of 750), the expected amount is overridden and balance is calculated correctly |
|
||||||
|
| `test_reconcile_fallback_to_attendance` | When no exception exists, the attendance-based fee is used |
|
||||||
|
|
||||||
|
**Why these tests matter**: The exception system is critical for correctness — an incorrect override could cause members to be shown incorrect amounts owed. These tests verify that:
|
||||||
|
- Exceptions properly override the attendance-based fee
|
||||||
|
- The absence of an exception correctly falls back to the standard calculation
|
||||||
|
- Balances are computed correctly against overridden amounts
|
||||||
|
|
||||||
|
## Test Data Patterns
|
||||||
|
|
||||||
|
The tests use minimal but representative data:
|
||||||
|
|
||||||
|
```python
|
||||||
|
# A member with attendance-based fee
|
||||||
|
members = [('Alice', 'A', {'2026-01': (750, 4)})]
|
||||||
|
|
||||||
|
# An exception reducing the fee
|
||||||
|
exceptions = {('alice', '2026-01'): {'amount': 400, 'note': 'Test exception'}}
|
||||||
|
|
||||||
|
# A matching payment
|
||||||
|
transactions = [{
|
||||||
|
'date': '2026-01-05',
|
||||||
|
'amount': 400,
|
||||||
|
'person': 'Alice',
|
||||||
|
'purpose': '2026-01',
|
||||||
|
'inferred_amount': 400,
|
||||||
|
'sender': 'Alice Sender',
|
||||||
|
'message': 'fee'
|
||||||
|
}]
|
||||||
|
```
|
||||||
|
|
||||||
|
## What's Not Tested
|
||||||
|
|
||||||
|
| Area | Status | Notes |
|
||||||
|
|------|--------|-------|
|
||||||
|
| Name matching logic | ❌ Not tested | `match_members()`, `_build_name_variants()` |
|
||||||
|
| Czech month parsing | ❌ Not tested | `parse_month_references()` |
|
||||||
|
| Fio bank data fetching | ❌ Not tested | Both API and HTML scraping |
|
||||||
|
| Sync deduplication | ❌ Not tested | `generate_sync_id()` |
|
||||||
|
| QR code generation | ❌ Not tested | `/qr` route |
|
||||||
|
| Payment inference | ❌ Not tested | `infer_payments.py` logic |
|
||||||
|
| Multi-person payment splitting | ❌ Not tested | Even split across members/months |
|
||||||
|
| Edge cases | ❌ Not tested | Empty sheets, malformed dates, etc. |
|
||||||
|
|
||||||
|
## Writing New Tests
|
||||||
|
|
||||||
|
### Adding a Flask route test
|
||||||
|
|
||||||
|
```python
|
||||||
|
# In tests/test_app.py
|
||||||
|
|
||||||
|
@patch('app.some_function')
|
||||||
|
def test_new_route(self, mock_fn):
|
||||||
|
mock_fn.return_value = expected_data
|
||||||
|
response = self.client.get('/new-route')
|
||||||
|
self.assertEqual(response.status_code, 200)
|
||||||
|
self.assertIn(b'expected content', response.data)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Adding a reconciliation logic test
|
||||||
|
|
||||||
|
```python
|
||||||
|
# In tests/test_reconcile_exceptions.py (or a new test file)
|
||||||
|
|
||||||
|
def test_multi_month_payment(self):
|
||||||
|
members = [('Bob', 'A', {
|
||||||
|
'2026-01': (750, 3),
|
||||||
|
'2026-02': (750, 4)
|
||||||
|
})]
|
||||||
|
transactions = [{
|
||||||
|
'date': '2026-02-01',
|
||||||
|
'amount': 1500,
|
||||||
|
'person': 'Bob',
|
||||||
|
'purpose': '2026-01, 2026-02',
|
||||||
|
'inferred_amount': 1500,
|
||||||
|
'sender': 'Bob',
|
||||||
|
'message': 'leden+unor'
|
||||||
|
}]
|
||||||
|
result = reconcile(members, ['2026-01', '2026-02'], transactions)
|
||||||
|
bob = result['members']['Bob']
|
||||||
|
self.assertEqual(bob['months']['2026-01']['paid'], 750)
|
||||||
|
self.assertEqual(bob['months']['2026-02']['paid'], 750)
|
||||||
|
self.assertEqual(bob['total_balance'], 0)
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
*Testing documentation generated from comprehensive code analysis on 2026-03-03.*
|
||||||
166
docs/by-claude-opus/user-guide.md
Normal file
166
docs/by-claude-opus/user-guide.md
Normal file
@@ -0,0 +1,166 @@
|
|||||||
|
# User Guide — FUJ Web Dashboard
|
||||||
|
|
||||||
|
## Getting Started
|
||||||
|
|
||||||
|
Start the dashboard with:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
make web
|
||||||
|
```
|
||||||
|
|
||||||
|
The dashboard is available at **http://localhost:5001** and provides three pages accessible via the green navigation bar at the top.
|
||||||
|
|
||||||
|
## Page 1: Attendance & Fees (`/fees`)
|
||||||
|
|
||||||
|
This page answers the question: **"How much does each member owe this month?"**
|
||||||
|
|
||||||
|
### What You See
|
||||||
|
|
||||||
|
A table with one row per adult member and one column per month. Each cell shows:
|
||||||
|
|
||||||
|
| Cell | Meaning |
|
||||||
|
|------|---------|
|
||||||
|
| `750 CZK (4)` | Member owes 750 CZK (attended 4 practices that month) |
|
||||||
|
| `200 CZK (1)` | Member owes 200 CZK (attended 1 practice) |
|
||||||
|
| `-` | Member didn't attend — no fee |
|
||||||
|
| `400 (750) CZK (3)` | Fee **overridden** from 750 to 400 CZK (shown in orange) |
|
||||||
|
|
||||||
|
The bottom row shows **monthly totals** — the total amount expected from all adult members.
|
||||||
|
|
||||||
|
### Fee Rules
|
||||||
|
|
||||||
|
| Practices in a month | Monthly fee |
|
||||||
|
|----------------------|-------------|
|
||||||
|
| 0 | 0 CZK (no charge) |
|
||||||
|
| 1 | 200 CZK |
|
||||||
|
| 2 or more | 750 CZK |
|
||||||
|
|
||||||
|
### Source Links
|
||||||
|
|
||||||
|
At the top, you'll find direct links to:
|
||||||
|
- **Attendance Sheet** — the Google Sheet with raw attendance data
|
||||||
|
- **Payments Ledger** — the Google Sheet with bank transactions
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Page 2: Payment Reconciliation (`/reconcile`)
|
||||||
|
|
||||||
|
This page answers: **"Who has paid, who hasn't, and who owes extra?"**
|
||||||
|
|
||||||
|
### Main Table
|
||||||
|
|
||||||
|
Each cell in the matrix shows the payment status for a member × month combination:
|
||||||
|
|
||||||
|
| Cell | Color | Meaning |
|
||||||
|
|------|-------|---------|
|
||||||
|
| `OK` | 🟢 Green | Fully paid |
|
||||||
|
| `UNPAID 750` | 🔴 Red | Haven't paid at all |
|
||||||
|
| `300/750` | 🔴 Red | Partially paid (300 out of 750) |
|
||||||
|
| `-` | Gray | No fee expected |
|
||||||
|
| `PAID 200` | — | Payment received but no fee expected |
|
||||||
|
|
||||||
|
The rightmost column shows each member's **total balance**:
|
||||||
|
- **Positive** (green): Member has overpaid / has credit
|
||||||
|
- **Negative** (red): Member still owes money
|
||||||
|
- **Zero**: Fully settled
|
||||||
|
|
||||||
|
### Search Filter
|
||||||
|
|
||||||
|
Type in the search box at the top to filter members by name. The search is **diacritic-insensitive** — typing "novak" will match "Novák".
|
||||||
|
|
||||||
|
### Member Details
|
||||||
|
|
||||||
|
Click the **`[i]`** icon next to any member's name to open a detailed popup:
|
||||||
|
|
||||||
|
1. **Status Summary** — Month-by-month breakdown with attendance count, expected fee, paid amount, and status. Overridden fees are marked with an amber asterisk.
|
||||||
|
|
||||||
|
2. **Fee Exceptions** — If any months have manual fee overrides, they're listed here with the override amount and reason.
|
||||||
|
|
||||||
|
3. **Payment History** — Every bank transaction matched to this member, showing the date, amount, sender, and payment message.
|
||||||
|
|
||||||
|
**Keyboard shortcuts** (when the popup is open):
|
||||||
|
- `↑` / `↓` — Navigate to the previous/next member
|
||||||
|
- `Escape` — Close the popup
|
||||||
|
|
||||||
|
### QR Code Payments
|
||||||
|
|
||||||
|
When you hover over an unpaid or partially paid cell, a red **"Pay"** button appears. Clicking it opens a QR code that can be scanned with any Czech banking app. The QR code is pre-filled with:
|
||||||
|
|
||||||
|
- The club's bank account number
|
||||||
|
- The exact amount owed
|
||||||
|
- A payment message identifying the member and month
|
||||||
|
|
||||||
|
This makes it trivial to send a payment link to a member who owes money.
|
||||||
|
|
||||||
|
### Summary Sections
|
||||||
|
|
||||||
|
Below the main table, three additional sections may appear:
|
||||||
|
|
||||||
|
| Section | Shows |
|
||||||
|
|---------|-------|
|
||||||
|
| **Credits** | Members with positive balances (advance payments or overpayments) |
|
||||||
|
| **Debts** | Members with negative balances (outstanding fees) |
|
||||||
|
| **Unmatched Transactions** | Bank transactions that couldn't be automatically matched to any member |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Page 3: Payments Ledger (`/payments`)
|
||||||
|
|
||||||
|
This page answers: **"What payments has each member made?"**
|
||||||
|
|
||||||
|
### What You See
|
||||||
|
|
||||||
|
Transactions grouped by member name, each showing:
|
||||||
|
- **Date** — When the payment was received
|
||||||
|
- **Amount** — How much was paid (in CZK)
|
||||||
|
- **Purpose** — Which month(s) the payment covers
|
||||||
|
- **Bank Message** — The original message from the bank transfer
|
||||||
|
|
||||||
|
Transactions are sorted newest-first within each member's section.
|
||||||
|
|
||||||
|
### Unmatched Payments
|
||||||
|
|
||||||
|
Transactions that couldn't be assigned to a member appear under **"Unmatched / Unknown"** — these typically need manual review in the Google Sheet.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Performance Footer
|
||||||
|
|
||||||
|
Every page shows a **render time** in the bottom-right corner (very small, gray text). This tells you how long the page took to generate.
|
||||||
|
|
||||||
|
Click on it to reveal a detailed breakdown showing how much time was spent on each step (fetching members, fetching payments, reconciliation, template rendering, etc.). This is mostly useful for debugging slow page loads.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Common Workflows
|
||||||
|
|
||||||
|
### "A member asks how much they owe"
|
||||||
|
|
||||||
|
1. Open `/reconcile`
|
||||||
|
2. Search for the member's name
|
||||||
|
3. Their row shows the exact status per month
|
||||||
|
4. Click `[i]` for detailed payment history
|
||||||
|
|
||||||
|
### "A member wants to pay"
|
||||||
|
|
||||||
|
1. Open `/reconcile`
|
||||||
|
2. Find the unpaid cell
|
||||||
|
3. Hover and click the red **Pay** button
|
||||||
|
4. Share the QR code with the member (screenshot or show on screen)
|
||||||
|
|
||||||
|
### "I want to see all payments from one person"
|
||||||
|
|
||||||
|
1. Open `/payments`
|
||||||
|
2. Scroll to the member's section (alphabetically sorted)
|
||||||
|
|
||||||
|
### "A transaction wasn't matched correctly"
|
||||||
|
|
||||||
|
1. Open the **Payments Ledger** Google Sheet (link at the top of any page)
|
||||||
|
2. Find the row
|
||||||
|
3. Manually correct the **Person** and/or **Purpose** columns
|
||||||
|
4. Put any marker in the **manual fix** column to prevent the inference script from overwriting your edit
|
||||||
|
5. Refresh the web dashboard
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
*User guide generated from comprehensive code analysis on 2026-03-03.*
|
||||||
256
docs/by-claude-opus/web-app.md
Normal file
256
docs/by-claude-opus/web-app.md
Normal file
@@ -0,0 +1,256 @@
|
|||||||
|
# Web Application Documentation
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
The FUJ Management web application is a Flask-based dashboard that provides real-time visibility into club finances. It renders server-side HTML with embedded CSS and JavaScript — no build tools, no npm, no framework. The UI follows a distinctive **terminal-inspired aesthetic** with monospace fonts, green-on-black colors, and dashed borders.
|
||||||
|
|
||||||
|
## Routes
|
||||||
|
|
||||||
|
### `GET /` — Index (Redirect)
|
||||||
|
|
||||||
|
Redirects to `/fees` via an HTML meta refresh tag. This exists so the root URL always leads somewhere useful.
|
||||||
|
|
||||||
|
### `GET /fees` — Attendance & Fees Dashboard
|
||||||
|
|
||||||
|
**Template**: `templates/fees.html`
|
||||||
|
|
||||||
|
Displays a table of all adult members with their calculated monthly fees based on attendance. Each cell shows the fee amount (in CZK), the number of practices attended, or a dash for months with zero attendance.
|
||||||
|
|
||||||
|
**Data pipeline**:
|
||||||
|
```
|
||||||
|
attendance.py::get_members_with_fees() → Filter to tier "A" (adults)
|
||||||
|
match_payments.py::fetch_exceptions() → Check for fee overrides
|
||||||
|
→ Format cells with override indicators
|
||||||
|
→ Render fees.html with totals row
|
||||||
|
```
|
||||||
|
|
||||||
|
**Visual features**:
|
||||||
|
- Fee overrides shown in **orange** with the original amount in parentheses
|
||||||
|
- Empty months shown in muted gray
|
||||||
|
- Monthly totals row at the bottom
|
||||||
|
- Performance timing in the footer (click to expand breakdown)
|
||||||
|
|
||||||
|
**Template variables**:
|
||||||
|
|
||||||
|
| Variable | Type | Content |
|
||||||
|
|----------|------|---------|
|
||||||
|
| `months` | `list[str]` | Month labels like "Jan 2026" |
|
||||||
|
| `results` | `list[dict]` | `{name, months: [{cell, overridden}]}` |
|
||||||
|
| `totals` | `list[str]` | Monthly total strings like "3750 CZK" |
|
||||||
|
| `attendance_url` | `str` | Link to the attendance Google Sheet |
|
||||||
|
| `payments_url` | `str` | Link to the payments Google Sheet |
|
||||||
|
|
||||||
|
### `GET /reconcile` — Payment Reconciliation
|
||||||
|
|
||||||
|
**Template**: `templates/reconcile.html` (802 lines — the most complex template)
|
||||||
|
|
||||||
|
The centerpiece of the application. Shows a matrix of members × months with payment status, plus summary sections for credits, debts, and unmatched transactions.
|
||||||
|
|
||||||
|
**Data pipeline**:
|
||||||
|
```
|
||||||
|
attendance.py::get_members_with_fees() → All members + fees
|
||||||
|
match_payments.py::fetch_sheet_data() → All payment transactions
|
||||||
|
match_payments.py::fetch_exceptions() → Fee overrides
|
||||||
|
match_payments.py::reconcile() → Match payments ↔ fees
|
||||||
|
→ Render reconcile.html
|
||||||
|
```
|
||||||
|
|
||||||
|
**Cell statuses**:
|
||||||
|
|
||||||
|
| Status | CSS Class | Display | Meaning |
|
||||||
|
|--------|-----------|---------|---------|
|
||||||
|
| `empty` | `cell-empty` | `-` | No fee expected, no payment |
|
||||||
|
| `ok` | `cell-ok` | `OK` | Paid in full (green) |
|
||||||
|
| `partial` | `cell-unpaid` | `300/750` | Partially paid (red) |
|
||||||
|
| `unpaid` | `cell-unpaid` | `UNPAID 750` | Nothing paid (red) |
|
||||||
|
| `surplus` | — | `PAID 200` | Payment received but no fee expected |
|
||||||
|
|
||||||
|
**Interactive features**:
|
||||||
|
|
||||||
|
1. **Member detail modal** — Click the `[i]` icon next to any member name to see:
|
||||||
|
- Status summary table (month, attendance count, expected, paid, status)
|
||||||
|
- Fee exceptions (if any, shown in amber)
|
||||||
|
- Full payment history with dates, amounts, senders, and messages
|
||||||
|
|
||||||
|
2. **Keyboard navigation** — When a member modal is open:
|
||||||
|
- `↑` / `↓` arrows navigate between members (respecting search filter)
|
||||||
|
- `Escape` closes the modal
|
||||||
|
|
||||||
|
3. **Name search filter** — Type in the search box to filter members. Uses diacritic-insensitive matching (e.g., typing "novak" matches "Novák").
|
||||||
|
|
||||||
|
4. **QR Payment** — Hover over an unpaid/partial cell to reveal a "Pay" button. Clicking it opens a QR code modal with:
|
||||||
|
- A Czech SPD-format QR code (scannable by Czech banking apps)
|
||||||
|
- Pre-filled account number, amount, and payment message
|
||||||
|
- The QR image is generated server-side via `GET /qr`
|
||||||
|
|
||||||
|
**Client-side data**:
|
||||||
|
|
||||||
|
The template receives a full JSON dump of member data (`member_data`) embedded in a `<script>` tag. This powers the modal without additional API calls:
|
||||||
|
|
||||||
|
```javascript
|
||||||
|
const memberData = {{ member_data | safe }};
|
||||||
|
const sortedMonths = {{ raw_months | tojson }};
|
||||||
|
```
|
||||||
|
|
||||||
|
**Summary sections** (rendered below the main table):
|
||||||
|
|
||||||
|
| Section | Shown when | Content |
|
||||||
|
|---------|-----------|---------|
|
||||||
|
| Credits | Any member has positive balance | Names with surplus amounts |
|
||||||
|
| Debts | Any member has negative balance | Names with outstanding amounts (red) |
|
||||||
|
| Unmatched Transactions | Any transaction couldn't be matched | Date, amount, sender, message |
|
||||||
|
|
||||||
|
### `GET /payments` — Payments Ledger
|
||||||
|
|
||||||
|
**Template**: `templates/payments.html`
|
||||||
|
|
||||||
|
Displays all bank transactions grouped by member name. Each member section shows their transactions in reverse chronological order.
|
||||||
|
|
||||||
|
**Data pipeline**:
|
||||||
|
```
|
||||||
|
match_payments.py::fetch_sheet_data() → All transactions
|
||||||
|
→ Group by Person column
|
||||||
|
→ Strip [?] markers
|
||||||
|
→ Handle comma-separated people
|
||||||
|
→ Sort by date descending
|
||||||
|
→ Render payments.html
|
||||||
|
```
|
||||||
|
|
||||||
|
**Multi-person handling**: If a transaction's "Person" field contains comma-separated names (e.g., "Alice, Bob"), the transaction appears under both Alice's and Bob's sections.
|
||||||
|
|
||||||
|
### `GET /qr` — QR Code Generator
|
||||||
|
|
||||||
|
Returns a PNG image containing a Czech SPD (Short Payment Descriptor) QR code.
|
||||||
|
|
||||||
|
**Query parameters**:
|
||||||
|
|
||||||
|
| Parameter | Default | Description |
|
||||||
|
|-----------|---------|-------------|
|
||||||
|
| `account` | `BANK_ACCOUNT` env var | IBAN or Czech account number |
|
||||||
|
| `amount` | `0` | Payment amount |
|
||||||
|
| `message` | *(empty)* | Payment message (max 60 chars) |
|
||||||
|
|
||||||
|
**SPD format**: `SPD*1.0*ACC:{account}*AM:{amount}*CC:CZK*MSG:{message}`
|
||||||
|
|
||||||
|
This format is recognized by all Czech banking apps and generates a pre-filled payment order when scanned.
|
||||||
|
|
||||||
|
## UI Design System
|
||||||
|
|
||||||
|
### Color Palette
|
||||||
|
|
||||||
|
| Element | Color | Hex |
|
||||||
|
|---------|-------|-----|
|
||||||
|
| Background | Near-black | `#0c0c0c` |
|
||||||
|
| Base text | Medium gray | `#cccccc` |
|
||||||
|
| Headings, accents, "OK" | Terminal green | `#00ff00` |
|
||||||
|
| Unpaid, debts | Alert red | `#ff3333` |
|
||||||
|
| Fee overrides | Amber/orange | `#ffa500` / `#ffaa00` |
|
||||||
|
| Empty/muted | Dark gray | `#444444` |
|
||||||
|
| Borders | Subtle gray | `#333` (dashed), `#555` (solid) |
|
||||||
|
|
||||||
|
### Typography
|
||||||
|
|
||||||
|
All text uses the system monospace font stack:
|
||||||
|
```css
|
||||||
|
font-family: ui-monospace, SFMono-Regular, Menlo, Monaco, Consolas,
|
||||||
|
"Liberation Mono", "Courier New", monospace;
|
||||||
|
```
|
||||||
|
|
||||||
|
Base font size is 11px with 1.2 line-height — intentionally dense for a data-heavy dashboard.
|
||||||
|
|
||||||
|
### Navigation
|
||||||
|
|
||||||
|
A persistent nav bar appears at the top of every page:
|
||||||
|
```
|
||||||
|
[Attendance/Fees] [Payment Reconciliation] [Payments Ledger]
|
||||||
|
```
|
||||||
|
The active page's link is highlighted with inverted colors (black text on green background).
|
||||||
|
|
||||||
|
### Shared Footer
|
||||||
|
|
||||||
|
Every page includes a click-to-expand performance timer showing total render time and a per-step breakdown.
|
||||||
|
|
||||||
|
## Flask Application Architecture
|
||||||
|
|
||||||
|
### Request Lifecycle
|
||||||
|
|
||||||
|
```
|
||||||
|
Request → @app.before_request (start timer) → Route handler → Template → Response
|
||||||
|
│ │
|
||||||
|
▼ ▼
|
||||||
|
g.start_time record_step("fetch_members")
|
||||||
|
g.steps = [] record_step("fetch_payments")
|
||||||
|
record_step("process_data")
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
@app.context_processor
|
||||||
|
inject_render_time()
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
{{ get_render_time() }}
|
||||||
|
in template footer
|
||||||
|
```
|
||||||
|
|
||||||
|
### Module Loading
|
||||||
|
|
||||||
|
The Flask app adds the `scripts/` directory to `sys.path` at startup, allowing direct imports from scripts:
|
||||||
|
|
||||||
|
```python
|
||||||
|
scripts_dir = Path(__file__).parent / "scripts"
|
||||||
|
sys.path.append(str(scripts_dir))
|
||||||
|
|
||||||
|
from attendance import get_members_with_fees, SHEET_ID
|
||||||
|
from match_payments import reconcile, fetch_sheet_data, fetch_exceptions, normalize
|
||||||
|
```
|
||||||
|
|
||||||
|
### Environment Variables
|
||||||
|
|
||||||
|
| Variable | Default | Purpose |
|
||||||
|
|----------|---------|---------|
|
||||||
|
| `BANK_ACCOUNT` | `CZ8520100000002800359168` | Bank account for QR code generation |
|
||||||
|
| `FIO_API_TOKEN` | *(none)* | Fio API token (used by `fio_utils.py`) |
|
||||||
|
|
||||||
|
### Error Handling
|
||||||
|
|
||||||
|
The application has minimal error handling:
|
||||||
|
- If Google Sheets returns no data, routes return a simple "No data." text response
|
||||||
|
- No custom error pages for 404/500
|
||||||
|
- Exceptions propagate to Flask's default error handler (debug mode in development, 500 in production)
|
||||||
|
|
||||||
|
## Template Architecture
|
||||||
|
|
||||||
|
All three page templates share a common structure:
|
||||||
|
|
||||||
|
```html
|
||||||
|
<!DOCTYPE html>
|
||||||
|
<html lang="en">
|
||||||
|
<head>
|
||||||
|
<meta charset="UTF-8">
|
||||||
|
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||||
|
<title>FUJ [Page Name]</title>
|
||||||
|
<style>
|
||||||
|
/* ALL CSS is inline — no external stylesheets */
|
||||||
|
/* ~150-400 lines of CSS per template */
|
||||||
|
</style>
|
||||||
|
</head>
|
||||||
|
<body>
|
||||||
|
<div class="nav"><!-- 3-link navigation --></div>
|
||||||
|
<h1>Page Title</h1>
|
||||||
|
<div class="description"><!-- Source links --></div>
|
||||||
|
|
||||||
|
<!-- Page-specific content -->
|
||||||
|
|
||||||
|
<div class="footer"><!-- Render time --></div>
|
||||||
|
|
||||||
|
<script>
|
||||||
|
/* Page-specific JavaScript (only in reconcile.html) */
|
||||||
|
</script>
|
||||||
|
</body>
|
||||||
|
</html>
|
||||||
|
```
|
||||||
|
|
||||||
|
There is no shared base template (no Jinja2 template inheritance). CSS is duplicated across templates with small variations.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
*Web application documentation generated from comprehensive code analysis on 2026-03-03.*
|
||||||
36
docs/by-gemini/README.md
Normal file
36
docs/by-gemini/README.md
Normal file
@@ -0,0 +1,36 @@
|
|||||||
|
# FUJ Management System
|
||||||
|
|
||||||
|
Welcome to the **FUJ Management System**, a streamlined solution for managing Ultimate Frisbee club finances, attendance, and member payments. This system automates the tedious parts of club management, keeping your ledger clean and your reconciliation painless.
|
||||||
|
|
||||||
|
## 🚀 Mission
|
||||||
|
|
||||||
|
The project's goal is to minimize manual entry and potential human error in club management by:
|
||||||
|
1. **Automating Bank Synchronization**: Periodically fetching transactions from Fio bank.
|
||||||
|
2. **Smart Inference**: Using heuristics to match bank transactions to members and payment periods.
|
||||||
|
3. **Visual Reconciliation**: Providing a clear, real-time web dashboard for managers to track who has paid and who is in debt.
|
||||||
|
|
||||||
|
## ✨ Key Features
|
||||||
|
|
||||||
|
- **Seamless Bank Integration**: Synchronize transactions directly from the Fio bank API into a Google Spreadsheet.
|
||||||
|
- **Intelligent Matching**: Automatic detection of member names and payment periods from transaction messages using diacritic-insensitive Czech text processing.
|
||||||
|
- **Dynamic Dashboard**: A Flask-powered web interface displaying monthly fees, payment status (OK, Partial, Unpaid), and total balances.
|
||||||
|
- **Manual Overrides**: Support for fee exceptions and manual payment matching when automation needs a human touch.
|
||||||
|
- **QR Payment Generation**: Integrated QR code generation to make paying outstanding fees trivial for members.
|
||||||
|
|
||||||
|
## 🛠 Tech Stack
|
||||||
|
|
||||||
|
- **Backend**: Python 3.12+ (managed with `uv`)
|
||||||
|
- **Web Framework**: Flask with Jinja2 templates
|
||||||
|
- **Data Storage**: Google Sheets (used as a collaborative database)
|
||||||
|
- **APIs**: Fio Bank API, Google Sheets API v4
|
||||||
|
- **Containerization**: Docker / OCI Images
|
||||||
|
- **Automation**: `Makefile` based workflow
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📂 Documentation Guide
|
||||||
|
|
||||||
|
- [Architecture](architecture.md): High-level system design and data flow.
|
||||||
|
- [User Guide](user-guide.md): How to operate the system as a club manager.
|
||||||
|
- [Support Scripts](scripts.md): Detailed reference for CLI tools.
|
||||||
|
- [Deployment](deployment.md): Technical setup and infrastructure instructions.
|
||||||
48
docs/by-gemini/architecture.md
Normal file
48
docs/by-gemini/architecture.md
Normal file
@@ -0,0 +1,48 @@
|
|||||||
|
# System Architecture
|
||||||
|
|
||||||
|
The FUJ Management system is designed around a **"Sheet-as-a-Database"** architecture. This allows for easy manual editing and transparency while enabling powerful automation.
|
||||||
|
|
||||||
|
## 🔄 High-Level Data Flow
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
graph TD
|
||||||
|
Fio[Fio Bank API] -->|Sync Script| GS(Google Spreadsheet)
|
||||||
|
Att[Attendance Sheet] -->|CSV Export| App(Flask Web App)
|
||||||
|
GS -->|API Fetch| App
|
||||||
|
App -->|Display| UI[Manager Dashboard]
|
||||||
|
GS -.->|Manual Edits| GS
|
||||||
|
App -->|Generate| QR[QR Codes for Members]
|
||||||
|
```
|
||||||
|
|
||||||
|
### 1. Data Ingestion (Bank to Sheet)
|
||||||
|
The synchronization pipeline moves raw bank data into a structured format:
|
||||||
|
- `sync_fio_to_sheets.py` fetches transactions and appends them to the "Transactions" sheet.
|
||||||
|
- Each transaction is assigned a unique `Sync ID` to prevent duplicates.
|
||||||
|
- `infer_payments.py` processes new rows, attempting to fill the `Person`, `Purpose`, and `Inferred Amount` columns based on the message and sender.
|
||||||
|
|
||||||
|
### 2. Logic & Reconciliation
|
||||||
|
The core logic resides in shared Python scripts:
|
||||||
|
- **Attendance**: `attendance.py` pulls the latest practice data from a separate attendance sheet and calculates expected fees (e.g., 0/200/750 CZK rules).
|
||||||
|
- **Matching**: `match_payments.py` performs the "heavy lifting" by correlating members, months, and payments. It handles partial payments, overpayments (credits), and manual exceptions.
|
||||||
|
|
||||||
|
### 3. Presentation Layer
|
||||||
|
The Flask application (`app.py`) serves as the primary interface:
|
||||||
|
- **Fees View**: Shows attendance-based charges.
|
||||||
|
- **Reconciliation View**: The main "truth" dashboard showing balance per member.
|
||||||
|
- **Payments View**: Historical list of transactions grouped by member.
|
||||||
|
|
||||||
|
## 🛡 Security & Authentication
|
||||||
|
|
||||||
|
- **Fio Bank**: Authorized via a private API token (kept in `.secret/`).
|
||||||
|
- **Google Sheets**: Authenticated via a **Service Account** or **OAuth2** (using `.secret/fuj-management-bot-credentials.json`).
|
||||||
|
- **Environment**: Secrets are never committed; the `.secret/` directory is git-ignored.
|
||||||
|
|
||||||
|
## 🧩 Key Components
|
||||||
|
|
||||||
|
| Component | Responsibility |
|
||||||
|
| :--- | :--- |
|
||||||
|
| **Google Spreadsheet** | Unified source of truth for transactions and manual overrides. |
|
||||||
|
| **scripts/** | A suite of CLI utilities for batch processing and data maintenance. |
|
||||||
|
| **Flask App** | Read-only views for state visualization and QR code generation. |
|
||||||
|
| **czech_utils.py** | Diacritic-normalization and NLP for Czech month/name parsing. |
|
||||||
|
```
|
||||||
72
docs/by-gemini/deployment.md
Normal file
72
docs/by-gemini/deployment.md
Normal file
@@ -0,0 +1,72 @@
|
|||||||
|
# Deployment & Technical Setup
|
||||||
|
|
||||||
|
This document provides instructions for developers and devops engineers to set up and deploy the FUJ Management system.
|
||||||
|
|
||||||
|
## 🛠 Prerequisites
|
||||||
|
|
||||||
|
- **Python 3.12+**: The project uses modern type hinting and syntax features.
|
||||||
|
- **uv**: High-performance Python package installer and resolver.
|
||||||
|
- Install via brew: `brew install uv`
|
||||||
|
- **Docker** (Optional): For containerized deployments.
|
||||||
|
|
||||||
|
## ⚙️ Initial Setup
|
||||||
|
|
||||||
|
1. **Clone the repository**:
|
||||||
|
```bash
|
||||||
|
git clone <repo-url>
|
||||||
|
cd fuj-management
|
||||||
|
```
|
||||||
|
|
||||||
|
2. **Install dependencies**:
|
||||||
|
Using `uv`, everything is handled automatically:
|
||||||
|
```bash
|
||||||
|
make venv
|
||||||
|
```
|
||||||
|
|
||||||
|
3. **Secrets Management**:
|
||||||
|
Create a `.secret/` directory. You will need two main credentials:
|
||||||
|
- `fuj-management-bot-credentials.json`: A Google Cloud Service Account key with access to the Sheets API.
|
||||||
|
- `fio-token.txt`: (Implicitly used by `fio_utils.py`) Your Fio bank API token.
|
||||||
|
|
||||||
|
Ensure these are never committed! They are ignored by `.gitignore`.
|
||||||
|
|
||||||
|
## 🐳 Containerization
|
||||||
|
|
||||||
|
The project can be built and run as an OCI image.
|
||||||
|
|
||||||
|
1. **Build the image**:
|
||||||
|
```bash
|
||||||
|
make image
|
||||||
|
```
|
||||||
|
This uses the `build/Dockerfile`, which is optimized for small size and security.
|
||||||
|
|
||||||
|
2. **Run the container**:
|
||||||
|
```bash
|
||||||
|
make run
|
||||||
|
```
|
||||||
|
The app exposes port `5001`.
|
||||||
|
|
||||||
|
## 🧪 Testing & Validation
|
||||||
|
|
||||||
|
The project includes a suite of infrastructure and logic tests.
|
||||||
|
|
||||||
|
- **Run all tests**:
|
||||||
|
```bash
|
||||||
|
make test
|
||||||
|
```
|
||||||
|
- **Verbose output**:
|
||||||
|
```bash
|
||||||
|
make test-v
|
||||||
|
```
|
||||||
|
|
||||||
|
Tests are located in the `tests/` directory and use the standard Python `unittest` framework. They cover:
|
||||||
|
- CSV parsing logic.
|
||||||
|
- Fee calculation rules.
|
||||||
|
- Name matching and normalization.
|
||||||
|
|
||||||
|
## 🚀 Future Roadmap
|
||||||
|
|
||||||
|
- **Automated Backups**: Regular snapshots of the Google Sheet.
|
||||||
|
- **Authentication Layer**: Login for the web dashboard (currently assumes internal VPN or trusted environment).
|
||||||
|
- **Gitea Actions**: Continuous Integration for building and testing images.
|
||||||
|
```
|
||||||
66
docs/by-gemini/scripts.md
Normal file
66
docs/by-gemini/scripts.md
Normal file
@@ -0,0 +1,66 @@
|
|||||||
|
# Support Scripts Reference
|
||||||
|
|
||||||
|
The project includes several CLI utilities located in the `scripts/` directory. Most are accessible via `make` targets.
|
||||||
|
|
||||||
|
## 🚀 Primary Scripts
|
||||||
|
|
||||||
|
### `sync_fio_to_sheets.py`
|
||||||
|
**Target**: `make sync` | `make sync-2026`
|
||||||
|
- **Purpose**: Downloads transactions from Fio bank via API and appends new ones to the Google Sheet.
|
||||||
|
- **Key Logic**: Uses a `Sync ID` (SHA-256 hash of transaction details) to ensure that even if the sync is run multiple times, no duplicate rows are created.
|
||||||
|
- **Arguments**:
|
||||||
|
- `--days`: How many days back to look (default 30).
|
||||||
|
- `--from/--to`: Specific date range.
|
||||||
|
- `--sort-by-date`: Re-sorts the spreadsheet after appending.
|
||||||
|
|
||||||
|
### `infer_payments.py`
|
||||||
|
**Target**: `make infer`
|
||||||
|
- **Purpose**: Processes the "Transactions" sheet to fill in `Person`, `Purpose`, and `Inferred Amount`.
|
||||||
|
- **Logic**:
|
||||||
|
- Analyzes the `Sender` and `Message` fields.
|
||||||
|
- Uses `match_payments.py` heuristics to find members.
|
||||||
|
- If confidence is low, prefixes the name with `[?]` to flag it for manual review.
|
||||||
|
- Won't overwrite cells that already have data (respecting your manual fixes).
|
||||||
|
|
||||||
|
### `match_payments.py`
|
||||||
|
**Target**: `make match` | `make reconcile`
|
||||||
|
- **Purpose**: The core "Reconciliation Engine".
|
||||||
|
- **Logic**:
|
||||||
|
- Fetches attendance fees (from `attendance.py`).
|
||||||
|
- Fetches transaction data.
|
||||||
|
- Correlates them based on inferred `Person` and `Purpose`.
|
||||||
|
- Handles "rollover" balances—extra money from one month is tracked as a credit for the next.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🛠 Utility Modules
|
||||||
|
|
||||||
|
### `attendance.py`
|
||||||
|
- Handles the connection to the Google Attendance sheet (exported as CSV).
|
||||||
|
- Implements the club's fee rules:
|
||||||
|
- 0 practices = 0 CZK
|
||||||
|
- 1 practice = 200 CZK
|
||||||
|
- 2+ practices = 750 CZK
|
||||||
|
- *Note*: Fee calculation only applies to members in Tier "A" (Adults).
|
||||||
|
|
||||||
|
### `czech_utils.py`
|
||||||
|
- **Normalization**: Strips diacritics and lowercases text (e.g., `František` -> `frantisek`).
|
||||||
|
- **Month Parsing**: Advanced regex to detect month references in Czech (e.g., "leden-brezen", "11+12/25", "na únor").
|
||||||
|
|
||||||
|
### `fio_utils.py`
|
||||||
|
- Low-level wrapper for the Fio Bank API.
|
||||||
|
- Handles HTTP requests and JSON parsing for transaction lists.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ⚙️ Makefile Summary
|
||||||
|
|
||||||
|
| Command | Action |
|
||||||
|
| :--- | :--- |
|
||||||
|
| `make fees` | Preview calculated fees based on attendance. |
|
||||||
|
| `make sync` | Sync last 30 days of bank data. |
|
||||||
|
| `make infer` | Run smart tagging on the sheet. |
|
||||||
|
| `make reconcile` | Run a text-based reconciliation report in terminal. |
|
||||||
|
| `make web` | Start the Flask dashboard. |
|
||||||
|
| `make test` | Run the test suite. |
|
||||||
|
```
|
||||||
61
docs/by-gemini/user-guide.md
Normal file
61
docs/by-gemini/user-guide.md
Normal file
@@ -0,0 +1,61 @@
|
|||||||
|
# User Guide
|
||||||
|
|
||||||
|
This guide is intended for club managers who use the FUJ Management system for day-to-day operations.
|
||||||
|
|
||||||
|
## 🛠 Operational Workflow
|
||||||
|
|
||||||
|
To keep the club finances up-to-date, follow these steps periodically (e.g., once a week):
|
||||||
|
|
||||||
|
1. **Sync Bank Transactions**:
|
||||||
|
Run the sync script to pull the latest payments from Fio.
|
||||||
|
```bash
|
||||||
|
make sync
|
||||||
|
```
|
||||||
|
2. **Infer Payments**:
|
||||||
|
Let the system automatically tag who paid for what.
|
||||||
|
```bash
|
||||||
|
make infer
|
||||||
|
```
|
||||||
|
3. **Manual Review (Google Sheets)**:
|
||||||
|
Open the Google Spreadsheet. Check rows with the `[?]` prefix in the `Person` column—these require human confirmation.
|
||||||
|
- If correct: Remove the `[?]` prefix.
|
||||||
|
- If incorrect: Manually fix the `Person` and `Purpose`.
|
||||||
|
- If a payment covers a special case: Use the **exceptions** sheet to override expected fees.
|
||||||
|
4. **Check Reconciliation Dashboard**:
|
||||||
|
Start the web app to see the final balance report.
|
||||||
|
```bash
|
||||||
|
make web
|
||||||
|
```
|
||||||
|
Navigate to `http://localhost:5001/reconcile`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📊 Understanding the Dashboard
|
||||||
|
|
||||||
|
### Reconciliation Page
|
||||||
|
- **Green (OK)**: Member has paid exactly what was expected (or more).
|
||||||
|
- **Orange (Partial)**: Some payment was received, but there's still a debt.
|
||||||
|
- **Red (UNPAID)**: No payment recorded for this month.
|
||||||
|
- **Blue (SURPLUS)**: Payment received for a month where no fee was expected.
|
||||||
|
|
||||||
|
### Handling Debts
|
||||||
|
If a member is in debt, you can click on the unpaid/partial cell to get a **QR Platba** link. You can send this link or screenshot to the member to facilitate quick payment.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ❓ FAQ & Troubleshooting
|
||||||
|
|
||||||
|
### Why is a payment "Unmatched"?
|
||||||
|
A payment stays unmatched if neither the sender name nor the message contains recognizable member names or nicknames.
|
||||||
|
- **Fix**: Manually enter the member's name in the `Person` column in the Google Sheet.
|
||||||
|
|
||||||
|
### How do I handle a "Family Discount" or "Prepaid Year"?
|
||||||
|
Use the `exceptions` sheet in the Google Spreadsheet.
|
||||||
|
1. Add the member's name (exactly as it appears in attendance).
|
||||||
|
2. Enter the month (e.g., `2026-03`).
|
||||||
|
3. Enter the new `Amount` (use `0` for prepaid).
|
||||||
|
4. Add a `Note` for clarity.
|
||||||
|
|
||||||
|
### The web app is slow to load.
|
||||||
|
The app fetches data from Google Sheets API on every request. This ensures real-time data but can take a few seconds. The "Performance Breakdown" footer shows exactly where the time was spent.
|
||||||
|
```
|
||||||
43
docs/index.html
Normal file
43
docs/index.html
Normal file
@@ -0,0 +1,43 @@
|
|||||||
|
<!DOCTYPE html>
|
||||||
|
<html lang="en">
|
||||||
|
|
||||||
|
<head>
|
||||||
|
<meta charset="UTF-8">
|
||||||
|
<title>FUJ Management - Documentation</title>
|
||||||
|
<meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1" />
|
||||||
|
<meta name="description" content="Documentation for FUJ Management Application">
|
||||||
|
<meta name="viewport" content="width=device-width, initial-scale=1.0, minimum-scale=1.0">
|
||||||
|
<link rel="stylesheet" href="//cdn.jsdelivr.net/npm/docsify@4/lib/themes/vue.css">
|
||||||
|
<style>
|
||||||
|
:root {
|
||||||
|
--theme-color: #42b983;
|
||||||
|
}
|
||||||
|
|
||||||
|
.sidebar {
|
||||||
|
background-color: #f8f9fa;
|
||||||
|
}
|
||||||
|
</style>
|
||||||
|
</head>
|
||||||
|
|
||||||
|
<body>
|
||||||
|
<div id="app">Loading documentation...</div>
|
||||||
|
<script>
|
||||||
|
window.$docsify = {
|
||||||
|
name: 'FUJ Management',
|
||||||
|
repo: '',
|
||||||
|
loadSidebar: true,
|
||||||
|
subMaxLevel: 2,
|
||||||
|
search: 'auto',
|
||||||
|
auto2top: true,
|
||||||
|
alias: {
|
||||||
|
'/.*/_sidebar.md': '/_sidebar.md'
|
||||||
|
}
|
||||||
|
}
|
||||||
|
</script>
|
||||||
|
<!-- Docsify v4 -->
|
||||||
|
<script src="//cdn.jsdelivr.net/npm/docsify@4"></script>
|
||||||
|
<script src="//cdn.jsdelivr.net/npm/docsify/lib/plugins/search.min.js"></script>
|
||||||
|
<script src="//cdn.jsdelivr.net/npm/docsify-copy-code"></script>
|
||||||
|
</body>
|
||||||
|
|
||||||
|
</html>
|
||||||
1
prompts/2026-03-09-add-pay-all.md
Normal file
1
prompts/2026-03-09-add-pay-all.md
Normal file
@@ -0,0 +1 @@
|
|||||||
|
Now on both reconiciliation pages in the balance column i want to have a button "Pay All" which will create a new row in the transactions table with amount equal to the balance and with a note same as for payment for single period but stating all periods debt consist of
|
||||||
7
prompts/2026-03-10-cache-data-from-google-sheets.md
Normal file
7
prompts/2026-03-10-cache-data-from-google-sheets.md
Normal file
@@ -0,0 +1,7 @@
|
|||||||
|
i would like to implement caching of data that we load from the google documents. For all of them. I do not need persistence across application restarts, so file of whatever format in tmp directory would be good enough. I think it would be good idea to read metadata about documents we access - last modified time? and reload these files only when document is newer than cached data.
|
||||||
|
|
||||||
|
Suggest solution, suggest file format for caching.
|
||||||
|
|
||||||
|
------------
|
||||||
|
|
||||||
|
i do not need caching for scripts, caching is relevant for web app only
|
||||||
Reference in New Issue
Block a user