fuj-management/docs/spec/fio_to_sheets_sync.md

Specification: Fio to Google Sheets Sync

Goal

Automatically sync incoming transactions from a Fio bank account to a Google Sheet to serve as an intermediary ledger. This ledger allows for manual tagging, note-taking, and further processing without risking data loss on subsequent syncs.

Data Source (Fio)

Scraping Mechanism

The script will fetch data from the Fio bank in two possible ways:

1. Transparent Account HTML (Default):
   • Fetches the HTML from https://ib.fio.cz/ib/transparent?a=2800359168&f={date_from}&t={date_to}.
   • Uses a custom HTMLParser to locate the second <table> with class table.
   • Extracts columns: Date (0), Amount (1), Typ (2), Sender (3), Message (4), KS (5), VS (6), SS (7), and Note (8).
   • Normalizes dates from DD.MM.YYYY and amounts from Czech formatting (e.g., 1 500,00 CZK).
2. Official API (If FIO_API_TOKEN is set):
   • Fetches JSON from https://fioapi.fio.cz/v1/rest/periods/{token}/{date_from}/{date_to}/transactions.json.
   • Provides more structured and reliable data, including exact sender account numbers.

Unique Key Generation

To prevent duplicate entries in the Google Sheet, each transaction is assigned a Transaction ID.

• Algorithm: SHA-256 hash of a concatenated string.
• Input String: date|amount|currency|sender|vs|message|bank_id
• Why?: Even if two payments have the same amount on the same day, they likely differ in the variable symbol (VS), sender account, or message. Including all these fields makes the ID stable. The bank_id (if available from the API) is the strongest unique identifier.

Google Sheet Schema

The ledger uses the following column structure (Columns A-K):

Col	Label	Description
A	Date	Transaction date (YYYY-MM-DD).
B	Amount	Bank transaction amount in CZK.
C	manual fix	If not empty, make infer will skip this row (Manual Override).
D	Person	Inferred or manually entered member name.
E	Purpose	Inferred or manually entered month(s) (e.g., 2026-01).
F	Inferred Amount	The amount to be used for reconciliation.
G	Sender	Sender account name/number.
H	VS	Variable Symbol.
I	Message	Message for recipient.
J	Bank ID	Official Fio Transaction ID.
K	Sync ID	Unique SHA-256 hash for deduplication.

Usage Workflow

To maintain the ledger and generate reports, follow these steps:

1. Sync: make sync
   • Pulls new transactions from Fio and appends them to the sheet.
2. Infer: make infer
   • Automatically fills Person, Purpose, and Inferred Amount for new rows.
   • Skips any rows where manual fix or Person/Purpose are already filled.
3. Manual Review (Optional):
   • Open the Google Sheet and check any rows marked with [?] (low confidence).
   • Correct the Person, Purpose, or Inferred Amount as needed.
   • Type anything into the manual fix column to prevent the script from changing your edits.
4. Reconcile: make reconcile
   • Reads the processed data from the sheet and prints a balance report showing who owes how much.

Note

The reconciliation report uses the Google Sheet as the source of truth, so any manual corrections you make are immediately reflected in the final balances.

Synchronization Logic

The script performs a "Pull and Append" workflow:

1. Read Existing: Connects to the target Google Sheet and reads all values in Column K (Sync ID).
2. Fetch New: Scrapes Fio for the specified range (default: last 30 days).
3. Deduplicate:
   • For each scraped transaction, generate its Sync ID.
   • If the Sync ID is not in the list of existing IDs, it is marked for upload.
4. Append: Uploads only the new transactions to the next available rows in the sheet.
5. Preserve Manual Edits: Because the script only appends and never updates or deletes existing rows, any manual notes or extra columns added in the Google Sheet remain untouched.