kacerrandClaude Fable 5
86a61da36f
Count paths in a map, copy into []KV, sort by count descending with cmp.Compare, truncate to n. Scanner buffer raised to 1MB for long lines; malformed lines (<3 fields) are skipped. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012MpK9niZm4jcs3mAJbxMfi
04 — Log Crunching: Top-N
Looks like a warm-up. It's actually a trap-density championship.
Trap 1: bufio.Scanner's 64KB guillotine
Scanner refuses tokens above 64KB by default and stops with
bufio.ErrTooLong — and if you forget to check sc.Err(), it stops
silently, and your report just... misses the tail of the file. Real-world
lines that exceed 64KB: JSON-formatted logs, stack traces, requests with fat
query strings. The fix:
sc.Buffer(make([]byte, 0, 1024*1024), 1024*1024)
An enormous number of production log processors have shipped with this bug.
Checking sc.Err() after the loop is the difference between a tool and a liar.
Trap 2: maps have no order — by design, with a great backstory
Go deliberately randomizes map iteration order (a random seed per map, and the runtime even randomizes the starting bucket) so nobody can accidentally depend on it. To rank by count you must extract into a slice and sort:
sort.Slice(out, func(i, j int) bool { return out[i].Count > out[j].Count })
Since 1.24 maps use a Swiss-table implementation (flat, SIMD-probed, cache friendlier) — fun fact to drop if asked "what's inside a Go map."
Trap 3: strings.Fields vs strings.Split
Split(s, " ") produces empty strings for repeated spaces; Fields splits on
any run of whitespace (incl. tabs) and never yields empties. For log parsing,
Fields is almost always what you mean.
The senior follow-up: "the file is 500 GB"
Have a ladder ready:
- This solution already streams — memory is O(distinct endpoints), not O(file). Say that first; many candidates don't realize their answer scales.
- Cardinality explosion (endpoints contain IDs): normalize paths
(
/users/123→/users/:id) or cap the map with count-min sketch / space-saving algorithm — approximate top-K in bounded memory. Naming "count-min sketch" in an SRE interview is a mic-drop; it's what real metrics pipelines do to survive high-cardinality labels. - Parallelize: shard by hash(endpoint) across workers, merge partials — map-reduce in one sentence.
- Only need top-N, not full sort? A min-heap of size N gives
O(distinct · log N) instead of O(distinct · log distinct).
container/heapexists; nobody remembers its interface; everyone respects you for trying.