zfin/AGENTS.md

AGENTS.md

⛔ ABSOLUTE PROHIBITIONS — READ FIRST ⛔

Zig 0.16.0 reference — read the release notes

This codebase is on Zig 0.16.0. The 0.16 release was a major I/O-as-an-interface refactor that reshaped the standard library. Before making non-trivial changes (especially anything touching std.Io, std.fs, std.process, std.http, std.Thread, allocators, or std.time), read the release notes at:

https://ziglang.org/download/0.16.0/release-notes.html

Key migrations that bit us repeatedly during the 0.16 upgrade and will bite future work too:

• std.io → std.Io (namespace rename, no deprecation alias).
• std.fs.cwd() → std.Io.Dir.cwd(). All file ops take io.
• std.process.Child.init(argv, alloc) → std.process.spawn(io, .{...}) or std.process.run(gpa, io, .{...}).
• std.time.timestamp() → std.Io.Timestamp.now(io, .real).toSeconds().
• std.Thread.sleep(ns) → std.Io.sleep(io, duration, clock).
• pub fn main gains a std.process.Init parameter ("Juicy Main"); provides pre-built gpa, io, arena, environ_map.
• std.heap.GeneralPurposeAllocator → std.heap.DebugAllocator.
• std.heap.ThreadSafeAllocator removed; ArenaAllocator is now lock-free thread-safe, DebugAllocator is thread-safe by default.
• std.mem.trimRight/trimLeft → trimEnd/trimStart.
• std.mem.indexOf* → find* (deprecation aliases still present, so old names work but warn).
• std.testing.refAllDeclsRecursive removed. Only refAllDecls remains. We use the bare std.testing.refAllDecls(@This()) in src/main.zig's test block — it sema-touches every top-level decl, which transitively pulls in every imported file's test blocks. No local reimplementation is needed. See "Test discovery" below.
• std.fs.File.readToEndAlloc(alloc, N) → two-step: file.reader(io, &.{}) then .interface.allocRemaining(alloc, .limited(N)).

For anything not on this list, read the release notes first. The notes are long but they're organized by section; searching for the specific symbol you're migrating is fast.

io vs today / now_s — design rule

The 0.16 upgrade made a deliberate choice about which Zig-0.16 Io calls to thread through and which to sidestep. This rule is load-bearing; please read before adding new code that needs the current time.

• io: std.Io is threaded through anything that actually does I/O — file reads/writes, stderr, HTTP, process spawn, terminal detection. A function taking io is announcing that it touches the outside world. The "code smell" is a feature.
• today: Date is passed as a value for functions that need "what day is it" but don't otherwise do I/O. Captured once at the top of the unit of work (runCli for CLI, App.init for TUI) and threaded through. Render output stays deterministic within a frame even if the clock ticks over mid-render.
• now_s: i64 (or similar before_s/after_s pairs) is passed as a value for sub-second-precision metadata fields like snapshot captured_at, rollup #!created=, audit cadence staleness math. Same single-capture-and-thread pattern as today.

When adding a new function that needs the current time, do NOT reach for std.Io.Timestamp.now(io, .real) inside the function. Take today: Date or now_s: i64 instead. Only take io if the function genuinely needs to do I/O for other reasons.

Legitimate Timestamp.now callers (each must have a // wall-clock required: <why> comment justifying the read):

• cache/store.zig — cache entry timestamps and TTL math
• service.zig — per-fetch FetchResult.timestamp
• net/RateLimiter.zig — token-bucket refill
• TUI per-frame "now" captures for relative-time display
• The single Timestamp.now capture in main.zig's dispatch entry that produces today and now_s for the rest of the invocation
• The format.todayDate(io) helper itself (the one legitimate capture function for unit-of-work entry points)

If you find yourself writing Timestamp.now(io, ...) somewhere not on that list, either add a justifying comment or refactor the function to take a value parameter.

NEVER invoke ripgrep. EVER.

Do not run rg in the Bash tool. Not for open-ended search, not for counting matches, not for "just this one quick check", not ever. Running ripgrep on this machine hammers the filesystem badly enough to degrade the whole system — this is a recurring, reproducible problem, not a hunch.

Use instead:

• Grep tool (built-in) for content search. It handles regex, file globs, and output shaping without spawning rg.
• Glob tool (built-in) for finding files by name pattern.
• Read tool for reading files (with offset/limit for large ones).
• Plain grep via the Bash tool is acceptable when the built-in Grep tool can't express what you need — but prefer the built-in first.

If you catch yourself typing rg in a Bash command: stop, delete it, use the Grep tool instead. The fact that rg is faster in the abstract does NOT matter here. This machine's filesystem + ripgrep's parallelism is a bad combination, full stop.

This applies to every variant: rg, ripgrep, piping through rg, backgrounded rg, rg --files, etc. All banned.

NEVER delete or modify build caches. EVER.

This means:

• NEVER run rm -rf .zig-cache or rm -rf .zig-cache/* or any variant.
• NEVER run rm -rf ~/.cache/zig or touch anything under ~/.cache/zig/.
• NEVER touch ~/.cache/zls/ or any other tool cache.
• NEVER suggest deleting the cache as a "fix" — it is not a fix, it is damage. Deleting .zig-cache while ZLS or another zig build is running creates a corrupt state where the build runner's expected cache entry (.zig-cache/o/<hash>/build) references a path that no longer exists, producing the error failed to spawn build runner .zig-cache/o/<hash>/build: FileNotFound. Recovering from this on the affected machine requires killing every concurrent zig process (including ZLS) and waiting for filesystem state to re-stabilize — it is NOT a simple retry.
• NEVER touch the cache "just to force a rebuild". Zig's cache is content-addressed. It does not get stale in a way that deletion fixes. If a build result looks wrong, the bug is in the source, not the cache. Use touch src/somefile.zig if you truly need to invalidate one file's cache line. Do not nuke the whole directory.
• NEVER suggest a "different cache directory" as a workaround without an explicit, specific reason and explicit user approval. --cache-dir and --global-cache-dir flags exist; they are not toys.

If a test result seems wrong or cached incorrectly: the answer is ALWAYS to investigate the source code or build graph, not to delete cache. See "Test discovery" below — 99% of the time the "cached wrong result" is actually a test discovery problem, not a cache problem.

If you find yourself typing rm -rf anywhere near a cache path: STOP. Ask the user instead.

NEVER run destructive git operations without explicit permission.

• No git reset --hard, git clean -fdx, git push --force, git checkout . on files with uncommitted work, unless the user asks for that specific operation by name.

NEVER run git add, git commit, or git push. EVER.

• The user commits. You do not. Do not stage files. Do not create commits. Do not amend commits. Do not push. Do not suggest running these commands yourself "to save a step". This includes git add -p, git add ., git add <file>, git commit -m ..., git commit --amend, git push, and any gh pr create that would auto-stage or auto-commit.
• If you are tempted to run any of these because "the work is done and it seems logical to commit" — STOP. The user has a review-and-commit workflow. Your job ends at a clean working tree with the changes ready to review.
• The ONLY exception is when the user says, verbatim in the current turn, "commit this" / "make a commit" / "push it" / similar direct imperative. Do not extrapolate from earlier intent, a plan that mentioned milestones, or any indirect signal. If in doubt, ask — don't commit.
• When a milestone plan says "STOP POINT — user reviews and commits": you stop. You do not commit. You do not prepare a commit. You hand off the working tree and wait.

Documentation-file conventions

• TODO.md holds only what's still open. Git already tracks what was done and when. Do NOT add "DONE" markers, completion status, strikethrough, or "shipped in …" blurbs to TODO entries — just delete the section when the work is finished. If follow-up work came out of the finished task, add it as a new top-level section; don't leave the parent entry around as a historical wrapper.
• REPORT.md is untracked on purpose. It's a personal workflow doc living in the repo root only until it moves to ~/finance. Edit it freely when asked; don't treat it as part of the repo surface. Don't mention it in commit messages for unrelated work.

---

Commands

zig build                    # build the zfin binary (output: zig-out/bin/zfin)
zig build test               # run all tests (single binary, discovers all tests via refAllDecls + the import graph)
zig build run -- <args>      # build and run CLI
zig build docs               # generate library documentation
zig build coverage -Dcoverage-threshold=60  # run tests with kcov coverage (Linux only)

Tooling (managed via .mise.toml):

• Zig 0.16.0 (minimum)
• ZLS 0.16.0
• zlint 0.7.9

Linting: zlint --deny-warnings --fix (runs via pre-commit on staged .zig files).

Formatting: zig fmt (enforced by pre-commit). Always run before committing.

Pre-commit hooks run in order: trailing-whitespace, end-of-file-fixer, zig-fmt, zlint, zig-build, then tests with coverage.

Architecture

Single binary (CLI + TUI) built from src/main.zig. No separate library binary for internal use — the library module (src/root.zig) exists only for downstream consumers and documentation generation.

Data flow

User input → main.zig (CLI dispatch) or tui.zig (TUI event loop)
  → commands/*.zig (CLI) or tui/*.zig (TUI tab renderers)
    → DataService (service.zig) — sole data access layer
      → Cache check (cache/store.zig, SRF files in ~/.cache/zfin/{SYMBOL}/)
      → Server sync (optional ZFIN_SERVER, parallel HTTP)
      → Provider fetch (providers/*.zig, rate-limited HTTP)
      → Cache write
    → analytics/*.zig (performance, risk, valuation calculations)
    → format.zig (shared formatters, braille charts)
    → views/*.zig (view models — renderer-agnostic display data)
    → stdout (CLI via buffered Writer) or vaxis (TUI terminal rendering)

Key design decisions

• Internal imports use file paths, not module names. Only external dependencies (srf, vaxis, z2d) use @import("name"). Internal code uses relative paths like @import("models/date.zig"). This is intentional — it lets refAllDecls in the test binary discover all tests across the entire source tree.

• DataService is the sole data source. Both CLI and TUI go through DataService for all fetched data. Never call provider APIs directly from commands or TUI tabs.

• Providers are lazily initialized. DataService fields like td, pg, fh start as null and are created on first use via getProvider(). The provider field name is derived from the type name at comptime.

• Cache uses SRF format. SRF (Simple Record Format) is a line-oriented key-value format. Cache layout: {cache_dir}/{SYMBOL}/{data_type}.srf. Freshness is determined by file mtime vs TTL.

• Candles use incremental updates. On cache miss, only candles newer than the last cached date are fetched (not the full 10-year history). The candles_meta.srf file tracks the last date and provider without deserializing the full candle file.

• View models separate data from rendering. views/portfolio_sections.zig produces renderer-agnostic structs with StyleIntent enums. CLI and TUI renderers are thin adapters that map StyleIntent to ANSI colors or vaxis styles.

• Negative cache entries. When a provider fetch fails permanently (not rate-limited), a negative cache entry is written to prevent repeated retries for nonexistent symbols.

Module map

Directory	Purpose
src/models/	Data types: Date (days since epoch), Candle, Dividend, Split, Lot, Portfolio, OptionContract, EarningsEvent, EtfProfile, Quote
src/providers/	API clients: each provider has its own struct with init(allocator, api_key) + fetch methods. json_utils.zig has shared JSON parsing helpers.
src/analytics/	Pure computation: performance.zig (Morningstar-style trailing returns), risk.zig (Sharpe, drawdown), valuation.zig (portfolio summary), analysis.zig (breakdowns by class/sector/geo)
src/commands/	CLI command handlers: each has a run() function taking (allocator, *DataService, symbol, color, *Writer). common.zig has shared CLI helpers and color constants.
src/tui/	TUI tab renderers: each tab (portfolio, quote, perf, options, earnings, analysis) is a separate file. keybinds.zig and theme.zig handle configurable input/colors. chart.zig renders pixel charts via Kitty graphics protocol.
src/views/	View models producing renderer-agnostic display data with StyleIntent
src/cache/	store.zig: SRF cache read/write with TTL freshness checks
src/net/	http.zig: HTTP client with retry and error classification. RateLimiter.zig: token-bucket rate limiter.
build/	Build-time support: Coverage.zig (kcov integration)

Code patterns and conventions

Error handling

• Provider HTTP errors are classified in net/http.zig: RequestFailed, RateLimited, Unauthorized, NotFound, ServerError, InvalidResponse.
• DataService wraps these into DataError: NoApiKey, FetchFailed, TransientError, AuthError, etc.
• Transient errors (server 5xx, connection failures) cause the refresh to stop. Non-transient errors (NotFound, ParseError) cause fallback to the next provider.
• Rate limit hits trigger a single retry after rateLimitBackoff().

The Date type

Date is an i32 of days since Unix epoch. It is used everywhere instead of timestamps. Construction: Date.fromYmd(2024, 1, 15) or Date.parse("2024-01-15"). Formatting: date.format(&buf) writes YYYY-MM-DD into a *[10]u8. The type has SRF serialization hooks (srfParse, srfFormat).

Formatting pattern

Functions in format.zig write into caller-provided buffers and return slices. They never allocate. Example: fmtMoneyAbs(&buf, amount) returns []const u8. The sign handling is always caller-side.

Provider pattern

Each provider in src/providers/ follows the same structure:

1. Struct with client: http.Client, allocator, api_key
2. init(allocator, api_key) / deinit()
3. fetch*(allocator, symbol, ...) methods that build a URL, call self.client.get(url), and parse the JSON response
4. Private parse* functions that handle the provider-specific JSON format
5. Shared JSON helpers from json_utils.zig (parseJsonFloat, optFloat, optUint, jsonStr)

Test pattern

All tests are inline (in test blocks within source files). There is a single test binary rooted at src/main.zig which uses std.testing.refAllDecls(@This()) to sema-touch every top-level decl in main.zig. Each decl that's a @import(...) of a source file pulls that file into compilation, which causes its test blocks to be collected by the test runner. The tests/ directory exists but fixtures are empty — all test data is defined inline.

Tests use std.testing.allocator (which detects leaks) and are structured as unit tests that verify individual functions. Network-dependent code is not tested (no mocking infrastructure).

⚠️ Test discovery — READ THIS BEFORE ADDING A NEW .zig FILE WITH TESTS ⚠️

This gets fucked up every single session. Read it. Do what it says.

zig build test runs tests from test blocks in files that are part of the test binary's compilation unit AND get sema-pulled by the import graph from src/main.zig. With the bare std.testing.refAllDecls(@This()) we use, a file's tests are collected as long as the file is imported (directly or transitively) from main.zig.

The failure mode: you add src/models/foo.zig with 20 tests. You wire it into src/service.zig only as a type extraction, e.g. const Bar = @import("models/foo.zig").Bar; (assigning the type, not the file struct). The file compiles because Bar is referenced, but the file struct itself was never sema-touched as a struct, so its test blocks are not collected.

The fix: ensure at least one importer assigns the file struct to a const, like const foo = @import("models/foo.zig");. Even if you only use a type from it, the const foo form pulls in the file's test blocks.

How to verify a new file's tests are discovered:

1. Before relying on the test count, add a canary that MUST fail:

   test "CANARY_DISCOVERY_CHECK_REMOVE_ME" {
       try std.testing.expect(false);
   }
   

2. Run zig build test --summary all 2>&1 | grep -E "tests passed|error:".
3. If the canary test appears in failures → discovery works, remove canary.
4. If the canary does NOT appear and total count is unchanged → ensure the file is imported via a const x = @import(...) form somewhere reachable from main.zig.

Fallback fix: if you can't fix the import shape, add an explicit import in the test block at the bottom of src/main.zig:

test {
    std.testing.refAllDecls(@This());
    _ = @import("models/foo.zig");  // ← orphaned file
}

Adding it inside the test block (not at file scope as a comptime block) keeps the non-test build unaffected while guaranteeing the test binary sema-reaches the file and collects its test blocks.

Rule of thumb: after adding ANY new .zig file under src/ that contains test blocks, run zig build test --summary all 2>&1 | grep "tests passed" BEFORE and AFTER the change. If the delta doesn't match rg -c "^test " path/to/new_file.zig, add the explicit import to main.zig's test block.

Do NOT, under any circumstance, try to "fix" this by clearing the cache. The cache is not the problem. The import graph is the problem. Re-read the prohibitions at the top of this file.

Adding a new CLI command

1. Create src/commands/newcmd.zig with a pub fn run(allocator, *DataService, symbol, color, *Writer) !void
2. Add the import to the commands struct in src/main.zig
3. Add the dispatch branch in main.zig's command matching chain
4. Update the usage string in main.zig

Adding a new provider

1. Create src/providers/newprovider.zig following the existing struct pattern
2. Add a field to DataService (e.g., np: ?NewProvider = null)
3. Add the API key to Config (e.g., newprovider_key: ?[]const u8 = null) — the field name must be the lowercased type name + _key for the comptime getProvider lookup to work
4. Wire resolve("NEWPROVIDER_API_KEY") in Config.fromEnv

Adding a new TUI tab

1. Create src/tui/newtab_tab.zig
2. Add the tab variant to tui.Tab enum
3. Wire rendering in tui.zig's draw and event handling

Command run() signatures — allocator as code smell

A CLI command's run() function that takes *DataService and *std.Io.Writer usually doesn't also need an std.mem.Allocator parameter. FetchResult(T) carries its own allocator and self-deinits (see src/service.zig), so callers never need to wire up matching allocators for payload cleanup. The writer owns its own buffer.

If a new run() signature still wants an allocator, ask whether the work it's funding is:

• Legitimate: file I/O for a secondary config (portfolio load, metadata), non-trivial intermediate computation, or an arena wrapping view-layer allocations. Keep it.
• Suspicious: freeing FetchResult.data manually instead of calling result.deinit(), or duplicating strings that could be borrowed. Drop the allocator and fix the leak-shaped helper.

Not a hard rule — just a signal worth questioning when reviewing a new command.

Gotchas

• Provider field naming is comptime-derived. DataService.getProvider(T) finds the ?T field by iterating struct fields at comptime, and the config key is derived by lowercasing the type name and appending _key. If you rename a provider struct, you must also rename the config field or the comptime logic breaks.

• Candle data has two cache files. candles_daily.srf holds the actual OHLCV data; candles_meta.srf holds metadata (last_date, provider, fail_count). When invalidating candles, both must be cleared (this is handled by DataService.invalidate).

• TwelveData candles are force-refetched. Cached candles from the TwelveData provider are treated as stale regardless of TTL because TwelveData's adj_close values were found to be unreliable. The code in getCandles explicitly checks m.provider == .twelvedata and falls through.

• Mutual fund detection is heuristic. isMutualFund checks if the symbol is exactly 5 chars ending in 'X'. This skips earnings fetching for mutual funds. It's imperfect but covers the common case.

• SRF string lifetimes. When reading SRF records, string fields point into the iterator's internal buffer. If you need strings to outlive the iterator, use a postProcess callback to allocator.dupe() them (see dividendPostProcess in service.zig).

• Buffered stdout. CLI output uses a single std.Io.Writer with a 4096-byte stack buffer, flushed once at the end of main(). Don't write to stdout through other means.

• The color parameter flows through everything. CLI commands accept a color: bool parameter. Don't use ANSI escapes unconditionally — always gate on the color flag.

• Portfolio auto-detection. Both CLI and TUI auto-load portfolio.srf from cwd if no explicit path is given. If not found in cwd, falls back to $ZFIN_HOME/portfolio.srf. watchlist.srf and .env follow the same cascade. metadata.srf and accounts.srf are loaded from the same directory as the resolved portfolio file.

• transaction_log.srf is a sibling file. Optional. Lives next to portfolio.srf / accounts.srf. Holds user-declared transfer:: records so the contributions pipeline can tell internal account-to-account movement apart from real external contributions. Only type::cash is wired in v1 — type::in_kind parses but is rejected downstream. Missing file → matcher is a no-op. See REPORT.md §5 "Transfer log" for the user-facing guide.

• Server sync is optional. The ZFIN_SERVER env var enables parallel cache syncing from a remote zfin-server instance. All server sync code silently no-ops when the URL is null.

Dependencies

Dependency	Purpose
SRF	Cache file format, portfolio/watchlist parsing, serialization
libvaxis (v0.5.1)	Terminal UI rendering
z2d (v0.10.0)	Pixel chart rendering (Kitty graphics protocol)

Build system rules

• Never use addAnonymousImport in build.zig. Always use b.addModule() + addImport(). Anonymous imports cause "file belongs to multiple modules" errors and make dependency wiring opaque.