lobo/zfin
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions
zfin/AGENTS.md

18 KiB
Raw Blame History

AGENTS.md

⛔ ABSOLUTE PROHIBITIONS — READ FIRST ⛔

NEVER delete or modify build caches. EVER.

This means:

  • NEVER run rm -rf .zig-cache or rm -rf .zig-cache/* or any variant.
  • NEVER run rm -rf ~/.cache/zig or touch anything under ~/.cache/zig/.
  • NEVER touch ~/.cache/zls/ or any other tool cache.
  • NEVER suggest deleting the cache as a "fix" — it is not a fix, it is damage. Deleting .zig-cache while ZLS or another zig build is running creates a corrupt state where the build runner's expected cache entry (.zig-cache/o/<hash>/build) references a path that no longer exists, producing the error failed to spawn build runner .zig-cache/o/<hash>/build: FileNotFound. Recovering from this on the affected machine requires killing every concurrent zig process (including ZLS) and waiting for filesystem state to re-stabilize — it is NOT a simple retry.
  • NEVER touch the cache "just to force a rebuild". Zig's cache is content-addressed. It does not get stale in a way that deletion fixes. If a build result looks wrong, the bug is in the source, not the cache. Use touch src/somefile.zig if you truly need to invalidate one file's cache line. Do not nuke the whole directory.
  • NEVER suggest a "different cache directory" as a workaround without an explicit, specific reason and explicit user approval. --cache-dir and --global-cache-dir flags exist; they are not toys.

If a test result seems wrong or cached incorrectly: the answer is ALWAYS to investigate the source code or build graph, not to delete cache. See "Test discovery" below — 99% of the time the "cached wrong result" is actually a test discovery problem, not a cache problem.

If you find yourself typing rm -rf anywhere near a cache path: STOP. Ask the user instead.

NEVER run destructive git operations without explicit permission.

  • No git reset --hard, git clean -fdx, git push --force, git checkout . on files with uncommitted work, unless the user asks for that specific operation by name.

NEVER run git add, git commit, or git push. EVER.

  • The user commits. You do not. Do not stage files. Do not create commits. Do not amend commits. Do not push. Do not suggest running these commands yourself "to save a step". This includes git add -p, git add ., git add <file>, git commit -m ..., git commit --amend, git push, and any gh pr create that would auto-stage or auto-commit.
  • If you are tempted to run any of these because "the work is done and it seems logical to commit" — STOP. The user has a review-and-commit workflow. Your job ends at a clean working tree with the changes ready to review.
  • The ONLY exception is when the user says, verbatim in the current turn, "commit this" / "make a commit" / "push it" / similar direct imperative. Do not extrapolate from earlier intent, a plan that mentioned milestones, or any indirect signal. If in doubt, ask — don't commit.
  • When a milestone plan says "STOP POINT — user reviews and commits": you stop. You do not commit. You do not prepare a commit. You hand off the working tree and wait.

Documentation-file conventions

  • TODO.md holds only what's still open. Git already tracks what was done and when. Do NOT add "DONE" markers, completion status, strikethrough, or "shipped in …" blurbs to TODO entries — just delete the section when the work is finished. If follow-up work came out of the finished task, add it as a new top-level section; don't leave the parent entry around as a historical wrapper.
  • REPORT.md is untracked on purpose. It's a personal workflow doc living in the repo root only until it moves to ~/finance. Edit it freely when asked; don't treat it as part of the repo surface. Don't mention it in commit messages for unrelated work.

Commands

zig build                    # build the zfin binary (output: zig-out/bin/zfin)
zig build test               # run all tests (single binary, discovers all tests via refAllDeclsRecursive)
zig build run -- <args>      # build and run CLI
zig build docs               # generate library documentation
zig build coverage -Dcoverage-threshold=60  # run tests with kcov coverage (Linux only)

Tooling (managed via .mise.toml):

  • Zig 0.15.2 (minimum)
  • ZLS 0.15.1
  • zlint 0.7.9

Linting: zlint --deny-warnings --fix (runs via pre-commit on staged .zig files).

Formatting: zig fmt (enforced by pre-commit). Always run before committing.

Pre-commit hooks run in order: trailing-whitespace, end-of-file-fixer, zig-fmt, zlint, zig-build, then tests with coverage.

Architecture

Single binary (CLI + TUI) built from src/main.zig. No separate library binary for internal use — the library module (src/root.zig) exists only for downstream consumers and documentation generation.

Data flow

User input → main.zig (CLI dispatch) or tui.zig (TUI event loop)
  → commands/*.zig (CLI) or tui/*.zig (TUI tab renderers)
    → DataService (service.zig) — sole data access layer
      → Cache check (cache/store.zig, SRF files in ~/.cache/zfin/{SYMBOL}/)
      → Server sync (optional ZFIN_SERVER, parallel HTTP)
      → Provider fetch (providers/*.zig, rate-limited HTTP)
      → Cache write
    → analytics/*.zig (performance, risk, valuation calculations)
    → format.zig (shared formatters, braille charts)
    → views/*.zig (view models — renderer-agnostic display data)
    → stdout (CLI via buffered Writer) or vaxis (TUI terminal rendering)

Key design decisions

  • Internal imports use file paths, not module names. Only external dependencies (srf, vaxis, z2d) use @import("name"). Internal code uses relative paths like @import("models/date.zig"). This is intentional — it lets refAllDeclsRecursive in the test binary discover all tests across the entire source tree.

  • DataService is the sole data source. Both CLI and TUI go through DataService for all fetched data. Never call provider APIs directly from commands or TUI tabs.

  • Providers are lazily initialized. DataService fields like td, pg, fh start as null and are created on first use via getProvider(). The provider field name is derived from the type name at comptime.

  • Cache uses SRF format. SRF (Simple Record Format) is a line-oriented key-value format. Cache layout: {cache_dir}/{SYMBOL}/{data_type}.srf. Freshness is determined by file mtime vs TTL.

  • Candles use incremental updates. On cache miss, only candles newer than the last cached date are fetched (not the full 10-year history). The candles_meta.srf file tracks the last date and provider without deserializing the full candle file.

  • View models separate data from rendering. views/portfolio_sections.zig produces renderer-agnostic structs with StyleIntent enums. CLI and TUI renderers are thin adapters that map StyleIntent to ANSI colors or vaxis styles.

  • Negative cache entries. When a provider fetch fails permanently (not rate-limited), a negative cache entry is written to prevent repeated retries for nonexistent symbols.

Module map

Directory Purpose
src/models/ Data types: Date (days since epoch), Candle, Dividend, Split, Lot, Portfolio, OptionContract, EarningsEvent, EtfProfile, Quote
src/providers/ API clients: each provider has its own struct with init(allocator, api_key) + fetch methods. json_utils.zig has shared JSON parsing helpers.
src/analytics/ Pure computation: performance.zig (Morningstar-style trailing returns), risk.zig (Sharpe, drawdown), valuation.zig (portfolio summary), analysis.zig (breakdowns by class/sector/geo)
src/commands/ CLI command handlers: each has a run() function taking (allocator, *DataService, symbol, color, *Writer). common.zig has shared CLI helpers and color constants.
src/tui/ TUI tab renderers: each tab (portfolio, quote, perf, options, earnings, analysis) is a separate file. keybinds.zig and theme.zig handle configurable input/colors. chart.zig renders pixel charts via Kitty graphics protocol.
src/views/ View models producing renderer-agnostic display data with StyleIntent
src/cache/ store.zig: SRF cache read/write with TTL freshness checks
src/net/ http.zig: HTTP client with retry and error classification. RateLimiter.zig: token-bucket rate limiter.
build/ Build-time support: Coverage.zig (kcov integration)

Code patterns and conventions

Error handling

  • Provider HTTP errors are classified in net/http.zig: RequestFailed, RateLimited, Unauthorized, NotFound, ServerError, InvalidResponse.
  • DataService wraps these into DataError: NoApiKey, FetchFailed, TransientError, AuthError, etc.
  • Transient errors (server 5xx, connection failures) cause the refresh to stop. Non-transient errors (NotFound, ParseError) cause fallback to the next provider.
  • Rate limit hits trigger a single retry after rateLimitBackoff().

The Date type

Date is an i32 of days since Unix epoch. It is used everywhere instead of timestamps. Construction: Date.fromYmd(2024, 1, 15) or Date.parse("2024-01-15"). Formatting: date.format(&buf) writes YYYY-MM-DD into a *[10]u8. The type has SRF serialization hooks (srfParse, srfFormat).

Formatting pattern

Functions in format.zig write into caller-provided buffers and return slices. They never allocate. Example: fmtMoneyAbs(&buf, amount) returns []const u8. The sign handling is always caller-side.

Provider pattern

Each provider in src/providers/ follows the same structure:

  1. Struct with client: http.Client, allocator, api_key
  2. init(allocator, api_key) / deinit()
  3. fetch*(allocator, symbol, ...) methods that build a URL, call self.client.get(url), and parse the JSON response
  4. Private parse* functions that handle the provider-specific JSON format
  5. Shared JSON helpers from json_utils.zig (parseJsonFloat, optFloat, optUint, jsonStr)

Test pattern

All tests are inline (in test blocks within source files). There is a single test binary rooted at src/main.zig which uses refAllDeclsRecursive(@This()) to discover all tests transitively via file imports. The tests/ directory exists but fixtures are empty — all test data is defined inline.

Tests use std.testing.allocator (which detects leaks) and are structured as unit tests that verify individual functions. Network-dependent code is not tested (no mocking infrastructure).

⚠️ Test discovery — READ THIS BEFORE ADDING A NEW .zig FILE WITH TESTS ⚠️

This gets fucked up every single session. Read it. Do what it says.

zig build test runs tests from test blocks in files that are part of the test binary's compilation unit AND are reachable from src/main.zig's import graph in a way that refAllDeclsRecursive actually visits the file struct itself (not just a type extracted from it).

The failure mode: you add src/models/foo.zig with 20 tests. You wire it into src/service.zig via const foo = @import("models/foo.zig"); and re-export a type from root.zig as pub const foo = @import("models/foo.zig");. You run zig build test and the test count does NOT go up. The file compiles (because foo.Bar is referenced as a function return type), but the test blocks inside it are never run.

Why: root.zig is imported into main.zig via const zfin = @import("root.zig"); — non-pub. refAllDeclsRecursive walks @typeInfo(@This()).@"struct".decls at main.zig, which only surfaces some of the decl graph. The pub const foo = @import(...) in root.zig is not reliably traversed from main.zig's test root, so foo.zig's test blocks aren't collected even though the file is compiled.

How to verify a new file's tests are discovered:

  1. Before relying on the test count, add a canary that MUST fail: 
    test "CANARY_DISCOVERY_CHECK_REMOVE_ME" {
    try std.testing.expect(false);
}
  2. Run zig build test --summary all 2>&1 | grep -E "tests passed|error:".
  3. If the canary test appears in failures → discovery works, remove canary.
  4. If the canary does NOT appear and total count is unchanged → see fix below.

Fix: add an explicit import in the test block at the bottom of src/main.zig:

test {
    std.testing.refAllDeclsRecursive(@This());
    _ = @import("models/foo.zig");  // ← new entry for each orphaned file
}

Adding it inside the test block (not at file scope as a comptime block) keeps the non-test build unaffected while guaranteeing the test binary sema-reaches the file and collects its test blocks.

Rule of thumb: after adding ANY new .zig file under src/ that contains test blocks, run zig build test --summary all 2>&1 | grep "tests passed" BEFORE and AFTER the change. If the delta doesn't match rg -c "^test " path/to/new_file.zig, add the explicit import to main.zig's test block.

Do NOT, under any circumstance, try to "fix" this by clearing the cache. The cache is not the problem. The import graph is the problem. Re-read the prohibitions at the top of this file.

Adding a new CLI command

  1. Create src/commands/newcmd.zig with a pub fn run(allocator, *DataService, symbol, color, *Writer) !void
  2. Add the import to the commands struct in src/main.zig
  3. Add the dispatch branch in main.zig's command matching chain
  4. Update the usage string in main.zig

Adding a new provider

  1. Create src/providers/newprovider.zig following the existing struct pattern
  2. Add a field to DataService (e.g., np: ?NewProvider = null)
  3. Add the API key to Config (e.g., newprovider_key: ?[]const u8 = null) — the field name must be the lowercased type name + _key for the comptime getProvider lookup to work
  4. Wire resolve("NEWPROVIDER_API_KEY") in Config.fromEnv

Adding a new TUI tab

  1. Create src/tui/newtab_tab.zig
  2. Add the tab variant to tui.Tab enum
  3. Wire rendering in tui.zig's draw and event handling

Command run() signatures — allocator as code smell

A CLI command's run() function that takes *DataService and *std.Io.Writer usually doesn't also need an std.mem.Allocator parameter. FetchResult(T) carries its own allocator and self-deinits (see src/service.zig), so callers never need to wire up matching allocators for payload cleanup. The writer owns its own buffer.

If a new run() signature still wants an allocator, ask whether the work it's funding is:

  • Legitimate: file I/O for a secondary config (portfolio load, metadata), non-trivial intermediate computation, or an arena wrapping view-layer allocations. Keep it.
  • Suspicious: freeing FetchResult.data manually instead of calling result.deinit(), or duplicating strings that could be borrowed. Drop the allocator and fix the leak-shaped helper.

Not a hard rule — just a signal worth questioning when reviewing a new command.

Gotchas

  • Provider field naming is comptime-derived. DataService.getProvider(T) finds the ?T field by iterating struct fields at comptime, and the config key is derived by lowercasing the type name and appending _key. If you rename a provider struct, you must also rename the config field or the comptime logic breaks.

  • Candle data has two cache files. candles_daily.srf holds the actual OHLCV data; candles_meta.srf holds metadata (last_date, provider, fail_count). When invalidating candles, both must be cleared (this is handled by DataService.invalidate).

  • TwelveData candles are force-refetched. Cached candles from the TwelveData provider are treated as stale regardless of TTL because TwelveData's adj_close values were found to be unreliable. The code in getCandles explicitly checks m.provider == .twelvedata and falls through.

  • Mutual fund detection is heuristic. isMutualFund checks if the symbol is exactly 5 chars ending in 'X'. This skips earnings fetching for mutual funds. It's imperfect but covers the common case.

  • SRF string lifetimes. When reading SRF records, string fields point into the iterator's internal buffer. If you need strings to outlive the iterator, use a postProcess callback to allocator.dupe() them (see dividendPostProcess in service.zig).

  • Buffered stdout. CLI output uses a single std.Io.Writer with a 4096-byte stack buffer, flushed once at the end of main(). Don't write to stdout through other means.

  • The color parameter flows through everything. CLI commands accept a color: bool parameter. Don't use ANSI escapes unconditionally — always gate on the color flag.

  • Portfolio auto-detection. Both CLI and TUI auto-load portfolio.srf from cwd if no explicit path is given. If not found in cwd, falls back to $ZFIN_HOME/portfolio.srf. watchlist.srf and .env follow the same cascade. metadata.srf and accounts.srf are loaded from the same directory as the resolved portfolio file.

  • transaction_log.srf is a sibling file. Optional. Lives next to portfolio.srf / accounts.srf. Holds user-declared transfer:: records so the contributions pipeline can tell internal account-to-account movement apart from real external contributions. Only type::cash is wired in v1 — type::in_kind parses but is rejected downstream. Missing file → matcher is a no-op. See REPORT.md §5 "Transfer log" for the user-facing guide.

  • Server sync is optional. The ZFIN_SERVER env var enables parallel cache syncing from a remote zfin-server instance. All server sync code silently no-ops when the URL is null.

Dependencies

Dependency Purpose
SRF Cache file format, portfolio/watchlist parsing, serialization
libvaxis (v0.5.1) Terminal UI rendering
z2d (v0.10.0) Pixel chart rendering (Kitty graphics protocol)

Build system rules

  • Never use addAnonymousImport in build.zig. Always use b.addModule() + addImport(). Anonymous imports cause "file belongs to multiple modules" errors and make dependency wiring opaque.