zfin/AGENTS.md

# AGENTS.md

## ⛔ ABSOLUTE PROHIBITIONS — READ FIRST ⛔

### NEVER delete or modify build caches. EVER.

**This means:**

- **NEVER** run `rm -rf .zig-cache` or `rm -rf .zig-cache/*` or any variant.
- **NEVER** run `rm -rf ~/.cache/zig` or touch anything under `~/.cache/zig/`.
- **NEVER** touch `~/.cache/zls/` or any other tool cache.
- **NEVER** suggest deleting the cache as a "fix" — it is not a fix, it is
  damage. Deleting `.zig-cache` while ZLS or another `zig build` is running
  creates a corrupt state where the build runner's expected cache entry
  (`.zig-cache/o/<hash>/build`) references a path that no longer exists,
  producing the error `failed to spawn build runner .zig-cache/o/<hash>/build:
  FileNotFound`. Recovering from this on the affected machine requires
  killing every concurrent `zig` process (including ZLS) and waiting for
  filesystem state to re-stabilize — it is NOT a simple retry.
- **NEVER** touch the cache "just to force a rebuild". Zig's cache is
  content-addressed. It does not get stale in a way that deletion fixes.
  If a build result looks wrong, the bug is in the source, not the cache.
  Use `touch src/somefile.zig` if you truly need to invalidate one file's
  cache line. Do not nuke the whole directory.
- **NEVER** suggest a "different cache directory" as a workaround without
  an explicit, specific reason and explicit user approval. `--cache-dir`
  and `--global-cache-dir` flags exist; they are not toys.

**If a test result seems wrong or cached incorrectly:** the answer is
ALWAYS to investigate the source code or build graph, not to delete
cache. See "Test discovery" below — 99% of the time the "cached wrong
result" is actually a test discovery problem, not a cache problem.

**If you find yourself typing `rm -rf` anywhere near a cache path: STOP.
Ask the user instead.**

### NEVER run destructive git operations without explicit permission.

- No `git reset --hard`, `git clean -fdx`, `git push --force`, `git checkout .`
  on files with uncommitted work, unless the user asks for that specific
  operation by name.

### NEVER run `git add`, `git commit`, or `git push`. EVER.

- **The user commits. You do not.** Do not stage files. Do not create commits.
  Do not amend commits. Do not push. Do not suggest running these commands
  yourself "to save a step". This includes `git add -p`, `git add .`,
  `git add <file>`, `git commit -m ...`, `git commit --amend`, `git push`,
  and any `gh pr create` that would auto-stage or auto-commit.
- If you are tempted to run any of these because "the work is done and it
  seems logical to commit" — STOP. The user has a review-and-commit workflow.
  Your job ends at a clean working tree with the changes ready to review.
- The ONLY exception is when the user says, verbatim in the current turn,
  "commit this" / "make a commit" / "push it" / similar direct imperative.
  Do not extrapolate from earlier intent, a plan that mentioned milestones,
  or any indirect signal. If in doubt, ask — don't commit.
- When a milestone plan says "STOP POINT — user reviews and commits": you
  stop. You do not commit. You do not prepare a commit. You hand off the
  working tree and wait.

### Documentation-file conventions

- **`TODO.md` holds only what's still open.** Git already tracks what was
  done and when. Do NOT add "DONE" markers, completion status, strikethrough,
  or "shipped in …" blurbs to TODO entries — just delete the section when
  the work is finished. If follow-up work came out of the finished task,
  add it as a new top-level section; don't leave the parent entry around
  as a historical wrapper.
- **`REPORT.md` is untracked on purpose.** It's a personal workflow doc
  living in the repo root only until it moves to `~/finance`. Edit it
  freely when asked; don't treat it as part of the repo surface. Don't
  mention it in commit messages for unrelated work.

---

## Commands

```bash
zig build                    # build the zfin binary (output: zig-out/bin/zfin)
zig build test               # run all tests (single binary, discovers all tests via refAllDeclsRecursive)
zig build run -- <args>      # build and run CLI
zig build docs               # generate library documentation
zig build coverage -Dcoverage-threshold=60  # run tests with kcov coverage (Linux only)
```

**Tooling** (managed via `.mise.toml`):
- Zig 0.15.2 (minimum)
- ZLS 0.15.1
- zlint 0.7.9

**Linting**: `zlint --deny-warnings --fix` (runs via pre-commit on staged `.zig` files).

**Formatting**: `zig fmt` (enforced by pre-commit). Always run before committing.

**Pre-commit hooks** run in order: trailing-whitespace, end-of-file-fixer, zig-fmt, zlint, zig-build, then tests with coverage.

## Architecture

Single binary (CLI + TUI) built from `src/main.zig`. No separate library binary for internal use — the library module (`src/root.zig`) exists only for downstream consumers and documentation generation.

### Data flow

```
User input → main.zig (CLI dispatch) or tui.zig (TUI event loop)
  → commands/*.zig (CLI) or tui/*.zig (TUI tab renderers)
    → DataService (service.zig) — sole data access layer
      → Cache check (cache/store.zig, SRF files in ~/.cache/zfin/{SYMBOL}/)
      → Server sync (optional ZFIN_SERVER, parallel HTTP)
      → Provider fetch (providers/*.zig, rate-limited HTTP)
      → Cache write
    → analytics/*.zig (performance, risk, valuation calculations)
    → format.zig (shared formatters, braille charts)
    → views/*.zig (view models — renderer-agnostic display data)
    → stdout (CLI via buffered Writer) or vaxis (TUI terminal rendering)
```

### Key design decisions

- **Internal imports use file paths, not module names.** Only external dependencies (`srf`, `vaxis`, `z2d`) use `@import("name")`. Internal code uses relative paths like `@import("models/date.zig")`. This is intentional — it lets `refAllDeclsRecursive` in the test binary discover all tests across the entire source tree.

- **DataService is the sole data source.** Both CLI and TUI go through `DataService` for all fetched data. Never call provider APIs directly from commands or TUI tabs.

- **Providers are lazily initialized.** `DataService` fields like `td`, `pg`, `fh` start as `null` and are created on first use via `getProvider()`. The provider field name is derived from the type name at comptime.

- **Cache uses SRF format.** [SRF](https://git.lerch.org/lobo/srf) (Simple Record Format) is a line-oriented key-value format. Cache layout: `{cache_dir}/{SYMBOL}/{data_type}.srf`. Freshness is determined by file mtime vs TTL.

- **Candles use incremental updates.** On cache miss, only candles newer than the last cached date are fetched (not the full 10-year history). The `candles_meta.srf` file tracks the last date and provider without deserializing the full candle file.

- **View models separate data from rendering.** `views/portfolio_sections.zig` produces renderer-agnostic structs with `StyleIntent` enums. CLI and TUI renderers are thin adapters that map `StyleIntent` to ANSI colors or vaxis styles.

- **Negative cache entries.** When a provider fetch fails permanently (not rate-limited), a negative cache entry is written to prevent repeated retries for nonexistent symbols.

### Module map

| Directory | Purpose |
|-----------|---------|
| `src/models/` | Data types: `Date` (days since epoch), `Candle`, `Dividend`, `Split`, `Lot`, `Portfolio`, `OptionContract`, `EarningsEvent`, `EtfProfile`, `Quote` |
| `src/providers/` | API clients: each provider has its own struct with `init(allocator, api_key)` + fetch methods. `json_utils.zig` has shared JSON parsing helpers. |
| `src/analytics/` | Pure computation: `performance.zig` (Morningstar-style trailing returns), `risk.zig` (Sharpe, drawdown), `valuation.zig` (portfolio summary), `analysis.zig` (breakdowns by class/sector/geo) |
| `src/commands/` | CLI command handlers: each has a `run()` function taking `(allocator, *DataService, symbol, color, *Writer)`. `common.zig` has shared CLI helpers and color constants. |
| `src/tui/` | TUI tab renderers: each tab (portfolio, quote, perf, options, earnings, analysis) is a separate file. `keybinds.zig` and `theme.zig` handle configurable input/colors. `chart.zig` renders pixel charts via Kitty graphics protocol. |
| `src/views/` | View models producing renderer-agnostic display data with `StyleIntent` |
| `src/cache/` | `store.zig`: SRF cache read/write with TTL freshness checks |
| `src/net/` | `http.zig`: HTTP client with retry and error classification. `RateLimiter.zig`: token-bucket rate limiter. |
| `build/` | Build-time support: `Coverage.zig` (kcov integration) |

## Code patterns and conventions

### Error handling

- Provider HTTP errors are classified in `net/http.zig`: `RequestFailed`, `RateLimited`, `Unauthorized`, `NotFound`, `ServerError`, `InvalidResponse`.
- `DataService` wraps these into `DataError`: `NoApiKey`, `FetchFailed`, `TransientError`, `AuthError`, etc.
- Transient errors (server 5xx, connection failures) cause the refresh to stop. Non-transient errors (NotFound, ParseError) cause fallback to the next provider.
- Rate limit hits trigger a single retry after `rateLimitBackoff()`.

### The `Date` type

`Date` is an `i32` of days since Unix epoch. It is used everywhere instead of timestamps. Construction: `Date.fromYmd(2024, 1, 15)` or `Date.parse("2024-01-15")`. Formatting: `date.format(&buf)` writes `YYYY-MM-DD` into a `*[10]u8`. The type has SRF serialization hooks (`srfParse`, `srfFormat`).

### Formatting pattern

Functions in `format.zig` write into caller-provided buffers and return slices. They never allocate. Example: `fmtMoneyAbs(&buf, amount)` returns `[]const u8`. The sign handling is always caller-side.

### Provider pattern

Each provider in `src/providers/` follows the same structure:
1. Struct with `client: http.Client`, `allocator`, `api_key`
2. `init(allocator, api_key)` / `deinit()`
3. `fetch*(allocator, symbol, ...)` methods that build a URL, call `self.client.get(url)`, and parse the JSON response
4. Private `parse*` functions that handle the provider-specific JSON format
5. Shared JSON helpers from `json_utils.zig` (`parseJsonFloat`, `optFloat`, `optUint`, `jsonStr`)

### Test pattern

All tests are inline (in `test` blocks within source files). There is a single test binary rooted at `src/main.zig` which uses `refAllDeclsRecursive(@This())` to discover all tests transitively via file imports. The `tests/` directory exists but fixtures are empty — all test data is defined inline.

Tests use `std.testing.allocator` (which detects leaks) and are structured as unit tests that verify individual functions. Network-dependent code is not tested (no mocking infrastructure).

### ⚠️ Test discovery — READ THIS BEFORE ADDING A NEW .zig FILE WITH TESTS ⚠️

**This gets fucked up every single session. Read it. Do what it says.**

`zig build test` runs tests from `test` blocks in files that are part of the
test binary's compilation unit AND are reachable from `src/main.zig`'s
import graph in a way that `refAllDeclsRecursive` actually visits the file
struct itself (not just a type extracted from it).

**The failure mode:** you add `src/models/foo.zig` with 20 tests. You wire
it into `src/service.zig` via `const foo = @import("models/foo.zig");` and
re-export a type from `root.zig` as `pub const foo = @import("models/foo.zig");`.
You run `zig build test` and the test count does NOT go up. The file
**compiles** (because `foo.Bar` is referenced as a function return type),
but the `test` blocks inside it are never run.

**Why:** `root.zig` is imported into `main.zig` via
`const zfin = @import("root.zig");` — non-pub. `refAllDeclsRecursive` walks
`@typeInfo(@This()).@"struct".decls` at main.zig, which only surfaces some
of the decl graph. The `pub const foo = @import(...)` in root.zig is not
reliably traversed from main.zig's test root, so `foo.zig`'s test blocks
aren't collected even though the file is compiled.

**How to verify a new file's tests are discovered:**

1. Before relying on the test count, add a canary that MUST fail:
   ```zig
   test "CANARY_DISCOVERY_CHECK_REMOVE_ME" {
       try std.testing.expect(false);
   }
   ```
2. Run `zig build test --summary all 2>&1 | grep -E "tests passed|error:"`.
3. If the canary test appears in failures → discovery works, remove canary.
4. If the canary does NOT appear and total count is unchanged → see fix below.

**Fix:** add an explicit import in the `test` block at the bottom of
`src/main.zig`:

```zig
test {
    std.testing.refAllDeclsRecursive(@This());
    _ = @import("models/foo.zig");  // ← new entry for each orphaned file
}
```

Adding it inside the `test` block (not at file scope as a `comptime` block)
keeps the non-test build unaffected while guaranteeing the test binary
sema-reaches the file and collects its test blocks.

**Rule of thumb:** after adding ANY new `.zig` file under `src/` that contains
`test` blocks, run `zig build test --summary all 2>&1 | grep "tests passed"`
BEFORE and AFTER the change. If the delta doesn't match
`rg -c "^test " path/to/new_file.zig`, add the explicit import to main.zig's
test block.

**Do NOT, under any circumstance, try to "fix" this by clearing the cache.**
The cache is not the problem. The import graph is the problem. Re-read
the prohibitions at the top of this file.

### Adding a new CLI command

1. Create `src/commands/newcmd.zig` with a `pub fn run(allocator, *DataService, symbol, color, *Writer) !void`
2. Add the import to the `commands` struct in `src/main.zig`
3. Add the dispatch branch in `main.zig`'s command matching chain
4. Update the `usage` string in `main.zig`

### Adding a new provider

1. Create `src/providers/newprovider.zig` following the existing struct pattern
2. Add a field to `DataService` (e.g., `np: ?NewProvider = null`)
3. Add the API key to `Config` (e.g., `newprovider_key: ?[]const u8 = null`) — the field name must be the lowercased type name + `_key` for the comptime `getProvider` lookup to work
4. Wire `resolve("NEWPROVIDER_API_KEY")` in `Config.fromEnv`

### Adding a new TUI tab

1. Create `src/tui/newtab_tab.zig`
2. Add the tab variant to `tui.Tab` enum
3. Wire rendering in `tui.zig`'s draw and event handling

### Command `run()` signatures — allocator as code smell

A CLI command's `run()` function that takes `*DataService` and `*std.Io.Writer`
usually doesn't also need an `std.mem.Allocator` parameter. `FetchResult(T)`
carries its own allocator and self-deinits (see `src/service.zig`), so callers
never need to wire up matching allocators for payload cleanup. The writer
owns its own buffer.

If a new `run()` signature still wants an allocator, ask whether the work
it's funding is:

- **Legitimate**: file I/O for a secondary config (portfolio load,
  metadata), non-trivial intermediate computation, or an arena wrapping
  view-layer allocations. Keep it.
- **Suspicious**: freeing `FetchResult.data` manually instead of calling
  `result.deinit()`, or duplicating strings that could be borrowed.
  Drop the allocator and fix the leak-shaped helper.

Not a hard rule — just a signal worth questioning when reviewing a new
command.

## Gotchas

- **Provider field naming is comptime-derived.** `DataService.getProvider(T)` finds the `?T` field by iterating struct fields at comptime, and the config key is derived by lowercasing the type name and appending `_key`. If you rename a provider struct, you must also rename the config field or the comptime logic breaks.

- **Candle data has two cache files.** `candles_daily.srf` holds the actual OHLCV data; `candles_meta.srf` holds metadata (last_date, provider, fail_count). When invalidating candles, both must be cleared (this is handled by `DataService.invalidate`).

- **TwelveData candles are force-refetched.** Cached candles from the TwelveData provider are treated as stale regardless of TTL because TwelveData's `adj_close` values were found to be unreliable. The code in `getCandles` explicitly checks `m.provider == .twelvedata` and falls through.

- **Mutual fund detection is heuristic.** `isMutualFund` checks if the symbol is exactly 5 chars ending in 'X'. This skips earnings fetching for mutual funds. It's imperfect but covers the common case.

- **SRF string lifetimes.** When reading SRF records, string fields point into the iterator's internal buffer. If you need strings to outlive the iterator, use a `postProcess` callback to `allocator.dupe()` them (see `dividendPostProcess` in `service.zig`).

- **Buffered stdout.** CLI output uses a single `std.Io.Writer` with a 4096-byte stack buffer, flushed once at the end of `main()`. Don't write to stdout through other means.

- **The `color` parameter flows through everything.** CLI commands accept a `color: bool` parameter. Don't use ANSI escapes unconditionally — always gate on the `color` flag.

- **Portfolio auto-detection.** Both CLI and TUI auto-load `portfolio.srf` from cwd if no explicit path is given. If not found in cwd, falls back to `$ZFIN_HOME/portfolio.srf`. `watchlist.srf` and `.env` follow the same cascade. `metadata.srf` and `accounts.srf` are loaded from the same directory as the resolved portfolio file.

- **`transaction_log.srf` is a sibling file.** Optional. Lives next to `portfolio.srf` / `accounts.srf`. Holds user-declared `transfer::` records so the contributions pipeline can tell internal account-to-account movement apart from real external contributions. Only `type::cash` is wired in v1 — `type::in_kind` parses but is rejected downstream. Missing file → matcher is a no-op. See `REPORT.md` §5 "Transfer log" for the user-facing guide.

- **Server sync is optional.** The `ZFIN_SERVER` env var enables parallel cache syncing from a remote zfin-server instance. All server sync code silently no-ops when the URL is null.

## Dependencies

| Dependency | Purpose |
|------------|---------|
| [SRF](https://git.lerch.org/lobo/srf) | Cache file format, portfolio/watchlist parsing, serialization |
| [libvaxis](https://github.com/rockorager/libvaxis) (v0.5.1) | Terminal UI rendering |
| [z2d](https://github.com/vancluever/z2d) (v0.10.0) | Pixel chart rendering (Kitty graphics protocol) |

## Build system rules

- **Never use `addAnonymousImport`** in `build.zig`. Always use `b.addModule()` + `addImport()`. Anonymous imports cause "file belongs to multiple modules" errors and make dependency wiring opaque.