764 lines
41 KiB
Markdown
764 lines
41 KiB
Markdown
# AGENTS.md
|
|
|
|
This file is zfin's source-of-truth for assistant agents. It
|
|
holds project-specific architecture, conventions, and gotchas
|
|
plus the universal hard rules that apply on top.
|
|
|
|
Read the ABSOLUTE PROHIBITIONS section first.
|
|
|
|
## ⛔ ABSOLUTE PROHIBITIONS — READ FIRST ⛔
|
|
|
|
|
|
### `io` vs `today` / `now_s` — design rule
|
|
|
|
The 0.16 upgrade made a deliberate choice about which Zig-0.16 `Io`
|
|
calls to thread through and which to sidestep. **This rule is
|
|
load-bearing**; please read before adding new code that needs the
|
|
current time.
|
|
|
|
- **`io: std.Io` is threaded through anything that actually does
|
|
I/O** — file reads/writes, stderr, HTTP, process spawn, terminal
|
|
detection. A function taking `io` is announcing that it touches
|
|
the outside world. The "code smell" is a feature.
|
|
- **`today: Date` is passed as a value** for functions that need
|
|
"what day is it" but don't otherwise do I/O. Captured once at
|
|
the top of the unit of work (`runCli` for CLI, `App.init` for
|
|
TUI) and threaded through. Render output stays deterministic
|
|
within a frame even if the clock ticks over mid-render.
|
|
|
|
- **Use the name `today` only when the parameter genuinely
|
|
means "the current calendar day."** That's the App-level
|
|
`app.today` field and the per-command `today` arg threaded
|
|
from `runCli`. The caller can't reasonably pass anything
|
|
other than the actual current day to these.
|
|
- **Use `as_of: Date` when the parameter is the reference
|
|
date for a computation, and the caller might legitimately
|
|
pass any value.** Examples: rolling-windows blocks (the
|
|
`--as-of` flag back-dates the view), CAGR / annualization
|
|
helpers, snapshot-resolution helpers. The function should
|
|
work correctly whether the caller passes today's date,
|
|
last week's date, or 2014. Calling such a parameter
|
|
`today` invites bugs because future maintainers will
|
|
assume the function uses it as "now" rather than "the
|
|
reference date you asked about."
|
|
- When in doubt: if the function's behavior would be
|
|
nonsensical with a date that isn't today, use `today`.
|
|
Otherwise use `as_of`. Most analytics functions are
|
|
`as_of`.
|
|
|
|
- **`now_s: i64` (or similar `before_s`/`after_s` pairs) is passed
|
|
as a value** for sub-second-precision metadata fields like
|
|
snapshot `captured_at`, rollup `#!created=`, audit cadence
|
|
staleness math. Same single-capture-and-thread pattern as
|
|
`today`.
|
|
|
|
When adding a new function that needs the current time, do NOT
|
|
reach for `std.Io.Timestamp.now(io, .real)` inside the function.
|
|
Take `today: Date` or `now_s: i64` instead. Only take `io` if the
|
|
function genuinely needs to do I/O for other reasons.
|
|
|
|
**Legitimate `Timestamp.now` callers** (each must have a
|
|
`// wall-clock required: <why>` comment justifying the read):
|
|
- `cache/store.zig` — cache entry timestamps and TTL math
|
|
- `service.zig` — per-fetch `FetchResult.timestamp`
|
|
- `net/RateLimiter.zig` — token-bucket refill
|
|
- `commands/audit.zig`, `commands/cache.zig`, `commands/history.zig`
|
|
— per-invocation `now_s` captures for staleness math, "X ago"
|
|
age displays, and rollup `#!created=` directives. Each call
|
|
site has a justifying comment.
|
|
- TUI per-frame "now" captures for relative-time display
|
|
(e.g. earnings, options, quote tabs)
|
|
- `tui.zig` `shouldDebounceWheel` — uses `.awake` (monotonic
|
|
clock) for sub-millisecond input-event debounce; resists
|
|
system clock jumps that `.real` would expose
|
|
- The single `Timestamp.now` capture in `main.zig`'s dispatch
|
|
entry that produces `today` and `now_s` for the rest of the
|
|
invocation
|
|
- The `format.todayDate(io)` helper itself (the one legitimate
|
|
capture function for unit-of-work entry points)
|
|
|
|
If you find yourself writing `Timestamp.now(io, ...)` somewhere
|
|
not on that list, either add a justifying comment or refactor
|
|
the function to take a value parameter.
|
|
|
|
### Time and money helpers — CHECK FIRST before adding any new function
|
|
|
|
This project is, at its core, thousands of lines of code about
|
|
time and money. Almost any helper you think you need to add for
|
|
date arithmetic, age calculation, dollar formatting, percentage
|
|
formatting, currency rounding, or trailing-zero handling **already
|
|
exists** under some name. Adding a near-duplicate is a recurring,
|
|
easy-to-make mistake that silently bloats the codebase with three
|
|
slightly-different ways to do the same thing.
|
|
|
|
**Before writing any new helper that touches time or money, you
|
|
MUST search for existing implementations.** Not "search if it
|
|
seems familiar" — search every time. Examples of helpers that
|
|
already exist and have caught me out:
|
|
|
|
- `Date.addYears` / `Date.subtractYears` — calendar-year math
|
|
with Feb 29 → Feb 28 clamping.
|
|
- `Date.yearsBetween` — 365.25-day approximation, returns f64.
|
|
- `Date.wholeYearsBetween` — floored, returns u16.
|
|
- `Date.ageOn` — calendar-precise age (handles "birthday hasn't
|
|
occurred this year yet"). Distinct from `wholeYearsBetween`.
|
|
- `Date.format` — Zig 0.15+ writer-style format method. Use `{f}`
|
|
to render "YYYY-MM-DD" directly into a writer.
|
|
- `Date.padRight(N)` / `Date.padLeft(N)` — column-aligned wrappers
|
|
for `{f}` rendering. Use these instead of `{s:>N}` when you
|
|
previously would have called a buffer-into-slice formatter and
|
|
passed the slice to a width-spec.
|
|
- For cases that need a `[]const u8` (URL params, struct fields),
|
|
call `std.fmt.bufPrint(&buf, "{f}", .{my_date})` into a `[10]u8`.
|
|
- `Money.from(amount)` with `{f}` — "$1,234.56" with commas,
|
|
always 2 dp. Standard format method (Zig 0.15+ format-method
|
|
protocol) — no buffer ceremony.
|
|
- `Money.from(amount).whole()` — "$1,234" rounded to whole
|
|
dollars. Returns a wrapper struct; render with `{f}`.
|
|
- `Money.from(amount).trim()` — like default but elides `.00`.
|
|
- `Money.from(amount).signed()` — "+$1,234.56" / "-$1,234.56".
|
|
- `Money.from(amount).padRight(N)` / `padLeft(N)` — column-aligned
|
|
output. Composes with `.whole()`/`.trim()`/`.signed()` (each
|
|
variant exposes its own `padRight`/`padLeft`). Generic over the
|
|
inner type via `Padded(T)` so the same wrapper works for any
|
|
`format`-bearing type.
|
|
- `format.fmtIntCommas` — "1,234,567" without `$`.
|
|
- `analytics/performance.formatReturn` — signed percent for
|
|
trailing-returns and gain/loss displays. (Lives in
|
|
`analytics/performance.zig` because returns are a
|
|
performance-domain concept; reusing it from elsewhere is fine.)
|
|
|
|
**Search recipes that catch the most cases:**
|
|
|
|
```
|
|
# Money — Money.zig should be your first stop. Search for callers:
|
|
grep -rn "Money.from\|fmt.fmtMoney" src/
|
|
|
|
# Bare-money formatter footguns (these existed pre-Money.zig and
|
|
# should be migrated to Money if found):
|
|
grep -rn "@round.*amount\|@intFromFloat(@round" src/
|
|
grep -rn "endsWith.*\\\".00\\\"\|lastIndexOfScalar.*'\\\\.'" src/
|
|
|
|
# Date / age helpers
|
|
grep -rn "fn age\|ageOn\|addYears\|subtractYears\|yearsBetween" src/
|
|
grep -rn "month() <\|day() <\|on\\.year() -" src/ # ad-hoc age math
|
|
|
|
# Time / now
|
|
grep -rn "Timestamp.now\|fromEpoch\|toEpoch" src/
|
|
```
|
|
|
|
If the search turns up an existing helper that does what you need,
|
|
**use it**. If it turns up something close-but-not-quite, prefer
|
|
extending the existing helper (new variant, optional parameter,
|
|
generalized signature) over adding a parallel one. Three closely
|
|
related functions in the same module is a smell; the user will
|
|
ask you to consolidate them and you'll have wasted both your time
|
|
and theirs.
|
|
|
|
If the search confirms nothing exists, add the new helper to the
|
|
right module:
|
|
|
|
- Date / calendar math → `src/Date.zig`, as a `Date` method
|
|
when the receiver is natural.
|
|
- Money formatting → `src/Money.zig`. New variants are wrapper
|
|
structs returned from `Money` methods; each implements
|
|
`format(self, *Writer) !void` so it works with `{f}`.
|
|
- Other number formatting (non-money) → `src/format.zig`.
|
|
- Per-domain formatting that wraps the above → keep in the
|
|
domain's view module.
|
|
|
|
Add tests in the same file. Money helpers belong next to
|
|
the other tests in `Money.zig`; date helpers belong next to
|
|
`yearsBetween`'s tests.
|
|
|
|
### Documentation-file conventions
|
|
|
|
- **`TODO.md` holds only what's still open.** Git already tracks
|
|
what was done and when. Do NOT add "DONE" markers, completion
|
|
status, strikethrough, or "shipped in ..." blurbs to TODO
|
|
entries. Just delete the section when the work is finished. If
|
|
follow-up work came out of the finished task, add it as a new
|
|
top-level section; don't leave the parent entry around as a
|
|
historical wrapper.
|
|
- **`REPORT.md` is untracked on purpose.** It's a personal workflow doc
|
|
living in the repo root only until it moves to the user's portfolio
|
|
directory. Edit it freely when asked; don't treat it as part of the
|
|
repo surface. Don't mention it in commit messages for unrelated work.
|
|
|
|
### Lint warnings — there are no "pre-existing" warnings
|
|
|
|
**Lint warnings get fixed, period.** Do NOT excuse a warning by
|
|
saying it was "pre-existing in the file" or "inherited from a
|
|
copy" or "the same style as elsewhere." That's how lint debt
|
|
accumulates to the point where the deny-warnings flag on the
|
|
pre-commit hook becomes a tripwire that everyone routes around.
|
|
|
|
The rule:
|
|
|
|
1. **Before any commit, run zlint on every file you touched
|
|
in the change.** If zlint reports any warnings on those
|
|
files, fix them in that change. There is no "I didn't
|
|
introduce it, not my problem." Once you've touched the file,
|
|
the warnings are yours.
|
|
|
|
2. **If you find pre-existing warnings in a file you didn't
|
|
intend to touch** (e.g. you ran zlint across the whole
|
|
tree for a sanity check), fix them in a SEPARATE commit and
|
|
call that out to the user explicitly so they can keep the
|
|
commits clean. Do not silently bundle drive-by lint fixes
|
|
into a feature commit.
|
|
|
|
3. **Never report a lint result by saying "0 errors, N warnings,
|
|
but they're all pre-existing."** Either fix the warnings in
|
|
this change OR report "0 errors, 0 warnings on the files I
|
|
touched. The wider tree has N warnings I haven't addressed
|
|
in this change; flagging for follow-up" — and only after
|
|
you've confirmed by file that none of the wider-tree warnings
|
|
are in files you modified.
|
|
|
|
zfin uses `zlint --deny-warnings` on the pre-commit hook. The
|
|
common zlint warning kinds and the right fix:
|
|
|
|
- `suppressed-errors` (`catch {}` / `catch "fallback"`):
|
|
replace with `try` (propagate to the caller). If the call
|
|
genuinely cannot propagate (e.g. an stderr write inside an
|
|
error-reporting path where the secondary error doesn't
|
|
matter), use `catch |err| std.log.debug(...)` or rewrite to
|
|
not need the catch. The `catch {}` form is almost never the
|
|
right answer.
|
|
- `unsafe-undefined`: add a `// SAFETY: <reason>` comment on
|
|
the line with `undefined` explaining why it's safe (e.g.
|
|
"buffer immediately overwritten by bufPrint below").
|
|
- `unused-decls`: delete the decl. Don't leave dead imports
|
|
or constants around "in case."
|
|
|
|
|
|
### Errors carry information — never throw it away
|
|
|
|
When you catch an error, the caller's first question is **"why
|
|
did this fail?"** A user-facing error message that says only
|
|
`"FetchFailed"` or `"could not parse portfolio file"` is failing
|
|
the user — they have to read source code to figure out what
|
|
happened. Three habits cause this:
|
|
|
|
**(1) Bare `catch {}` and `catch return error.X`.** Capture as
|
|
`catch |err|` and at minimum include `@errorName(err)` (or `{t}`
|
|
format) in any user-visible message. Better: re-raise a more
|
|
specific error variant from the caller's error set.
|
|
|
|
**(2) Generic `DataError.FetchFailed` collapse in service.zig.**
|
|
When `service.zig:getX` does `provider.fetchX(...) catch return
|
|
DataError.FetchFailed`, the provider's specific error
|
|
(`RateLimited` vs `Unauthorized` vs `NotFound`) is permanently
|
|
erased. Fix: either expand `DataError` to mirror the provider
|
|
distinctions, OR pass the inner error through via `catch |err| {
|
|
log.warn(... @errorName(err) ...); return ...; }` so stderr
|
|
tells the user something useful even when the typed return value
|
|
is collapsed.
|
|
|
|
**(3) User-facing stderr messages must name the error.** A CLI
|
|
command that catches an error MUST print `@errorName(err)` (or a
|
|
deliberate human translation per error kind) in the stderr
|
|
message. `"Error: Failed to fetch data for symbol"` is
|
|
unhelpful. `"Error fetching AAPL: RateLimited (free tier 5/min,
|
|
wait 60s)"` is helpful.
|
|
|
|
**The grep test:** `cli.stderrPrint` calls with hardcoded "Error:
|
|
failed to ..." strings and no error context fail this test.
|
|
|
|
### NEVER put PII in tests, fixtures, comments, or docs
|
|
|
|
This codebase is a **personal finance tool**. Real account names,
|
|
account numbers (full or partial trailing digits), real holders'
|
|
names, and any production-data identifiers are PII. Test
|
|
fixtures, doc comments, error-message examples, and sample
|
|
account-map entries must use **placeholder data only**.
|
|
|
|
The user has had to ask multiple times to scrub PII out of
|
|
tests after I introduced it via "this is real data so let me
|
|
write a test that uses it." That's wrong every time. Real data
|
|
should NEVER end up in source files.
|
|
|
|
**Approved placeholder vocabulary** (use these consistently
|
|
across the codebase so search-and-replace stays trivial):
|
|
|
|
- Account names:
|
|
- `Sample IRA`, `Sample Roth IRA`, `Sample Roth`, `Sample
|
|
Brokerage`, `Sample Trust`, `Sample HSA`, `Sample Source`,
|
|
`Sample Account`, `Sample Fid`, `Sample Fidelity Brokerage`
|
|
- With trailing digits: `Sample IRA *1234`, `Sample Brokerage
|
|
*5678`
|
|
- Filenames: `Sample_IRA_1234.txt`, `Sample_IRA_5678.txt`,
|
|
`smpl_1234`, `smpl-ira-1234`
|
|
- Account numbers: `1234`, `5678`, `9012`, `3456`, `7890`, or
|
|
alphanumeric like `Z123`, `Z111`, `Z222`. Do not use real
|
|
trailing-digit values from the user's actual accounts file.
|
|
- Portfolio file names: `portfolio_other.srf`, never
|
|
`portfolio_<real-name>.srf`.
|
|
- Schwab/Fidelity-style: `Schwab Trust`, `Sample Inherited IRA`, `Roth
|
|
IRA`, `Tax Loss` are generic enough to keep, but any
|
|
uniqueness suffix must be a placeholder.
|
|
|
|
**Things that are NEVER OK in source:**
|
|
|
|
- Real first names of family members.
|
|
- Real account-number trailing digits used in the user's actual
|
|
accounts file. Any number that came from a real brokerage entry
|
|
is PII, regardless of length or format.
|
|
- Real portfolio filenames matching `portfolio_<real-name>.srf`.
|
|
- Composite identifiers that combine real names and real
|
|
account-number digits.
|
|
- The user's actual portfolio directory path (leaks filesystem
|
|
layout) — refer generically to "the portfolio's `history/`
|
|
directory" or similar.
|
|
|
|
**Workflow rule when adding a test based on a real-world
|
|
scenario:**
|
|
|
|
1. Reproduce the bug locally with real data (the user's portfolio
|
|
directory; never staged).
|
|
2. Write the test using **placeholder names and numbers** that
|
|
preserve the structural shape of the bug (same string
|
|
lengths, same pattern of digits-vs-letters, same separator
|
|
characters, etc.) but contain no real-world identifiers.
|
|
3. Verify the test still reproduces the bug. If it doesn't, the
|
|
bug was tied to specific real-world content — investigate
|
|
whether that's a real signal (e.g. a Unicode-handling issue)
|
|
and either fix the underlying bug or find a placeholder that
|
|
exhibits the same shape.
|
|
|
|
**Workflow rule when finding existing PII:**
|
|
|
|
If you grep for PII while working in any file and find a real
|
|
name or number that snuck in, fix it in the same change. Don't
|
|
add to TODO; PII removal is never optional, and it never lands
|
|
in a separate commit unless the user explicitly asks.
|
|
|
|
**Pre-commit grep recipe.** Read the user's actual accounts
|
|
metadata locally (NOT staged, NOT in this file), extract the
|
|
identifying tokens, and grep `src/` for any of them. Conceptual
|
|
shape:
|
|
|
|
```
|
|
# Build the alternation FROM the user's accounts file at runtime —
|
|
# do NOT hardcode the values into source-tracked tooling.
|
|
local_pii_tokens=$(awk -F: '...' "$ZFIN_HOME/accounts.srf" ...)
|
|
grep -rn -E "$local_pii_tokens" src/ | grep -v ie_data.csv
|
|
```
|
|
|
|
The grep should always return zero non-`ie_data.csv` hits before
|
|
committing. The `ie_data.csv` exclusion is because the Shiller
|
|
dataset contains coincidental numeric matches in historical-year
|
|
fields that aren't PII.
|
|
|
|
If you're uncertain whether something is PII, **ask before
|
|
committing.** PII can be surgically removed from a working
|
|
tree, but once it's in `git log` it's effectively permanent.
|
|
**That includes this file** — never put real identifiers in
|
|
AGENTS.md or any other tracked file as "examples of what to look
|
|
for." The placeholder vocabulary above is the only safe way to
|
|
illustrate the patterns.
|
|
|
|
If you're uncertain whether something is PII, **ask before
|
|
committing.** PII can be surgically removed from a working
|
|
tree, but once it's in `git log` it's effectively permanent.
|
|
|
|
---
|
|
|
|
## Commands
|
|
|
|
```bash
|
|
zig build # build the zfin binary (output: zig-out/bin/zfin)
|
|
zig build test # run all tests (single binary, discovers all tests via refAllDecls + the import graph)
|
|
zig build run -- <args> # build and run CLI
|
|
zig build docs # generate library documentation
|
|
zig build coverage # run tests with kcov coverage (Linux only). See "Coverage" section.
|
|
zig build coverage -Dcoverage-threshold=72 # fail build if coverage < N% (see .pre-commit-config.yaml for current floor)
|
|
```
|
|
|
|
**Tooling** (managed via `.mise.toml`):
|
|
- Zig 0.16.0 (minimum)
|
|
- ZLS 0.16.0
|
|
- zlint 0.7.9
|
|
|
|
**Linting**: `zlint --deny-warnings --fix` (runs via pre-commit on staged `.zig` files).
|
|
|
|
**Formatting**: `zig fmt` (enforced by pre-commit). Always run before committing.
|
|
|
|
**Pre-commit hooks** run in order: trailing-whitespace, end-of-file-fixer, zig-fmt, zlint, zig-build, then tests with coverage.
|
|
|
|
## Architecture
|
|
|
|
Single binary (CLI + TUI) built from `src/main.zig`. No separate library binary for internal use — the library module (`src/root.zig`) exists only for downstream consumers and documentation generation.
|
|
|
|
### Data flow
|
|
|
|
```
|
|
User input → main.zig (CLI dispatch) or tui.zig (TUI event loop)
|
|
→ commands/*.zig (CLI) or tui/*.zig (TUI tab renderers)
|
|
→ DataService (service.zig) — sole data access layer
|
|
→ Cache check (cache/store.zig, SRF files in ~/.cache/zfin/{SYMBOL}/)
|
|
→ Server sync (optional ZFIN_SERVER, parallel HTTP)
|
|
→ Provider fetch (providers/*.zig, rate-limited HTTP)
|
|
→ Cache write
|
|
→ analytics/*.zig (performance, risk, valuation calculations)
|
|
→ format.zig (shared formatters, braille charts)
|
|
→ views/*.zig (view models — renderer-agnostic display data)
|
|
→ stdout (CLI via buffered Writer) or vaxis (TUI terminal rendering)
|
|
```
|
|
|
|
### Key design decisions
|
|
|
|
- **Internal imports use file paths, not module names.** Only external dependencies (`srf`, `vaxis`, `z2d`) use `@import("name")`. Internal code uses relative paths like `@import("Date.zig")` or `@import("models/portfolio.zig")`. This is intentional — it lets `refAllDecls` in the test binary discover all tests across the entire source tree.
|
|
|
|
- **DataService is the sole data source.** Both CLI and TUI go through `DataService` for all fetched data. Never call provider APIs directly from commands or TUI tabs.
|
|
|
|
- **Providers are lazily initialized.** `DataService` fields like `td`, `pg`, `fh` start as `null` and are created on first use via `getProvider()`. The provider field name is derived from the type name at comptime.
|
|
|
|
- **Cache uses SRF format.** [SRF](https://git.lerch.org/lobo/srf) (Simple Record Format) is a line-oriented key-value format. Cache layout: `{cache_dir}/{SYMBOL}/{data_type}.srf`. Freshness is determined by file mtime vs TTL.
|
|
|
|
- **Candles use incremental updates.** On cache miss, only candles newer than the last cached date are fetched (not the full 10-year history). The `candles_meta.srf` file tracks the last date and provider without deserializing the full candle file.
|
|
|
|
- **View models separate data from rendering.** `views/portfolio_sections.zig` produces renderer-agnostic structs with `StyleIntent` enums. CLI and TUI renderers are thin adapters that map `StyleIntent` to ANSI colors or vaxis styles.
|
|
|
|
- **Negative cache entries.** When a provider fetch fails permanently (not rate-limited), a negative cache entry is written to prevent repeated retries for nonexistent symbols.
|
|
|
|
- **TUI tab framework.** The TUI is a registry-driven framework with eight tabs sharing infrastructure. The single source of truth is `tab_modules` in `src/tui.zig` (an anonymous struct literal mapping tag → module). Everything else — the `Tab` enum, tab-bar labels, the `TabStates` aggregator, key/mouse dispatch, help overlay rows, status-line hints, draw routing — is derived from `tab_modules` at comptime. Each tab module conforms to a contract documented in `src/tui/tab_framework.zig`: declare an `Action` enum, a `State` struct, a `tab` namespace with required hooks, and exactly one of `buildStyledLines` or `drawContent`. A comptime validator (`tab_framework.validateTabModule`) checks every registered module at build time, including hook signatures and a "tabs cannot bind globally-bound keys" rule. See "Adding a new TUI tab" below for the workflow.
|
|
|
|
### Module map
|
|
|
|
| Directory | Purpose |
|
|
|-----------|---------|
|
|
| `src/models/` | Data types: `Candle`, `Dividend`, `Split`, `Lot`, `Portfolio`, `OptionContract`, `EarningsEvent`, `EtfProfile`, `Quote`. (`Date` and `Money` are top-level types in `src/Date.zig` and `src/Money.zig`.) |
|
|
| `src/providers/` | API clients: each provider has its own struct with `init(allocator, api_key)` + fetch methods. `json_utils.zig` has shared JSON parsing helpers. |
|
|
| `src/analytics/` | Pure computation: `performance.zig` (Morningstar-style trailing returns), `risk.zig` (Sharpe, drawdown), `valuation.zig` (portfolio summary), `analysis.zig` (breakdowns by class/sector/geo), `benchmark.zig` (per-position benchmark returns), `indicators.zig` (SMA/Bollinger/RSI), `milestones.zig` (retirement-attainment grid), `projections.zig` (Monte Carlo + percentile bands), `timeline.zig` (history-tab tier rollup). |
|
|
| `src/data/` | Static / semi-static datasets: `imported_values.zig` (back-history values), `shiller.zig` (S&P + CPI series, Shiller's data set), `staleness.zig` (account-cadence checks). `ie_data.csv` is the raw Shiller dataset. |
|
|
| `src/views/` | View models producing renderer-agnostic display data with `StyleIntent`: `compare.zig`, `history.zig`, `portfolio_sections.zig`, `projections.zig`. |
|
|
| `src/commands/` | CLI command handlers: each has a `pub fn run(...)` entry point. Signatures vary by command's needs (some take `as_of`, `now_s`, `args`, etc.); `common.zig` has shared CLI helpers and color constants. See "Adding a new CLI command" below. |
|
|
| `src/tui/` | Eight-tab interactive TUI. Each tab is a separate file conforming to the framework contract documented in `tab_framework.zig`: `portfolio_tab.zig`, `quote_tab.zig`, `performance_tab.zig`, `options_tab.zig`, `earnings_tab.zig`, `analysis_tab.zig`, `history_tab.zig`, `projections_tab.zig`. Plus `keybinds.zig` (configurable input + scoped bindings), `theme.zig` (configurable colors), `chart.zig` (Kitty graphics chart renderer), `projection_chart.zig` (percentile-band overlay), `input_buffer.zig` (modal text-input state machine). The `App` orchestrator lives in the parent `src/tui.zig`. |
|
|
| `src/cache/` | `store.zig`: SRF cache read/write with TTL freshness checks. |
|
|
| `src/net/` | `http.zig`: HTTP client with retry and error classification. `RateLimiter.zig`: token-bucket rate limiter. |
|
|
| `build/` | Build-time support: `Coverage.zig` (kcov integration), `download_kcov.zig` (kcov binary fetcher), `gen_shiller.zig` (CSV → comptime data converter), `bcov.css` (kcov report styling). |
|
|
|
|
## Code patterns and conventions
|
|
|
|
### Error handling
|
|
|
|
- Provider HTTP errors are classified in `net/http.zig`: `RequestFailed`, `RateLimited`, `Unauthorized`, `NotFound`, `ServerError`, `InvalidResponse`.
|
|
- `DataService` wraps these into `DataError`: `NoApiKey`, `FetchFailed`, `TransientError`, `AuthError`, etc.
|
|
- Transient errors (server 5xx, connection failures) cause the refresh to stop. Non-transient errors (NotFound, ParseError) cause fallback to the next provider.
|
|
- Rate limit hits trigger a single retry after `rateLimitBackoff()`.
|
|
|
|
### The `Date` type
|
|
|
|
`Date` is an `i32` of days since Unix epoch. It is used everywhere instead of timestamps. Construction: `Date.fromYmd(2024, 1, 15)` or `Date.parse("2024-01-15")`. Formatting: `try writer.print("{f}", .{date})` writes `YYYY-MM-DD` directly to a writer (Zig 0.15+ format method). For column-aligned output, use `date.padLeft(12)` / `date.padRight(12)` with `{f}`. For cases that need a `[]const u8` (URL params, struct fields), call `std.fmt.bufPrint(&buf, "{f}", .{date})` into a `[10]u8` buffer. The type has SRF serialization hooks (`srfParse`, `srfFormat`).
|
|
|
|
### Formatting pattern
|
|
|
|
Functions in `format.zig` write into caller-provided buffers and return slices. They never allocate. Example: `fmtIntCommas(&buf, value)` returns `[]const u8`. Money formatting now lives in `src/Money.zig` and uses the `{f}` format-method protocol — see the "Time and money helpers" prohibition section above.
|
|
|
|
### Provider pattern
|
|
|
|
Each provider in `src/providers/` follows the same structure:
|
|
1. Struct with `client: http.Client`, `allocator`, `api_key`
|
|
2. `init(allocator, api_key)` / `deinit()`
|
|
3. `fetch*(allocator, symbol, ...)` methods that build a URL, call `self.client.get(url)`, and parse the JSON response
|
|
4. Private `parse*` functions that handle the provider-specific JSON format
|
|
5. Shared JSON helpers from `json_utils.zig` (`parseJsonFloat`, `optFloat`, `optUint`, `jsonStr`)
|
|
|
|
### Test pattern
|
|
|
|
All tests are inline (in `test` blocks within source files). There is a single test binary rooted at `src/main.zig` which uses `std.testing.refAllDecls(@This())` to sema-touch every top-level decl in main.zig. Each decl that's a `@import(...)` of a source file pulls that file into compilation, which causes its `test` blocks to be collected by the test runner. The `tests/` directory exists but fixtures are empty — all test data is defined inline.
|
|
|
|
Tests use `std.testing.allocator` (which detects leaks) and are structured as unit tests that verify individual functions. Network-dependent code is not tested (no mocking infrastructure).
|
|
|
|
### Test discovery
|
|
|
|
`zig build test` runs every `test` block in every file reachable from
|
|
`src/main.zig` via the import graph, courtesy of
|
|
`std.testing.refAllDecls(@This())` in main.zig's bottom `test {}` block.
|
|
"Reachable" means: somewhere in the chain there's a
|
|
`const foo = @import("foo.zig");` (or any other binding to the imported
|
|
file struct, including `pub const`, struct field, etc).
|
|
|
|
**The verification workflow is dead simple:**
|
|
|
|
```bash
|
|
zig build test --summary all 2>&1 | grep "tests passed"
|
|
```
|
|
|
|
Run it before and after a change. If the count moved the way you expected,
|
|
you're done. If it didn't, fix it. There is no further analysis required —
|
|
no manual graph walking, no canary tests, no dependency archaeology.
|
|
|
|
**The one gotcha:** if you import a file purely as a type extraction —
|
|
`const T = @import("foo.zig").T;` — the test blocks in `foo.zig` are NOT
|
|
collected. The fix is to bind the file struct itself somewhere:
|
|
`const foo = @import("foo.zig");`. You'll know this happened because the
|
|
test count won't go up after adding tests to a new file. The `_ = @import(...)`
|
|
escape hatch in main.zig's test block is also fine if reshaping imports is
|
|
inconvenient — `refAllDecls` will sema-touch it.
|
|
|
|
**Do NOT clear the cache "to be sure."** Cache is content-addressed; it
|
|
isn't the problem. See the prohibitions at the top of this file.
|
|
|
|
### Coverage
|
|
|
|
Test coverage is measured by `zig build coverage`, which runs the
|
|
test binary under [kcov](https://simonkagstrom.github.io/kcov/) and
|
|
emits an HTML report under `coverage/` plus a one-line summary on
|
|
stdout:
|
|
|
|
```
|
|
Total test coverage: 65.15% (15399/23638)
|
|
```
|
|
|
|
**The pre-commit hook enforces a coverage floor.** The exact
|
|
threshold lives in `.pre-commit-config.yaml` as the
|
|
`-Dcoverage-threshold=N` flag on the `test` hook — that's the
|
|
source of truth, always. The hook runs
|
|
`zig build coverage -Dcoverage-threshold=N` and fails the commit
|
|
if coverage drops below `N`. Bumping the floor over time is
|
|
encouraged — every time we push the actual coverage materially
|
|
higher, raise the threshold in the pre-commit config in the same
|
|
commit so the gain is locked in.
|
|
|
|
**Coverage expectations for new work:**
|
|
|
|
1. **For most features, coverage should go UP, or you should be
|
|
able to explain why not.** New analytics modules, parsers,
|
|
loaders, formatters, and pure-domain transforms are easy to
|
|
cover and should be — they're the load-bearing logic. New
|
|
tests on existing files also nudge the percentage up by
|
|
exercising more lines of the same code.
|
|
|
|
2. **It's fine for some additions to push coverage flat or
|
|
slightly down.** Examples:
|
|
- **New TUI rendering paths.** TUI code that needs an
|
|
interactive vaxis context (event handlers, draw callbacks,
|
|
mouse handling) is hard to test without a vaxis-aware test
|
|
harness, which we don't have. Adding a new tab will add
|
|
uncovered lines unless you can extract a pure render
|
|
function and test it in isolation.
|
|
- **New provider HTTP code.** We don't mock providers; live
|
|
network calls aren't run in tests. Provider request/response
|
|
parsers ARE testable (and SHOULD be tested) — extract them
|
|
from the HTTP-bound code so they can be exercised with
|
|
fixture bytes.
|
|
- **CLI command dispatch glue in `src/main.zig`.** The command
|
|
match chain itself doesn't need tests; the command's
|
|
`run()` function and helpers should.
|
|
|
|
3. **If coverage drops, document why in the commit message.** A
|
|
single sentence — "Adds TUI tab; pure render fn covered;
|
|
event handlers and mouse handlers uncovered, no test
|
|
harness" — is enough. Future-you will thank present-you.
|
|
|
|
**How to investigate uncovered lines:**
|
|
|
|
```bash
|
|
zig build coverage
|
|
# Open coverage/index.html in a browser, or:
|
|
ls coverage/ # per-file HTML reports
|
|
```
|
|
|
|
Each file's report shows red lines (uncovered) and green lines
|
|
(covered). For a quick numeric breakdown by file, the kcov JSON
|
|
output under `coverage/kcov-merged/coverage.json` is greppable.
|
|
|
|
**Common reasons coverage looks lower than expected:**
|
|
|
|
- A new `.zig` file's tests aren't being discovered. Check the
|
|
Test discovery section above. The tests pass-or-fail report
|
|
will say "0 tests" for that module if it's orphaned.
|
|
- A function has many branches but the tests only exercise the
|
|
happy path. Add error-path tests with `expectError`.
|
|
- A switch over many enum variants only tests one. Loop the test
|
|
over all variants.
|
|
- Code is dead. Either start using it or delete it.
|
|
|
|
**Don't game the metric.** If you find yourself adding tests that
|
|
don't actually verify behavior just to pump the percentage —
|
|
`try expect(true)` calls, tests that only construct types and
|
|
check field defaults, etc. — stop. The gate exists to catch real
|
|
regressions in test discipline; gaming it produces tests that
|
|
will fail to catch real bugs later.
|
|
|
|
### Adding a new CLI command
|
|
|
|
1. Create `src/commands/newcmd.zig` with a `pub fn run(allocator, *DataService, symbol, color, *Writer) !void`
|
|
2. Add the import to the `commands` struct in `src/main.zig`
|
|
3. Add the dispatch branch in `main.zig`'s command matching chain
|
|
4. Update the `usage` string in `main.zig`
|
|
|
|
### Loading the portfolio (live mode)
|
|
|
|
For commands that operate on the live portfolio, **always use
|
|
`cli.loadPortfolio(ctx, as_of)`**. It resolves `-p` patterns
|
|
through `framework.resolvePatterns` and union-merges every
|
|
matching `portfolio_*.srf` file in `ZFIN_HOME` (or cwd), so the
|
|
CLI sees the same merged view as the TUI.
|
|
|
|
Do not introduce a new "load just the first resolved file"
|
|
helper. There used to be a `loadPortfolioFromFile(io, alloc,
|
|
path, as_of)` convenience for exactly that — it was deleted
|
|
because every production caller had the same bug: a user with
|
|
`portfolio.srf` plus sibling `portfolio_NNNN.srf` files saw
|
|
silently-different totals from the CLI vs the TUI. The
|
|
union-merge contract is now enforced by the absence of any
|
|
single-file alternative; keep it that way.
|
|
|
|
When you need historical (snapshot) data instead of the live
|
|
portfolio, that's a different code path entirely:
|
|
`history.loadSnapshotAt(io, alloc, hist_dir, date)` reads a
|
|
single `*-portfolio.srf` snapshot file produced by `zfin
|
|
snapshot`, and that is the correct single-file load. The
|
|
distinction: snapshot files are immutable historical records,
|
|
not the user's currently-edited portfolio.
|
|
|
|
### Adding a new provider
|
|
|
|
1. Create `src/providers/newprovider.zig` following the existing struct pattern
|
|
2. Add a field to `DataService` (e.g., `np: ?NewProvider = null`)
|
|
3. Add the API key to `Config` (e.g., `newprovider_key: ?[]const u8 = null`) — the field name must be the lowercased type name + `_key` for the comptime `getProvider` lookup to work
|
|
4. Wire `resolve("NEWPROVIDER_API_KEY")` in `Config.fromEnv`
|
|
|
|
### Adding a new TUI tab
|
|
|
|
The TUI uses a comptime-derived tab registry (`tab_modules` in
|
|
`src/tui.zig`). Adding a new tab is one append to the registry
|
|
plus a tab module that conforms to the framework contract. The
|
|
`Tab` enum, tab-bar label, `TabStates` aggregator, key/mouse
|
|
dispatch, help overlay, status hint, and draw routing all flow
|
|
from the registry at comptime — App needs no hand-edits.
|
|
|
|
1. **Create `src/tui/newtab_tab.zig`** with:
|
|
- `pub const Action = enum { ... };` — tab-local keybind
|
|
actions (or empty).
|
|
- `pub const State = struct { ... };` — tab-private state
|
|
(cursor, expansion flags, cached load state, etc).
|
|
- `pub const tab = struct { ... };` — the framework
|
|
contract: `label`, `default_bindings`, `action_labels`,
|
|
`status_hints`, lifecycle hooks (`init`, `deinit`,
|
|
`activate`, `deactivate`, `reload`, `tick`),
|
|
`handleAction`, optional event hooks (`handleKey`,
|
|
`handleMouse`, `handlePaste`, `statusOverride`,
|
|
`onSymbolChange`, `onScroll`, `onCursorMove`,
|
|
`isDisabled`).
|
|
- **Exactly one** of `pub fn buildStyledLines(state, app, arena)`
|
|
(line-list rendering) or `pub fn drawContent(state, app,
|
|
arena, buf, width, height)` (direct cell-buffer rendering,
|
|
for charts). The framework validator enforces this.
|
|
2. **Append to `tab_modules`** at the top of `src/tui.zig`:
|
|
`.newtab = @import("tui/newtab_tab.zig"),`
|
|
3. Done. The compiler's framework validator (in
|
|
`tab_framework.validateTabModule`) will reject the build
|
|
with a precise message if any required hook is missing or
|
|
has the wrong signature. There's also a comptime check that
|
|
the tab's keybindings don't conflict with the global keymap.
|
|
|
|
For the contract details, read the doc-block at the top of
|
|
`src/tui/tab_framework.zig` — it shows the full required
|
|
shape with example signatures.
|
|
|
|
### `anytype` is almost never the right answer — pause and ask first
|
|
|
|
Empirically, every time `anytype` looked necessary in this
|
|
codebase it turned out not to be. Concrete-typed parameters
|
|
always worked once we actually tried them. **Before adding any
|
|
new `anytype` parameter, stop and reconsider.** The discussion
|
|
that surfaces "no, this can be `*App` after all" is short and
|
|
worth having every time.
|
|
|
|
Common reasons people reach for `anytype` and what to do instead:
|
|
|
|
- **"I want to avoid a circular import."** Test the assumption.
|
|
Zig resolves `a.zig ↔ b.zig` cycles fine in most cases — file
|
|
structs are evaluated lazily, and the cycle only fails if
|
|
evaluation actually loops. Just write the concrete type and
|
|
run `zig build`. If it fails, the answer is usually to extract
|
|
the shared type to a third file, not to weaken the contract
|
|
with `anytype`. (See: the `tab_framework.zig` ↔ `tui.zig` cycle
|
|
— caught me once; turned out Zig handled it cleanly.)
|
|
- **"This function is genuinely polymorphic over many types."**
|
|
In Zig, the right shape for runtime polymorphism is usually
|
|
`*anyopaque` + an explicit cast at the boundary, paired with a
|
|
vtable struct of fn pointers. That's harder to abuse than
|
|
`anytype` and the cast site documents the type contract
|
|
unambiguously. Compile-time polymorphism over a known set of
|
|
types is better expressed with `comptime T: type` parameters.
|
|
- **"It's an event union and I don't want to pattern-match in
|
|
every caller."** Split the contract by event class
|
|
(`handleKey`, `handleMouse`, `handlePaste`) and let the
|
|
dispatcher do the discrimination once. Each method gets a
|
|
concrete type; tabs that don't care about a class simply omit
|
|
the method.
|
|
- **"It's a test helper that takes any struct."** This is the
|
|
one case where `anytype` is sometimes OK — generic test
|
|
utilities like `std.testing.expectEqual`. But check whether a
|
|
concrete type would do.
|
|
|
|
The real cost of `anytype` is that it punches a hole in the type
|
|
contract: the compiler can't catch wrong-shape arguments at the
|
|
boundary, signatures stop being self-documenting, and each
|
|
callsite gets monomorphized into its own instantiation that may
|
|
or may not behave the same as the others. The framework's whole
|
|
point is "let the compiler enforce the shape." `anytype` defeats
|
|
that.
|
|
|
|
If you've considered the alternatives above and still believe
|
|
`anytype` is correct, **flag it in your message to the user
|
|
before writing the code.** Phrase it as "I think this needs
|
|
`anytype` because X — does that match your intuition?" so the
|
|
default is discussion, not silently-typed-loose code.
|
|
|
|
### Command `run()` signatures — allocator as code smell
|
|
|
|
A CLI command's `run()` function that takes `*DataService` and `*std.Io.Writer`
|
|
usually doesn't also need an `std.mem.Allocator` parameter. `FetchResult(T)`
|
|
carries its own allocator and self-deinits (see `src/service.zig`), so callers
|
|
never need to wire up matching allocators for payload cleanup. The writer
|
|
owns its own buffer.
|
|
|
|
If a new `run()` signature still wants an allocator, ask whether the work
|
|
it's funding is:
|
|
|
|
- **Legitimate**: file I/O for a secondary config (portfolio load,
|
|
metadata), non-trivial intermediate computation, or an arena wrapping
|
|
view-layer allocations. Keep it.
|
|
- **Suspicious**: freeing `FetchResult.data` manually instead of calling
|
|
`result.deinit()`, or duplicating strings that could be borrowed.
|
|
Drop the allocator and fix the leak-shaped helper.
|
|
|
|
Not a hard rule — just a signal worth questioning when reviewing a new
|
|
command.
|
|
|
|
## Gotchas
|
|
|
|
- **Provider field naming is comptime-derived.** `DataService.getProvider(T)` finds the `?T` field by iterating struct fields at comptime, and the config key is derived by lowercasing the type name and appending `_key`. If you rename a provider struct, you must also rename the config field or the comptime logic breaks.
|
|
|
|
- **Candle data has two cache files.** `candles_daily.srf` holds the actual OHLCV data; `candles_meta.srf` holds metadata (last_date, provider, fail_count). When invalidating candles, both must be cleared (this is handled by `DataService.invalidate`).
|
|
|
|
- **TwelveData candles are force-refetched.** Cached candles from the TwelveData provider are treated as stale regardless of TTL because TwelveData's `adj_close` values were found to be unreliable. The code in `getCandles` explicitly checks `m.provider == .twelvedata` and falls through.
|
|
|
|
- **Mutual fund detection is heuristic.** `isMutualFund` checks if the symbol is exactly 5 chars ending in 'X'. This skips earnings fetching for mutual funds. It's imperfect but covers the common case.
|
|
|
|
- **SRF string lifetimes.** When reading SRF records, string fields point into the iterator's internal buffer. If you need strings to outlive the iterator, use a `postProcess` callback to `allocator.dupe()` them (see `dividendPostProcess` in `service.zig`).
|
|
|
|
- **Buffered stdout.** CLI output uses a single `std.Io.Writer` with a 4096-byte stack buffer, flushed once at the end of `main()`. Don't write to stdout through other means.
|
|
|
|
- **The `color` parameter flows through everything.** CLI commands accept a `color: bool` parameter. Don't use ANSI escapes unconditionally — always gate on the `color` flag.
|
|
|
|
- **Portfolio auto-detection.** Both CLI and TUI auto-load `portfolio.srf` from cwd if no explicit path is given. If not found in cwd, falls back to `$ZFIN_HOME/portfolio.srf`. `watchlist.srf` and `.env` follow the same cascade. `metadata.srf` and `accounts.srf` are loaded from the same directory as the resolved portfolio file.
|
|
|
|
- **`transaction_log.srf` is a sibling file.** Optional. Lives next to `portfolio.srf` / `accounts.srf`. Holds user-declared `transfer::` records so the contributions pipeline can tell internal account-to-account movement apart from real external contributions. Only `type::cash` is wired in v1 — `type::in_kind` parses but is rejected downstream. Missing file → matcher is a no-op. See `REPORT.md` §5 "Transfer log" for the user-facing guide.
|
|
|
|
- **Server sync is optional.** The `ZFIN_SERVER` env var enables parallel cache syncing from a remote zfin-server instance. All server sync code silently no-ops when the URL is null.
|
|
|
|
## Dependencies
|
|
|
|
| Dependency | Purpose |
|
|
|------------|---------|
|
|
| [SRF](https://git.lerch.org/lobo/srf) | Cache file format, portfolio/watchlist parsing, serialization |
|
|
| [libvaxis](https://github.com/rockorager/libvaxis) (v0.6.0) | Terminal UI rendering |
|
|
| [z2d](https://github.com/vancluever/z2d) (v0.11.0) | Pixel chart rendering (Kitty graphics protocol) |
|