From 641a88b0b768769a33e3f95fa6f1736fa58e4a6f Mon Sep 17 00:00:00 2001 From: Emil Lerch Date: Sun, 31 May 2026 11:30:12 -0700 Subject: [PATCH] README update --- README.md | 178 ++++++++++++++++++++++++++++++++++++------------------ 1 file changed, 118 insertions(+), 60 deletions(-) diff --git a/README.md b/README.md index b312632..66680ef 100644 --- a/README.md +++ b/README.md @@ -4,28 +4,52 @@ A financial data library, CLI, and terminal UI written in Zig. Tracks portfolios ## Quick start +### macOS (Apple Silicon) -- pre-built binary + ```bash -# Set at least one API key (see "API keys" below) -export TIINGO_API_KEY=your_key - -# Build -zig build - -# CLI usage -zig build run -- perf VTI # trailing returns -zig build run -- quote AAPL # real-time quote -zig build run -- options AAPL # options chains -zig build run -- earnings MSFT # earnings history -zig build run -- portfolio # portfolio summary (reads portfolio.srf) -zig build run -- analysis # portfolio analysis (reads portfolio.srf + metadata.srf) - -# Interactive TUI -zig build run -- i # auto-loads portfolio.srf from cwd -zig build run -- i -p portfolio.srf -w watchlist.srf -zig build run -- i -s AAPL # start with a symbol, no portfolio +# Download the latest aarch64-macos binary, make it executable, drop on $PATH +curl -L -o zfin \ + https://git.lerch.org/api/packages/lobo/generic/zfin-aarch64-macos/latest/zfin-aarch64-macos +chmod +x zfin +sudo mv zfin /usr/local/bin/ # or anywhere on $PATH ``` -Requires Zig 0.15.2 or later. +### Linux / building from source + +```bash +# Requires Zig 0.16.0. With mise installed, `mise install` will pick the right +# version from .mise.toml. Otherwise grab Zig 0.16.0 manually. +zig build +# Binary lands at zig-out/bin/zfin +``` + +### Configure + +```bash +# Set at least one API key plus your contact email (see "API keys" below). +# The email is required for ETF profiles + `enrich` because SEC EDGAR mandates +# a User-Agent contact. +export TIINGO_API_KEY=your_key +export ZFIN_USER_EMAIL=you@example.com +``` + +### Use it + +```bash +zfin perf VTI # trailing returns +zfin quote AAPL # real-time quote +zfin options AAPL # options chains +zfin earnings MSFT # earnings history +zfin portfolio # portfolio summary (reads portfolio.srf) +zfin analysis # portfolio analysis (reads portfolio.srf + metadata.srf) + +# Interactive TUI +zfin i # auto-loads portfolio.srf from cwd +zfin i -p portfolio.srf -w watchlist.srf +zfin i -s AAPL # start with a symbol, no portfolio +``` + +Building from source requires Zig 0.16.0. ## Data providers @@ -33,15 +57,17 @@ zfin aggregates data from multiple free-tier APIs. Each provider is used for the ### Provider summary - primary providers -| Data type | Provider | Auth | Free-tier limit | Cache TTL | -|-----------------------|---------------|------------------------|----------------------------|--------------| -| Daily candles (OHLCV) | Tiingo | `TIINGO_API_KEY` | 1,000 req/day | 23h45m | -| Real-time quotes | Yahoo Finance | None required | Unofficial | Never cached | -| Dividends | Polygon | `POLYGON_API_KEY` | 5 req/min | 14 days | -| Splits | Polygon | `POLYGON_API_KEY` | 5 req/min | 14 days | -| Options chains | CBOE | None required | ~30 req/min (self-imposed) | 1 hour | -| Earnings | FMP | `FMP_API_KEY` | 250 req/day | 30 days* | -| ETF profiles | Alpha Vantage | `ALPHAVANTAGE_API_KEY` | 25 req/day | 30 days | +| Data type | Provider | Auth | Free-tier limit | Cache TTL | +|-----------------------|------------------|----------------------|----------------------------|--------------| +| Daily candles (OHLCV) | Tiingo | `TIINGO_API_KEY` | 1,000 req/day | 23h45m | +| Real-time quotes | Yahoo Finance | None required | Unofficial | Never cached | +| Quote fallback | TwelveData | `TWELVEDATA_API_KEY` | 8 req/min, 800/day | Never cached | +| Dividends | Polygon | `POLYGON_API_KEY` | 5 req/min | 14 days | +| Splits | Polygon | `POLYGON_API_KEY` | 5 req/min | 14 days | +| Options chains | CBOE | None required | ~30 req/min (self-imposed) | 1 hour | +| Earnings | FMP | `FMP_API_KEY` | 250 req/day | 30 days* | +| ETF profiles | SEC EDGAR | `ZFIN_USER_EMAIL` | 10 req/sec | ~90 days | +| Classification | Wikidata + EDGAR | `ZFIN_USER_EMAIL` | No per-day quota | Long-lived | ### Tiingo @@ -89,36 +115,51 @@ zfin aggregates data from multiple free-tier APIs. Each provider is used for the - History depth: full — often back to the 1980s for long-listed tickers. - Coverage: US stocks with real earnings. ETFs, mutual funds, CUSIPs, and some dual-class shares (BRK.B, GOOG) return 402 on the free tier and show up as "no earnings data" in the UI — a documented limitation, not a bug. -### Alpha Vantage +### SEC EDGAR + Wikidata -**Used for:** ETF profiles (expense ratio, holdings, sector weights). +**Used for:** ETF profiles (NPORT-P holdings, sector weights, AUM, inception +dates) and the `enrich` flow that bootstraps `metadata.srf` (sector / industry +/ country / asset-class classification). -- Endpoint: `https://www.alphavantage.co/query?function=ETF_PROFILE` -- Free tier: 25 requests per day. Used sparingly -- ETF profiles rarely change. +- Endpoints: + - `https://data.sec.gov/...` -- XBRL company facts, NPORT-P primary + documents, mutual-fund ticker map. + - `https://www.wikidata.org/sparql` -- sector / industry / country + statements. +- Free, but the SEC requires a contact email in the User-Agent header. zfin + reads this from `ZFIN_USER_EMAIL` (`.env` or environment). Without it, ETF + profiles and `enrich` are unavailable; everything else still works. +- The SEC caps requests at 10/sec; the rate limiter respects this. Wikidata + has no per-day quota. +- The `enrich` command queries Wikidata first (rich classification metadata); + when Wikidata has no entry for a symbol (common for managed funds, UITs), + EDGAR's mutual-fund ticker map is the fallback. Symbols that miss both fall + through as TODO entries the user fills in by hand. ## API keys Set keys as environment variables or in a `.env` file (searched in the executable's parent directory, then cwd): ```bash -TIINGO_API_KEY=your_key # Required for candles (primary provider) -TWELVEDATA_API_KEY=your_key # Quote fallback (after Yahoo) -POLYGON_API_KEY=your_key # Required for dividends/splits (total returns) -FMP_API_KEY=your_key # Required for earnings data -ALPHAVANTAGE_API_KEY=your_key # Required for ETF profiles +TIINGO_API_KEY=your_key # Required for candles (primary provider) +TWELVEDATA_API_KEY=your_key # Quote fallback (after Yahoo) +POLYGON_API_KEY=your_key # Required for dividends/splits (total returns) +FMP_API_KEY=your_key # Required for earnings data +ZFIN_USER_EMAIL=you@example.com # Required for ETF profiles + `enrich` + # (SEC EDGAR mandates a User-Agent contact) ``` The cache directory defaults to `~/.cache/zfin` and can be overridden with `ZFIN_CACHE_DIR`. Not all keys are required. Without a key, the corresponding data simply won't be available: -| Key | Without it | -|------------------------|-------------------------------------------------------------------------------------| -| `TIINGO_API_KEY` | Candles fall back to Yahoo only; some symbols (especially mutual funds) won't work | -| `TWELVEDATA_API_KEY` | No quote fallback after Yahoo | -| `POLYGON_API_KEY` | No forward-looking dividends; trailing total returns may use only Tiingo's view | -| `FMP_API_KEY` | No earnings data (tab disabled) | -| `ALPHAVANTAGE_API_KEY` | No ETF profiles | +| Key | Without it | +|----------------------|-------------------------------------------------------------------------------------------| +| `TIINGO_API_KEY` | Candles fall back to Yahoo only; some symbols (especially mutual funds) won't work | +| `TWELVEDATA_API_KEY` | No quote fallback after Yahoo | +| `POLYGON_API_KEY` | No forward-looking dividends; trailing total returns may use only Tiingo's view | +| `FMP_API_KEY` | No earnings data (tab disabled) | +| `ZFIN_USER_EMAIL` | No ETF profiles; `enrich` cannot bootstrap metadata. (Used as the EDGAR User-Agent value) | CBOE options require no API key. @@ -150,14 +191,15 @@ Manual refresh (`r` / `F5` in TUI) invalidates the cache for the current tab's d Each provider has a client-side token-bucket rate limiter that prevents exceeding free-tier limits: -| Provider | Rate limit | -|---------------|-------------| -| Tiingo | 1,000/day | -| TwelveData | 8/minute | -| Polygon | 5/minute | -| FMP | 250/day | -| CBOE | 30/minute | -| Alpha Vantage | 25/day | +| Provider | Rate limit | +|---------------|---------------------| +| Tiingo | 1,000/day | +| TwelveData | 8/minute | +| Polygon | 5/minute | +| FMP | 250/day | +| CBOE | 30/minute | +| SEC EDGAR | 10/second | +| Wikidata | (no enforced limit) | The limiter blocks until a token is available, spreading bursts of requests automatically rather than failing with 429 errors. @@ -500,7 +542,9 @@ Cash and CD lots are automatically classified as "Cash & CDs" without needing me ### Bootstrapping metadata -Use the `enrich` command to generate a starting `metadata.srf` from Alpha Vantage company overview data: +Use the `enrich` command to generate a starting `metadata.srf` from Wikidata ++ SEC EDGAR data. Requires `ZFIN_USER_EMAIL` set so EDGAR will accept the +request. ```bash # Enrich an entire portfolio (generates full metadata.srf) @@ -510,7 +554,12 @@ zfin enrich portfolio.srf > metadata.srf zfin enrich SCHD >> metadata.srf ``` -When given a file path, it fetches all stock symbols and outputs a complete SRF file with headers. When given a symbol, it outputs just the classification lines (no header), so you can append directly with `>>`. +When given a file path, it fetches all stock symbols and outputs a complete +SRF file with headers. When given a symbol, it outputs just the +classification lines (no header), so you can append directly with `>>`. +Wikidata is queried first; symbols not found in Wikidata fall through to +EDGAR's mutual-fund ticker map. Anything that misses both shows up as a +TODO entry to fill in by hand. ## Account metadata (accounts.srf) @@ -832,7 +881,7 @@ Commands: analysis [FILE] Portfolio analysis breakdowns (default: portfolio.srf) snapshot [opts] Write a daily portfolio snapshot to history/ compare [] Compare portfolio state across two dates - enrich Generate metadata.srf from Alpha Vantage + enrich Generate metadata.srf from Wikidata + SEC EDGAR lookup CUSIP to ticker lookup via OpenFIGI cache stats Show cached symbols cache clear Delete all cached data @@ -872,9 +921,11 @@ src/ polygon.zig Polygon: dividends, splits (primary, with forward-looking entries) fmp.zig FMP: earnings (actuals + estimates) cboe.zig CBOE: options chains (no API key) - alphavantage.zig Alpha Vantage: ETF profiles, company overview + Edgar.zig SEC EDGAR: ETF profiles (NPORT-P), mutual-fund ticker map, XBRL company facts + Wikidata.zig Wikidata SPARQL: classification metadata for `enrich` yahoo.zig Yahoo Finance: quotes (primary), candles (Tiingo fallback) openfigi.zig OpenFIGI: CUSIP to ticker lookup + xml.zig Vendored XML parser used by Edgar.zig (see "Vendored code") analytics/ indicators.zig SMA, Bollinger Bands, RSI performance.zig Trailing returns (as-of-date + month-end) @@ -907,14 +958,21 @@ accounts.srf Account to tax type mapping for analysis ### Dependencies -| Dependency | Source | Purpose | -|----------------------------------------------------|---------------------|---------------------------------------------------| -| [SRF](https://git.lerch.org/lobo/srf) | Git | Cache file format and portfolio/watchlist parsing | -| [libvaxis](https://github.com/rockorager/libvaxis) | Git (v0.5.1) | Terminal UI rendering | +| Dependency | Source | Purpose | +|----------------------------------------------------|----------------|---------------------------------------------------| +| [SRF](https://git.lerch.org/lobo/srf) | Git | Cache file format and portfolio/watchlist parsing | +| [libvaxis](https://github.com/rockorager/libvaxis) | Git (v0.6.0) | Terminal UI rendering | +| [z2d](https://github.com/vancluever/z2d) | Git (v0.11.0) | Pixel chart rendering (Kitty graphics protocol) | ## Building +Requires Zig 0.16.0. The canonical version is pinned in `.mise.toml`: + ```bash +# Recommended: use mise to install the right Zig and ZLS automatically +mise install + +# Or, with Zig 0.16.0 already on PATH: zig build # build the zfin binary zig build test # run all tests zig build run -- # build and run