From 3aff2e61b41c62a751ca69b455042a72f4cb1819 Mon Sep 17 00:00:00 2001 From: Emil Lerch Date: Fri, 26 Jun 2026 14:53:59 -0700 Subject: [PATCH] update TODO with more comprehensive Tiingo description --- TODO.md | 98 +++++++++++++++++++++++++++++++++++++++++++-------------- 1 file changed, 75 insertions(+), 23 deletions(-) diff --git a/TODO.md b/TODO.md index aabc028..b6e14b1 100644 --- a/TODO.md +++ b/TODO.md @@ -209,41 +209,93 @@ populate. This could be solved on the server by spawning a thread to fetch the data, then returning 202 Accepted, which could then be polled client side. Maybe this is a better long term approach? -## Configurable live-quote provider (Tiingo IEX) - priority LOW +## Support Tiingo paid plan - priority LOW + +zfin hardwires Tiingo to free-tier assumptions: the provider is +constructed with `RateLimiter.perHour(io, 50)` in `Tiingo.init` +(`providers/tiingo.zig`), and the only Tiingo surface is end-of-day +candles plus the corporate actions that ride along in the same +response. A user who pays for a Tiingo plan ($30/mo Power tier and +up) gets nothing for it today - the same 50/hour throttle, the same +EOD-only data. "Support the paid plan" is the umbrella for unlocking +what that subscription actually buys: higher rate limits and +real-time IEX quotes. The two are coupled (real-time polling only +makes sense once the limit is raised), which is why they belong in +one entry rather than two. + +### Tier-aware rate limiting + +The 50/hour cap is hardcoded in `Tiingo.init` +(`RateLimiter.perHour(io, 50)`), and the module docstring bakes in +"Free tier: 50 requests/hour and 1,000 requests/day." Paid tiers +raise both ceilings substantially, so a paying subscriber is being +throttled far below their entitlement. Today only the hourly bucket +is wired; the daily ceiling isn't enforced at all (the docstring +notes it's "far from binding" for bursty EOD usage - real-time +polling changes that calculus). + +Work: + +- Make the Tiingo limits configurable instead of hardcoded. Options: + explicit `ZFIN_TIINGO_RATE_PER_HOUR` (and per-day) numeric env + knobs, or a coarser `ZFIN_TIINGO_PLAN` = `free` (default) | + `power` | ... that maps to known limits. Lean toward explicit + numeric overrides so we aren't chasing Tiingo's published per-tier + numbers as they drift. +- `RateLimiter` already supports arbitrary `init(io, max, window_ns)` + plus `perDay`/`perHour` convenience ctors, so the limiter side is + cheap. Decide whether a paid plan needs both an hourly and a daily + bucket enforced, or whether hourly alone stays sufficient. +- Caveat from `RateLimiter`'s own docs: the bucket is in-memory and + per-process - it caps a single run's burst, not usage across + separate launches in the same window. Sustained real-time polling + (below) makes cross-process usage likelier, so revisit whether + per-process accounting is still good enough. + +### Real-time IEX quotes (was: configurable live-quote provider) The TUI refresh key (`r`) values the portfolio with live intraday -quotes via `DataService.loadLiveQuotes`, which is Yahoo-only: Yahoo is -keyless, consolidated, and stays off every rate-limit budget, so bursty -refresh traffic costs nothing. The tradeoffs are that Yahoo's -unofficial feed is ~15-minute delayed and "can break without notice." +quotes via `DataService.loadLiveQuotes` (`service.zig`), which is +Yahoo-only: Yahoo is keyless, consolidated, and stays off every +rate-limit budget, so bursty refresh traffic costs nothing. The +tradeoffs are that Yahoo's unofficial feed is ~15-minute delayed and +"can break without notice." Tiingo's IEX endpoint (`/iex/?tickers=A,B,C`) is a strong opt-in -alternative: it's genuinely real-time (IEX last-sale, no 15-min delay), -official/keyed, and bills per HTTP request - one call returns the whole -portfolio (confirmed empirically: a 2-ticker batch decrements the daily -quota by 1, not 2). Fields map cleanly: `tngoLast` to price, `prevClose` -to day-change. Caveats: IEX is a single venue (~2-3% of volume), so +alternative for a paid subscriber: it's genuinely real-time (IEX +last-sale, no 15-min delay), official/keyed, and bills per HTTP +request - one call returns the whole portfolio (confirmed +empirically: a 2-ticker batch decrements the daily quota by 1, not +2). Fields map cleanly: `tngoLast` to price, `prevClose` to +day-change. Caveats: IEX is a single venue (~2-3% of volume), so `tngoLast` can sit stale between prints on illiquid names, and IEX doesn't trade mutual funds, so those fall back to the candle close. Proposal: a config knob (env var, e.g. `ZFIN_LIVE_QUOTE_PROVIDER` = `yahoo` (default) | `tiingo`) that switches `loadLiveQuotes` to a new -`Tiingo.fetchQuotes(tickers)` batched call. Someone on Tiingo's Power -tier ($30/mo, higher limits) who wants real-time and mashes `r` a lot -(or once we add streaming) reuses their existing `TIINGO_API_KEY` and -gets real-time coverage; everyone else keeps the keyless Yahoo default. +`Tiingo.fetchQuotes(tickers)` batched call. A paid subscriber who +wants real-time and mashes `r` a lot (or once we add streaming) +reuses their existing `TIINGO_API_KEY` and gets real-time coverage; +everyone else keeps the keyless Yahoo default. Implementation notes: -- `Tiingo.fetchQuotes` returns an array whose order is NOT guaranteed to - match the request order, so key results by the returned `ticker` - field, not by position. -- Tiingo-sourced live quotes would share Tiingo's 50/hour token bucket - (`RateLimiter.perHour`, wired into the provider). A batched quote - call is 1 request, but heavy `r` use plus candle refreshes draw from - the same hourly budget, so watch for contention. - - Tiingo websocket streaming would be the natural follow-on for true - push-based real-time, replacing poll-on-`r` entirely. +- `Tiingo.fetchQuotes` returns an array whose order is NOT guaranteed + to match the request order, so key results by the returned + `ticker` field, not by position. +- Live quotes share Tiingo's token bucket, so this is the concrete + reason the tier-aware rate-limiting work above has to land first + (or alongside): a batched quote call is only 1 request, but heavy + `r` use plus candle refreshes draining the free 50/hour bucket is + exactly the contention that raising the paid-tier limit relieves. + +### Websocket streaming (follow-on) + +Tiingo's IEX websocket would be the natural follow-on for true +push-based real-time, replacing poll-on-`r` entirely. Materially +bigger than the REST quote path (persistent connection, reconnect +handling, a background task feeding the TUI) and squarely a +paid-plan feature. Sequence it after the REST quote path proves out. ## Analysis: dividend equity / income-shaped equity - think about it