update TODO with more comprehensive Tiingo description

This commit is contained in:
Emil Lerch 2026-06-26 14:53:59 -07:00
parent 1d89b68da4
commit 3aff2e61b4
Signed by: lobo
GPG key ID: A7B62D657EF764F8

98
TODO.md
View file

@ -209,41 +209,93 @@ populate. This could be solved on the server by spawning a thread to fetch the
data, then returning 202 Accepted, which could then be polled client side. Maybe
this is a better long term approach?
## Configurable live-quote provider (Tiingo IEX) - priority LOW
## Support Tiingo paid plan - priority LOW
zfin hardwires Tiingo to free-tier assumptions: the provider is
constructed with `RateLimiter.perHour(io, 50)` in `Tiingo.init`
(`providers/tiingo.zig`), and the only Tiingo surface is end-of-day
candles plus the corporate actions that ride along in the same
response. A user who pays for a Tiingo plan ($30/mo Power tier and
up) gets nothing for it today - the same 50/hour throttle, the same
EOD-only data. "Support the paid plan" is the umbrella for unlocking
what that subscription actually buys: higher rate limits and
real-time IEX quotes. The two are coupled (real-time polling only
makes sense once the limit is raised), which is why they belong in
one entry rather than two.
### Tier-aware rate limiting
The 50/hour cap is hardcoded in `Tiingo.init`
(`RateLimiter.perHour(io, 50)`), and the module docstring bakes in
"Free tier: 50 requests/hour and 1,000 requests/day." Paid tiers
raise both ceilings substantially, so a paying subscriber is being
throttled far below their entitlement. Today only the hourly bucket
is wired; the daily ceiling isn't enforced at all (the docstring
notes it's "far from binding" for bursty EOD usage - real-time
polling changes that calculus).
Work:
- Make the Tiingo limits configurable instead of hardcoded. Options:
explicit `ZFIN_TIINGO_RATE_PER_HOUR` (and per-day) numeric env
knobs, or a coarser `ZFIN_TIINGO_PLAN` = `free` (default) |
`power` | ... that maps to known limits. Lean toward explicit
numeric overrides so we aren't chasing Tiingo's published per-tier
numbers as they drift.
- `RateLimiter` already supports arbitrary `init(io, max, window_ns)`
plus `perDay`/`perHour` convenience ctors, so the limiter side is
cheap. Decide whether a paid plan needs both an hourly and a daily
bucket enforced, or whether hourly alone stays sufficient.
- Caveat from `RateLimiter`'s own docs: the bucket is in-memory and
per-process - it caps a single run's burst, not usage across
separate launches in the same window. Sustained real-time polling
(below) makes cross-process usage likelier, so revisit whether
per-process accounting is still good enough.
### Real-time IEX quotes (was: configurable live-quote provider)
The TUI refresh key (`r`) values the portfolio with live intraday
quotes via `DataService.loadLiveQuotes`, which is Yahoo-only: Yahoo is
keyless, consolidated, and stays off every rate-limit budget, so bursty
refresh traffic costs nothing. The tradeoffs are that Yahoo's
unofficial feed is ~15-minute delayed and "can break without notice."
quotes via `DataService.loadLiveQuotes` (`service.zig`), which is
Yahoo-only: Yahoo is keyless, consolidated, and stays off every
rate-limit budget, so bursty refresh traffic costs nothing. The
tradeoffs are that Yahoo's unofficial feed is ~15-minute delayed and
"can break without notice."
Tiingo's IEX endpoint (`/iex/?tickers=A,B,C`) is a strong opt-in
alternative: it's genuinely real-time (IEX last-sale, no 15-min delay),
official/keyed, and bills per HTTP request - one call returns the whole
portfolio (confirmed empirically: a 2-ticker batch decrements the daily
quota by 1, not 2). Fields map cleanly: `tngoLast` to price, `prevClose`
to day-change. Caveats: IEX is a single venue (~2-3% of volume), so
alternative for a paid subscriber: it's genuinely real-time (IEX
last-sale, no 15-min delay), official/keyed, and bills per HTTP
request - one call returns the whole portfolio (confirmed
empirically: a 2-ticker batch decrements the daily quota by 1, not
2). Fields map cleanly: `tngoLast` to price, `prevClose` to
day-change. Caveats: IEX is a single venue (~2-3% of volume), so
`tngoLast` can sit stale between prints on illiquid names, and IEX
doesn't trade mutual funds, so those fall back to the candle close.
Proposal: a config knob (env var, e.g. `ZFIN_LIVE_QUOTE_PROVIDER` =
`yahoo` (default) | `tiingo`) that switches `loadLiveQuotes` to a new
`Tiingo.fetchQuotes(tickers)` batched call. Someone on Tiingo's Power
tier ($30/mo, higher limits) who wants real-time and mashes `r` a lot
(or once we add streaming) reuses their existing `TIINGO_API_KEY` and
gets real-time coverage; everyone else keeps the keyless Yahoo default.
`Tiingo.fetchQuotes(tickers)` batched call. A paid subscriber who
wants real-time and mashes `r` a lot (or once we add streaming)
reuses their existing `TIINGO_API_KEY` and gets real-time coverage;
everyone else keeps the keyless Yahoo default.
Implementation notes:
- `Tiingo.fetchQuotes` returns an array whose order is NOT guaranteed to
match the request order, so key results by the returned `ticker`
field, not by position.
- Tiingo-sourced live quotes would share Tiingo's 50/hour token bucket
(`RateLimiter.perHour`, wired into the provider). A batched quote
call is 1 request, but heavy `r` use plus candle refreshes draw from
the same hourly budget, so watch for contention.
- Tiingo websocket streaming would be the natural follow-on for true
push-based real-time, replacing poll-on-`r` entirely.
- `Tiingo.fetchQuotes` returns an array whose order is NOT guaranteed
to match the request order, so key results by the returned
`ticker` field, not by position.
- Live quotes share Tiingo's token bucket, so this is the concrete
reason the tier-aware rate-limiting work above has to land first
(or alongside): a batched quote call is only 1 request, but heavy
`r` use plus candle refreshes draining the free 50/hour bucket is
exactly the contention that raising the paid-tier limit relieves.
### Websocket streaming (follow-on)
Tiingo's IEX websocket would be the natural follow-on for true
push-based real-time, replacing poll-on-`r` entirely. Materially
bigger than the REST quote path (persistent connection, reconnect
handling, a background task feeding the TUI) and squarely a
paid-plan feature. Sequence it after the REST quote path proves out.
## Analysis: dividend equity / income-shaped equity - think about it