lobo/zfin
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages 2 Wiki Activity Actions

update TODO with more comprehensive Tiingo description

Browse source
This commit is contained in:
Emil Lerch 2026-06-26 14:53:59 -07:00
parent 1d89b68da4
commit 3aff2e61b4
Signed by: lobo
GPG key ID: A7B62D657EF764F8
1 changed files with 75 additions and 23 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

98
TODO.md
View file

@ -209,41 +209,93 @@ populate. This could be solved on the server by spawning a thread to fetch the
data, then returning 202 Accepted, which could then be polled client side. Maybe data, then returning 202 Accepted, which could then be polled client side. Maybe
this is a better long term approach? this is a better long term approach?
## Configurable live-quote provider (Tiingo IEX) - priority LOW ## Support Tiingo paid plan - priority LOW
zfin hardwires Tiingo to free-tier assumptions: the provider is
constructed with `RateLimiter.perHour(io, 50)` in `Tiingo.init`
(`providers/tiingo.zig`), and the only Tiingo surface is end-of-day
candles plus the corporate actions that ride along in the same
response. A user who pays for a Tiingo plan ($30/mo Power tier and
up) gets nothing for it today - the same 50/hour throttle, the same
EOD-only data. "Support the paid plan" is the umbrella for unlocking
what that subscription actually buys: higher rate limits and
real-time IEX quotes. The two are coupled (real-time polling only
makes sense once the limit is raised), which is why they belong in
one entry rather than two.
### Tier-aware rate limiting
The 50/hour cap is hardcoded in `Tiingo.init`
(`RateLimiter.perHour(io, 50)`), and the module docstring bakes in
"Free tier: 50 requests/hour and 1,000 requests/day." Paid tiers
raise both ceilings substantially, so a paying subscriber is being
throttled far below their entitlement. Today only the hourly bucket
is wired; the daily ceiling isn't enforced at all (the docstring
notes it's "far from binding" for bursty EOD usage - real-time
polling changes that calculus).
Work:
- Make the Tiingo limits configurable instead of hardcoded. Options:
explicit `ZFIN_TIINGO_RATE_PER_HOUR` (and per-day) numeric env
knobs, or a coarser `ZFIN_TIINGO_PLAN` = `free` (default) |
`power` | ... that maps to known limits. Lean toward explicit
numeric overrides so we aren't chasing Tiingo's published per-tier
numbers as they drift.
- `RateLimiter` already supports arbitrary `init(io, max, window_ns)`
plus `perDay`/`perHour` convenience ctors, so the limiter side is
cheap. Decide whether a paid plan needs both an hourly and a daily
bucket enforced, or whether hourly alone stays sufficient.
- Caveat from `RateLimiter`'s own docs: the bucket is in-memory and
per-process - it caps a single run's burst, not usage across
separate launches in the same window. Sustained real-time polling
(below) makes cross-process usage likelier, so revisit whether
per-process accounting is still good enough.
### Real-time IEX quotes (was: configurable live-quote provider)
The TUI refresh key (`r`) values the portfolio with live intraday The TUI refresh key (`r`) values the portfolio with live intraday
quotes via `DataService.loadLiveQuotes`, which is Yahoo-only: Yahoo is quotes via `DataService.loadLiveQuotes` (`service.zig`), which is
keyless, consolidated, and stays off every rate-limit budget, so bursty Yahoo-only: Yahoo is keyless, consolidated, and stays off every
refresh traffic costs nothing. The tradeoffs are that Yahoo's rate-limit budget, so bursty refresh traffic costs nothing. The
unofficial feed is ~15-minute delayed and "can break without notice." tradeoffs are that Yahoo's unofficial feed is ~15-minute delayed and
"can break without notice."
Tiingo's IEX endpoint (`/iex/?tickers=A,B,C`) is a strong opt-in Tiingo's IEX endpoint (`/iex/?tickers=A,B,C`) is a strong opt-in
alternative: it's genuinely real-time (IEX last-sale, no 15-min delay), alternative for a paid subscriber: it's genuinely real-time (IEX
official/keyed, and bills per HTTP request - one call returns the whole last-sale, no 15-min delay), official/keyed, and bills per HTTP
portfolio (confirmed empirically: a 2-ticker batch decrements the daily request - one call returns the whole portfolio (confirmed
quota by 1, not 2). Fields map cleanly: `tngoLast` to price, `prevClose` empirically: a 2-ticker batch decrements the daily quota by 1, not
to day-change. Caveats: IEX is a single venue (~2-3% of volume), so 2). Fields map cleanly: `tngoLast` to price, `prevClose` to
day-change. Caveats: IEX is a single venue (~2-3% of volume), so
`tngoLast` can sit stale between prints on illiquid names, and IEX `tngoLast` can sit stale between prints on illiquid names, and IEX
doesn't trade mutual funds, so those fall back to the candle close. doesn't trade mutual funds, so those fall back to the candle close.
Proposal: a config knob (env var, e.g. `ZFIN_LIVE_QUOTE_PROVIDER` = Proposal: a config knob (env var, e.g. `ZFIN_LIVE_QUOTE_PROVIDER` =
`yahoo` (default) | `tiingo`) that switches `loadLiveQuotes` to a new `yahoo` (default) | `tiingo`) that switches `loadLiveQuotes` to a new
`Tiingo.fetchQuotes(tickers)` batched call. Someone on Tiingo's Power `Tiingo.fetchQuotes(tickers)` batched call. A paid subscriber who
tier ($30/mo, higher limits) who wants real-time and mashes `r` a lot wants real-time and mashes `r` a lot (or once we add streaming)
(or once we add streaming) reuses their existing `TIINGO_API_KEY` and reuses their existing `TIINGO_API_KEY` and gets real-time coverage;
gets real-time coverage; everyone else keeps the keyless Yahoo default. everyone else keeps the keyless Yahoo default.
Implementation notes: Implementation notes:
- `Tiingo.fetchQuotes` returns an array whose order is NOT guaranteed to - `Tiingo.fetchQuotes` returns an array whose order is NOT guaranteed
match the request order, so key results by the returned `ticker` to match the request order, so key results by the returned
field, not by position. `ticker` field, not by position.
- Tiingo-sourced live quotes would share Tiingo's 50/hour token bucket - Live quotes share Tiingo's token bucket, so this is the concrete
(`RateLimiter.perHour`, wired into the provider). A batched quote reason the tier-aware rate-limiting work above has to land first
call is 1 request, but heavy `r` use plus candle refreshes draw from (or alongside): a batched quote call is only 1 request, but heavy
the same hourly budget, so watch for contention. `r` use plus candle refreshes draining the free 50/hour bucket is
- Tiingo websocket streaming would be the natural follow-on for true exactly the contention that raising the paid-tier limit relieves.
push-based real-time, replacing poll-on-`r` entirely.
### Websocket streaming (follow-on)
Tiingo's IEX websocket would be the natural follow-on for true
push-based real-time, replacing poll-on-`r` entirely. Materially
bigger than the REST quote path (persistent connection, reconnect
handling, a background task feeding the TUI) and squarely a
paid-plan feature. Sequence it after the REST quote path proves out.
## Analysis: dividend equity / income-shaped equity - think about it ## Analysis: dividend equity / income-shaped equity - think about it