79 lines
5.4 KiB
Markdown
79 lines
5.4 KiB
Markdown
# Data providers and API keys
|
|
|
|
zfin aggregates several free-tier data sources, using each for what it
|
|
does best and caching aggressively to stay within limits. This page is
|
|
the reference for who supplies what, the free-tier ceilings, and where
|
|
to get keys. For the design rationale see
|
|
[Why multiple data providers](../explanation/data-providers.md); for
|
|
how keys are configured see
|
|
[environment variables](config/environment.md).
|
|
|
|
## Summary
|
|
|
|
| Data | Provider | Auth | Free-tier limit | Cache TTL |
|
|
|-----------------------|------------------|-------------------------------|------------------------------------------------|------------|
|
|
| Daily candles (OHLCV) | Tiingo | `TIINGO_API_KEY` | 1,000/day, 50/hour (free); 10,000/hour (Power) | ~24h |
|
|
| Real-time quotes | Yahoo | none | unofficial | never |
|
|
| Real-time IEX quotes | Tiingo | `TIINGO_API_KEY` | any tier (1 req per connect/batch) | never |
|
|
| Quote fallback | TwelveData | `TWELVEDATA_API_KEY` | 8/min, 800/day | never |
|
|
| Dividends | Polygon | `POLYGON_API_KEY` | 5/min | 14 days |
|
|
| Splits | Polygon | `POLYGON_API_KEY` | 5/min | 14 days |
|
|
| Options chains | CBOE | none | ~30/min (self-imposed) | 1 hour |
|
|
| Earnings | FMP | `FMP_API_KEY` | 250 req/day | 30 days |
|
|
| ETF profiles | SEC EDGAR | `ZFIN_USER_EMAIL` | 10/sec | ~90 days |
|
|
| Classification | Wikidata + EDGAR | `ZFIN_USER_EMAIL` | no daily quota | long-lived |
|
|
| CUSIP lookup | OpenFIGI | `OPENFIGI_API_KEY` (optional) | higher with key | indefinite |
|
|
|
|
## Where to get a key
|
|
|
|
| Provider | Sign up | Notes |
|
|
|------------|-------------------------------------|--------------------------------------------------------------------------------------|
|
|
| Tiingo | <https://tiingo.com> | The one key worth setting first -- it's the primary price source. |
|
|
| Polygon | <https://polygon.io> | Enables total return (dividends) and forward-looking dividend dates. |
|
|
| FMP | <https://financialmodelingprep.com> | Earnings actuals + estimates. |
|
|
| TwelveData | <https://twelvedata.com> | Optional quote fallback. |
|
|
| OpenFIGI | <https://www.openfigi.com/api> | Optional; raises CUSIP-lookup rate limits. |
|
|
| SEC EDGAR | (no signup) | Set `ZFIN_USER_EMAIL` to your address; EDGAR requires a contact in its `User-Agent`. |
|
|
|
|
## Per-provider notes
|
|
|
|
- **Tiingo** -- primary for candles across stocks, ETFs, and mutual
|
|
funds. Candles are fetched from a fixed 2000-01-01 start so the cache
|
|
supports long `--as-of` projections. Its price series also carries
|
|
per-row dividend/split data that zfin merges into the Polygon view.
|
|
Also serves **real-time IEX quotes** (the `tngoLast` reference price)
|
|
for the `r` refresh and the live stream when
|
|
[`ZFIN_LIVE_QUOTE_PROVIDER=tiingo`](config/environment.md#live-quotes-and-streaming);
|
|
the IEX reference feed works on any tier (no exchange license). Rate
|
|
limits are tier-aware via `ZFIN_TIINGO_PLAN` (free 50/hour, Power
|
|
10,000/hour) -- one batched quote call or one stream connect is a
|
|
single request.
|
|
- **Yahoo** -- primary, keyless quote source, the candle fallback when
|
|
Tiingo is unavailable, and the default live-quote/stream feed
|
|
(~15-min delayed, unofficial).
|
|
- **TwelveData** -- quote fallback only. It is no longer used for
|
|
candles (its split-adjusted closes proved unreliable for return math).
|
|
- **Polygon** -- primary for dividends and splits, including
|
|
forward-looking declared events.
|
|
- **CBOE** -- keyless options chains, 15-minute delayed during market
|
|
hours, with greeks and open interest.
|
|
- **FMP** -- earnings history and estimates. ETFs, mutual funds,
|
|
CUSIPs, and some dual-class shares return no earnings on the free
|
|
tier (a documented limitation).
|
|
- **SEC EDGAR + Wikidata** -- ETF profiles (NPORT-P holdings, sector
|
|
weights, AUM) and the classification data behind
|
|
[`enrich`](cli/enrich.md). Needs `ZFIN_USER_EMAIL`, not a key.
|
|
|
|
## Rate limiting
|
|
|
|
Each provider has a client-side token-bucket limiter sized to its
|
|
free-tier ceiling. When you'd exceed the rate, zfin blocks until a
|
|
token frees up rather than firing a request that would 429 -- so a
|
|
large `--refresh-data=force` run paces itself. See
|
|
[Caching](../explanation/caching.md#rate-limiting).
|
|
|
|
## See also
|
|
|
|
- [Environment variables](config/environment.md) -- setting each key.
|
|
- [Getting started](../getting-started.md) -- the minimal first-run setup.
|
|
- [Why multiple data providers](../explanation/data-providers.md) -- fallback behavior.
|