72 lines
3.6 KiB
Markdown
72 lines
3.6 KiB
Markdown
# Why multiple data providers
|
|
|
|
No single free data source does everything well. zfin aggregates
|
|
several, using each for what it's best at, and falls back gracefully
|
|
when one is unavailable. This page explains the design; for signup
|
|
links, limits, and the full table, see
|
|
[Data providers and API keys](../reference/providers.md).
|
|
|
|
## One source per job
|
|
|
|
Each data type has a primary provider chosen for coverage and quality:
|
|
|
|
| Data | Primary | Notes |
|
|
|-------------------------------|----------------------|-----------------------------------------------|
|
|
| Daily candles | Tiingo | Deep history; stocks, ETFs, mutual funds |
|
|
| Real-time quotes | Yahoo | No key required |
|
|
| Dividends / splits | Polygon | Carries forward-looking declared events |
|
|
| Options chains | CBOE | No key; 15-minute delayed |
|
|
| Earnings | FMP | Actuals + analyst estimates |
|
|
| ETF profiles / classification | SEC EDGAR + Wikidata | Authoritative holdings; needs a contact email |
|
|
|
|
The [data service](concepts.md#the-data-service-and-caching) hides this
|
|
behind one interface -- commands ask for "candles for VTI," not "call
|
|
Tiingo." That's also what makes the cache and rate limiting uniform
|
|
across providers.
|
|
|
|
## Fallback, not single-point-of-failure
|
|
|
|
Where a second source can stand in, zfin uses it:
|
|
|
|
- **Candles:** Tiingo is primary; **Yahoo** is the fallback if Tiingo
|
|
is unavailable or lacks the symbol.
|
|
- **Quotes:** Yahoo is primary; **TwelveData** is the fallback.
|
|
- **Dividends/splits:** Polygon is primary, but Tiingo's price-series
|
|
response carries per-row dividend and split data that zfin merges in
|
|
to rescue events Polygon's reference endpoints occasionally miss.
|
|
|
|
How a failure propagates depends on its kind. **Transient** errors
|
|
(server 5xx, connection drops) stop a refresh so you don't get a
|
|
half-updated view. **Permanent** errors (not-found, parse failures)
|
|
fall through to the next provider, and a rate-limit hit triggers a
|
|
single backoff-and-retry.
|
|
|
|
## Why some data needs a key and some doesn't
|
|
|
|
Quotes (Yahoo) and options (CBOE) are unauthenticated. The rest want a
|
|
free API key, and SEC EDGAR wants a contact email in its `User-Agent`
|
|
(set via `ZFIN_USER_EMAIL`) rather than a key. None are individually
|
|
required -- missing a key just removes that one data type, and
|
|
everything else keeps working. See
|
|
[which key unlocks what](../reference/config/environment.md#api-keys).
|
|
|
|
## A few deliberate quirks
|
|
|
|
- **Yahoo is unofficial.** Yahoo Finance has no public API; zfin reads
|
|
an undocumented endpoint that needs no key but carries no guarantees
|
|
-- it can change shape or stop responding at any time. Quotes (and
|
|
the candle fallback) ride on it, so keeping `TIINGO_API_KEY` set for
|
|
candles and `TWELVEDATA_API_KEY` for quote fallback cushions a Yahoo
|
|
outage.
|
|
- **TwelveData is no longer used for candles.** Its split-adjusted
|
|
closes proved unreliable for the return math, so it's quote-fallback
|
|
only now. Yahoo is the candle fallback.
|
|
- **Mutual-fund and dual-class coverage varies.** Some symbols (ETFs,
|
|
CUSIPs, a few dual-class shares like BRK.B) have no earnings on the
|
|
free FMP tier and show "no earnings data" -- a documented limitation,
|
|
not a bug. See the [FAQ](faq-troubleshooting.md).
|
|
|
|
## See also
|
|
|
|
- [Data providers and API keys](../reference/providers.md) -- the reference table.
|
|
- [Caching and data freshness](caching.md) -- how fetched data is reused.
|