lobo/zfin
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

add tiingo provider as primary for candles

Browse source
This commit is contained in:
Emil Lerch 2026-03-17 08:52:25 -07:00
parent 3e13faa66f
commit 2846b7f3a3
Signed by: lobo
GPG key ID: A7B62D657EF764F8
6 changed files with 301 additions and 54 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

81
README.md
View file

@ -6,7 +6,7 @@ A financial data library, CLI, and terminal UI written in Zig. Tracks portfolios
```bash
# Set at least one API key (see "API keys" below)
export TWELVEDATA_API_KEY=your_key
export TIINGO_API_KEY=your_key
# Build
zig build
@ -31,31 +31,39 @@ Requires Zig 0.15.2 or later.
zfin aggregates data from multiple free-tier APIs. Each provider is used for the data it does best, and aggressive caching keeps usage well within free-tier limits.
### Provider summary
### Provider summary - primary providers
| Data type | Provider | Auth | Free-tier limit | Cache TTL |
|-----------------------|---------------|------------------------|----------------------------|--------------|
| Daily candles (OHLCV) | TwelveData | `TWELVEDATA_API_KEY` | 8 req/min, 800/day | 23h45m |
| Real-time quotes | TwelveData | `TWELVEDATA_API_KEY` | 8 req/min, 800/day | Never cached |
| Daily candles (OHLCV) | Tiingo | `TIINGO_API_KEY` | 1,000 req/day | 23h45m |
| Real-time quotes | Yahoo Finance | None required | Unofficial | Never cached |
| Dividends | Polygon | `POLYGON_API_KEY` | 5 req/min | 14 days |
| Splits | Polygon | `POLYGON_API_KEY` | 5 req/min | 14 days |
| Options chains | CBOE | None required | ~30 req/min (self-imposed) | 1 hour |
| Earnings | Finnhub | `FINNHUB_API_KEY` | 60 req/min | 30 days* |
| ETF profiles | Alpha Vantage | `ALPHAVANTAGE_API_KEY` | 25 req/day | 30 days |
### Tiingo
**Used for:** daily candles (primary provider for all symbols).
- Endpoint: `https://api.tiingo.com/tiingo/daily/{symbol}/prices`
- Free tier: 1,000 requests per day, no per-minute restriction.
- Covers stocks, ETFs, and mutual funds. Mutual fund NAVs are available after midnight ET.
- Candles are fetched with a 10-year + 60-day lookback window for trailing return calculations.
- Returns split-adjusted prices with `adjClose` for dividend-adjusted values.
### TwelveData
**Used for:** daily candles and real-time quotes.
**Used for:** candle fallback (when Tiingo fails), real-time quotes (fallback after Yahoo).
- Endpoint: `https://api.twelvedata.com/time_series` and `/quote`
- Free tier: 8 API credits per minute, 800 per day. Each symbol in a request costs 1 credit (batch requests do NOT reduce credit cost).
- Candles are fetched with a 10-year + 60-day lookback window for trailing return calculations.
- Returns split-adjusted but NOT dividend-adjusted prices. Total returns are computed separately using Polygon dividend data.
- Quotes are never cached (always a live fetch, ~15 min delay on free tier).
- Free tier: 8 API credits per minute, 800 per day. Each symbol in a request costs 1 credit.
- Mutual fund NAV updates can lag by a full trading day compared to Tiingo.
### Polygon
**Used for:** dividend history and stock splits.
**Used for:** dividend and stock splits information, both historical and upcoming.
- Endpoints: `https://api.polygon.io/v3/reference/dividends` and `/v3/reference/splits`
- Free tier: 5 requests per minute, unlimited daily. Full historical dividend/split data.
@ -91,10 +99,11 @@ zfin aggregates data from multiple free-tier APIs. Each provider is used for the
Set keys as environment variables or in a `.env` file (searched in the executable's parent directory, then cwd):
```bash
TWELVEDATA_API_KEY=your_key # Required for candles and quotes
POLYGON_API_KEY=your_key # Required for dividends/splits (total returns)
FINNHUB_API_KEY=your_key # Required for earnings data
ALPHAVANTAGE_API_KEY=your_key # Required for ETF profiles
TIINGO_API_KEY=your_key # Required for candles (primary provider)
TWELVEDATA_API_KEY=your_key # Candle fallback, quote fallback
POLYGON_API_KEY=your_key # Required for dividends/splits (total returns)
FINNHUB_API_KEY=your_key # Required for earnings data
ALPHAVANTAGE_API_KEY=your_key # Required for ETF profiles
```
The cache directory defaults to `~/.cache/zfin` and can be overridden with `ZFIN_CACHE_DIR`.
@ -103,7 +112,8 @@ Not all keys are required. Without a key, the corresponding data simply won't be
| Key | Without it |
|------------------------|--------------------------------------------------------------------|
| `TWELVEDATA_API_KEY` | No candles, quotes, or trailing returns |
| `TIINGO_API_KEY` | Candles fall back to TwelveData, then Yahoo |
| `TWELVEDATA_API_KEY` | No candle fallback after Tiingo, no quote fallback after Yahoo |
| `POLYGON_API_KEY` | No dividends -- trailing returns show price-only (no total return) |
| `FINNHUB_API_KEY` | No earnings data (tab disabled) |
| `ALPHAVANTAGE_API_KEY` | No ETF profiles |
@ -120,17 +130,17 @@ Every data fetch follows the same pattern:
Cache files use [SRF](https://github.com/lobo/srf) (Simple Record Format), a line-oriented key-value format. Freshness is determined by file modification time vs. the TTL for that data type.
| Data type | TTL | Rationale |
|---------------|--------------|--------------------------------------------------|
| Daily candles | 23h45m | Slightly under 24h for cron jitter tolerance |
| Dividends | 14 days | Declared well in advance |
| Splits | 14 days | Rare corporate events |
| Options | 1 hour | Prices change continuously during market hours |
| Data type | TTL | Rationale |
|---------------|--------------|-----------------------------------------------------|
| Daily candles | 23h45m | Slightly under 24h for cron jitter tolerance |
| Dividends | 14 days | Declared well in advance |
| Splits | 14 days | Rare corporate events |
| Options | 1 hour | Prices change continuously during market hours |
| Earnings | 30 days* | Quarterly events; smart refresh after announcements |
| ETF profiles | 30 days | Holdings/weights change slowly |
| Quotes | Never cached | Intended for live price checks |
| ETF profiles | 30 days | Holdings/weights change slowly |
| Quotes | Never cached | Intended for live price checks |
\* **Earnings smart refresh:** Even within the 30-day TTL, cached earnings are automatically re-fetched when an earnings date has passed but the cache still has no actual results for it. This ensures results appear promptly after an announcement without wasteful daily polling.
**Earnings smart refresh:** Even within the 30-day TTL, cached earnings are automatically re-fetched when an earnings date has passed but the cache still has no actual results for it. This ensures results appear promptly after an announcement without wasteful daily polling.
Manual refresh (`r` / `F5` in TUI) invalidates the cache for the current tab's data before re-fetching.
@ -138,13 +148,14 @@ Manual refresh (`r` / `F5` in TUI) invalidates the cache for the current tab's d
Each provider has a client-side token-bucket rate limiter that prevents exceeding free-tier limits:
| Provider | Rate limit |
|---------------|------------|
| TwelveData | 8/minute |
| Polygon | 5/minute |
| Finnhub | 60/minute |
| CBOE | 30/minute |
| Alpha Vantage | 25/day |
| Provider | Rate limit |
|---------------|-------------|
| Tiingo | 1,000/day |
| TwelveData | 8/minute |
| Polygon | 5/minute |
| Finnhub | 60/minute |
| CBOE | 30/minute |
| Alpha Vantage | 25/day |
The limiter blocks until a token is available, spreading bursts of requests automatically rather than failing with 429 errors.
@ -316,7 +327,7 @@ security_type::watch,symbol::TSLA
### Security types
- **stock** (default) -- Stocks, ETFs, and mutual funds. Prices are fetched from TwelveData. Positions are aggregated by symbol and shown with gain/loss.
- **stock** (default) -- Stocks, ETFs, and mutual funds. Prices are fetched from Tiingo (primary), TwelveData, or Yahoo (fallbacks). Positions are aggregated by symbol and shown with gain/loss.
- **option** -- Option contracts. Shown in a separate "Options" section. Shares can be negative for short positions.
- **cd** -- Certificates of deposit. Shown sorted by maturity date with rate and face value.
- **cash** -- Cash, money market, and settlement balances. Shown grouped by account with optional notes.
@ -327,7 +338,7 @@ security_type::watch,symbol::TSLA
For stock lots, prices are resolved in this order:
1. **Live API** -- Latest close from TwelveData cached candles
1. **Live API** -- Latest close from cached candles (Tiingo/TwelveData/Yahoo)
2. **Manual price** -- `price::` field on the lot (for securities without API coverage, e.g. 401k CIT share classes)
3. **Average cost** -- Falls back to the position's `open_price` as a last resort
@ -495,11 +506,13 @@ src/
classification.zig Classification metadata parser
quote.zig Real-time quote data
providers/
twelvedata.zig TwelveData: candles, quotes
tiingo.zig Tiingo: daily candles (primary)
twelvedata.zig TwelveData: candles (fallback), quotes (fallback)
polygon.zig Polygon: dividends, splits
finnhub.zig Finnhub: earnings
cboe.zig CBOE: options chains (no API key)
alphavantage.zig Alpha Vantage: ETF profiles, company overview
yahoo.zig Yahoo Finance: quotes (primary), candles (last resort)
openfigi.zig OpenFIGI: CUSIP to ticker lookup
analytics/
indicators.zig SMA, Bollinger Bands, RSI

4
src/cache/store.zig vendored
View file

@ -368,15 +368,17 @@ pub const Store = struct {
last_date: Date,
/// Which provider sourced the candle data. Used during incremental refresh
/// to go directly to the right provider instead of trying TwelveData first.
provider: CandleProvider = .twelvedata,
provider: CandleProvider = .tiingo,
};
pub const CandleProvider = enum {
twelvedata,
yahoo,
tiingo,
pub fn fromString(s: []const u8) CandleProvider {
if (std.mem.eql(u8, s, "yahoo")) return .yahoo;
if (std.mem.eql(u8, s, "tiingo")) return .tiingo;
return .twelvedata;
}
};

5
src/config.zig
View file

@ -7,6 +7,7 @@ pub const Config = struct {
polygon_key: ?[]const u8 = null,
finnhub_key: ?[]const u8 = null,
alphavantage_key: ?[]const u8 = null,
tiingo_key: ?[]const u8 = null,
openfigi_key: ?[]const u8 = null,
/// URL of a zfin-server instance for lazy cache sync (e.g. "https://zfin.lerch.org")
server_url: ?[]const u8 = null,
@ -33,6 +34,7 @@ pub const Config = struct {
self.polygon_key = self.resolve("POLYGON_API_KEY");
self.finnhub_key = self.resolve("FINNHUB_API_KEY");
self.alphavantage_key = self.resolve("ALPHAVANTAGE_API_KEY");
self.tiingo_key = self.resolve("TIINGO_API_KEY");
self.openfigi_key = self.resolve("OPENFIGI_API_KEY");
self.server_url = self.resolve("ZFIN_SERVER");
@ -71,7 +73,8 @@ pub const Config = struct {
return self.twelvedata_key != null or
self.polygon_key != null or
self.finnhub_key != null or
self.alphavantage_key != null;
self.alphavantage_key != null or
self.tiingo_key != null;
}
/// Look up a key: environment variable first, then .env file fallback.

198
src/providers/tiingo.zig Normal file
View file

@ -0,0 +1,198 @@
//! Tiingo provider -- official REST API for end-of-day prices.
//!
//! Free tier: 1,000 requests/day, no per-minute restriction.
//! Covers stocks, ETFs, and mutual funds with same-day NAV updates
//! (mutual fund NAVs available after midnight ET).
//!
//! API docs: https://www.tiingo.com/documentation/end-of-day
const std = @import("std");
const http = @import("../net/http.zig");
const Date = @import("../models/date.zig").Date;
const Candle = @import("../models/candle.zig").Candle;
const json_utils = @import("json_utils.zig");
const optFloat = json_utils.optFloat;
const jsonStr = json_utils.jsonStr;
const base_url = "https://api.tiingo.com/tiingo/daily";
pub const Tiingo = struct {
client: http.Client,
allocator: std.mem.Allocator,
api_key: []const u8,
pub fn init(allocator: std.mem.Allocator, api_key: []const u8) Tiingo {
return .{
.client = http.Client.init(allocator),
.allocator = allocator,
.api_key = api_key,
};
}
pub fn deinit(self: *Tiingo) void {
self.client.deinit();
}
/// Fetch daily candles for a symbol between two dates (inclusive).
/// Returns candles sorted oldest-first.
pub fn fetchCandles(
self: *Tiingo,
allocator: std.mem.Allocator,
symbol: []const u8,
from: Date,
to: Date,
) ![]Candle {
var from_buf: [10]u8 = undefined;
var to_buf: [10]u8 = undefined;
const from_str = from.format(&from_buf);
const to_str = to.format(&to_buf);
const symbol_url = try std.fmt.allocPrint(allocator, base_url ++ "/{s}/prices", .{symbol});
defer allocator.free(symbol_url);
const url = try http.buildUrl(allocator, symbol_url, &.{
.{ "startDate", from_str },
.{ "endDate", to_str },
.{ "token", self.api_key },
});
defer allocator.free(url);
var response = try self.client.get(url);
defer response.deinit();
return parseCandles(allocator, response.body);
}
};
/// Parse Tiingo's JSON array of price objects into Candles.
/// Tiingo returns oldest-first, which matches our convention.
fn parseCandles(allocator: std.mem.Allocator, body: []const u8) ![]Candle {
const parsed = std.json.parseFromSlice(std.json.Value, allocator, body, .{}) catch
return error.ParseError;
defer parsed.deinit();
const items = switch (parsed.value) {
.array => |a| a.items,
// Tiingo returns an object with "detail" on errors (bad symbol, auth, etc.)
else => return error.RequestFailed,
};
var candles: std.ArrayList(Candle) = .empty;
errdefer {
candles.deinit(allocator);
}
for (items) |item| {
const obj = switch (item) {
.object => |o| o,
else => continue,
};
const date = parseDate(obj.get("date")) orelse continue;
const close = optFloat(obj.get("close")) orelse continue;
try candles.append(allocator, .{
.date = date,
.open = optFloat(obj.get("open")) orelse close,
.high = optFloat(obj.get("high")) orelse close,
.low = optFloat(obj.get("low")) orelse close,
.close = close,
.adj_close = optFloat(obj.get("adjClose")) orelse close,
.volume = blk: {
const v = optFloat(obj.get("volume")) orelse break :blk 0;
break :blk @intFromFloat(@max(0, v));
},
});
}
return candles.toOwnedSlice(allocator);
}
/// Parse a Tiingo date string (e.g. "2026-03-16T00:00:00.000Z") into a Date.
fn parseDate(val: ?std.json.Value) ?Date {
const s = jsonStr(val) orelse return null;
if (s.len < 10) return null;
return Date.parse(s[0..10]) catch null;
}
// -- Tests --
test "parseCandles basic" {
const body =
\\[
\\ {
\\ "date": "2026-03-13T00:00:00.000Z",
\\ "close": 42.41, "high": 42.41, "low": 42.41, "open": 42.41,
\\ "volume": 0, "adjClose": 42.41, "adjHigh": 42.41,
\\ "adjLow": 42.41, "adjOpen": 42.41, "adjVolume": 0,
\\ "divCash": 0.0, "splitFactor": 1.0
\\ },
\\ {
\\ "date": "2026-03-16T00:00:00.000Z",
\\ "close": 42.74, "high": 42.74, "low": 42.74, "open": 42.74,
\\ "volume": 0, "adjClose": 42.74, "adjHigh": 42.74,
\\ "adjLow": 42.74, "adjOpen": 42.74, "adjVolume": 0,
\\ "divCash": 0.0, "splitFactor": 1.0
\\ }
\\]
;
const allocator = std.testing.allocator;
const candles = try parseCandles(allocator, body);
defer allocator.free(candles);
try std.testing.expectEqual(@as(usize, 2), candles.len);
// Oldest first
try std.testing.expectEqual(@as(i16, 2026), candles[0].date.year());
try std.testing.expectEqual(@as(u8, 3), candles[0].date.month());
try std.testing.expectEqual(@as(u8, 13), candles[0].date.day());
try std.testing.expectApproxEqAbs(@as(f64, 42.41), candles[0].close, 0.01);
try std.testing.expectEqual(@as(u8, 16), candles[1].date.day());
try std.testing.expectApproxEqAbs(@as(f64, 42.74), candles[1].close, 0.01);
try std.testing.expectApproxEqAbs(@as(f64, 42.74), candles[1].adj_close, 0.01);
}
test "parseCandles stock with volume" {
const body =
\\[
\\ {
\\ "date": "2026-03-16T00:00:00.000Z",
\\ "close": 183.22, "high": 185.10, "low": 180.50, "open": 181.00,
\\ "volume": 217307380, "adjClose": 183.22, "adjHigh": 185.10,
\\ "adjLow": 180.50, "adjOpen": 181.00, "adjVolume": 217307380,
\\ "divCash": 0.0, "splitFactor": 1.0
\\ }
\\]
;
const allocator = std.testing.allocator;
const candles = try parseCandles(allocator, body);
defer allocator.free(candles);
try std.testing.expectEqual(@as(usize, 1), candles.len);
try std.testing.expectApproxEqAbs(@as(f64, 181.00), candles[0].open, 0.01);
try std.testing.expectApproxEqAbs(@as(f64, 185.10), candles[0].high, 0.01);
try std.testing.expectApproxEqAbs(@as(f64, 180.50), candles[0].low, 0.01);
try std.testing.expectApproxEqAbs(@as(f64, 183.22), candles[0].close, 0.01);
try std.testing.expectEqual(@as(u64, 217307380), candles[0].volume);
}
test "parseCandles error response" {
const body =
\\{"detail": "Not found."}
;
const allocator = std.testing.allocator;
const result = parseCandles(allocator, body);
try std.testing.expectError(error.RequestFailed, result);
}
test "parseCandles empty array" {
const body = "[]";
const allocator = std.testing.allocator;
const candles = try parseCandles(allocator, body);
defer allocator.free(candles);
try std.testing.expectEqual(@as(usize, 0), candles.len);
}

2
src/root.zig
View file

@ -1,7 +1,7 @@
//! zfin -- Zig Financial Data Library
//!
//! Fetches, caches, and analyzes US equity/ETF financial data from
//! multiple free-tier API providers (Twelve Data, Polygon, Finnhub,
//! multiple free-tier API providers (Tiingo, Twelve Data, Polygon, Finnhub,
//! Alpha Vantage). Includes Morningstar-style performance calculations.
//!
//! ## Getting Started

65
src/service.zig
View file

@ -28,6 +28,7 @@ const AlphaVantage = @import("providers/alphavantage.zig").AlphaVantage;
const alphavantage = @import("providers/alphavantage.zig");
const OpenFigi = @import("providers/openfigi.zig");
const Yahoo = @import("providers/yahoo.zig").Yahoo;
const Tiingo = @import("providers/tiingo.zig").Tiingo;
const fmt = @import("format.zig");
const performance = @import("analytics/performance.zig");
const http = @import("net/http.zig");
@ -94,6 +95,7 @@ pub const DataService = struct {
cboe: ?Cboe = null,
av: ?AlphaVantage = null,
yh: ?Yahoo = null,
tg: ?Tiingo = null,
pub fn init(allocator: std.mem.Allocator, config: Config) DataService {
return .{
@ -109,6 +111,7 @@ pub const DataService = struct {
if (self.cboe) |*c| c.deinit();
if (self.av) |*av| av.deinit();
if (self.yh) |*yh| yh.deinit();
if (self.tg) |*tg| tg.deinit();
}
// ── Provider accessor ──────────────────────────────────────────
@ -231,7 +234,7 @@ pub const DataService = struct {
// ── Public data methods ──────────────────────────────────────
/// Fetch candles from providers with fallback logic.
/// Tries the provider recorded in meta (if any), then TwelveData, then Yahoo.
/// Tries the provider recorded in meta (if any), then Tiingo (primary), then TwelveData, then Yahoo.
/// Returns the candles and which provider succeeded.
fn fetchCandlesFromProviders(
self: *DataService,
@ -250,23 +253,51 @@ pub const DataService = struct {
} else |_| {}
}
// Try TwelveData
if (self.getProvider(TwelveData)) |td| {
if (td.fetchCandles(self.allocator, symbol, from, to)) |candles| {
log.debug("{s}: candles from TwelveData", .{symbol});
return .{ .candles = candles, .provider = .twelvedata };
} else |err| {
if (err == error.RateLimited) {
self.rateLimitBackoff();
if (td.fetchCandles(self.allocator, symbol, from, to)) |candles| {
log.debug("{s}: candles from TwelveData (after rate limit retry)", .{symbol});
return .{ .candles = candles, .provider = .twelvedata };
} else |_| {}
// If preferred is TwelveData, try it before Tiingo
if (preferred == .twelvedata) {
if (self.getProvider(TwelveData)) |td| {
if (td.fetchCandles(self.allocator, symbol, from, to)) |candles| {
log.debug("{s}: candles from TwelveData (preferred)", .{symbol});
return .{ .candles = candles, .provider = .twelvedata };
} else |err| {
if (err == error.RateLimited) {
self.rateLimitBackoff();
if (td.fetchCandles(self.allocator, symbol, from, to)) |candles| {
log.debug("{s}: candles from TwelveData (preferred, after rate limit retry)", .{symbol});
return .{ .candles = candles, .provider = .twelvedata };
} else |_| {}
}
}
}
} else |_| {}
}
// Primary: Tiingo (1000 req/day, no per-minute limit)
if (self.getProvider(Tiingo)) |tg| {
if (tg.fetchCandles(self.allocator, symbol, from, to)) |candles| {
log.debug("{s}: candles from Tiingo", .{symbol});
return .{ .candles = candles, .provider = .tiingo };
} else |_| {}
} else |_| {}
// Fallback: Yahoo (if not already tried as preferred)
// Fallback: TwelveData (if not already tried as preferred)
if (preferred != .twelvedata) {
if (self.getProvider(TwelveData)) |td| {
if (td.fetchCandles(self.allocator, symbol, from, to)) |candles| {
log.debug("{s}: candles from TwelveData (fallback)", .{symbol});
return .{ .candles = candles, .provider = .twelvedata };
} else |err| {
if (err == error.RateLimited) {
self.rateLimitBackoff();
if (td.fetchCandles(self.allocator, symbol, from, to)) |candles| {
log.debug("{s}: candles from TwelveData (fallback, after rate limit retry)", .{symbol});
return .{ .candles = candles, .provider = .twelvedata };
} else |_| {}
}
}
} else |_| {}
}
// Last resort: Yahoo (if not already tried as preferred)
if (preferred != .yahoo) {
if (self.getProvider(Yahoo)) |yh| {
if (yh.fetchCandles(self.allocator, symbol, from, to)) |candles| {
@ -280,7 +311,7 @@ pub const DataService = struct {
}
/// Fetch daily candles for a symbol (10+ years for trailing returns).
/// Checks cache first; fetches from TwelveData if stale/missing.
/// Checks cache first; fetches from Tiingo (primary), TwelveData, or Yahoo if stale/missing.
/// Uses incremental updates: when the cache is stale, only fetches
/// candles newer than the last cached date rather than re-fetching
/// the entire history.
@ -363,7 +394,7 @@ pub const DataService = struct {
log.debug("{s}: fetching full candle history from provider", .{symbol});
const from = today.addDays(-3700);
const result = self.fetchCandlesFromProviders(symbol, from, today, .twelvedata) catch {
const result = self.fetchCandlesFromProviders(symbol, from, today, .tiingo) catch {
s.writeNegative(symbol, .candles_daily);
return DataError.FetchFailed;
};