# zfin-server

HTTP data service backed by [zfin](https://git.lerch.org/lobo/zfin)'s financial
data providers. Serves cached market data over HTTP for two consumers:

1. **LibreCalc spreadsheets** — trailing returns via `WEBSERVICE`/`FILTERXML`
2. **zfin CLI/TUI** — raw SRF cache files as an L1 data cache

## Quick start

```sh
# Set API keys (same as zfin)
export TWELVEDATA_API_KEY=...
export POLYGON_API_KEY=...

# Start the server
zig build run -- serve --port=8080

# In another terminal
curl http://localhost:8080/AAPL/returns
curl http://localhost:8080/AAPL/returns?fmt=xml
```

## Endpoints

| Route | Content-Type | Description |
|-------|-------------|-------------|
| `GET /` | `text/html` | Landing page |
| `GET /help` | `text/plain` | Endpoint documentation |
| `GET /symbols` | `application/json` | List of tracked symbols |
| `GET /:symbol/returns` | `application/json` | Trailing 1/3/5/10yr returns + volatility |
| `GET /:symbol/returns?fmt=xml` | `application/xml` | Same, XML for LibreCalc |
| `GET /:symbol/watch` | `application/json` | Add the symbol to the watchlist (authenticated) |
| `GET /:symbol/quote` | `application/json` | Latest quote |
| `GET /:symbol/candles` | `application/x-srf` | Raw SRF cache file |
| `GET /:symbol/dividends` | `application/x-srf` | Raw SRF cache file |
| `GET /:symbol/earnings` | `application/x-srf` | Raw SRF cache file |
| `GET /:symbol/options` | `application/x-srf` | Raw SRF cache file |
| `GET /:symbol/classification` | `application/x-srf` | Wikidata classification (SRF) |
| `GET /:symbol/etf_metrics` | `application/x-srf` | EDGAR NPORT-P fund metrics; 404 for non-funds |
| `GET /:cik/entity_facts` | `application/x-srf` | EDGAR XBRL entity facts (CIK-keyed) |
| `GET /_edgar/tickers_funds` | `application/x-srf` | EDGAR mutual-fund ticker map (~3 MB) |
| `GET /_edgar/tickers_companies` | `application/x-srf` | EDGAR company ticker map (~5 MB) |

The SRF endpoints behave as an L2 cache. A present cache file is served
as-is (even if stale -- the `refresh` cron is the freshness authority).
On a miss the server fetches once from the upstream provider, fills the
cache, and serves the result; it returns 404 only if that fetch also
fails. A miss never adds the symbol to the tracked set, so an ad-hoc
lookup doesn't create recurring `refresh` load.

## Authentication

`/`, `/help`, and `/:symbol/returns` (the LibreCalc endpoint) are always
public. Every other endpoint -- the raw SRF cache files, `/quote`,
`/symbols`, and the EDGAR ticker maps -- is gated behind a shared key
when `ZFIN_SERVER_API_KEY` is set:

```sh
export ZFIN_SERVER_API_KEY=your-secret

# Supply the key as a header (preferred)...
curl -H "X-API-Key: your-secret" http://localhost:8080/AAPL/candles
# ...or as a query parameter (curl convenience)
curl "http://localhost:8080/AAPL/candles?api_key=your-secret"
```

If `ZFIN_SERVER_API_KEY` is unset or empty, the server is fully open --
a soft cutover so the key can roll out to clients before enforcement is
switched on.

## Watchlist

Adding a symbol to the watchlist enrolls it in the `refresh` cron, so it
gets fetched on every run from then on -- the one operation that creates
recurring provider load. There are two ways in, gated to fit each
consumer:

- **`GET /:symbol/watch`** -- authenticated (API key), for your own
  tooling deliberately growing the tracked set.
- **`GET /:symbol/returns?watch=true`** -- public, but the add only
  happens when the request carries LibreOffice's `WEBSERVICE`
  User-Agent. This lets the non-technical user add symbols from a
  spreadsheet while keeping random callers from growing the cron set.
  (Obscurity-grade: a User-Agent is trivially spoofable, which is
  acceptable under the casual-traffic threat model.)

Adds are serialized and written atomically, and implausible symbols are
rejected, so a stray request can't corrupt or balloon `portfolio.srf`.

## LibreCalc usage

```
=FILTERXML(WEBSERVICE("http://localhost:8080/AAPL/returns?fmt=xml"), "//trailing1YearReturn")
```

XML response format:

```xml
<returns>
  <ticker>AAPL</ticker>
  <returnDate>2026-03-07</returnDate>
  <trailing1YearReturn>12.34000</trailing1YearReturn>
  <trailing3YearReturn>8.56000</trailing3YearReturn>
  <trailing5YearReturn>15.78000</trailing5YearReturn>
  <trailing10YearReturn>22.10000</trailing10YearReturn>
  <volatility>18.50000</volatility>
</returns>
```

## Cache refresh

Use cron to keep the cache warm for all symbols in your portfolio:

```sh
# Refresh all portfolio symbols daily at 5pm ET (after market close)
0 17 * * 1-5 cd /path/to/workdir && ZFIN_PORTFOLIO=portfolio.srf zfin-server refresh
```

The candle cache TTL is 23h45m (not a full 24h), providing a 15-minute jitter
buffer so daily cron runs at the same time always see stale data and trigger a
fresh fetch. Dividend and earnings TTLs are 14 and 30 days respectively.

The `refresh` command reads the portfolio SRF file (defaults to `portfolio.srf`
in the current directory if `ZFIN_PORTFOLIO` is not set), extracts all stock and
watch symbols, and fetches candles, dividends, and earnings for each.

## Building

Requires [Zig 0.16.0](https://ziglang.org/download/).

```sh
zig build        # build
zig build test   # run tests
zig build run -- serve --port=8080
```

## Configuration

All configuration is via environment variables:

| Variable | Required | Description |
|----------|----------|-------------|
| `TWELVEDATA_API_KEY` | Yes | TwelveData API key |
| `POLYGON_API_KEY` | No | Polygon API key |
| `FINNHUB_API_KEY` | No | Finnhub API key |
| `ZFIN_USER_EMAIL` | Yes | Contact email for SEC EDGAR User-Agent header |
| `ZFIN_PORTFOLIO` | No | Path to portfolio SRF (default: `portfolio.srf`) |
| `ZFIN_CACHE_DIR` | No | Cache directory (default: `~/.cache/zfin`) |
| `ZFIN_SERVER_API_KEY` | No | If set, require this key on every endpoint except `/`, `/help`, and `/:symbol/returns` |

## License

MIT