lobo/zfin
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages 2 Wiki Activity Actions

add diagnose/server reference pages

Browse source
This commit is contained in:
Emil Lerch 2026-08-19 15:18:37 -07:00
parent 65fefdb566
commit 0dbac7d416
Signed by: lobo
GPG key ID: A7B62D657EF764F8
6 changed files with 204 additions and 14 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
docs/explanation/caching.md
View file

@ -128,8 +128,8 @@ payer that's about four full fetches a year, each within a day or so of
an ex-date. It's why a refresh run occasionally takes noticeably longer
than the usual top-up.
`zfin diagnose SYMBOL` reports the basis for one symbol and says whether
a restatement is pending.
[`zfin diagnose SYMBOL`](../reference/cli/diagnose.md) reports the basis
for one symbol and says whether a restatement is pending.
## Negative caching

3
docs/guides/offline-and-refresh.md
View file

@ -36,7 +36,8 @@ the bars behind it need their adjusted closes marked down, and zfin
re-downloads that symbol's full history to pick up the corrected values
on its own -- see
[Incremental candle updates](../explanation/caching.md#incremental-candle-updates).
`zfin diagnose SYMBOL` says whether one is pending.
[`zfin diagnose SYMBOL`](../reference/cli/diagnose.md) says whether one is
pending.
`zfin cache clear` remains the blunt instrument if you want to discard
everything and start over.

34
docs/reference/cli/cache.md
View file

@ -3,32 +3,56 @@
Inspect or clear the local provider-data cache.
```
Usage: zfin cache <stats|clear>
Usage: zfin cache <stats|stale|clear> | zfin cache refresh [SYMBOL...]
```
| Subcommand | Does |
|------------|------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `stats` | List every cached symbol with per-data-type size, age, and freshness state. Stale entries (past TTL) are flagged. Includes `cusip_tickers.srf` if present. |
| `clear` | Delete every file under the cache directory. No confirmation; the next provider call re-fetches everything. |
| `stale` | Find symbols whose newest candle is behind their peers'. Read-only; reports, never fetches. |
| `refresh` | Force-refresh candle data in the **local** cache, bypassing the TTL and the shared server. No arguments refreshes exactly what `stale` reports. |
| `clear` | Delete every file under the cache directory. No confirmation; the next provider call re-fetches everything. |
The cache directory is `$ZFIN_CACHE_DIR` if set, otherwise
`~/.cache/zfin`.
## `stale` compares against peers, not the clock
A symbol is called stale when other cached symbols **of the same kind**
(equity vs mutual fund) hold a newer bar than it does. That is a
different question from "is this bar old?", and the peer framing is what
makes it trustworthy: an un-modeled market closure moves every symbol
together, so it cannot be mistaken for one frozen cache entry.
## `refresh` deliberately skips the server
`zfin cache refresh` goes straight to the provider. It does **not** ask
`ZFIN_SERVER`, because its job is to repair *your* copy. To refresh the
server's copy instead, use
[`zfin server refresh`](server.md); to find out which side is behind in
the first place, use [`zfin diagnose SYMBOL`](diagnose.md).
## Examples
```bash
zfin cache stats
zfin cache clear # wipe; everything re-fetches on next use
zfin cache stats # what's cached, sizes, and ages
zfin cache stale # which symbols are behind their peers
zfin cache refresh # re-fetch exactly what `stale` reported
zfin cache refresh SPY AGG # re-fetch these two
zfin cache clear # wipe; everything re-fetches on next use
```
`clear` is safe -- it only removes cached copies of public market data.
Reach for it when you suspect corrupt cached data; routine staleness is
handled automatically by the `auto` refresh policy.
handled automatically by the `auto` refresh policy, and a stale
adjustment basis repairs itself on the next refresh.
## See also
- [Caching and data freshness](../../explanation/caching.md) -- TTLs and the fetch model.
- [Offline use and refreshing data](../../guides/offline-and-refresh.md) -- the `--refresh-data` flag.
- [`diagnose`](diagnose.md) -- trace one symbol through every tier.
- [`server`](server.md) -- refresh the shared server's cache instead.
---

98
docs/reference/cli/diagnose.md Normal file
View file

@ -0,0 +1,98 @@
# `zfin diagnose`
Trace one symbol's candle data through every tier -- local cache, shared
server, upstream provider -- and say where the chain breaks.
```
Usage: zfin diagnose SYMBOL
```
`diagnose` answers one question: **is the provider behind for this
symbol, or is it us?** It is the counterpart to
[`doctor`](doctor.md), and deliberately its opposite on both axes --
`doctor` checks the whole setup and never touches the network, while
`diagnose` checks a single symbol and queries every tier including the
provider.
It is **read-only**: no cache writes, and the provider query does not
populate the cache. Safe to run while investigating.
## What it reports
Seven lines, in the order the data flows:
| Line | Says |
|-------------|-------------------------------------------------------------------------------------------------------|
| `local` | Newest cached bar, TTL state, which provider sourced it, consecutive failures, any Tiingo backoff |
| `adj basis` | How current the cached adjusted closes are, against the newest dividend or split that has already gone ex |
| `peers` | Newest bar held by other cached symbols of the same kind (equity vs mutual fund) |
| `tracked` | Whether a normal run fetches this symbol, or it is only pulled on demand |
| `server` | What `ZFIN_SERVER` offers, whether it is ahead or behind you, and whether it refreshes this symbol |
| `provider` | What the upstream provider actually has right now |
| `verdict` | Where the chain breaks, and what to do about it |
## Example
```bash
zfin diagnose SPY
```
```
local newest 2026-08-19, TTL still in the future, tiingo
adj basis 2026-08-17 - current through the newest corporate action (2026-06-18)
peers 23 other equity cached, newest 2026-08-19
tracked on demand - a projections benchmark symbol
server offers 2026-08-19 - same as local, written 2026-08-19, tracked: yes
provider tiingo newest 2026-08-19 (8 bars in the last 10 days)
verdict up to date with everything available
```
## Reading the `adj basis` line
This is the least obvious line, so it is worth spelling out. Cached
price history grows by appending, which cannot retroactively correct the
*adjusted* closes of bars already on disk. When a dividend or split goes
ex, those earlier bars need marking down, and until they are, total
returns spanning that ex-date read low by roughly the missed yield. See
[Incremental candle updates](../../explanation/caching.md#incremental-candle-updates).
Three forms:
```
adj basis 2026-08-17 - current through the newest corporate action (2026-06-18)
adj basis 1970-01-01 - STALE, 2026-06-01 went ex behind it; total returns read low until restated
adj basis 2026-08-19 - no dividends or splits cached, nothing to restate
```
`STALE` is not something to act on -- the next refresh re-downloads that
symbol's full history on its own. It explains a total return that looks
slightly low in the meantime.
Two things the date is **not**: it is not when the restatement ran (it is
the newest bar present at the time, which lags when a session's bar has
not posted yet), and it is not an ex-date. `1970-01-01` means "never
restated", which is what a cache predating the feature reads as.
The compared-against date is the newest ex-date that has **already
passed**. A declared-but-not-yet-ex dividend is deliberately ignored:
no provider has applied it either, so waiting on it would refetch
forever.
## Rate limit caveat
The provider query uses its own rate limiter, separate from the budget
the rest of zfin accounts for -- so it is invisible to that budget. On a
free tier close to its cap, `diagnose` can be the request that trips it.
One request per invocation.
## See also
- [`doctor`](doctor.md) -- whole-setup health check, no network.
- [`cache`](cache.md) -- `stale` finds symbols behind their peers; `refresh` forces a local re-fetch.
- [`server`](server.md) -- force a refresh in the shared server's cache instead.
- [Caching and data freshness](../../explanation/caching.md) -- the tiers and TTLs.
---
[CLI command reference](index.md)

14
docs/reference/cli/index.md
View file

@ -55,12 +55,14 @@ Get help at any time with `zfin help` or per command with
## Infrastructure
| Command | Does |
|---------------------------------|------------------------------------------------|
| [`cache`](cache.md) | Inspect or clear the local data cache |
| [`doctor`](doctor.md) | Health-check files and environment (read-only) |
| [`version`](version.md) | Show version and build info |
| [`interactive`](interactive.md) | Launch the interactive TUI (alias `i`) |
| Command | Does |
|---------------------------------|--------------------------------------------------|
| [`cache`](cache.md) | Inspect or clear the local data cache |
| [`diagnose`](diagnose.md) | Trace one symbol through cache, server, provider |
| [`server`](server.md) | Ask the shared server to refresh its own cache |
| [`doctor`](doctor.md) | Health-check files and environment (read-only) |
| [`version`](version.md) | Show version and build info |
| [`interactive`](interactive.md) | Launch the interactive TUI (alias `i`) |
## Global options

65
docs/reference/cli/server.md Normal file
View file

@ -0,0 +1,65 @@
# `zfin server`
Ask the shared server (`ZFIN_SERVER`) to refresh its own cache.
```
Usage: zfin server refresh SYMBOL [SYMBOL...]
```
Force-refreshes candle data in the **server's** cache, bypassing its
TTL. Nothing local changes -- your next normal run picks up the server's
new copy through the usual sync.
## Which side are you refreshing?
Two commands look similar and do opposite things:
| Command | Refreshes | Goes through the server? |
|-------------------------------|----------------------|--------------------------|
| `zfin cache refresh SYMBOL` | your **local** cache | No, deliberately |
| `zfin server refresh SYMBOL` | the **server's** cache | That is the point |
Run [`zfin diagnose SYMBOL`](diagnose.md) first -- its `local` and
`server` lines say which side is actually behind, so you do not refresh
the wrong one.
## `moved` vs `unchanged`
The per-symbol result reports whether the newest bar actually **moved**,
not merely whether the fetch succeeded. A successful refresh that changes
nothing is the signature of a provider with no newer data -- which is
usually the finding you came for, so it is not reported as success.
```bash
zfin server refresh SPY AGG
```
```
Asking https://zfin.example.org to refresh 2 symbol(s)...
SPY moved 2026-08-19
AGG unchanged 2026-08-18
1 moved, 1 unchanged, 0 failed
Your local cache is untouched - a normal run will sync the new copy.
```
## Requirements and limits
- **`ZFIN_SERVER`** must be set, and **`ZFIN_SERVER_API_KEY`** when the
server enforces one. Without a server configured there is nothing to
ask, and the command says so rather than falling back to a local fetch.
- The server caps a single request at **25 symbols**.
- It **refuses a second concurrent refresh** rather than queueing it, so
a refresh already in flight (a cron run, say) returns a conflict rather
than doubling the provider load.
## See also
- [`diagnose`](diagnose.md) -- which tier is behind, before you pick a side.
- [`cache`](cache.md) -- `refresh` for the local cache instead.
- [Caching and data freshness](../../explanation/caching.md) -- the server as an optional second tier.
- [Environment variables](../config/environment.md) -- `ZFIN_SERVER`, `ZFIN_SERVER_API_KEY`.
---
[CLI command reference](index.md)