add diagnose/server reference pages
This commit is contained in:
parent
65fefdb566
commit
0dbac7d416
6 changed files with 204 additions and 14 deletions
|
|
@ -128,8 +128,8 @@ payer that's about four full fetches a year, each within a day or so of
|
||||||
an ex-date. It's why a refresh run occasionally takes noticeably longer
|
an ex-date. It's why a refresh run occasionally takes noticeably longer
|
||||||
than the usual top-up.
|
than the usual top-up.
|
||||||
|
|
||||||
`zfin diagnose SYMBOL` reports the basis for one symbol and says whether
|
[`zfin diagnose SYMBOL`](../reference/cli/diagnose.md) reports the basis
|
||||||
a restatement is pending.
|
for one symbol and says whether a restatement is pending.
|
||||||
|
|
||||||
## Negative caching
|
## Negative caching
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -36,7 +36,8 @@ the bars behind it need their adjusted closes marked down, and zfin
|
||||||
re-downloads that symbol's full history to pick up the corrected values
|
re-downloads that symbol's full history to pick up the corrected values
|
||||||
on its own -- see
|
on its own -- see
|
||||||
[Incremental candle updates](../explanation/caching.md#incremental-candle-updates).
|
[Incremental candle updates](../explanation/caching.md#incremental-candle-updates).
|
||||||
`zfin diagnose SYMBOL` says whether one is pending.
|
[`zfin diagnose SYMBOL`](../reference/cli/diagnose.md) says whether one is
|
||||||
|
pending.
|
||||||
|
|
||||||
`zfin cache clear` remains the blunt instrument if you want to discard
|
`zfin cache clear` remains the blunt instrument if you want to discard
|
||||||
everything and start over.
|
everything and start over.
|
||||||
|
|
|
||||||
|
|
@ -3,32 +3,56 @@
|
||||||
Inspect or clear the local provider-data cache.
|
Inspect or clear the local provider-data cache.
|
||||||
|
|
||||||
```
|
```
|
||||||
Usage: zfin cache <stats|clear>
|
Usage: zfin cache <stats|stale|clear> | zfin cache refresh [SYMBOL...]
|
||||||
```
|
```
|
||||||
|
|
||||||
| Subcommand | Does |
|
| Subcommand | Does |
|
||||||
|------------|------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
|------------|------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||||
| `stats` | List every cached symbol with per-data-type size, age, and freshness state. Stale entries (past TTL) are flagged. Includes `cusip_tickers.srf` if present. |
|
| `stats` | List every cached symbol with per-data-type size, age, and freshness state. Stale entries (past TTL) are flagged. Includes `cusip_tickers.srf` if present. |
|
||||||
| `clear` | Delete every file under the cache directory. No confirmation; the next provider call re-fetches everything. |
|
| `stale` | Find symbols whose newest candle is behind their peers'. Read-only; reports, never fetches. |
|
||||||
|
| `refresh` | Force-refresh candle data in the **local** cache, bypassing the TTL and the shared server. No arguments refreshes exactly what `stale` reports. |
|
||||||
|
| `clear` | Delete every file under the cache directory. No confirmation; the next provider call re-fetches everything. |
|
||||||
|
|
||||||
The cache directory is `$ZFIN_CACHE_DIR` if set, otherwise
|
The cache directory is `$ZFIN_CACHE_DIR` if set, otherwise
|
||||||
`~/.cache/zfin`.
|
`~/.cache/zfin`.
|
||||||
|
|
||||||
|
## `stale` compares against peers, not the clock
|
||||||
|
|
||||||
|
A symbol is called stale when other cached symbols **of the same kind**
|
||||||
|
(equity vs mutual fund) hold a newer bar than it does. That is a
|
||||||
|
different question from "is this bar old?", and the peer framing is what
|
||||||
|
makes it trustworthy: an un-modeled market closure moves every symbol
|
||||||
|
together, so it cannot be mistaken for one frozen cache entry.
|
||||||
|
|
||||||
|
## `refresh` deliberately skips the server
|
||||||
|
|
||||||
|
`zfin cache refresh` goes straight to the provider. It does **not** ask
|
||||||
|
`ZFIN_SERVER`, because its job is to repair *your* copy. To refresh the
|
||||||
|
server's copy instead, use
|
||||||
|
[`zfin server refresh`](server.md); to find out which side is behind in
|
||||||
|
the first place, use [`zfin diagnose SYMBOL`](diagnose.md).
|
||||||
|
|
||||||
## Examples
|
## Examples
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
zfin cache stats
|
zfin cache stats # what's cached, sizes, and ages
|
||||||
zfin cache clear # wipe; everything re-fetches on next use
|
zfin cache stale # which symbols are behind their peers
|
||||||
|
zfin cache refresh # re-fetch exactly what `stale` reported
|
||||||
|
zfin cache refresh SPY AGG # re-fetch these two
|
||||||
|
zfin cache clear # wipe; everything re-fetches on next use
|
||||||
```
|
```
|
||||||
|
|
||||||
`clear` is safe -- it only removes cached copies of public market data.
|
`clear` is safe -- it only removes cached copies of public market data.
|
||||||
Reach for it when you suspect corrupt cached data; routine staleness is
|
Reach for it when you suspect corrupt cached data; routine staleness is
|
||||||
handled automatically by the `auto` refresh policy.
|
handled automatically by the `auto` refresh policy, and a stale
|
||||||
|
adjustment basis repairs itself on the next refresh.
|
||||||
|
|
||||||
## See also
|
## See also
|
||||||
|
|
||||||
- [Caching and data freshness](../../explanation/caching.md) -- TTLs and the fetch model.
|
- [Caching and data freshness](../../explanation/caching.md) -- TTLs and the fetch model.
|
||||||
- [Offline use and refreshing data](../../guides/offline-and-refresh.md) -- the `--refresh-data` flag.
|
- [Offline use and refreshing data](../../guides/offline-and-refresh.md) -- the `--refresh-data` flag.
|
||||||
|
- [`diagnose`](diagnose.md) -- trace one symbol through every tier.
|
||||||
|
- [`server`](server.md) -- refresh the shared server's cache instead.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|
|
||||||
98
docs/reference/cli/diagnose.md
Normal file
98
docs/reference/cli/diagnose.md
Normal file
|
|
@ -0,0 +1,98 @@
|
||||||
|
# `zfin diagnose`
|
||||||
|
|
||||||
|
Trace one symbol's candle data through every tier -- local cache, shared
|
||||||
|
server, upstream provider -- and say where the chain breaks.
|
||||||
|
|
||||||
|
```
|
||||||
|
Usage: zfin diagnose SYMBOL
|
||||||
|
```
|
||||||
|
|
||||||
|
`diagnose` answers one question: **is the provider behind for this
|
||||||
|
symbol, or is it us?** It is the counterpart to
|
||||||
|
[`doctor`](doctor.md), and deliberately its opposite on both axes --
|
||||||
|
`doctor` checks the whole setup and never touches the network, while
|
||||||
|
`diagnose` checks a single symbol and queries every tier including the
|
||||||
|
provider.
|
||||||
|
|
||||||
|
It is **read-only**: no cache writes, and the provider query does not
|
||||||
|
populate the cache. Safe to run while investigating.
|
||||||
|
|
||||||
|
## What it reports
|
||||||
|
|
||||||
|
Seven lines, in the order the data flows:
|
||||||
|
|
||||||
|
| Line | Says |
|
||||||
|
|-------------|-------------------------------------------------------------------------------------------------------|
|
||||||
|
| `local` | Newest cached bar, TTL state, which provider sourced it, consecutive failures, any Tiingo backoff |
|
||||||
|
| `adj basis` | How current the cached adjusted closes are, against the newest dividend or split that has already gone ex |
|
||||||
|
| `peers` | Newest bar held by other cached symbols of the same kind (equity vs mutual fund) |
|
||||||
|
| `tracked` | Whether a normal run fetches this symbol, or it is only pulled on demand |
|
||||||
|
| `server` | What `ZFIN_SERVER` offers, whether it is ahead or behind you, and whether it refreshes this symbol |
|
||||||
|
| `provider` | What the upstream provider actually has right now |
|
||||||
|
| `verdict` | Where the chain breaks, and what to do about it |
|
||||||
|
|
||||||
|
## Example
|
||||||
|
|
||||||
|
```bash
|
||||||
|
zfin diagnose SPY
|
||||||
|
```
|
||||||
|
|
||||||
|
```
|
||||||
|
local newest 2026-08-19, TTL still in the future, tiingo
|
||||||
|
adj basis 2026-08-17 - current through the newest corporate action (2026-06-18)
|
||||||
|
peers 23 other equity cached, newest 2026-08-19
|
||||||
|
tracked on demand - a projections benchmark symbol
|
||||||
|
server offers 2026-08-19 - same as local, written 2026-08-19, tracked: yes
|
||||||
|
provider tiingo newest 2026-08-19 (8 bars in the last 10 days)
|
||||||
|
|
||||||
|
verdict up to date with everything available
|
||||||
|
```
|
||||||
|
|
||||||
|
## Reading the `adj basis` line
|
||||||
|
|
||||||
|
This is the least obvious line, so it is worth spelling out. Cached
|
||||||
|
price history grows by appending, which cannot retroactively correct the
|
||||||
|
*adjusted* closes of bars already on disk. When a dividend or split goes
|
||||||
|
ex, those earlier bars need marking down, and until they are, total
|
||||||
|
returns spanning that ex-date read low by roughly the missed yield. See
|
||||||
|
[Incremental candle updates](../../explanation/caching.md#incremental-candle-updates).
|
||||||
|
|
||||||
|
Three forms:
|
||||||
|
|
||||||
|
```
|
||||||
|
adj basis 2026-08-17 - current through the newest corporate action (2026-06-18)
|
||||||
|
adj basis 1970-01-01 - STALE, 2026-06-01 went ex behind it; total returns read low until restated
|
||||||
|
adj basis 2026-08-19 - no dividends or splits cached, nothing to restate
|
||||||
|
```
|
||||||
|
|
||||||
|
`STALE` is not something to act on -- the next refresh re-downloads that
|
||||||
|
symbol's full history on its own. It explains a total return that looks
|
||||||
|
slightly low in the meantime.
|
||||||
|
|
||||||
|
Two things the date is **not**: it is not when the restatement ran (it is
|
||||||
|
the newest bar present at the time, which lags when a session's bar has
|
||||||
|
not posted yet), and it is not an ex-date. `1970-01-01` means "never
|
||||||
|
restated", which is what a cache predating the feature reads as.
|
||||||
|
|
||||||
|
The compared-against date is the newest ex-date that has **already
|
||||||
|
passed**. A declared-but-not-yet-ex dividend is deliberately ignored:
|
||||||
|
no provider has applied it either, so waiting on it would refetch
|
||||||
|
forever.
|
||||||
|
|
||||||
|
## Rate limit caveat
|
||||||
|
|
||||||
|
The provider query uses its own rate limiter, separate from the budget
|
||||||
|
the rest of zfin accounts for -- so it is invisible to that budget. On a
|
||||||
|
free tier close to its cap, `diagnose` can be the request that trips it.
|
||||||
|
One request per invocation.
|
||||||
|
|
||||||
|
## See also
|
||||||
|
|
||||||
|
- [`doctor`](doctor.md) -- whole-setup health check, no network.
|
||||||
|
- [`cache`](cache.md) -- `stale` finds symbols behind their peers; `refresh` forces a local re-fetch.
|
||||||
|
- [`server`](server.md) -- force a refresh in the shared server's cache instead.
|
||||||
|
- [Caching and data freshness](../../explanation/caching.md) -- the tiers and TTLs.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
[CLI command reference](index.md)
|
||||||
|
|
@ -55,12 +55,14 @@ Get help at any time with `zfin help` or per command with
|
||||||
|
|
||||||
## Infrastructure
|
## Infrastructure
|
||||||
|
|
||||||
| Command | Does |
|
| Command | Does |
|
||||||
|---------------------------------|------------------------------------------------|
|
|---------------------------------|--------------------------------------------------|
|
||||||
| [`cache`](cache.md) | Inspect or clear the local data cache |
|
| [`cache`](cache.md) | Inspect or clear the local data cache |
|
||||||
| [`doctor`](doctor.md) | Health-check files and environment (read-only) |
|
| [`diagnose`](diagnose.md) | Trace one symbol through cache, server, provider |
|
||||||
| [`version`](version.md) | Show version and build info |
|
| [`server`](server.md) | Ask the shared server to refresh its own cache |
|
||||||
| [`interactive`](interactive.md) | Launch the interactive TUI (alias `i`) |
|
| [`doctor`](doctor.md) | Health-check files and environment (read-only) |
|
||||||
|
| [`version`](version.md) | Show version and build info |
|
||||||
|
| [`interactive`](interactive.md) | Launch the interactive TUI (alias `i`) |
|
||||||
|
|
||||||
## Global options
|
## Global options
|
||||||
|
|
||||||
|
|
|
||||||
65
docs/reference/cli/server.md
Normal file
65
docs/reference/cli/server.md
Normal file
|
|
@ -0,0 +1,65 @@
|
||||||
|
# `zfin server`
|
||||||
|
|
||||||
|
Ask the shared server (`ZFIN_SERVER`) to refresh its own cache.
|
||||||
|
|
||||||
|
```
|
||||||
|
Usage: zfin server refresh SYMBOL [SYMBOL...]
|
||||||
|
```
|
||||||
|
|
||||||
|
Force-refreshes candle data in the **server's** cache, bypassing its
|
||||||
|
TTL. Nothing local changes -- your next normal run picks up the server's
|
||||||
|
new copy through the usual sync.
|
||||||
|
|
||||||
|
## Which side are you refreshing?
|
||||||
|
|
||||||
|
Two commands look similar and do opposite things:
|
||||||
|
|
||||||
|
| Command | Refreshes | Goes through the server? |
|
||||||
|
|-------------------------------|----------------------|--------------------------|
|
||||||
|
| `zfin cache refresh SYMBOL` | your **local** cache | No, deliberately |
|
||||||
|
| `zfin server refresh SYMBOL` | the **server's** cache | That is the point |
|
||||||
|
|
||||||
|
Run [`zfin diagnose SYMBOL`](diagnose.md) first -- its `local` and
|
||||||
|
`server` lines say which side is actually behind, so you do not refresh
|
||||||
|
the wrong one.
|
||||||
|
|
||||||
|
## `moved` vs `unchanged`
|
||||||
|
|
||||||
|
The per-symbol result reports whether the newest bar actually **moved**,
|
||||||
|
not merely whether the fetch succeeded. A successful refresh that changes
|
||||||
|
nothing is the signature of a provider with no newer data -- which is
|
||||||
|
usually the finding you came for, so it is not reported as success.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
zfin server refresh SPY AGG
|
||||||
|
```
|
||||||
|
|
||||||
|
```
|
||||||
|
Asking https://zfin.example.org to refresh 2 symbol(s)...
|
||||||
|
SPY moved 2026-08-19
|
||||||
|
AGG unchanged 2026-08-18
|
||||||
|
|
||||||
|
1 moved, 1 unchanged, 0 failed
|
||||||
|
Your local cache is untouched - a normal run will sync the new copy.
|
||||||
|
```
|
||||||
|
|
||||||
|
## Requirements and limits
|
||||||
|
|
||||||
|
- **`ZFIN_SERVER`** must be set, and **`ZFIN_SERVER_API_KEY`** when the
|
||||||
|
server enforces one. Without a server configured there is nothing to
|
||||||
|
ask, and the command says so rather than falling back to a local fetch.
|
||||||
|
- The server caps a single request at **25 symbols**.
|
||||||
|
- It **refuses a second concurrent refresh** rather than queueing it, so
|
||||||
|
a refresh already in flight (a cron run, say) returns a conflict rather
|
||||||
|
than doubling the provider load.
|
||||||
|
|
||||||
|
## See also
|
||||||
|
|
||||||
|
- [`diagnose`](diagnose.md) -- which tier is behind, before you pick a side.
|
||||||
|
- [`cache`](cache.md) -- `refresh` for the local cache instead.
|
||||||
|
- [Caching and data freshness](../../explanation/caching.md) -- the server as an optional second tier.
|
||||||
|
- [Environment variables](../config/environment.md) -- `ZFIN_SERVER`, `ZFIN_SERVER_API_KEY`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
[CLI command reference](index.md)
|
||||||
Loading…
Add table
Reference in a new issue