610 lines
29 KiB
Markdown
610 lines
29 KiB
Markdown
# Future Work
|
||
|
||
No work here is blocking — we're in a good state. Items below are
|
||
ordered roughly by priority within each section. Priority labels
|
||
(`HIGH` / `MEDIUM` / `LOW`) mark items that deserve explicit
|
||
ranking; unlabeled items are "someday, if the mood strikes."
|
||
|
||
**Next up:** configurable benchmark symbols (low-effort win) and the
|
||
manual-check accounts mechanism (medium effort, real user value).
|
||
|
||
## Projections: future enhancements
|
||
|
||
- **Configurable benchmark symbols — priority MEDIUM.** Currently
|
||
hardcoded SPY + AGG. Route through `projections.srf` as a
|
||
`type::config,benchmark::SYMBOL` record (or similar). Low effort.
|
||
- **Configurable return cap per position — priority MEDIUM.**
|
||
Default: none; cap outliers like NVDA. Should route through
|
||
`projections.srf` cleanly.
|
||
- **Accumulation-mode SWR rate column is misleading — priority LOW.**
|
||
When `retirement_age`/`retirement_at` is configured, the "Safe
|
||
Withdrawal" table's % column divides the SWR amount by the
|
||
CURRENT portfolio value, not the post-accumulation portfolio
|
||
value. The dollar amount is correct (it's the safe spending in
|
||
retirement, given the projected accumulation), but the % rate
|
||
comes out absurdly high (e.g., 22% of today's portfolio). The
|
||
Accumulation phase block already shows the median portfolio at
|
||
retirement, so the user can compute the real rate themselves —
|
||
but the SWR table's rate column should ideally divide by the
|
||
median post-accumulation value, or be suppressed when accumulation
|
||
is active. Decide which.
|
||
- **Chart vertical line at retirement boundary — priority LOW.**
|
||
The accumulation-phase spec called this "mandatory" but it was
|
||
explicitly deferred during implementation. The chart currently
|
||
shows the full `accumulation_years + horizon` span without a
|
||
visual marker for where accumulation ends and distribution
|
||
begins. Easier to add to the kitty-graphics chart than the braille
|
||
one.
|
||
- **Goal-seek over distribution horizon for W1 — priority LOW.**
|
||
Today the W1 ("set spending, find date") workflow reports the
|
||
earliest retirement at each user-configured `(horizon, confidence)`
|
||
cell. The philosophically correct version asks "when have I
|
||
accumulated enough wealth that the projection shows a 95%
|
||
probability of success withdrawing X per year from retirement
|
||
until age-of-death?" — i.e. goal-seek across both `accumulation_years`
|
||
AND `distribution_years` simultaneously, anchored to a configured
|
||
age-of-death. NP-shaped search; not worth optimizing until
|
||
someone wants it.
|
||
- **Per-person retirement_age — priority LOW.**
|
||
V1 of the accumulation-phase spec chose Option A: a single
|
||
household retirement boundary derived from the oldest configured
|
||
birthdate. Households where one earner retires significantly
|
||
earlier than the other would benefit from per-person
|
||
`retirement_age` fields on each `type::birthdate` record, with
|
||
contributions stopped per-person.
|
||
- **Configurable max_accumulation_years — priority LOW.**
|
||
Hardcoded at 50 years. Route through `projections.srf` if anyone
|
||
hits the cap.
|
||
- Configurable MIN period selection (currently 3Y/5Y/10Y, exclude 1Y)
|
||
- Multiple spending models: flat (current), decreasing (1-2% real annual decrease,
|
||
Blanchett "spending smile"). Late-life healthcare better modeled as a life event.
|
||
- Unclassified position handling in allocation split (warn user)
|
||
- **Historical projection overlay follow-ups.** The base
|
||
`--overlay-actuals` overlay shipped (CLI tip + TUI primary surface).
|
||
Open enhancements:
|
||
- Historical `metadata.srf` / `projections.srf` for back-dated
|
||
runs. Today the overlay re-runs against current classifications
|
||
and assumptions; for historically faithful what-the-model-said-then
|
||
output we'd check out the git-tracked versions of those files
|
||
at the as-of commit and load those instead. Edge case until
|
||
classifications materially drift.
|
||
- Contribution-attribution overlay. Today's actuals line includes
|
||
contributions implicitly; the bands assume modeled contributions
|
||
that may or may not match reality. A "decompose actuals into
|
||
market return vs contributions" annotation would clarify how
|
||
much of the trajectory was the model being right vs new money
|
||
arriving on schedule.
|
||
- Mosaic mode: overlay multiple as-of starting points on one chart
|
||
("show me 1Y, 3Y, 5Y, 10Y projections all at once") so the user
|
||
can see how the projection envelope tightened as data came in.
|
||
- **Better composition basis for imported-only as-of.** Today
|
||
the imported-only path uses today's allocations scaled by
|
||
`imported_liquid / today_total_liquid`. That's the simplest
|
||
thing that could work, but it's "today's mix back-dated" —
|
||
it ignores everything we know about the historical context.
|
||
Specifically: `imported_values.srf` already carries an
|
||
`expected_return` field per row that the user captured at
|
||
that date in their source spreadsheet. We could:
|
||
- Use the imported `expected_return` as a sanity check
|
||
against the simulation's per-position weighted return
|
||
(warn or clamp if they diverge wildly — the spreadsheet's
|
||
number reflects what the user actually saw at the time).
|
||
- Use the imported `expected_return` to bias the
|
||
stock/bond split inference: a higher expected return
|
||
implies a higher historical equity weighting than today's
|
||
mix probably reflects.
|
||
- Reach further: derive a synthetic stock/bond split from
|
||
the imported `expected_return` directly, treating it as
|
||
a weighted average of SPY and AGG returns at that date
|
||
and solving for the weights. That gives a per-imported-
|
||
row composition that's locally faithful instead of
|
||
one-mix-fits-all.
|
||
None of these are urgent — the current "today's mix scaled"
|
||
approximation is documented as such and the bands still
|
||
render meaningfully — but each would tighten the historical
|
||
faithfulness one notch. Pick whichever has the highest
|
||
payoff vs. complexity when this gets revisited.
|
||
- **Chart zoom for short-history overlays.** With a 50-year
|
||
projection horizon and only ~10 years of imported actuals,
|
||
the actuals line is squashed into the first 20% of the
|
||
chart and the comparison-against-bands story is hard to
|
||
read. Two design directions:
|
||
- **Auto-zoom**: when the overlay is on, the chart's
|
||
x-axis defaults to `[as_of, today + N years]` (where
|
||
N is small, e.g. 2x the actuals span) instead of
|
||
`[as_of, as_of + horizon]`. The bands beyond `today
|
||
+ N` are still computed but clipped from view. The
|
||
tradeoff: the user loses the long-tail terminal-value
|
||
context unless they toggle back out.
|
||
- **Toggle**: a separate keybind (e.g. `z` for zoom)
|
||
flips between full-horizon and zoomed views. Default
|
||
off so the bands tell their full story; user opts in
|
||
when they want overlay legibility.
|
||
Auto-zoom is more invasive (changes the default chart
|
||
semantics for everyone running with overlay-on) but better
|
||
matches what the user actually wants when they toggle the
|
||
overlay. Toggle is safer but requires the user to know the
|
||
feature exists. Probably do auto-zoom but expose a toggle
|
||
to escape it ("show full horizon").
|
||
|
||
## Export chart as PNG (`--export-chart <path>`) — priority MEDIUM
|
||
|
||
z2d already supports PNG export natively. Today the chart-bearing
|
||
commands (`quote`, `history`, `projections`, plus the equivalent TUI
|
||
tabs) render to braille (CLI) or Kitty graphics (TUI). Adding a
|
||
`--export-chart <path>` flag would land just the chart (not the
|
||
surrounding text output) as a PNG file at the given path, at full
|
||
fidelity, regardless of which surface invoked it.
|
||
|
||
Driver: when reviewing a back-dated projection or a notable price
|
||
move, capturing the chart as an image (e.g. for a write-up, an email
|
||
to the household, or a wiki page) is currently a screenshot-and-crop
|
||
chore. PNG export makes it a one-shot CLI invocation.
|
||
|
||
Sketch:
|
||
- `zfin quote AAPL --export-chart aapl.png` → just the price+
|
||
Bollinger chart as a PNG, no other output.
|
||
- `zfin projections --as-of 1Y --overlay-actuals --export-chart projection.png`
|
||
→ the projection-bands chart plus overlay, no other output.
|
||
- The chart code already produces RGB pixel buffers via z2d; replace
|
||
the `transmitPreEncodedImage` call (TUI) or the braille text path
|
||
(CLI) with a `Surface.write_png` call when the flag is present.
|
||
|
||
Plumbing: a thin "chart-only render" entry point in each chart
|
||
module (`projection_chart.zig`, `chart.zig` for symbols), called
|
||
from the relevant command's `run()` when `--export-chart` is set.
|
||
Exits before the rest of the text output renders.
|
||
|
||
Out of scope for V1: file-format alternatives (SVG, PDF), themed
|
||
color overrides for export (always uses the active terminal theme),
|
||
non-chart command output as PNG.
|
||
|
||
## Refactor: trim `src/format.zig` once Money / Date have absorbed their helpers — priority LOW
|
||
|
||
`src/format.zig` is still a ~1700-line grab-bag, but the money- and
|
||
date-shaped helpers that used to live there have been moved out:
|
||
money formatting now lives in `src/Money.zig` (with `{f}` /
|
||
`whole()` / `trim()` / `signed()` / `padRight(N)` / `padLeft(N)`),
|
||
and date formatting lives in `src/Date.zig` (with `{f}` /
|
||
`padRight(N)` / `padLeft(N)`). What's left in `format.zig` is the
|
||
genuinely-format-domain stuff: braille charts, return formatters,
|
||
allocation notes, signed-percent rendering.
|
||
|
||
If the file ever grows enough to be annoying again, consider
|
||
renaming to `src/render.zig` to better describe what's left, or
|
||
splitting the braille chart out (it's ~600 lines on its own).
|
||
Not blocking — file it as cleanup if and when it bites.
|
||
|
||
## Refactor: TUI App struct knows too much about each tab — priority MEDIUM
|
||
|
||
`src/tui.zig`'s `App` struct currently has dozens of tab-specific
|
||
fields scattered across its top level — `projections_loaded`,
|
||
`projections_disabled`, `projections_config`, `projections_ctx`,
|
||
`projections_horizon_idx`, `projections_image_id`,
|
||
`projections_image_width`, `projections_image_height`,
|
||
`projections_chart_dirty`, `projections_chart_visible`,
|
||
`projections_events_enabled`, `projections_value_min`,
|
||
`projections_value_max`, `projections_as_of`,
|
||
`projections_as_of_requested`, `projections_overlay_actuals`,
|
||
plus equivalents for portfolio, history, options, earnings,
|
||
analysis, perf, and quote tabs.
|
||
|
||
This couples `App` to the implementation details of every tab.
|
||
Touching a single tab's state shape requires editing the central
|
||
struct, which makes refactors noisy and discourages tab modules
|
||
from owning their own state cleanly.
|
||
|
||
**Proposal: each tab gets exactly ONE field on `App`** — a
|
||
struct (or pointer to a struct) defined in that tab's own file.
|
||
The tab module owns its state shape; `App` only carries the
|
||
top-level reference.
|
||
|
||
Sketch:
|
||
|
||
```zig
|
||
// src/tui/projections_tab.zig
|
||
pub const State = struct {
|
||
loaded: bool = false,
|
||
disabled: bool = false,
|
||
config: projections.UserConfig = .{},
|
||
ctx: ?ProjectionContext = null,
|
||
horizon_idx: usize = 0,
|
||
chart: ChartState = .{},
|
||
as_of: AsOfState = .{},
|
||
overlay_actuals: bool = false,
|
||
// ...
|
||
};
|
||
|
||
// src/tui.zig
|
||
pub const App = struct {
|
||
// ... shared cross-tab state (today, allocator, vx_app, ...) ...
|
||
portfolio: portfolio_tab.State = .{},
|
||
quote: quote_tab.State = .{},
|
||
perf: perf_tab.State = .{},
|
||
history: history_tab.State = .{},
|
||
projections: projections_tab.State = .{},
|
||
options: options_tab.State = .{},
|
||
earnings: earnings_tab.State = .{},
|
||
analysis: analysis_tab.State = .{},
|
||
};
|
||
```
|
||
|
||
After the migration, `app.projections_overlay_actuals` becomes
|
||
`app.projections.overlay_actuals`, etc. Each tab's State struct
|
||
documents its own invariants without polluting `App`.
|
||
|
||
**Benefits:**
|
||
|
||
- Tab modules become genuinely self-contained. Adding new state
|
||
to a tab only touches that tab's file.
|
||
- `App`'s field count drops from ~80 to ~10 (cross-cutting state
|
||
+ 8 tab-state fields).
|
||
- Clearer ownership: when reading `App`, you can tell at a
|
||
glance which fields are shared vs tab-private.
|
||
- Easier to reason about lifetimes — each tab's `freeLoaded()`
|
||
operates on its own struct rather than reaching into App.
|
||
- Onboarding new contributors: "to add a tab feature, define
|
||
state in the tab file" instead of "edit two files in lockstep."
|
||
|
||
**Migration approach:**
|
||
|
||
This is a large mechanical refactor (~80+ field renames across
|
||
all tab files plus `tui.zig`). Best done as one focused PR:
|
||
|
||
1. Define `State` in each tab file with all current fields.
|
||
2. Add the eight new fields to `App`; flip them on as defaults
|
||
(so old `app.projections_*` accesses temporarily still work
|
||
if we add accessor shims, but cleaner to just rip the
|
||
bandaid).
|
||
3. Sweep all `app.<tab>_<field>` references → `app.<tab>.<field>`.
|
||
4. Delete the old top-level fields from `App`.
|
||
5. Verify `zig build test` is unchanged. The refactor should be
|
||
strictly behavior-preserving.
|
||
|
||
Risks: large diff (touches every tab file plus tui.zig), but
|
||
mechanical — no logic changes, no tests should move. The pre-
|
||
commit hooks catch any miss instantly.
|
||
|
||
**While we're at it: action handler bodies should also move.**
|
||
The same shape problem shows up in `tui.zig`'s keybind-action
|
||
dispatch — `sort_reverse`, `toggle_chart`, `toggle_events`,
|
||
`account_filter`, and the per-tab branches inside them are
|
||
~100 lines of tab-specific logic living in the central event
|
||
loop. Concretely (line numbers as of writing):
|
||
|
||
- `sort_reverse` (~line 1328) dual-dispatches by active tab:
|
||
portfolio flips sort direction + calls
|
||
`sortPortfolioAllocations` / `rebuildPortfolioRows`;
|
||
projections toggles overlay-actuals + reloads data + sets
|
||
status. None of that body is `App`-level concern; it's two
|
||
separate tabs' private state mutations.
|
||
- `toggle_chart` (~line 1382) flips
|
||
`projections_chart_visible` and resets `scroll_offset`. Pure
|
||
projections-tab business.
|
||
- `toggle_events` (~line 1390) flips
|
||
`projections_events_enabled` and triggers a reload. Same.
|
||
- `account_filter` (~line 1366) opens the account picker mode
|
||
with portfolio-tab-specific cursor positioning.
|
||
|
||
The cleaner shape: each tab module exposes a
|
||
`handleAction(app, action) bool` (or similar) that returns
|
||
true when it consumed the action. `tui.zig`'s dispatch becomes
|
||
a thin "ask the active tab if it wants this action; otherwise
|
||
fall through to global handlers." The body of each `case`
|
||
shrinks to a one-liner that dispatches to
|
||
`tab_module.handleAction(self)`.
|
||
|
||
This pairs naturally with the State-struct migration above —
|
||
the tab module's `handleAction` operates on its own State
|
||
struct rather than reaching into `App`. Some keybinds are
|
||
genuinely cross-cutting (quit, refresh, tab navigation, scroll)
|
||
and stay in the central handler. The split is "App owns
|
||
chrome; tab owns content."
|
||
|
||
Driver: every overlay-actuals-shaped feature added to a tab
|
||
recently has involved adding 1–2 fields to `App`, and the
|
||
struct keeps growing. Eventually it becomes unreviewable.
|
||
|
||
## Bug: braille charts use raw `close`, not `adj_close` — cliff at splits
|
||
|
||
**Reproduction:** `zfin quote SOXX` (or the TUI quote tab). The
|
||
braille chart drops sharply on **2024-03-07**, which is the
|
||
iShares Semiconductor ETF's 3-for-1 split date:
|
||
|
||
- 2024-03-06 close: $689.60
|
||
- 2024-03-07 close: $237.75 (≈ $689.60 / 2.9)
|
||
|
||
The `adj_close` column in `~/.cache/zfin/SOXX/candles_daily.srf`
|
||
tracks correctly through the split (~$226 → ~$234), so the
|
||
provider data is fine. The bug is purely cosmetic: the chart
|
||
renders the *unadjusted* close price.
|
||
|
||
**Root cause:** `computeBrailleChart` in `src/format.zig:888`
|
||
indexes `data[i].close` instead of `data[i].adj_close`. Lines 901,
|
||
902, 904, 905, 935 all use `.close`.
|
||
|
||
**Independent confirmation:** `zfin splits SOXX` returns
|
||
`2024-03-07 3:1` from Polygon. So the split data exists in the
|
||
provider layer (and gets cached as `splits.srf` once requested),
|
||
but the charting code path doesn't consult it.
|
||
|
||
**Fix candidates:**
|
||
|
||
A. **Switch `computeBrailleChart` to consume `adj_close` directly.**
|
||
Simplest. Affects every chart caller (quote, history, projections
|
||
median band, TUI quote/projections tabs). Cosmetic only — no
|
||
computation depends on it. The price-axis labels would render
|
||
adjusted prices, which may surprise users used to seeing the
|
||
raw last-close. Mitigate with a comment in the chart's right-edge
|
||
label region or a header note.
|
||
|
||
B. **Pass a flag to `computeBrailleChart` selecting `close` vs
|
||
`adj_close`.** Default to adjusted; let the quote tab show raw.
|
||
More flexible, marginally more code.
|
||
|
||
C. **Add a `chart_close` accessor to `Candle` that returns
|
||
`adj_close` if non-zero, else `close`.** Same effect as (A) with a
|
||
defensive fallback.
|
||
|
||
D. **Apply split adjustments at chart-data prep time using
|
||
`splits.srf`.** Walk the candle slice with the split history and
|
||
pre-multiply pre-split closes by the cumulative ratio. More
|
||
work, but produces a chart-axis dollar value the user expects
|
||
("today's last close was $X, the chart starts at $Y from N
|
||
years ago"). This is what most charting libraries do.
|
||
Requires plumbing `DataService.getSplits` into the chart-prep
|
||
path on every chart caller, OR doing the adjustment once in the
|
||
service layer alongside candle fetching. Not all callers have a
|
||
`DataService` reference today (e.g., `runProjection`'s synthetic
|
||
median-band candles).
|
||
|
||
**Recommendation:** Start with (A) or (C) — single-line fix, gets
|
||
the cliff out of all charts immediately. (D) is the "correct" fix
|
||
but a bigger refactor; file as a follow-up if (A)/(C) lands first.
|
||
|
||
**Other affected symbols:** Any held position with a split in the
|
||
last 10 years will have the same artifact. Check NVDA (10:1 split
|
||
on 2024-06-10) for a louder example.
|
||
|
||
**Priority:** LOW. Cosmetic only — analytics already use
|
||
`adj_close` correctly via the per-position trailing-returns path.
|
||
But it's confusing when scanning a chart and seeing a 50% drop
|
||
that isn't real.
|
||
|
||
## Audit: manual-check accounts mechanism (NYL, Kelly's ESPP, etc.) — priority HIGH
|
||
|
||
Some accounts/positions can't be reconciled from broker CSVs and need a
|
||
human-in-the-loop reminder at the audit step. Examples:
|
||
|
||
- **NY Life** — no CSV export at all. Values only live in periodic
|
||
statements.
|
||
- **Kelly's ESPP** — accrued payroll-deduction cash doesn't appear in the
|
||
Fidelity positions CSV until the purchase date hits (typically every
|
||
6 months). Between purchases the cash is a real contribution that
|
||
`zfin audit` can't see.
|
||
- Future: treat as an open category.
|
||
|
||
The existing `update_cadence::weekly|monthly|quarterly|none` field already
|
||
sort-of covers this, but has two gaps:
|
||
|
||
1. It fires off the last *git-detected change*, not the last *human
|
||
review*. For NYL, the value sometimes hasn't changed in months — so
|
||
git never fires, cadence never trips.
|
||
2. ESPP needs weekly-ish attention while accumulating cash between
|
||
purchases, but the accrued balance is invisible to the CSV audit.
|
||
|
||
### Options
|
||
|
||
A. **New `update_cadence::manual` variant** — always fires every audit
|
||
run until silenced. Blunt but zero design work.
|
||
|
||
B. **`last_refreshed::YYYY-MM-DD` field on `accounts.srf`** — explicit
|
||
human-review timestamp, decoupled from git-detected changes. Audit
|
||
compares `today - last_refreshed` against the cadence. User bumps
|
||
the field when they check the statement. Probably the most
|
||
correct fit for NYL.
|
||
|
||
C. **Sticky TODO list** — a `todos.srf` or `todo::` field on accounts
|
||
that audit always surfaces until cleared. General-purpose; also
|
||
covers "remember to rebalance on 5/15".
|
||
|
||
### ESPP-specific follow-through
|
||
|
||
ESPP is also a contribution-attribution blind spot. If Kelly's paystub
|
||
deducts $X/week but the cash lot doesn't reach `portfolio.srf` until
|
||
the purchase date, the attribution math is under-counting contributions
|
||
and over-counting the purchase-week gain. Possible fixes are discussed
|
||
in the "Contributions diff" TODO below — option C there (per-account
|
||
`cash_is_contribution`) would make manually-entered ESPP
|
||
cash additions count correctly.
|
||
|
||
## In-kind transfer support (`type::in_kind`) — priority MEDIUM
|
||
|
||
`transaction_log.srf` parses `type::in_kind` records but the
|
||
contributions matcher always rejects them with "in-kind transfers
|
||
not yet supported in v1." In-kind movements need per-symbol
|
||
matching across accounts: an in-kind transfer of 100 VTI shares
|
||
from Acct A to Acct B shows up as `lot_removed` on A + `new_stock`
|
||
on B (or a `rollup_delta` share increase if B already had a VTI
|
||
lot), neither of which can be matched by the current
|
||
amount-based cash matcher.
|
||
|
||
Proposed: a second pass in `matchTransfers` that iterates
|
||
`type::in_kind` records and looks for same-symbol matches across
|
||
`lot_removed` on `from` + `new_stock`/`rollup_delta` on `to`
|
||
within the window. Gated on share-count and open_price sanity so
|
||
a partial transfer doesn't false-positive against an unrelated
|
||
edit.
|
||
|
||
Driver: when the user starts moving positions between accounts
|
||
directly (e.g. Roth conversion of already-held shares, 401k →
|
||
rollover IRA in-kind) rather than liquidating and re-buying.
|
||
|
||
## Torn SRF files from server sync (root cause unknown)
|
||
|
||
**Status:** Root cause still unidentified. We have mitigations and
|
||
diagnostics in place that keep torn responses from corrupting the
|
||
cache, but we don't yet know *why* responses arrive torn. Until we
|
||
have a root cause, this is not resolved — it's mitigated.
|
||
|
||
Mitigations landed so far:
|
||
|
||
- `syncFromServer` (`src/service.zig`) validates responses via
|
||
`cache.Store.looksCompleteSrf` before `writeRaw`. Torn HTTP bodies
|
||
(empty, missing `#!srfv1` header, or no trailing newline) are
|
||
rejected with a warn-level log and NOT written to cache.
|
||
- HTTP responses are checked for an `ETag` sha256 header; on mismatch
|
||
we retry the request once before giving up and falling back to the
|
||
provider.
|
||
- Read-path self-heal: on SRF parse failure during read, the cache
|
||
entry is invalidated so a subsequent refresh can repair without
|
||
user intervention.
|
||
- Diagnostics: richer error capture around the sync path. So far,
|
||
HTTP transit is the dominant source of torn responses — but that's
|
||
an observation, not a root cause.
|
||
|
||
**Remaining work:**
|
||
|
||
- Identify root cause. Candidates to investigate: proxy/load-balancer
|
||
behavior, HTTP keepalive reuse, partial reads on the server side,
|
||
client-side buffer handling. The etag retry tells us whether the
|
||
problem is per-request or persistent; dig into the diagnostics
|
||
output when the next occurrence is captured.
|
||
- Once root cause is known, decide whether the current mitigations
|
||
are sufficient or whether a targeted fix is needed. The
|
||
mitigations may end up being the whole answer, but we can't
|
||
conclude that without understanding the underlying cause.
|
||
|
||
(Content-Length validation was considered and rejected: once the
|
||
server starts compressing response bodies, Content-Length reflects
|
||
the compressed byte count, not the decoded payload, so it's not a
|
||
reliable integrity check.)
|
||
|
||
## Market-aware cache TTL for daily candles
|
||
|
||
Daily candle TTL is currently 23h45m, but candle data only becomes meaningful
|
||
after the market close. Investigate keying the cache freshness to ~4:30 PM
|
||
Eastern rather than a rolling window. This would avoid unnecessary refetches
|
||
during the trading day and ensure a fetch shortly after close gets fresh data.
|
||
Probably alleviated by the cron job approach.
|
||
|
||
## On-demand server-side fetch for new symbols
|
||
|
||
Currently the server's SRF endpoints (`/candles`, `/dividends`, etc.) are pure
|
||
cache reads — they 404 if the data isn't already on disk. New symbols only get
|
||
populated when added to the portfolio and picked up by the next cron refresh.
|
||
|
||
Consider: on a cache miss, instead of blocking the HTTP response with a
|
||
multi-second provider fetch, kick off an async background fetch (or just
|
||
auto-add the symbol to the portfolio) and return 404 as usual. The next
|
||
request — or the next cron run — would then have the data. This gives
|
||
"instant-ish gratification" for new symbols without the downsides of
|
||
synchronous fetch-on-miss (latency, rate limit contention, unbounded cache
|
||
growth from arbitrary tickers).
|
||
|
||
Note that this process doesn't do anything to eliminate all the API keys
|
||
that are necessary for a fully functioning system. A more aggressive view
|
||
would be to treat ZFIN_SERVER as a 100% source of record, but that would
|
||
introduce some opacity to the process as we wait for candles (for example) to
|
||
populate. This could be solved on the server by spawning a thread to fetch the
|
||
data, then returning 202 Accepted, which could then be polled client side. Maybe
|
||
this is a better long term approach?
|
||
|
||
## Low-priority items
|
||
|
||
The following items are acknowledged but not prioritized. Listed here
|
||
so they don't get lost; pick up opportunistically.
|
||
|
||
### UX
|
||
|
||
- **CLI options command UX.** The `options` command auto-expands only
|
||
the nearest monthly expiration and lists others collapsed. Reconsider
|
||
the interaction model — e.g. allow specifying an expiration date,
|
||
showing all monthlies expanded by default, or filtering by strategy
|
||
(covered calls, spreads).
|
||
- **TUI: toggle to last symbol keybind.** A single-key toggle that
|
||
flips between the current symbol and the previously selected one
|
||
(like `cd -` in bash or `Ctrl+^` in vim). Store `last_symbol` on
|
||
`App`; on symbol change, stash the previous. Useful for
|
||
eyeball-comparing performance/risk data between two symbols.
|
||
|
||
### Data quality
|
||
|
||
- **Fix `enrich` command for international funds.** `deriveMetadata`
|
||
in `src/commands/enrich.zig` misclassifies international ETFs:
|
||
1. `geo` uses Alpha Vantage's `Country` field, which is the *fund
|
||
issuer's* domicile (USA for all US-listed ETFs), not the fund's
|
||
investment geography. Every US-domiciled international fund gets
|
||
`geo::US`.
|
||
2. `asset_class` short-circuits to `"ETF"` when
|
||
`asset_type == "ETF"`, or falls through to a US-market-cap
|
||
heuristic that always produces `"US Large Cap"` /
|
||
`"US Mid Cap"` / `"US Small Cap"`.
|
||
|
||
Known misclassified tickers (all came back as
|
||
`geo::US, asset_class::US Large Cap`):
|
||
- **FRDM** — Freedom 100 Emerging Markets ETF → should be
|
||
`geo::Emerging Markets, asset_class::Emerging Markets`
|
||
- **HFXI** — NYLI FTSE International Equity Currency Neutral ETF
|
||
→ should be
|
||
`geo::International Developed, asset_class::International Developed`
|
||
- **IDMO** — Invesco S&P International Developed Momentum ETF →
|
||
should be
|
||
`geo::International Developed, asset_class::International Developed`
|
||
- **IVLU** — iShares MSCI International Developed Value Factor
|
||
ETF → should be
|
||
`geo::International Developed, asset_class::International Developed`
|
||
|
||
The Alpha Vantage OVERVIEW endpoint doesn't provide fund geography
|
||
data. Options: use the ETF_PROFILE holdings/country data to infer
|
||
geography, parse the fund name for keywords ("International",
|
||
"Emerging", "ex-US"), or accept that `enrich` is a scaffold and
|
||
emit a `# TODO` comment for ETFs instead of silently misclassifying.
|
||
|
||
### Options / valuation
|
||
|
||
- **Per-account covered call adjustment.** `adjustForCoveredCalls` in
|
||
`valuation.zig` operates on portfolio-wide aggregated allocations.
|
||
It matches sold calls against total underlying shares across all
|
||
accounts. This is wrong — calls in one account can only cover
|
||
shares in that same account. Fixing means restructuring
|
||
`portfolioSummary`, since `Allocation` is currently
|
||
account-agnostic. Low priority — naked calls are rare, and calls
|
||
are typically in the same account as the underlying.
|
||
- **Covered call adjustment O(N*M) loop.** `adjustForCoveredCalls`
|
||
has a nested loop — for each allocation, it iterates all lots to
|
||
find matching option contracts. Fine for personal portfolios
|
||
(<1000 lots). Pre-indexing options by underlying would help if
|
||
someone had a very large options-heavy portfolio.
|
||
|
||
### Analysis / correctness
|
||
|
||
- **Analysis account/asset-class total mismatch.** The "By Account"
|
||
and "By Tax Type" sections in the analysis command sum to slightly
|
||
more than "Asset Class" (~0.6% error). Likely a discrepancy between
|
||
how the lot-level account loop values cash, CDs, or options vs how
|
||
the asset-class section computes them via `portfolio.totalCash()` /
|
||
`totalCdFaceValue()`. Per-account values themselves are correct
|
||
after the price_ratio fix.
|
||
|
||
### Audit
|
||
|
||
- **Audit large-lot threshold tuning.** `src/commands/audit.zig` uses
|
||
`audit_large_lot_threshold: f64 = 10_000.0` as the cutoff for
|
||
"surface this new lot for confirmation." Revisit if $10k proves too
|
||
aggressive (ESPP accruals spam the report) or too permissive (large
|
||
DRIP confirmations slip past). If runtime tuning becomes necessary,
|
||
a `--large-lot <amount>` flag or a global
|
||
`audit_large_lot_threshold` field on `accounts.srf` would be
|
||
reasonable extensions.
|
||
|
||
### Infra / performance
|
||
|
||
- **HTTP connection pooling.** Parallel server sync in `loadAllPrices`
|
||
spawns up to 8 threads, each with its own HTTP connection. Could
|
||
reuse connections to reduce TCP handshake overhead. Only matters
|
||
with very large portfolios (100+ symbols) hitting ZFIN_SERVER.
|
||
- **Streaming cache deserialization.** Cache store reads entire files
|
||
into memory (`readFileAlloc` with 50MB limit). For portfolios with
|
||
10+ years of daily candles, this could use significant memory.
|
||
Keep current approach unless memory becomes a real problem.
|