zfin/TODO.md

Future Work

No work here is blocking - we're in a good state. Items below are ordered roughly by priority within each section. Priority labels (HIGH / MEDIUM / LOW) mark items that deserve explicit ranking; unlabeled items are "someday, if the mood strikes."

Investigate: detailed 401(k) contributions data source

Found a more detailed contributions screen on at least one employer-sponsored 401(k) provider portal - distinct from the standard positions/holdings view we already pull from. Worth investigating whether this unlocks better attribution than what we get from the positions CSV alone, and whether other 401(k) providers expose similar screens.

Open questions to answer when picking this up:

• Which screen specifically (path / URL within the portal)? Is there an export option, or is it view-only / scrape territory?
• What fields does it expose (employee pre-tax, employer match, after-tax / mega-backdoor, by-pay-period dates, per-fund allocations)?
• Refresh cadence - per-paycheck, daily, on-demand?
• Can it be auto-discovered like the existing audit CSVs, or is it manual-entry territory?

If the export is structured and recurring, this could feed a 401(k)-specific contributions classifier that bypasses the lot-diff heuristic for that account, similar to how cash_is_contribution opts ESPP/HSA accounts into cash-based attribution.

Related: ESPP-style accrual blind spot in the "Audit: manual-check accounts mechanism" section above.

On-demand server-side fetch for new symbols

Currently the server's SRF endpoints (/candles, /dividends, etc.) are pure cache reads - they 404 if the data isn't already on disk. New symbols only get populated when added to the portfolio and picked up by the next cron refresh.

Consider: on a cache miss, instead of blocking the HTTP response with a multi-second provider fetch, kick off an async background fetch (or just auto-add the symbol to the portfolio) and return 404 as usual. The next request - or the next cron run - would then have the data. This gives "instant-ish gratification" for new symbols without the downsides of synchronous fetch-on-miss (latency, rate limit contention, unbounded cache growth from arbitrary tickers).

Note that this process doesn't do anything to eliminate all the API keys that are necessary for a fully functioning system. A more aggressive view would be to treat ZFIN_SERVER as a 100% source of record, but that would introduce some opacity to the process as we wait for candles (for example) to populate. This could be solved on the server by spawning a thread to fetch the data, then returning 202 Accepted, which could then be polled client side. Maybe this is a better long term approach?

Infra / performance

• HTTP connection pooling. Parallel server sync in loadAllPrices spawns up to 8 threads, each with its own HTTP connection. Could reuse connections to reduce TCP handshake overhead. Only matters with very large portfolios (100+ symbols) hitting ZFIN_SERVER.
• Streaming cache deserialization. Cache store reads entire files into memory (readFileAlloc with 50MB limit). For portfolios with 10+ years of daily candles, this could use significant memory. Keep current approach unless memory becomes a real problem.