67 lines
3.2 KiB
Markdown
67 lines
3.2 KiB
Markdown
# Future Work
|
|
|
|
No work here is blocking - we're in a good state. Items below are
|
|
ordered roughly by priority within each section. Priority labels
|
|
(`HIGH` / `MEDIUM` / `LOW`) mark items that deserve explicit
|
|
ranking; unlabeled items are "someday, if the mood strikes."
|
|
|
|
## Investigate: detailed 401(k) contributions data source
|
|
|
|
Found a more detailed contributions screen on at least one
|
|
employer-sponsored 401(k) provider portal - distinct from the
|
|
standard positions/holdings view we already pull from. Worth
|
|
investigating whether this unlocks better attribution than what
|
|
we get from the positions CSV alone, and whether other 401(k)
|
|
providers expose similar screens.
|
|
|
|
Open questions to answer when picking this up:
|
|
|
|
- Which screen specifically (path / URL within the portal)? Is there
|
|
an export option, or is it view-only / scrape territory?
|
|
- What fields does it expose (employee pre-tax, employer match,
|
|
after-tax / mega-backdoor, by-pay-period dates, per-fund
|
|
allocations)?
|
|
- Refresh cadence - per-paycheck, daily, on-demand?
|
|
- Can it be auto-discovered like the existing audit CSVs, or
|
|
is it manual-entry territory?
|
|
|
|
If the export is structured and recurring, this could feed a
|
|
401(k)-specific contributions classifier that bypasses the lot-diff
|
|
heuristic for that account, similar to how `cash_is_contribution`
|
|
opts ESPP/HSA accounts into cash-based attribution.
|
|
|
|
Related: ESPP-style accrual blind spot in the "Audit: manual-check
|
|
accounts mechanism" section above.
|
|
|
|
## On-demand server-side fetch for new symbols
|
|
|
|
Currently the server's SRF endpoints (`/candles`, `/dividends`, etc.) are pure
|
|
cache reads - they 404 if the data isn't already on disk. New symbols only get
|
|
populated when added to the portfolio and picked up by the next cron refresh.
|
|
|
|
Consider: on a cache miss, instead of blocking the HTTP response with a
|
|
multi-second provider fetch, kick off an async background fetch (or just
|
|
auto-add the symbol to the portfolio) and return 404 as usual. The next
|
|
request - or the next cron run - would then have the data. This gives
|
|
"instant-ish gratification" for new symbols without the downsides of
|
|
synchronous fetch-on-miss (latency, rate limit contention, unbounded cache
|
|
growth from arbitrary tickers).
|
|
|
|
Note that this process doesn't do anything to eliminate all the API keys
|
|
that are necessary for a fully functioning system. A more aggressive view
|
|
would be to treat ZFIN_SERVER as a 100% source of record, but that would
|
|
introduce some opacity to the process as we wait for candles (for example) to
|
|
populate. This could be solved on the server by spawning a thread to fetch the
|
|
data, then returning 202 Accepted, which could then be polled client side. Maybe
|
|
this is a better long term approach?
|
|
|
|
## Infra / performance
|
|
|
|
- **HTTP connection pooling.** Parallel server sync in `loadAllPrices`
|
|
spawns up to 8 threads, each with its own HTTP connection. Could
|
|
reuse connections to reduce TCP handshake overhead. Only matters
|
|
with very large portfolios (100+ symbols) hitting ZFIN_SERVER.
|
|
- **Streaming cache deserialization.** Cache store reads entire files
|
|
into memory (`readFileAlloc` with 50MB limit). For portfolios with
|
|
10+ years of daily candles, this could use significant memory.
|
|
Keep current approach unless memory becomes a real problem.
|