Compare commits
6 commits
df204a3eb2
...
a621f2932b
| Author | SHA1 | Date | |
|---|---|---|---|
| a621f2932b | |||
| 3f9cb33be0 | |||
| efb4d57c23 | |||
| 770ceb8777 | |||
| 0964da08d5 | |||
| 3fc1be8073 |
16 changed files with 687 additions and 64 deletions
|
|
@ -463,6 +463,23 @@ touched while `ZFIN_SERVER` is unreachable is poisoned until
|
|||
re-hit the network for every legitimately candle-less symbol on every
|
||||
run. It belongs with the deferred routing work.
|
||||
|
||||
### The label is not sticky, and must not become sticky
|
||||
|
||||
`external` describes where a series came from *now*, not a permanent
|
||||
property of the symbol. If a provider ever does serve it, the bars in
|
||||
the cache genuinely came from that provider, and `applyTiingoCoverage`
|
||||
overwriting the label with `.tiingo` / `.yahoo` is the correct outcome:
|
||||
the provenance changed, and the symbol has rejoined the ordinary
|
||||
provider path under its own steam. That is a good day, not a bug.
|
||||
|
||||
So do **not** add a guard preserving `.external` across a successful
|
||||
fetch, and any future routing built on the variant must keep that escape
|
||||
hatch open. Making it a one-way door recreates the latch bug that keying
|
||||
routing off `provider == .yahoo` already produced - see [the
|
||||
`tiingo_retry_after_s` rationale](#getcandles-single-symbol) - except
|
||||
worse, because the pinned symbol would be pinned to a tier that only one
|
||||
machine can populate.
|
||||
|
||||
### Adding a `CandleProvider` variant is a coordinated deployment
|
||||
|
||||
Not a backward-compatible change. `provider` has no default, so it is
|
||||
|
|
|
|||
|
|
@ -66,7 +66,7 @@ the attribution will lie to you.
|
|||
One command gives you the whole "since last review" picture:
|
||||
|
||||
```bash
|
||||
zfin compare 1W --projections --commit-before HEAD
|
||||
zfin compare 1W --projections
|
||||
```
|
||||
|
||||
- **`1W`** is the point of comparison -- the snapshot from one week ago.
|
||||
|
|
@ -75,8 +75,8 @@ zfin compare 1W --projections --commit-before HEAD
|
|||
- **`--projections`** folds in projected-return and safe-withdrawal
|
||||
(SWR@99%) deltas, then vs. now. (Costs ~1-2s per endpoint for the
|
||||
Monte Carlo search; add `--no-events` to exclude life events.)
|
||||
- **`--commit-before HEAD`** pins the contributions/gains attribution to
|
||||
your latest reconciliation commit. This matters -- see
|
||||
- **No `--commit-before` needed.** Attribution now anchors itself on the
|
||||
commit that *recorded* the comparison snapshot. See
|
||||
[Attribution and commit timing](#attribution-and-commit-timing).
|
||||
|
||||
Read off the liquid-total delta, the contributions-vs-gains split, the
|
||||
|
|
@ -122,16 +122,51 @@ your backup and your audit trail.
|
|||
|
||||
`compare` and `contributions` work out "contributions vs. gains" by
|
||||
walking the git history of `portfolio.srf`. The positional date (`1W`)
|
||||
picks the *snapshot* whose prices you compare against; **`--commit-before`**
|
||||
picks which *commit* anchors the attribution. Those two can drift apart.
|
||||
picks the *snapshot* whose prices you compare against; the attribution
|
||||
needs a *commit* to anchor on. If those two describe different periods,
|
||||
the split is wrong -- and it is wrong silently, because gains are derived
|
||||
as a residual (`delta - contributions`), so anything the attribution
|
||||
window misses is reported as market performance.
|
||||
|
||||
If you reconcile on Saturday but don't commit until Monday, next
|
||||
Saturday's `1W` snapshot lands *before* your last commit -- so a bare
|
||||
`compare 1W` would attribute **two weeks** of contributions to one week.
|
||||
`--commit-before HEAD` sidesteps this by pinning attribution to your
|
||||
most recent reconciliation commit regardless of the snapshot date. When
|
||||
the dates line up anyway, the flag is harmless -- which is why it's
|
||||
worth making a habit.
|
||||
**This is now handled for you.** The default anchor is the commit that
|
||||
ADDED `history/<date>-portfolio.srf` for the snapshot being compared
|
||||
against. Because a review commits its snapshots together with the
|
||||
reconciled `portfolio.srf`, that commit *is* that week's
|
||||
reconciliation -- no naming convention, no dependence on how many times
|
||||
you committed, and unaffected by later edits to the file.
|
||||
|
||||
**Do not pass `--commit-before HEAD`.** It used to be the recommended
|
||||
workaround for reconciling on Saturday and committing on Monday, and it
|
||||
worked only while `HEAD` happened to be the *previous* week's
|
||||
reconciliation commit. Commit anything during a review -- a data fix, a
|
||||
metadata tweak -- and the attribution window collapses to "uncommitted
|
||||
edits only" while the value window still spans the week. One observed
|
||||
case reported ~$23k of vested shares as market gains.
|
||||
|
||||
The flag remains for pinning a window deliberately; it is no longer a
|
||||
habit worth having.
|
||||
|
||||
### Re-snapshot after reconciling
|
||||
|
||||
One ordering detail matters more than it looks. If a scheduled job
|
||||
writes daily snapshots, the one for the last market day is captured
|
||||
*before* you reconcile -- so it holds pre-reconcile share counts while
|
||||
`portfolio.srf` ends up holding post-reconcile ones. The two then
|
||||
disagree at the same commit, permanently, by one reconciliation:
|
||||
|
||||
```
|
||||
history/2026-08-21-portfolio.srf AMZN 1569 (pre-reconcile)
|
||||
portfolio.srf @ that same commit AMZN 1656 (post-reconcile)
|
||||
```
|
||||
|
||||
`compare` reads the snapshot for "then" and the live portfolio for
|
||||
"now", so those 87 shares appear in the value delta. Attribution walks
|
||||
`portfolio.srf` across commits, where they were already present -- so no
|
||||
contribution is found and the residual absorbs them.
|
||||
|
||||
Fix: run `zfin snapshot --force` after reconciling and before
|
||||
committing. On a non-trading day it rewrites the last trading day's
|
||||
file, so it never invents a weekend snapshot.
|
||||
|
||||
## Relative dates
|
||||
|
||||
|
|
|
|||
|
|
@ -23,7 +23,7 @@ silently.
|
|||
| `--projections` | Add projected-return and 99% safe-withdrawal deltas (adds ~1-2s per endpoint). |
|
||||
| `--no-events` | With `--projections`, exclude life events. |
|
||||
| `--snapshot-before <DATE>` / `--snapshot-after <DATE>` | Override a side's snapshot (`--snapshot-after live` for the current portfolio). |
|
||||
| `--commit-before <SPEC>` / `--commit-after <SPEC>` | Pin the git commit for the attribution block (`HEAD`, `HEAD~N`, SHA, or `working`). Useful when a review date and its commit diverge. |
|
||||
| `--commit-before <SPEC>` / `--commit-after <SPEC>` | Pin the git commit for the attribution block (`HEAD`, `HEAD~N`, SHA, or `working`). Rarely needed: by default each side anchors on the commit that *recorded* its snapshot. Passing `HEAD` is usually wrong -- see [Attribution and commit timing](../../guides/periodic-review.md#attribution-and-commit-timing). |
|
||||
|
||||
## Example
|
||||
|
||||
|
|
|
|||
|
|
@ -59,6 +59,17 @@ leg's trailing returns, the bond leg's, a blend weighted by your actual
|
|||
equity/fixed-income split, and your portfolio's own weighted return. The
|
||||
two legs default to `SPY` and `AGG`:
|
||||
|
||||
The blend is **renormalized** to the weight actually present. Your
|
||||
equity and fixed-income fractions do not sum to 1 -- the remainder is
|
||||
cash, CDs, options, and anything with no `metadata.srf` row -- so a raw
|
||||
`stock*w1 + bond*w2` would describe a benchmark holding a zero-return
|
||||
sleeve, and would not be comparable against the portfolio row (which is
|
||||
renormalized). Both rows therefore describe invested assets only.
|
||||
|
||||
The year columns are **total return** (dividend-reinvested). The `Week`
|
||||
column is price-only, because dividends over seven days are negligible;
|
||||
the table header says so.
|
||||
|
||||
```
|
||||
type::config,benchmark_stock::SPYM
|
||||
type::config,benchmark_bond::BND
|
||||
|
|
|
|||
|
|
@ -209,7 +209,8 @@ pub fn toReturnsByPeriod(tr: TrailingReturns) ReturnsByPeriod {
|
|||
};
|
||||
}
|
||||
|
||||
/// Compute the weighted average of two ReturnsByPeriod values.
|
||||
/// Weighted average of two `ReturnsByPeriod`, renormalized to the weight actually
|
||||
/// present. See `blendOptional`.
|
||||
fn blendReturns(a: ReturnsByPeriod, a_weight: f64, b: ReturnsByPeriod, b_weight: f64) ReturnsByPeriod {
|
||||
return .{
|
||||
.one_year = blendOptional(a.one_year, a_weight, b.one_year, b_weight),
|
||||
|
|
@ -220,11 +221,39 @@ fn blendReturns(a: ReturnsByPeriod, a_weight: f64, b: ReturnsByPeriod, b_weight:
|
|||
};
|
||||
}
|
||||
|
||||
/// Blend two optional returns by weight, RENORMALIZED by the weight that
|
||||
/// contributed.
|
||||
///
|
||||
/// The renormalization is the whole point and it used to be missing. `a_weight`
|
||||
/// and `b_weight` are the portfolio's own stock and bond fractions, which do NOT
|
||||
/// sum to 1: the remainder is cash, CDs, options, `other`, and anything with no
|
||||
/// `metadata.srf` row (see `deriveAllocationSplit`). On a real portfolio that
|
||||
/// remainder was 4.7%, so `stock*0.851 + bond*0.102` described a benchmark holding
|
||||
/// a 4.7% sleeve returning exactly zero. The 1-year figure printed 18.91% where
|
||||
/// the two-fund blend it claims to be is 19.84%.
|
||||
///
|
||||
/// It also made the table internally inconsistent, which is how it was found:
|
||||
/// `portfolioWeightedReturns` DOES renormalize (see the comment there), so
|
||||
/// "Benchmark" carried a cash drag that "Your Portfolio" did not, and every
|
||||
/// comparison between the two rows was biased in the portfolio's favour by the
|
||||
/// same 4.7%. Now both rows describe invested assets only and are comparable.
|
||||
///
|
||||
/// The single-sided branches were worse than the two-sided one: with no bond data
|
||||
/// at all, the "Benchmark" row was `stock * 0.851` - not a benchmark of anything.
|
||||
/// Renormalizing collapses that to just `stock`, which is the honest answer when
|
||||
/// the bond leg is unavailable.
|
||||
///
|
||||
/// Callers wanting the un-renormalized figure want a different thing (a
|
||||
/// total-portfolio return including a cash drag), and that needs a declared cash
|
||||
/// return rather than an implicit zero. Not modelled here.
|
||||
fn blendOptional(a: ?f64, a_w: f64, b: ?f64, b_w: f64) ?f64 {
|
||||
if (a != null and b != null)
|
||||
return a.? * a_w + b.? * b_w;
|
||||
if (a != null) return a.? * a_w;
|
||||
if (b != null) return b.? * b_w;
|
||||
if (a != null and b != null) {
|
||||
const w = a_w + b_w;
|
||||
if (w <= 0) return null;
|
||||
return (a.? * a_w + b.? * b_w) / w;
|
||||
}
|
||||
if (a != null) return if (a_w > 0) a.? else null;
|
||||
if (b != null) return if (b_w > 0) b.? else null;
|
||||
return null;
|
||||
}
|
||||
|
||||
|
|
@ -395,14 +424,29 @@ test "blendReturns weighted average" {
|
|||
try std.testing.expectApproxEqAbs(@as(f64, 0.1225), result.three_year.?, 0.0001);
|
||||
}
|
||||
|
||||
test "blendReturns handles null" {
|
||||
test "blendReturns with one leg missing reports the surviving leg" {
|
||||
const a = ReturnsByPeriod{ .one_year = 0.20, .three_year = null };
|
||||
const b = ReturnsByPeriod{ .one_year = null, .three_year = 0.04 };
|
||||
const result = blendReturns(a, 0.75, b, 0.25);
|
||||
// Only a has 1Y: 0.20 * 0.75 = 0.15
|
||||
try std.testing.expectApproxEqAbs(@as(f64, 0.15), result.one_year.?, 0.0001);
|
||||
// Only b has 3Y: 0.04 * 0.25 = 0.01
|
||||
try std.testing.expectApproxEqAbs(@as(f64, 0.01), result.three_year.?, 0.0001);
|
||||
|
||||
// This test used to assert 0.15 and 0.01 - the surviving leg scaled by its own
|
||||
// weight. That figure is neither the leg's return nor a blend of anything: with
|
||||
// no bond data there is nothing to blend WITH, so 75% of the stock return
|
||||
// describes no portfolio. It also made "Benchmark" quietly understate itself by
|
||||
// the missing leg's weight whenever one series was unavailable.
|
||||
//
|
||||
// Renormalizing gives the only defensible answer - the leg that exists.
|
||||
try std.testing.expectApproxEqAbs(@as(f64, 0.20), result.one_year.?, 0.0001);
|
||||
try std.testing.expectApproxEqAbs(@as(f64, 0.04), result.three_year.?, 0.0001);
|
||||
|
||||
// Neither leg present stays null rather than becoming zero.
|
||||
const empty = blendReturns(.{}, 0.75, .{}, 0.25);
|
||||
try std.testing.expect(empty.one_year == null);
|
||||
|
||||
// A zero-weight leg cannot carry the blend either: weight 0 means "not part of
|
||||
// this portfolio", which is different from "return of 0%".
|
||||
const zero_w = blendReturns(a, 0, .{}, 0);
|
||||
try std.testing.expect(zero_w.one_year == null);
|
||||
}
|
||||
|
||||
test "portfolioWeightedReturns basic" {
|
||||
|
|
@ -619,6 +663,48 @@ test "conservativeWeightedReturn single position single period" {
|
|||
try std.testing.expectApproxEqAbs(@as(f64, 0.125), result, 0.001);
|
||||
}
|
||||
|
||||
test "buildComparison renormalizes the benchmark blend to the weight present" {
|
||||
// The regression. Weights are the portfolio's OWN stock/bond fractions and do
|
||||
// not sum to 1 - the remainder is cash, CDs, options and unclassified
|
||||
// holdings. Every existing test here happens to use weights summing to exactly
|
||||
// 1.0, where renormalized and un-renormalized agree, which is why this went
|
||||
// unnoticed.
|
||||
//
|
||||
// Figures are the real portfolio's: 85.1% stock, 10.2% bond, SPYM 1Y 21.92%,
|
||||
// AGG 1Y 2.47%.
|
||||
const stock_tr = TrailingReturns{ .one_year = makePR(0.2192, 0.2192), .week = -0.0138 };
|
||||
const bond_tr = TrailingReturns{ .one_year = makePR(0.0247, 0.0247), .week = -0.0013 };
|
||||
const positions = [_]PositionReturn{};
|
||||
const result = buildComparison(stock_tr, bond_tr, 0.851, 0.102, &positions, null);
|
||||
|
||||
// Un-renormalized this printed 0.851*0.2192 + 0.102*0.0247 = 0.18906 -> 18.91%,
|
||||
// i.e. a two-fund blend carrying a 4.7% sleeve returning zero. The blend it
|
||||
// claims to be is 0.18906 / 0.953 = 0.19839.
|
||||
try std.testing.expectApproxEqAbs(@as(f64, 0.19839), result.benchmark_returns.one_year.?, 0.0001);
|
||||
try std.testing.expect(result.benchmark_returns.one_year.? > 0.18906);
|
||||
|
||||
// Same for Week: -0.0138*0.851 + -0.0013*0.102 = -0.0118776 -> /0.953.
|
||||
try std.testing.expectApproxEqAbs(@as(f64, -0.012463), result.benchmark_returns.week.?, 0.0001);
|
||||
|
||||
// A renormalized blend must sit BETWEEN its two legs. The old form did not:
|
||||
// 18.91% was below AGG-weighted-plus-SPYM-weighted precisely because the
|
||||
// missing weight dragged it down.
|
||||
try std.testing.expect(result.benchmark_returns.one_year.? < 0.2192);
|
||||
try std.testing.expect(result.benchmark_returns.one_year.? > 0.0247);
|
||||
}
|
||||
|
||||
test "buildComparison with only one benchmark leg reports that leg, not a fraction of it" {
|
||||
// With no bond data the old code returned `stock * 0.851` - 85% of SPY's
|
||||
// return, presented as "Benchmark". Not a benchmark of anything.
|
||||
const stock_tr = TrailingReturns{ .one_year = makePR(0.20, 0.20) };
|
||||
const bond_tr = TrailingReturns{}; // no data at all
|
||||
const positions = [_]PositionReturn{};
|
||||
const result = buildComparison(stock_tr, bond_tr, 0.851, 0.102, &positions, null);
|
||||
|
||||
try std.testing.expectApproxEqAbs(@as(f64, 0.20), result.benchmark_returns.one_year.?, 0.0001);
|
||||
try std.testing.expect(result.benchmark_returns.week == null);
|
||||
}
|
||||
|
||||
test "buildComparison with week returns" {
|
||||
// Week returns now flow through `TrailingReturns.week` rather
|
||||
// than separate parameters - `performance.totalReturns`
|
||||
|
|
|
|||
|
|
@ -487,15 +487,33 @@ pub fn formatReturn(buf: []u8, value: f64) []const u8 {
|
|||
|
||||
/// Compute 1-week return from candle data: (latest_close / close_7_days_ago) - 1.
|
||||
/// Candles must be sorted by date ascending.
|
||||
///
|
||||
/// Bounded by `max_snap_days`, same as `findNearestCandle`. The backward scan used
|
||||
/// to be unbounded with a comment asserting "at most ~10 steps for daily candles"
|
||||
/// - an assumption about the data, not a constraint on the code. A hole in the
|
||||
/// candle cache therefore turned this into a month's return still labelled "Week",
|
||||
/// silently and with no way to tell from the output. That is exactly the failure
|
||||
/// `max_snap_days` was introduced to prevent ("Beyond this, the data is likely
|
||||
/// missing and the result would be misleading"); this function simply never
|
||||
/// applied it.
|
||||
///
|
||||
/// Real caches do have holes: a snapshot taken 2026-08-29 carried
|
||||
/// `stale_count:num:3`, with three symbols a full day behind their peers.
|
||||
///
|
||||
/// Returns null rather than a wrong number, so the caller renders "--" and the
|
||||
/// gap is visible instead of plausible.
|
||||
pub fn weekReturn(candles: []const Candle) ?f64 {
|
||||
if (candles.len < 2) return null;
|
||||
const latest = candles[candles.len - 1];
|
||||
const target_date = latest.date.addDays(-7);
|
||||
|
||||
// Linear scan backward (at most ~10 steps for daily candles)
|
||||
var i: usize = candles.len - 2;
|
||||
while (true) {
|
||||
if (candles[i].date.days <= target_date.days) {
|
||||
// How far BEFORE the target we landed. A weekend or holiday puts the
|
||||
// nearest bar a few days early, which is normal and fine; a cache hole
|
||||
// puts it weeks early, which is not a week's return.
|
||||
if (target_date.days - candles[i].date.days > max_snap_days) return null;
|
||||
if (candles[i].close == 0) return null;
|
||||
return (latest.close / candles[i].close) - 1.0;
|
||||
}
|
||||
|
|
@ -543,6 +561,33 @@ test "weekReturn snaps to nearest trading day" {
|
|||
try std.testing.expectApproxEqAbs(@as(f64, 0.1667), r, 0.001);
|
||||
}
|
||||
|
||||
test "weekReturn refuses a gap wider than max_snap_days" {
|
||||
// A hole in the candle cache. Latest is Feb 10; 7 days back is Feb 3, but the
|
||||
// previous bar is Jan 1 - forty days earlier. The old unbounded scan happily
|
||||
// returned (105/90 - 1) = +16.67% and labelled it "Week".
|
||||
const holed = [_]Candle{
|
||||
.{ .date = Date.fromYmd(2024, 1, 1), .open = 90, .high = 90, .low = 90, .close = 90, .adj_close = 90, .volume = 0 },
|
||||
.{ .date = Date.fromYmd(2024, 2, 10), .open = 105, .high = 105, .low = 105, .close = 105, .adj_close = 105, .volume = 0 },
|
||||
};
|
||||
try std.testing.expect(weekReturn(&holed) == null);
|
||||
|
||||
// The boundary is inclusive and generous enough for a holiday week: landing
|
||||
// exactly `max_snap_days` before the target still counts. Target is Jan 21 - 7
|
||||
// = Jan 14; Jan 4 is 10 days before that.
|
||||
const at_limit = [_]Candle{
|
||||
.{ .date = Date.fromYmd(2024, 1, 4), .open = 90, .high = 90, .low = 90, .close = 90, .adj_close = 90, .volume = 0 },
|
||||
.{ .date = Date.fromYmd(2024, 1, 21), .open = 99, .high = 99, .low = 99, .close = 99, .adj_close = 99, .volume = 0 },
|
||||
};
|
||||
try std.testing.expectApproxEqAbs(@as(f64, 0.10), weekReturn(&at_limit).?, 0.001);
|
||||
|
||||
// One day past it is refused, so the cutoff is actually enforced rather than
|
||||
// merely present.
|
||||
const past_limit = [_]Candle{
|
||||
.{ .date = Date.fromYmd(2024, 1, 3), .open = 90, .high = 90, .low = 90, .close = 90, .adj_close = 90, .volume = 0 },
|
||||
.{ .date = Date.fromYmd(2024, 1, 21), .open = 99, .high = 99, .low = 99, .close = 99, .adj_close = 99, .volume = 0 },
|
||||
};
|
||||
try std.testing.expect(weekReturn(&past_limit) == null);
|
||||
}
|
||||
test "weekReturn zero close returns null" {
|
||||
const candles = [_]Candle{
|
||||
.{ .date = Date.fromYmd(2024, 1, 1), .open = 0, .high = 0, .low = 0, .close = 0, .adj_close = 0, .volume = 0 },
|
||||
|
|
|
|||
20
src/cache/store.zig
vendored
20
src/cache/store.zig
vendored
|
|
@ -1550,11 +1550,27 @@ pub const Store = struct {
|
|||
/// motivating case is a unitized trust with no ticker and no
|
||||
/// CUSIP, whose daily unit values come from a plan
|
||||
/// recordkeeper's feed; Tiingo, Yahoo and TwelveData all 404
|
||||
/// it. So zfin can never fetch or restate such a series
|
||||
/// itself. It arrives either by `ZFIN_SERVER` sync (the client
|
||||
/// it. So zfin cannot fetch or restate such a series itself,
|
||||
/// and it arrives either by `ZFIN_SERVER` sync (the client
|
||||
/// case) or by direct population of the cache directory (the
|
||||
/// server case).
|
||||
///
|
||||
/// **This is a statement about the present, not a permanent
|
||||
/// property of the symbol - and the label is deliberately not
|
||||
/// sticky.** If a provider ever does serve the symbol, the
|
||||
/// bars in the cache genuinely came from that provider, so
|
||||
/// `applyTiingoCoverage` overwriting this label with `.tiingo`
|
||||
/// or `.yahoo` is *correct*: the provenance changed, and the
|
||||
/// symbol has rejoined the ordinary provider path. Do not add
|
||||
/// a guard that preserves `.external` across a successful
|
||||
/// fetch. Doing so would make it a one-way door, which is the
|
||||
/// exact shape of the latch bug that keying routing off
|
||||
/// `provider == .yahoo` already produced (see
|
||||
/// `tiingo_retry_after_s`) - a symbol that gained coverage
|
||||
/// would be pinned to the external regime forever. Any future
|
||||
/// routing built on this variant must preserve that escape
|
||||
/// hatch.
|
||||
///
|
||||
/// **A negative-cache entry on such a symbol is unrecoverable**,
|
||||
/// not merely inconvenient. `writeNegative` overwrites
|
||||
/// `candles_daily.srf` with a marker, negative entries never
|
||||
|
|
|
|||
|
|
@ -353,19 +353,19 @@ pub fn run(ctx: *framework.RunCtx, parsed: ParsedArgs) !void {
|
|||
|
||||
// ── Snap dates to nearest-earlier snapshots ──────────────
|
||||
//
|
||||
// The requested date may not correspond to an actual snapshot
|
||||
// file. Snap to the most recent snapshot at-or-before. Record
|
||||
// the original requested date so we can pass it to the
|
||||
// attribution git-window independently (liquid uses the snap'd
|
||||
// date; attribution uses the requested date unless --commit-*
|
||||
// overrides).
|
||||
// The requested date may not correspond to an actual snapshot file. Snap to the
|
||||
// most recent snapshot at-or-before.
|
||||
//
|
||||
// The snapped dates are then used for BOTH the value window and the attribution
|
||||
// window. They used to diverge - values on the snapped date, attribution on the
|
||||
// requested one - which quietly guaranteed the two windows described different
|
||||
// periods whenever a snap occurred, and `gains` (a residual) absorbed the
|
||||
// difference. Snapping both keeps them aligned; see the `attr_before`
|
||||
// construction below.
|
||||
var arena_state = std.heap.ArenaAllocator.init(allocator);
|
||||
defer arena_state.deinit();
|
||||
const arena = arena_state.allocator();
|
||||
|
||||
const then_date_requested = then_date;
|
||||
const now_date_requested = now_date;
|
||||
|
||||
const then_resolved = cli.resolveSnapshotOrExplain(io, arena, hist_dir, then_date) catch return error.SnapshotNotFound;
|
||||
if (!then_resolved.exact) {
|
||||
var stderr_buf: [256]u8 = undefined;
|
||||
|
|
@ -445,18 +445,29 @@ pub fn run(ctx: *framework.RunCtx, parsed: ParsedArgs) !void {
|
|||
}
|
||||
}
|
||||
|
||||
// Build the CommitSpecs for attribution. Explicit --commit-*
|
||||
// overrides win; otherwise fall back to the requested snapshot
|
||||
// date. now_is_live implies `null` for after (= working copy
|
||||
// default handling downstream).
|
||||
// Build the CommitSpecs for attribution. Explicit --commit-* overrides win;
|
||||
// otherwise anchor on the commit that RECORDED each side's snapshot.
|
||||
//
|
||||
// `snapshot_add` rather than `date_at_or_before`, and it matters. The value
|
||||
// window's endpoints are snapshot files, so the attribution window has to be
|
||||
// "the commit that recorded that snapshot" for the two to describe the same
|
||||
// period. A date lands on whatever commit happened to be nearest, which drifts
|
||||
// whenever a review is done on one day and committed on another - and `HEAD`
|
||||
// (the previous workaround for that drift) collapses the window entirely as
|
||||
// soon as anything is committed mid-review. See `git.commitThatAdded`.
|
||||
//
|
||||
// Uses the SNAPPED dates, not the requested ones: the snapped date is the
|
||||
// snapshot that actually supplied the values, so it is the one whose recording
|
||||
// commit bounds them. `snapshot_add` degrades to the old date behaviour when no
|
||||
// snapshot for that date was committed.
|
||||
const attr_before: git.CommitSpec = commit_before_override orelse
|
||||
.{ .date_at_or_before = then_date_requested };
|
||||
.{ .snapshot_add = then_date };
|
||||
const attr_after_opt: ?git.CommitSpec = if (commit_after_override) |s|
|
||||
s
|
||||
else if (now_is_live)
|
||||
null
|
||||
else
|
||||
.{ .date_at_or_before = now_date_requested };
|
||||
.{ .snapshot_add = now_date };
|
||||
|
||||
// Opt-in: bring the "then" snapshot into the current split basis so
|
||||
// a split between the two dates doesn't read as a phantom position
|
||||
|
|
@ -598,6 +609,8 @@ fn renderFromParts(
|
|||
cv.attribution = .{
|
||||
.contributions = a.total(),
|
||||
.gains = cv.liquid.delta - a.total(),
|
||||
.uncounted_in = a.uncounted_in,
|
||||
.uncounted_out = a.uncounted_out,
|
||||
};
|
||||
}
|
||||
|
||||
|
|
@ -854,6 +867,30 @@ fn renderAttributionLine(out: *std.Io.Writer, color: bool, delta: f64, attributi
|
|||
try cli.printGainLoss(out, color, attribution.gains, "{f}\n", .{Money.from(attribution.gains).signed()});
|
||||
try cli.printFg(out, color, cli.CLR_MUTED, " {s:<19} ", .{"Cash contributions:"});
|
||||
try cli.printGainLoss(out, color, attribution.contributions, "{f}\n", .{Money.from(attribution.contributions).signed()});
|
||||
|
||||
// Uncounted movement, when there is any.
|
||||
//
|
||||
// `gains` above is a RESIDUAL (`delta - contributions`), so anything the
|
||||
// classifier declines to count is reported as market performance. Raw
|
||||
// cash-balance changes and share reductions are both excluded by design - see
|
||||
// `AttributionSummary.uncounted_in` - and on a week where they do not cancel,
|
||||
// that error lands silently in the gains figure.
|
||||
//
|
||||
// Printed as two gross figures rather than one net, because the net hides the
|
||||
// scale: one real week netted to -$31.82 out of $25k moving each way, and
|
||||
// "-$31.82" would have said nothing useful about it.
|
||||
//
|
||||
// Suppressed entirely on a clean week so the common case stays two lines.
|
||||
if (attribution.hasUncounted()) {
|
||||
try cli.printFg(out, color, cli.CLR_MUTED, " {s:<19} ", .{"Uncounted in/out:"});
|
||||
try cli.printGainLoss(out, color, attribution.uncounted_in, "{f}", .{Money.from(attribution.uncounted_in).signed()});
|
||||
try cli.printFg(out, color, cli.CLR_MUTED, " / ", .{});
|
||||
try cli.printGainLoss(out, color, attribution.uncounted_out, "{f}", .{Money.from(attribution.uncounted_out).signed()});
|
||||
const net = attribution.uncounted_in + attribution.uncounted_out;
|
||||
try cli.printFg(out, color, cli.CLR_MUTED, " (net ", .{});
|
||||
try cli.printGainLoss(out, color, net, "{f}", .{Money.from(net).signed()});
|
||||
try cli.printFg(out, color, cli.CLR_MUTED, "; inside `Investment gains` - see `zfin contributions`)\n", .{});
|
||||
}
|
||||
}
|
||||
|
||||
fn renderSymbolRow(out: *std.Io.Writer, color: bool, s: view.SymbolChange) !void {
|
||||
|
|
@ -1176,6 +1213,81 @@ test "renderCompare: attribution line when attribution is set" {
|
|||
try testing.expect(std.mem.indexOf(u8, out, "+$7,512.02") != null);
|
||||
// The old `Attribution:` prefix is gone.
|
||||
try testing.expect(std.mem.indexOf(u8, out, "Attribution:") == null);
|
||||
|
||||
// A clean week has nothing uncounted, so the third line stays off and the
|
||||
// common case remains two rows.
|
||||
try testing.expect(std.mem.indexOf(u8, out, "Uncounted") == null);
|
||||
}
|
||||
|
||||
test "renderCompare: uncounted flows are surfaced gross, both directions" {
|
||||
// The real week that motivated this. `gains` is a residual, so a $25k cash
|
||||
// increase and a $25k share reduction - both deliberately uncounted - were
|
||||
// folded into "Investment gains: +$173.85" with nothing to indicate it. They
|
||||
// happened to be two legs of one internal move and nearly cancelled; an
|
||||
// unmatched leg of that size would have read as market performance.
|
||||
const cv = view.CompareView{
|
||||
.then_date = Date.fromYmd(2026, 8, 21),
|
||||
.now_date = Date.fromYmd(2026, 8, 30),
|
||||
.days_between = 9,
|
||||
.now_is_live = true,
|
||||
.liquid = view.buildTotalsRow(8_986_340.92, 8_987_832.16),
|
||||
.symbols = &.{},
|
||||
.held_count = 0,
|
||||
.added_count = 0,
|
||||
.removed_count = 0,
|
||||
.attribution = .{
|
||||
.contributions = 1_317.40,
|
||||
.gains = 173.85,
|
||||
.uncounted_in = 25_079.42,
|
||||
.uncounted_out = -25_111.24,
|
||||
},
|
||||
};
|
||||
|
||||
var buf: [4096]u8 = undefined;
|
||||
var stream = std.Io.Writer.fixed(&buf);
|
||||
try renderCompare(&stream, false, cv, null);
|
||||
const out = stream.buffered();
|
||||
|
||||
// Gross both ways, because the net alone conceals the scale.
|
||||
try testing.expect(std.mem.indexOf(u8, out, "+$25,079.42") != null);
|
||||
try testing.expect(std.mem.indexOf(u8, out, "-$25,111.24") != null);
|
||||
// And the net, so a cancelling pair is visibly a cancelling pair.
|
||||
try testing.expect(std.mem.indexOf(u8, out, "-$31.82") != null);
|
||||
// Says where the money currently sits, so the residual is not read as clean.
|
||||
try testing.expect(std.mem.indexOf(u8, out, "Investment gains") != null);
|
||||
try testing.expect(std.mem.indexOf(u8, out, "zfin contributions") != null);
|
||||
}
|
||||
|
||||
test "renderCompare: a one-directional uncounted flow still reports" {
|
||||
// The dangerous shape: nothing to cancel against, so the full amount is inside
|
||||
// `gains`. Must not be suppressed just because one side is zero.
|
||||
const cv = view.CompareView{
|
||||
.then_date = Date.fromYmd(2026, 8, 21),
|
||||
.now_date = Date.fromYmd(2026, 8, 28),
|
||||
.days_between = 7,
|
||||
.now_is_live = false,
|
||||
.liquid = view.buildTotalsRow(1_000_000, 1_020_000),
|
||||
.symbols = &.{},
|
||||
.held_count = 0,
|
||||
.added_count = 0,
|
||||
.removed_count = 0,
|
||||
.attribution = .{
|
||||
.contributions = 0,
|
||||
.gains = 20_000,
|
||||
.uncounted_in = 18_000,
|
||||
.uncounted_out = 0,
|
||||
},
|
||||
};
|
||||
|
||||
var buf: [4096]u8 = undefined;
|
||||
var stream = std.Io.Writer.fixed(&buf);
|
||||
try renderCompare(&stream, false, cv, null);
|
||||
const out = stream.buffered();
|
||||
|
||||
try testing.expect(std.mem.indexOf(u8, out, "Uncounted") != null);
|
||||
try testing.expect(std.mem.indexOf(u8, out, "+$18,000.00") != null);
|
||||
// Net equals the single leg; 90% of the reported "gains" is unexplained flow.
|
||||
try testing.expect(std.mem.indexOf(u8, out, "net +$18,000.00") != null);
|
||||
}
|
||||
|
||||
test "renderCompare: no attribution line when attribution is null" {
|
||||
|
|
|
|||
|
|
@ -744,7 +744,7 @@ fn specDisplayString(spec: ?git.CommitSpec, date_buf: *[10]u8) []const u8 {
|
|||
const s = spec orelse return "(unset)";
|
||||
return switch (s) {
|
||||
.git_ref => |r| r,
|
||||
.date_at_or_before => |d| std.fmt.bufPrint(date_buf, "{f}", .{d}) catch "????-??-??",
|
||||
.date_at_or_before, .snapshot_add => |d| std.fmt.bufPrint(date_buf, "{f}", .{d}) catch "????-??-??",
|
||||
.working_copy => "working",
|
||||
};
|
||||
}
|
||||
|
|
@ -845,6 +845,10 @@ fn specLabel(arena: std.mem.Allocator, spec: ?git.CommitSpec, resolved_ref: []co
|
|||
return switch (s) {
|
||||
.git_ref => |r| arena.dupe(u8, r),
|
||||
.date_at_or_before => |d| std.fmt.allocPrint(arena, "commit at-or-before {f}", .{d}),
|
||||
// Names the anchor, because "the commit that recorded this snapshot" is a
|
||||
// materially different claim from "some commit near this date" and the
|
||||
// header is where a reader checks which window they got.
|
||||
.snapshot_add => |d| std.fmt.allocPrint(arena, "the commit recording {f}", .{d}),
|
||||
.working_copy => arena.dupe(u8, "working copy"),
|
||||
};
|
||||
}
|
||||
|
|
@ -923,10 +927,43 @@ pub const AttributionSummary = struct {
|
|||
/// contribution-vs-DRIP cases, treated as DRIP here to avoid
|
||||
/// double-counting with cash contributions).
|
||||
drip: f64,
|
||||
/// Gross value of movements this pipeline deliberately does NOT count, split by
|
||||
/// direction. Reported, never added to `total()`.
|
||||
///
|
||||
/// ## Why these are surfaced at all
|
||||
///
|
||||
/// `compare` derives market performance as a RESIDUAL: `gains = delta -
|
||||
/// contributions`. That makes every classification gap indistinguishable from
|
||||
/// market movement. Raw cash-balance changes are noise by default (interest,
|
||||
/// DRIP legs, settlement sweeps), and share reductions are treated as a funding
|
||||
/// source rather than an outflow - both defensible, and both invisible once
|
||||
/// folded into a residual.
|
||||
///
|
||||
/// On one real week that hid a $25,079.42 cash increase against a $25,111.24
|
||||
/// share reduction. They were two legs of one internal move and netted to
|
||||
/// -$31.82, so `gains` was very nearly right - but nothing on screen said so,
|
||||
/// and an unmatched leg of that size would have been reported as market
|
||||
/// performance without comment.
|
||||
///
|
||||
/// ## Why gross rather than net
|
||||
///
|
||||
/// A single net figure would have read "-$31.82" for that week: accurate, and
|
||||
/// useless. It conceals that $25k moved. Two gross figures say "$25k came in,
|
||||
/// $25k went out, they cancel", which is the fact worth knowing.
|
||||
uncounted_in: f64 = 0,
|
||||
/// Gross uncounted outflows, as a NEGATIVE number so the sign carries meaning
|
||||
/// at the callsite without a naming convention to remember.
|
||||
uncounted_out: f64 = 0,
|
||||
|
||||
pub fn total(self: AttributionSummary) f64 {
|
||||
return self.new_contributions + self.drip;
|
||||
}
|
||||
|
||||
/// True when there is any uncounted movement worth showing. A cent threshold,
|
||||
/// because float noise on a quiet week should not produce a line.
|
||||
pub fn hasUncounted(self: AttributionSummary) bool {
|
||||
return @abs(self.uncounted_in) >= 0.005 or @abs(self.uncounted_out) >= 0.005;
|
||||
}
|
||||
};
|
||||
|
||||
/// Run the contributions pipeline over a commit window and return the
|
||||
|
|
@ -1161,10 +1198,35 @@ fn summarizeAttribution(ctx: ReportContext) AttributionSummary {
|
|||
// `partial_transfer_in` -> residual only (attributedValue()).
|
||||
var new_contributions: f64 = 0;
|
||||
var drip: f64 = 0;
|
||||
var uncounted_in: f64 = 0;
|
||||
var uncounted_out: f64 = 0;
|
||||
for (ctx.report.changes) |c| switch (c.kind) {
|
||||
.new_stock, .new_cash, .new_cd, .new_option, .cash_contribution => new_contributions += c.attributedValue(),
|
||||
.new_drip_lot, .drip_confirmed, .rollup_delta => drip += c.value(),
|
||||
.partial_transfer_in => new_contributions += c.attributedValue(),
|
||||
|
||||
// Uncounted, and now reported rather than silently dropped. See
|
||||
// `AttributionSummary.uncounted_in`.
|
||||
//
|
||||
// `cash_delta` is the raw-balance-change bucket (an opted-in account's
|
||||
// positive delta is reclassified to `cash_contribution` at diff time and
|
||||
// counted above, so it cannot be double-counted here). The share-reduction
|
||||
// kinds are the other side of the same coin: `attributedValue()` pins them
|
||||
// to 0 on the grounds that they are a funding source rather than an outflow,
|
||||
// which is right for attribution and still worth SEEING.
|
||||
.cash_delta, .drip_negative, .lot_removed, .position_closed, .lot_edited, .flagged => {
|
||||
const v = c.value();
|
||||
if (v >= 0) uncounted_in += v else uncounted_out += v;
|
||||
},
|
||||
|
||||
// Deliberately NOT in the uncounted totals:
|
||||
// - `cd_matured` / `cd_removed_early` have their own report section and
|
||||
// always pair with a cash increase in the same account, so counting both
|
||||
// legs would double-report one internal move.
|
||||
// - `transfer_in` / `transfer_out` / `unmatched_transfer` are DECLARED
|
||||
// internal moves (`transaction_log.srf`). The point of this line is
|
||||
// UNdeclared movement; a declared transfer is not a surprise.
|
||||
// - `price_only` carries no share change, so its value is zero anyway.
|
||||
else => {},
|
||||
};
|
||||
// Cash-dest transfer attribution is already removed by `attributedValue()` on
|
||||
|
|
@ -1172,7 +1234,12 @@ fn summarizeAttribution(ctx: ReportContext) AttributionSummary {
|
|||
// `transfer_attributed`, so a fully-attributed cash Change contributes zero
|
||||
// here without any separate per-account subtraction.
|
||||
|
||||
return .{ .new_contributions = new_contributions, .drip = drip };
|
||||
return .{
|
||||
.new_contributions = new_contributions,
|
||||
.drip = drip,
|
||||
.uncounted_in = uncounted_in,
|
||||
.uncounted_out = uncounted_out,
|
||||
};
|
||||
}
|
||||
|
||||
// ── Git discovery / invocation ───────────────────────────────
|
||||
|
|
|
|||
|
|
@ -490,7 +490,7 @@ fn renderWindowsBlock(out: *std.Io.Writer, color: bool, ws: timeline.WindowSet)
|
|||
// snapshot-to-snapshot Liquid deltas (or whichever metric is
|
||||
// focused) - they include contributions, withdrawals, and
|
||||
// weight drift, distinct from the `projections` benchmark
|
||||
// table which reports price-only weighted returns and so will
|
||||
// table which reports weighted total returns and so will
|
||||
// disagree on weeks with significant cash movement or
|
||||
// rebalancing.
|
||||
try cli.printFg(out, color, cli.CLR_MUTED, " (snapshot-to-snapshot Δ; includes contributions, withdrawals, weight drift)\n", .{});
|
||||
|
|
|
|||
|
|
@ -841,15 +841,52 @@ pub fn runBands(
|
|||
// comparing these numbers against the `history` tab's window
|
||||
// table doesn't get tripped up by the (legitimate)
|
||||
// disagreement: this table reports each row as a weighted
|
||||
// price-only return per period (per-symbol price change ×
|
||||
// current weight, summed) so SPY/AGG/Benchmark/Your
|
||||
// Portfolio rows are apples-to-apples; history's window
|
||||
// table reports snapshot-to-snapshot Liquid value deltas
|
||||
// that include contributions, withdrawals, and weight drift.
|
||||
// return per period (per-symbol return × current weight,
|
||||
// summed) so SPY/AGG/Benchmark/Your Portfolio rows are
|
||||
// apples-to-apples; history's window table reports
|
||||
// snapshot-to-snapshot Liquid value deltas that include
|
||||
// contributions, withdrawals, and weight drift.
|
||||
//
|
||||
// "total return", not "price-only", and the distinction is not
|
||||
// cosmetic. The year columns come from `performance.totalReturns`,
|
||||
// which is dividend-reinvested - see the comment in
|
||||
// `views/projections.zig` recording why: calling
|
||||
// `performance.trailingReturns` directly used the adj_close series
|
||||
// alone, so a stale adjustment basis understated the benchmark, the
|
||||
// portfolio-weighted return, AND `conservative_return`, which drives
|
||||
// the Monte Carlo. This title was never updated after that fix and
|
||||
// spent a while contradicting three of its own five columns.
|
||||
//
|
||||
// Week is genuinely price-only: `performance.weekReturn` uses raw
|
||||
// `close`. Left that way deliberately - dividends over seven days are
|
||||
// negligible, and reconstructing them would add a dividend lookup to
|
||||
// the one column nobody annualizes. Called out here so the mismatch
|
||||
// is documented rather than discovered.
|
||||
try cli.setBold(out, color);
|
||||
try out.print("Benchmark comparison (price-only weighted return)\n", .{});
|
||||
if (resolution) |r| {
|
||||
try out.print("Benchmark comparison as of {f} (total return, weighted; Week is price-only)\n", .{r.actual});
|
||||
} else {
|
||||
try out.print("Benchmark comparison (total return, weighted; Week is price-only)\n", .{});
|
||||
}
|
||||
try cli.reset(out, color);
|
||||
|
||||
// Under `--as-of`, say out loud which week the Week column covers.
|
||||
//
|
||||
// This exists because of a specific hour lost to it. `--as-of 1W` truncates the
|
||||
// candle series at the nearest snapshot a week back, so every column - Week
|
||||
// included - ends THERE, not today. Read beside a `compare` run covering the
|
||||
// current week, the two disagree by a whole week's market movement and look
|
||||
// like a bug in one of them. They were both right: SPYM returned -1.38% in the
|
||||
// week ending 2026-08-21 and +0.49% in the week ending 2026-08-28.
|
||||
//
|
||||
// The header above already carried the as-of date, but a table pasted into an
|
||||
// email loses it, and the reader is then holding two contradictory Week figures
|
||||
// with nothing on screen to reconcile them. So the date goes on the section
|
||||
// itself, and the window gets stated explicitly rather than implied.
|
||||
if (resolution) |r| {
|
||||
try cli.printFg(out, color, cli.CLR_MUTED, "(every column ends {f}; Week is the 7 days ENDING then, not the current week)\n", .{r.actual});
|
||||
}
|
||||
|
||||
// Header row
|
||||
try cli.printFg(out, color, cli.CLR_MUTED, "{s: <32}{s: >8}{s: >9}{s: >9}{s: >10}{s: >9}\n", .{
|
||||
"", "1 Year", "3 Year", "5 Year", "10 Year", "Week",
|
||||
|
|
|
|||
153
src/git.zig
153
src/git.zig
|
|
@ -78,6 +78,13 @@ pub const RepoInfo = struct {
|
|||
pub const CommitSpec = union(enum) {
|
||||
git_ref: []const u8,
|
||||
date_at_or_before: Date,
|
||||
/// The commit that ADDED `history/<date>-portfolio.srf`.
|
||||
///
|
||||
/// The stable anchor for a weekly window - see `commitThatAdded` for why this
|
||||
/// beats a bare date or `HEAD`. Falls back to `date_at_or_before` when no
|
||||
/// snapshot for that date was ever committed, so a portfolio with no history
|
||||
/// directory keeps working unchanged.
|
||||
snapshot_add: Date,
|
||||
working_copy,
|
||||
};
|
||||
|
||||
|
|
@ -408,6 +415,68 @@ pub fn lastCommitTimestampForPath(
|
|||
return std.fmt.parseInt(i64, trimmed, 10) catch return null;
|
||||
}
|
||||
|
||||
/// Return the SHA of the commit that ADDED `rel_path`, or null if the path was
|
||||
/// never added (or does not exist in history).
|
||||
///
|
||||
/// `--diff-filter=A` is the whole point. It matches only the commit that
|
||||
/// introduced the file, ignoring every later modification - which is what makes
|
||||
/// this a stable anchor where a date or `HEAD` is not.
|
||||
///
|
||||
/// ## Why anchoring on a snapshot's add-commit is the right window
|
||||
///
|
||||
/// The weekly review writes `history/<date>-portfolio.srf` for each market day and
|
||||
/// commits the whole batch together with the reconciled `portfolio.srf`. So the
|
||||
/// commit that ADDED a given Friday's snapshot IS that week's reconciliation
|
||||
/// commit, by construction. Resolving it needs no commit-message convention, is
|
||||
/// unaffected by how many other commits landed that day, and survives later
|
||||
/// rewrites of the file's contents.
|
||||
///
|
||||
/// The alternatives all failed in practice:
|
||||
///
|
||||
/// - `HEAD` assumes HEAD is the PREVIOUS week's reconcile commit. Commit anything
|
||||
/// during a review - a data fix, a metadata tweak - and the attribution window
|
||||
/// silently collapses to "uncommitted edits only" while the value window still
|
||||
/// spans the week. Observed: five commits landed during one review, and ~$23k
|
||||
/// of vested shares got reported as market gains.
|
||||
/// - A date (`--until=<DATE> -- portfolio.srf`) is close, but drifts whenever a
|
||||
/// review is done on one day and committed on another, which is the documented
|
||||
/// "committed mid-week after the review date" edge case.
|
||||
/// - "Last commit touching history/" picks up retroactive restatements. One
|
||||
/// commit rewrote 97 historical snapshots at once; `--diff-filter=A` ignored it
|
||||
/// correctly, a plain path filter would not have.
|
||||
///
|
||||
/// Caller owns the returned string.
|
||||
pub fn commitThatAdded(
|
||||
io: std.Io,
|
||||
allocator: std.mem.Allocator,
|
||||
env: *const std.process.Environ.Map,
|
||||
root: []const u8,
|
||||
rel_path: []const u8,
|
||||
) Error!?[]const u8 {
|
||||
// `--follow` is deliberately NOT used. It would chase renames and could walk
|
||||
// back into a differently-named predecessor, which is exactly the kind of
|
||||
// silent window-widening this function exists to prevent.
|
||||
const result = runGit(io, allocator, env, &.{
|
||||
"git", "-C", root,
|
||||
"log", "--diff-filter=A", "-1",
|
||||
"--format=%H", "--", rel_path,
|
||||
}, .limited(64 * 1024)) catch return error.GitUnavailable;
|
||||
defer allocator.free(result.stdout);
|
||||
defer allocator.free(result.stderr);
|
||||
|
||||
switch (result.term) {
|
||||
.exited => |code| if (code != 0) return error.GitLogFailed,
|
||||
else => return error.GitLogFailed,
|
||||
}
|
||||
|
||||
const trimmed = std.mem.trim(u8, result.stdout, " \t\r\n");
|
||||
if (trimmed.len == 0) return null;
|
||||
// Same hash-shape guard as `commitAtOrBeforeDate`.
|
||||
if (trimmed.len < 40) return error.GitLogFailed;
|
||||
for (trimmed) |c| if (!std.ascii.isHex(c)) return error.GitLogFailed;
|
||||
return try allocator.dupe(u8, trimmed);
|
||||
}
|
||||
|
||||
/// Return the SHA of the most recent commit that touched any of
|
||||
/// `rel_paths` at or before `date_iso` (YYYY-MM-DD, inclusive
|
||||
/// end-of-day semantics via `git log --until`).
|
||||
|
|
@ -651,10 +720,36 @@ fn resolveSpec(io: std.Io, arena: std.mem.Allocator, env: *const std.process.Env
|
|||
return error.NoCommitAtOrBefore;
|
||||
break :blk sha;
|
||||
},
|
||||
.snapshot_add => |d| blk: {
|
||||
const snap_rel = try snapshotRelPath(arena, rel_paths, d);
|
||||
if (try commitThatAdded(io, arena, env, repo.root, snap_rel)) |sha| break :blk sha;
|
||||
// No committed snapshot for that date. Fall back to the date behaviour
|
||||
// rather than erroring: a portfolio kept without a history directory is
|
||||
// a legitimate configuration, and this spec should degrade to the old
|
||||
// resolution instead of breaking it.
|
||||
var buf: [10]u8 = undefined;
|
||||
const date_str = std.fmt.bufPrint(&buf, "{f}", .{d}) catch buf[0..];
|
||||
const sha = (try commitAtOrBeforeDate(io, arena, env, repo.root, rel_paths, date_str)) orelse
|
||||
return error.NoCommitAtOrBefore;
|
||||
break :blk sha;
|
||||
},
|
||||
.working_copy => error.InvalidArg,
|
||||
};
|
||||
}
|
||||
|
||||
/// `history/<date>-portfolio.srf`, relative to the repo root.
|
||||
///
|
||||
/// Derived from the portfolio's own repo-relative directory so a portfolio kept in
|
||||
/// a subdirectory resolves its sibling history dir rather than one at the root.
|
||||
fn snapshotRelPath(arena: std.mem.Allocator, rel_paths: []const []const u8, d: Date) Error![]const u8 {
|
||||
const first = if (rel_paths.len > 0) rel_paths[0] else "portfolio.srf";
|
||||
const dir = std.fs.path.dirname(first);
|
||||
return if (dir) |dd|
|
||||
std.fmt.allocPrint(arena, "{s}/history/{f}-portfolio.srf", .{ dd, d }) catch error.OutOfMemory
|
||||
else
|
||||
std.fmt.allocPrint(arena, "history/{f}-portfolio.srf", .{d}) catch error.OutOfMemory;
|
||||
}
|
||||
|
||||
/// Back-compat wrapper for the original `Date`-based API. Existing
|
||||
/// callers (legacy `zfin contributions --since / --until`) keep
|
||||
/// working unchanged. New callers using explicit commit refs go
|
||||
|
|
@ -913,3 +1008,61 @@ test "resolveCommitRange: --since with no earlier commit -> NoCommitAtOrBefore"
|
|||
);
|
||||
try std.testing.expectError(error.NoCommitAtOrBefore, result);
|
||||
}
|
||||
|
||||
test "commitThatAdded finds the introducing commit and ignores later edits" {
|
||||
// Uses the ambient zfin checkout: `build.zig` was added once and modified many
|
||||
// times since, which is exactly the shape `--diff-filter=A` must see through.
|
||||
const allocator = std.testing.allocator;
|
||||
var env = gitTestEnv(allocator);
|
||||
defer env.deinit();
|
||||
const info = findRepo(std.testing.io, allocator, &env, "build.zig") catch return;
|
||||
defer allocator.free(info.root);
|
||||
defer allocator.free(info.rel_path);
|
||||
|
||||
const add_sha = (commitThatAdded(std.testing.io, allocator, &env, info.root, info.rel_path) catch return) orelse return;
|
||||
defer allocator.free(add_sha);
|
||||
try std.testing.expect(add_sha.len == 40 or add_sha.len == 64);
|
||||
for (add_sha) |c| try std.testing.expect(std.ascii.isHex(c));
|
||||
|
||||
// The add-commit must be at-or-older than the newest commit touching the file.
|
||||
// If `--diff-filter=A` were being ignored these would be equal for a file with
|
||||
// any edit history, which `build.zig` certainly has.
|
||||
const newest = (commitAtOrBeforeDate(std.testing.io, allocator, &env, info.root, &.{info.rel_path}, "2099-01-01") catch return) orelse return;
|
||||
defer allocator.free(newest);
|
||||
try std.testing.expect(!std.mem.eql(u8, add_sha, newest));
|
||||
}
|
||||
|
||||
test "commitThatAdded returns null for a path never committed" {
|
||||
const allocator = std.testing.allocator;
|
||||
var env = gitTestEnv(allocator);
|
||||
defer env.deinit();
|
||||
const info = findRepo(std.testing.io, allocator, &env, "build.zig") catch return;
|
||||
defer allocator.free(info.root);
|
||||
defer allocator.free(info.rel_path);
|
||||
|
||||
// Null, not an error: `resolveSpec` relies on this to fall back to the date
|
||||
// behaviour for a portfolio kept without a committed history directory.
|
||||
const sha_opt = commitThatAdded(std.testing.io, allocator, &env, info.root, "history/1970-01-01-portfolio.srf") catch return;
|
||||
try std.testing.expect(sha_opt == null);
|
||||
}
|
||||
|
||||
test "snapshotRelPath places history beside the portfolio, not at the repo root" {
|
||||
var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator);
|
||||
defer arena_state.deinit();
|
||||
const arena = arena_state.allocator();
|
||||
const d = Date.fromYmd(2026, 8, 21);
|
||||
|
||||
// Portfolio at the repo root.
|
||||
const at_root = try snapshotRelPath(arena, &.{"portfolio.srf"}, d);
|
||||
try std.testing.expectEqualStrings("history/2026-08-21-portfolio.srf", at_root);
|
||||
|
||||
// Portfolio in a subdirectory: the history dir is its SIBLING. Resolving to a
|
||||
// root-level `history/` would silently find nothing and fall back to the date
|
||||
// behaviour, reintroducing the drift this spec exists to remove.
|
||||
const nested = try snapshotRelPath(arena, &.{"accounts/portfolio.srf"}, d);
|
||||
try std.testing.expectEqualStrings("accounts/history/2026-08-21-portfolio.srf", nested);
|
||||
|
||||
// Empty path list is defensive, not expected; must not crash.
|
||||
const empty = try snapshotRelPath(arena, &.{}, d);
|
||||
try std.testing.expectEqualStrings("history/2026-08-21-portfolio.srf", empty);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -4379,9 +4379,10 @@ test "getCandles offline never escalates a stale adjustment basis" {
|
|||
// The label is pure provenance and must stay behavior-free, with one
|
||||
// exception: it vetoes the negative-cache write, because for a series
|
||||
// no provider carries, that marker is unrecoverable rather than merely
|
||||
// sticky. These tests pin both halves - the label changes nothing about
|
||||
// how a cache is served, and it does change whether a unanimous 404
|
||||
// gets remembered.
|
||||
// sticky. These tests pin all three halves - the label changes nothing
|
||||
// about how a cache is served, it does change whether a unanimous 404
|
||||
// gets remembered, and it is itself overwritten the moment a provider
|
||||
// starts serving the symbol.
|
||||
|
||||
test "shouldNegativeCache refuses an external-provider symbol" {
|
||||
// Looped over every variant so a fifth one has to come here and
|
||||
|
|
@ -4403,6 +4404,41 @@ test "shouldNegativeCache refuses an external-provider symbol" {
|
|||
try std.testing.expect(DataService.shouldNegativeCache(null));
|
||||
}
|
||||
|
||||
test "a successful provider fetch relabels an external symbol (the label is not sticky)" {
|
||||
// `external` records where the bars came from NOW, not a permanent
|
||||
// property of the symbol. A provider serving it means the cached
|
||||
// bars genuinely came from that provider, so relabelling is the
|
||||
// honest outcome - and it is the good outcome, because the symbol
|
||||
// has rejoined the ordinary provider path under its own steam.
|
||||
//
|
||||
// This test exists to stop a well-meaning future guard that
|
||||
// preserves `.external` across a successful fetch. That would make
|
||||
// the label a one-way door: a symbol that gained coverage would be
|
||||
// pinned to a tier only one machine can populate, which is the
|
||||
// latch bug that routing off `provider == .yahoo` already caused.
|
||||
const was_external = cache.Store.CandleMeta{
|
||||
.last_close = 25,
|
||||
.last_date = Date.fromYmd(2026, 8, 14),
|
||||
.provider = .external,
|
||||
};
|
||||
|
||||
// Tiingo started carrying it.
|
||||
const via_tiingo = DataService.applyTiingoCoverage(was_external, "XTRN", 1_787_000_000, .tiingo, .covered);
|
||||
try std.testing.expectEqual(cache.Store.CandleProvider.tiingo, via_tiingo.provider);
|
||||
|
||||
// Yahoo did, with Tiingo still disclaiming it. Provenance follows
|
||||
// whoever actually answered, same as for any other symbol.
|
||||
const via_yahoo = DataService.applyTiingoCoverage(was_external, "XTRN", 1_787_000_000, .yahoo, .not_found);
|
||||
try std.testing.expectEqual(cache.Store.CandleProvider.yahoo, via_yahoo.provider);
|
||||
|
||||
// And once relabelled, the negative-cache veto no longer applies -
|
||||
// the symbol is an ordinary provider-sourced series again, so a
|
||||
// later unanimous 404 is a real verdict worth remembering.
|
||||
try std.testing.expect(DataService.shouldNegativeCache(via_tiingo));
|
||||
try std.testing.expect(DataService.shouldNegativeCache(via_yahoo));
|
||||
try std.testing.expect(!DataService.shouldNegativeCache(was_external));
|
||||
}
|
||||
|
||||
test "getCandles serves an external-provider cache without touching the network" {
|
||||
// `provider::external` must be inert on the serve path. In
|
||||
// particular it must not fall into the `.twelvedata` carve-out,
|
||||
|
|
|
|||
|
|
@ -32,14 +32,6 @@ const srf = @import("srf");
|
|||
/// every `zfin portfolio` run; the fix was to turn this on for
|
||||
/// `portfolio.srf` alone, which left every other hand-edited file
|
||||
/// exposed to the same typo.
|
||||
///
|
||||
/// This is mitigation, not a cure. Two holes remain, both needing an
|
||||
/// upstream fix in SRF's `coerce`:
|
||||
///
|
||||
/// - a non-string, non-number value in a numeric field (say
|
||||
/// `harvested:bool:true`) still reaches the unchecked access;
|
||||
/// - enum fields ignore this option entirely, so a typo like
|
||||
/// `security_type::stok` still hits `stringToEnum(...).?`.
|
||||
pub const user_edited: srf.CoercionOptions = .{ .strings_to_numbers = true };
|
||||
|
||||
/// For files ZFIN writes: the candle, quote, and options caches,
|
||||
|
|
|
|||
|
|
@ -1124,7 +1124,9 @@ fn buildHeaderSection(state: *State, app: *App, arena: std.mem.Allocator, lines:
|
|||
const stock_pct = pctx.stock_pct;
|
||||
|
||||
try lines.append(arena, .{ .text = "", .style = th.contentStyle() });
|
||||
try lines.append(arena, .{ .text = " Benchmark Comparison (price-only weighted return)", .style = th.headerStyle() });
|
||||
// See `commands/projections.zig` for why this says total return: the year
|
||||
// columns are dividend-reinvested, only Week is price-only.
|
||||
try lines.append(arena, .{ .text = " Benchmark Comparison (total return, weighted; Week is price-only)", .style = th.headerStyle() });
|
||||
try lines.append(arena, .{ .text = "", .style = th.contentStyle() });
|
||||
|
||||
// Column headers
|
||||
|
|
@ -1872,7 +1874,7 @@ fn buildLines(state: *State, app: *App, arena: std.mem.Allocator) ![]const Style
|
|||
|
||||
// Header
|
||||
try lines.append(arena, .{
|
||||
.text = " Benchmark Comparison (price-only weighted return)",
|
||||
.text = " Benchmark Comparison (total return, weighted; Week is price-only)",
|
||||
.style = th.headerStyle(),
|
||||
});
|
||||
try lines.append(arena, .{ .text = "", .style = th.contentStyle() });
|
||||
|
|
|
|||
|
|
@ -175,8 +175,22 @@ pub const Attribution = struct {
|
|||
/// `zfin contributions` reports as "money in").
|
||||
contributions: f64,
|
||||
/// `TotalsRow.delta - contributions`. The residual - what the
|
||||
/// market actually did.
|
||||
/// market actually did, PLUS anything the classifier declined to
|
||||
/// count. See `uncounted_in` / `uncounted_out`.
|
||||
gains: f64,
|
||||
/// Gross movement the contributions pipeline deliberately excluded, split by
|
||||
/// direction (`out` is negative). Present so the residual above can be read
|
||||
/// with its error bars visible: raw cash-balance changes and share reductions
|
||||
/// are excluded by design, and folding them into `gains` unannounced makes a
|
||||
/// classification gap look like market performance.
|
||||
///
|
||||
/// Both zero on a clean week, in which case the renderer omits the line.
|
||||
uncounted_in: f64 = 0,
|
||||
uncounted_out: f64 = 0,
|
||||
|
||||
pub fn hasUncounted(self: Attribution) bool {
|
||||
return @abs(self.uncounted_in) >= 0.005 or @abs(self.uncounted_out) >= 0.005;
|
||||
}
|
||||
};
|
||||
|
||||
/// Complete compare view. `symbols` is caller-owned; call `deinit()`.
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue