Compare commits

...

4 commits

Author SHA1 Message Date
8de28d73bc
agents.md update for kcov/pii/env vars
All checks were successful
Generic zig build / build (push) Successful in 6m35s
Generic zig build / publish-macos (push) Successful in 12s
Generic zig build / deploy (push) Successful in 21s
2026-07-25 11:09:07 -07:00
143395e7ce
remove comment reference to REPORT.md 2026-07-25 11:08:24 -07:00
1bea4d621a
delete dead code 2026-07-25 11:07:08 -07:00
dcd5885fcc
document in-kind transfer implementation 2026-07-25 11:04:23 -07:00
6 changed files with 142 additions and 99 deletions

View file

@ -372,10 +372,46 @@ local_pii_tokens=$(awk -F: '...' "$ZFIN_HOME/accounts.srf" ...)
grep -rn -E "$local_pii_tokens" src/ | grep -v ie_data.csv
```
The grep should always return zero non-`ie_data.csv` hits before
committing. The `ie_data.csv` exclusion is because the Shiller
dataset contains coincidental numeric matches in historical-year
fields that aren't PII.
The grep is a **starting point, not a pass/fail gate** - it produces
false positives that must be triaged rather than dismissed wholesale.
Two known classes:
- **Short numeric tokens.** A 3- or 4-digit account number matches
coincidentally inside Unix timestamps, trade volumes, hex color
literals, fake git SHAs, dollar amounts in CSV fixtures, and even
file line numbers. The `ie_data.csv` exclusion exists for the same
reason (Shiller's historical-year fields).
- **Generic account-type names.** An account literally named for an
IRS category (`Roth IRA`, `Inherited IRA`, `HSA`) matches inside the
approved placeholder vocabulary above - e.g. the real name
`Inherited IRA` is a substring of the sanctioned fixture value
`Sample Inherited IRA`.
So the rule is: **inspect the context of every hit**, and confirm each
is either coincidental or a generic category before dismissing it. A
hit whose surrounding context is a real-looking account name, a
composite identifier, or a value that could only have come from the
user's data is PII and must be fixed in the same change.
When reporting the result, redact: print the matching lines with the
sensitive token substituted out, or report only `file:line`. Do not
paste raw matches into a transcript.
**`ZFIN_HOME` does NOT isolate you for smoke tests.** Pointing
`ZFIN_HOME` at a scratch directory does not sandbox the whole tool:
`ZFIN_AUDIT_FILES` is read independently (`audit/hygiene.zig`) and
points at wherever real brokerage exports get downloaded. A flagless
`zfin audit` will discover, parse, and print **real account numbers and
balances** from that directory regardless of `ZFIN_HOME`. This has
already caused one leak into a session transcript.
So when smoke-testing against scratch data:
- Run under `env -u ZFIN_AUDIT_FILES` (and unset any other
path-carrying `ZFIN_*` var the command consumes).
- Bound the output. Never pipe a whole command's output through a broad
`grep`/`head` - `sed -n '/Section Header/,/^$/p'` the one section
under test, so an unexpected section can't spill into the transcript.
If you're uncertain whether something is PII, **ask before
committing.** PII can be surgically removed from a working
@ -592,6 +628,28 @@ Each file's report shows red lines (uncovered) and green lines
(covered). For a quick numeric breakdown by file, the kcov JSON
output under `coverage/kcov-merged/coverage.json` is greppable.
**Two traps if you script against the coverage output.** Both have
already produced a confidently-wrong "fully covered" claim:
1. **Per-line data lives in the `.js` files, not the `.html`.** Each
`coverage/<binary>/<file>.<hash>.html` has only a handful of
`class="lineNum"` nodes; the real per-line records are in the
sibling `.js` as
`{"lineNum":" 42","line":"...","class":"lineCov|lineNoCov|linePartCov"}`.
A scraper pointed at the HTML matches **zero** lines, so
"0 uncovered" silently means "0 tracked." Always assert that the
tracked-line count is non-zero before trusting a coverage verdict,
and sanity-check the scraper against a line you know is uncovered.
2. **`coverage/` accumulates stale per-binary directories across
runs.** Old reports for a file you just edited stay on disk with the
previous source text. Filter to reports whose embedded `"line"`
content matches the current file (grep for a string you just added)
before unioning them.
Note that per-file numbers are a union across binaries: the same source
file appears in several `coverage/<binary>/` directories, and a line
covered by any of them counts.
**Common reasons coverage looks lower than expected:**
- A new `.zig` file's tests aren't being discovered. Check the
@ -777,7 +835,7 @@ command.
- **Portfolio auto-detection.** Both CLI and TUI resolve `portfolio*.srf` the same way, and it is ZFIN_HOME-exclusive when set: if `$ZFIN_HOME` is set, only that directory is searched (cwd is NOT a fallback - a project directory that incidentally ships a `portfolio.srf` must not shadow the user's canonical data); if `$ZFIN_HOME` is unset, cwd is searched. An explicit `-p` path resolves the same exclusive way. `watchlist.srf` and the `.env` file follow the same rule (`.env` adds process environment variables as a higher-priority tier on top). `metadata.srf` and `accounts.srf` are loaded from the same directory as the resolved portfolio file.
- **`transaction_log.srf` is a sibling file.** Optional. Lives next to `portfolio.srf` / `accounts.srf`. Holds user-declared `transfer::` records so the contributions pipeline can tell internal account-to-account movement apart from real external contributions. Only `type::cash` is wired in v1 - `type::in_kind` parses but is rejected downstream. Missing file -> matcher is a no-op. See `REPORT.md` section 5 "Transfer log" for the user-facing guide.
- **`transaction_log.srf` is a sibling file.** Optional. Lives next to `portfolio.srf` / `accounts.srf`. Holds user-declared `transfer::` records so the contributions pipeline can tell internal account-to-account movement apart from real external contributions. Both `type::cash` (verified against the destination's dollar value, $1 tolerance) and `type::in_kind` (verified against share counts, 1%/0.01-share tolerance) are wired into the classifier. Missing file -> matcher is a no-op. See `docs/reference/config/transaction-log-srf.md` for the user-facing guide.
- **Server sync is optional.** The `ZFIN_SERVER` env var enables parallel cache syncing from a remote zfin-server instance. All server sync code silently no-ops when the URL is null.

View file

@ -89,8 +89,21 @@ transfer::2026-05-02,type::cash,amount:num:50000,from::Joint taxable,to::Pat Rot
```
zfin matches the transfer against the diff and removes it from the
attribution total. Only `type::cash` is wired up today; `in_kind`
parses but isn't yet supported.
attribution total.
If the **securities** moved rather than cash -- an ACAT transfer or an
in-kind rollover -- use `type::in_kind` and point `dest_lot` at the lot
that arrived:
```srf
transfer::2026-05-02,type::in_kind,amount:num:87000,from::Old 401k,to::Pat Roth,dest_lot::VTI@2024-01-15
```
In-kind records are verified by share count rather than by `amount`
(which is informational there), so the shares that left have to match
the shares that arrived. See the
[reference](../reference/config/transaction-log-srf.md#typecash-vs-typein_kind)
for the exact tolerances and what happens when they don't line up.
## Cash that *is* a contribution

View file

@ -2,8 +2,8 @@
`transaction_log.srf` is an optional sibling of `portfolio.srf` that
declares real-world transactions which change how zfin interprets the
portfolio diff. In v1 it holds exactly one kind of record:
**transfers** -- money you moved between accounts you own.
portfolio diff. It holds exactly one kind of record: **transfers** --
money or securities you moved between accounts you own.
## Why it exists
@ -42,21 +42,63 @@ transfer::2026-05-02,type::cash,amount:num:4700,from::Sample IRA,to::Sample Brok
| Field | Type | Required | Description |
|------------|--------|----------|---------------------------------------------------------------------|
| `transfer` | date | Yes | Transfer date (`YYYY-MM-DD`); the record key. |
| `type` | string | Yes | `cash` or `in_kind` (see v1 scope below). |
| `type` | string | Yes | `cash` (money moved) or `in_kind` (securities moved). See below. |
| `amount` | num | Yes | Dollar amount transferred to this destination. |
| `from` | string | Yes | Source account name (matches an `account::` in your portfolio). |
| `to` | string | Yes | Destination account name. |
| `dest_lot` | string | Yes | Where it landed: `cash`, or `SYMBOL@YYYY-MM-DD` for a specific lot. |
## v1 scope and limits
## `type::cash` vs `type::in_kind`
Both are matched by the contributions classifier; they differ in what
moved and therefore in how the record is verified.
**`type::cash`** -- dollars left one account and arrived at another,
possibly getting invested on arrival. The `amount` is load-bearing: it's
matched against the destination's value, and the three outcomes are
- **within $1** -- fully a transfer; contributes $0 to attribution.
- **destination worth more than `amount`** -- a *partial* transfer. Only
`amount` is cancelled out; the excess is still counted as new money.
That's the "I moved $5k in and also added $2k of my own" case.
- **`amount` exceeds the destination's value by more than $1** --
rejected, and reported under **Flagged for review**. You can't have
moved more into a lot than the lot is worth, so the record is
presumed wrong rather than trusted.
**`type::in_kind`** -- the securities themselves moved (an ACAT
transfer, an in-kind rollover); no cash changed hands. Requires
`dest_lot::SYMBOL@YYYY-MM-DD` -- `dest_lot::cash` is rejected, since
nothing can land as cash in a transfer where no cash moved.
```srf
#!srfv1
# 300 shares of VTI moved from one account to the other, no cash involved
transfer::2026-05-02,type::in_kind,amount:num:87000,from::Sample IRA,to::Sample Brokerage,dest_lot::VTI@2024-01-15
```
Here `amount` is informational only. The moved value comes from the
destination lot itself, and verification is by **share count**: the
shares that left `from` must match the shares that arrived at `to`,
within 1% or 0.01 shares (whichever is larger). A mismatch is reported
under **Flagged for review** rather than silently absorbing what might
be a real contribution.
A missing source side is not an error for either type -- the sending
account may be untracked (an external rollover origin, a 401k you don't
model). Only the destination is required.
Because no cash funded an in-kind move, there is no partial outcome: a
destination is either fully a transfer or not one at all.
## Scope and limits
- Only `transfer::` records. Buys, sells, and dividends stay inferred
from the portfolio diff.
- Only `type::cash` is wired into the contributions classifier.
`type::in_kind` parses (the format is forward-compatible) but the
matcher rejects it with an "in-kind transfers not yet supported"
message.
- Forward-looking only -- there is no historical reconstruction.
- Account names are matched byte-exactly, so a
[renamed account](../../guides/set-up-accounts.md#renaming-an-account)
breaks records that reference the old name.
## See also

View file

@ -254,18 +254,6 @@ pub const AccountMap = struct {
return null;
}
/// Return all entries matching a given institution.
pub fn entriesForInstitution(self: AccountMap, institution: []const u8) []const AccountTaxEntry {
var count: usize = 0;
for (self.entries) |e| {
if (e.institution) |inst| {
if (std.mem.eql(u8, inst, institution)) count += 1;
}
}
if (count == 0) return &.{};
return self.entries;
}
/// Is cash-balance movement on `account` treated as a real
/// contribution (vs. internal noise) for the attribution total?
/// Defaults to false when the account isn't in the map.

View file

@ -1633,9 +1633,11 @@ pub fn renderKeyComparisonRows(
now: KeyMetrics,
events_enabled: bool,
) !void {
// `then` and `now` are computed against the same projections.srf
// (REPORT.md §4 - the "then" side reuses today's config), so
// their horizons agree. Use whichever side is convenient.
// `then` and `now` are computed against the same projections.srf:
// the historical side re-runs today's config against the older
// portfolio snapshot rather than trying to recover the config as it
// was then (which isn't recorded anywhere). So both sides share a
// horizon, and either one can supply the label.
const events_label: []const u8 = if (events_enabled) "included" else "excluded";
try cli.printFg(out, color, cli.CLR_MUTED, " ({d}-year horizon, lifecycle events {s})\n", .{ now.horizon_years, events_label });

View file

@ -53,8 +53,9 @@
//! destination share-addition, per-symbol.
//! - No historical reconstruction - forward-looking only.
//!
//! See `REPORT.md` §5 for the full usage guide and
//! `src/commands/contributions.zig` for the classifier integration.
//! See `docs/reference/config/transaction-log-srf.md` for the full
//! usage guide and `src/commands/contributions.zig` for the classifier
//! integration.
const std = @import("std");
const builtin = @import("builtin");
@ -201,31 +202,6 @@ pub const TransactionLog = struct {
}
self.allocator.free(self.transfers);
}
/// Return transfers whose `transfer` date falls within `[start, end]`
/// inclusive. The returned slice is allocator-owned - caller must
/// free it.
///
/// Works only because `parseTransactionLogFile` preserves file
/// order. If callers ever need chronological ordering regardless
/// of file layout, sort on the way out instead of on the way in -
/// file order is sometimes meaningful for a human reviewer
/// (grouping related records together).
pub fn transfersInWindow(
self: *const TransactionLog,
allocator: std.mem.Allocator,
start: Date,
end: Date,
) ![]const TransferRecord {
var out: std.ArrayList(TransferRecord) = .empty;
errdefer out.deinit(allocator);
for (self.transfers) |r| {
if (r.transfer.days < start.days) continue;
if (r.transfer.days > end.days) continue;
try out.append(allocator, r);
}
return try out.toOwnedSlice(allocator);
}
};
/// Parse `data` (the contents of a `transaction_log.srf` file) into a
@ -238,6 +214,11 @@ pub const TransactionLog = struct {
/// only hard errors are allocator failures and SRF-level parse errors
/// that prevent the iterator from starting at all.
///
/// Records come out in **file order**, deliberately not sorted by date.
/// File order is often meaningful to a human reviewer - related records
/// get grouped together - and any consumer that needs chronological
/// ordering can sort on the way out. Pinned by a test below.
///
/// The SRF record layout is `transfer::<date>,type::<t>,amount:num:<n>,
/// from::<a>,to::<b>,dest_lot::<dl>[,note::<n>]`. SRF's
/// `fields.to(TransferRecord)` does the coercion: each key matches a
@ -677,47 +658,6 @@ test "parseTransactionLogFile: malformed record skipped, subsequent record survi
try testing.expectEqual(@as(f64, 3000), log.transfers[0].amount);
}
test "transfersInWindow: inclusive on both ends" {
var log = try parseTransactionLogFile(testing.allocator,
\\#!srfv1
\\transfer::2026-04-30,type::cash,amount:num:100,from::Acct A,to::Acct B,dest_lot::cash
\\transfer::2026-05-01,type::cash,amount:num:200,from::Acct A,to::Acct B,dest_lot::cash
\\transfer::2026-05-15,type::cash,amount:num:300,from::Acct A,to::Acct B,dest_lot::cash
\\transfer::2026-05-31,type::cash,amount:num:400,from::Acct A,to::Acct B,dest_lot::cash
\\transfer::2026-06-01,type::cash,amount:num:500,from::Acct A,to::Acct B,dest_lot::cash
\\
);
defer log.deinit();
const slice = try log.transfersInWindow(
testing.allocator,
Date.fromYmd(2026, 5, 1),
Date.fromYmd(2026, 5, 31),
);
defer testing.allocator.free(slice);
try testing.expectEqual(@as(usize, 3), slice.len);
try testing.expectEqual(@as(f64, 200), slice[0].amount);
try testing.expectEqual(@as(f64, 300), slice[1].amount);
try testing.expectEqual(@as(f64, 400), slice[2].amount);
}
test "transfersInWindow: empty window returns empty slice" {
var log = try parseTransactionLogFile(testing.allocator,
\\#!srfv1
\\transfer::2026-05-01,type::cash,amount:num:100,from::Acct A,to::Acct B,dest_lot::cash
\\
);
defer log.deinit();
const slice = try log.transfersInWindow(
testing.allocator,
Date.fromYmd(2027, 1, 1),
Date.fromYmd(2027, 12, 31),
);
defer testing.allocator.free(slice);
try testing.expectEqual(@as(usize, 0), slice.len);
}
test "parseTransactionLogFile: preserves file order (not sorted)" {
var log = try parseTransactionLogFile(testing.allocator,
\\#!srfv1
@ -728,7 +668,7 @@ test "parseTransactionLogFile: preserves file order (not sorted)" {
);
defer log.deinit();
try testing.expectEqual(@as(usize, 3), log.transfers.len);
// File order preserved - see `transfersInWindow` docstring for why.
// File order preserved, NOT date-sorted - see the parser's docstring.
try testing.expectEqual(@as(f64, 3), log.transfers[0].amount);
try testing.expectEqual(@as(f64, 1), log.transfers[1].amount);
try testing.expectEqual(@as(f64, 2), log.transfers[2].amount);