initial implementation of import command
This commit is contained in:
parent
09d26d1767
commit
21279c1aeb
3 changed files with 923 additions and 0 deletions
60
TODO.md
60
TODO.md
|
|
@ -5,6 +5,66 @@ ordered roughly by priority within each section. Priority labels
|
|||
(`HIGH` / `MEDIUM` / `LOW`) mark items that deserve explicit
|
||||
ranking; unlabeled items are "someday, if the mood strikes."
|
||||
|
||||
## `zfin import`: preserve prior `open_date` on re-import — priority MEDIUM
|
||||
|
||||
`zfin import` (PR 3) currently writes every synthesized lot with
|
||||
`open_date::1970-01-01` — an honest "we don't know" sentinel.
|
||||
Brokerage holdings CSVs don't carry per-buy dates, so a single
|
||||
import has no real signal to seed `open_date` from. But once a
|
||||
portfolio file exists, the *prior* import's lots ARE a signal: a
|
||||
position that was in the file last week with `open_date::2026-04-15`
|
||||
should keep that date on the next import, not get rewritten.
|
||||
|
||||
Without this, every re-import zeroes out trailing-return and
|
||||
ST/LT classifications for positions that have actually been held
|
||||
since the previous run. Sentinel epoch dodges the "today resets
|
||||
the clock" version of this bug, but it doesn't fix it — it just
|
||||
makes the broken state inert.
|
||||
|
||||
### Sketch
|
||||
|
||||
When `import` runs and the target file already exists:
|
||||
|
||||
1. Read and parse the existing portfolio file (cleanly, via
|
||||
`cache.deserializePortfolio`).
|
||||
2. Build a lookup: `(symbol, account) → existing Lot`.
|
||||
3. For each synthesized lot, look up by `(symbol, account)`:
|
||||
- **Match:** preserve `existing.open_date` and (probably)
|
||||
`existing.open_price`. The synthesized cost-basis-per-share
|
||||
is "current avg" not "what I paid for THIS lot," so keeping
|
||||
the existing open_price is more honest if the prior import
|
||||
captured it.
|
||||
- **No match:** new position. Use today as the open_date
|
||||
(we genuinely don't know but today is the best honest guess
|
||||
for "first time we saw it").
|
||||
4. Closed-lot detection: for each existing lot in the file with
|
||||
no matching brokerage row, decide what to do — append a
|
||||
`close_date::<today>,close_price::<last_known>` pair? Drop
|
||||
it silently? This is the trickier half; punt to a sub-task
|
||||
if needed.
|
||||
|
||||
### Match key
|
||||
|
||||
`(symbol, account)` pair. Discussed during PR 3:
|
||||
- Including `security_type` is overkill (we don't have stock and
|
||||
cash AAPL in the same account).
|
||||
- Symbol-only matching across accounts hides real errors (a
|
||||
position moving accounts is a transfer, not a no-op).
|
||||
- If the existing file has multiple lots for the same (symbol,
|
||||
account), preserve the EARLIEST `open_date` — that's the
|
||||
longest-standing buy, the right anchor for trailing-return math.
|
||||
|
||||
### Driver
|
||||
|
||||
Lossless re-import. Today's epoch sentinel is correct-but-
|
||||
inert; this PR makes import actually useful for tracking
|
||||
positions over time without forcing the user to hand-edit
|
||||
dates after every refresh.
|
||||
|
||||
This also gets us most of the way to "transactions import"
|
||||
without needing a separate transactions CSV — closed lots fall
|
||||
out of the symbol-set diff between two consecutive imports.
|
||||
|
||||
## Projections: future enhancements
|
||||
|
||||
- **Configurable return cap per position — priority MEDIUM.**
|
||||
|
|
|
|||
862
src/commands/import.zig
Normal file
862
src/commands/import.zig
Normal file
|
|
@ -0,0 +1,862 @@
|
|||
//! `zfin import` — synthesize a portfolio file from a brokerage
|
||||
//! holdings export.
|
||||
//!
|
||||
//! The first mutating CLI command in zfin. The use case is a managed
|
||||
//! account whose lot-level history isn't worth maintaining by hand —
|
||||
//! direct-indexing accounts that don't track an underlying ETF, my
|
||||
//! mother's brokerage, etc. Each run replaces the target file's
|
||||
//! contents with one synthetic lot per (account, symbol) drawn from
|
||||
//! the brokerage's positions export. We trade lot-level fidelity for
|
||||
//! "look at the brokerage CSV; rerun import; commit" simplicity.
|
||||
//!
|
||||
//! ## Synthetic lots
|
||||
//!
|
||||
//! Brokerage holdings exports give us "100 AAPL @ $150 avg cost" —
|
||||
//! aggregate, no buy date. We synthesize one `Lot` per row:
|
||||
//!
|
||||
//! - `symbol::` from the export
|
||||
//! - `shares::` from the export's quantity
|
||||
//! - `open_date::` = `1970-01-01` (sentinel, see below)
|
||||
//! - `open_price::` = `cost_basis / quantity` if both > 0, else
|
||||
//! `current_value / quantity`, else 0
|
||||
//! - `account::` resolved via `accounts.srf` from the export's
|
||||
//! account_number; refuse to import if any number is unmapped
|
||||
//! - `note::` is deliberately unset for now. The natural value
|
||||
//! would be `"imported <broker> <date>"`, but that changes on
|
||||
//! every re-import and would dirty `git diff` for held
|
||||
//! positions that didn't actually change. The merge-aware
|
||||
//! follow-up PR will reintroduce a refresh-date note at the
|
||||
//! same time it preserves prior `open_date` values across
|
||||
//! re-imports.
|
||||
//! - `security_type::cash` for cash-classified positions
|
||||
//!
|
||||
//! ### Why the epoch sentinel for `open_date`?
|
||||
//!
|
||||
//! Brokerage holdings CSVs don't carry per-lot buy dates, so any
|
||||
//! synthesized `open_date` is a guess. Earlier versions used
|
||||
//! `today`, which is actively misleading: every re-import resets
|
||||
//! the clock, blowing away trailing-return and ST/LT
|
||||
//! classifications for positions that have actually been held for
|
||||
//! years. Using `1970-01-01` is honest — "we don't know" — and
|
||||
//! makes a `git diff` between two imports trivial: the only thing
|
||||
//! that changes for a held position is `shares` and possibly
|
||||
//! `open_price`, never the date column.
|
||||
//!
|
||||
//! Preserving the prior `open_date` from an existing portfolio
|
||||
//! file (so the first import seeds it and subsequent imports
|
||||
//! leave it alone) is a tracked follow-up; it requires reading
|
||||
//! the existing file at import time, which the current
|
||||
//! "blow-away-and-rewrite" model avoids.
|
||||
//!
|
||||
//! This loses per-buy lot history, which is acceptable for managed
|
||||
//! accounts where we don't track buys/sells anyway. Git serves as
|
||||
//! the history record for the file itself: prior states are visible
|
||||
//! in `git log` and recoverable with `git show`.
|
||||
//!
|
||||
//! ## Workflow
|
||||
//!
|
||||
//! ```
|
||||
//! # 1. Download Fidelity positions CSV from the website.
|
||||
//! # 2. Run import; review the diff in git.
|
||||
//! zfin -p portfolio_mom.srf import --fidelity ~/Downloads/positions.csv
|
||||
//! # 3. Inspect changes; commit when satisfied.
|
||||
//! git diff portfolio_mom.srf
|
||||
//! git add portfolio_mom.srf && git commit -m "Update mom's portfolio"
|
||||
//! ```
|
||||
//!
|
||||
//! ## Safety
|
||||
//!
|
||||
//! - `-p`/`--portfolio` is REQUIRED — we never guess which file to
|
||||
//! overwrite. The pattern must resolve to a single concrete path
|
||||
//! (no globs, no multi-match).
|
||||
//! - If the target file exists, prompt on stderr: `Overwrite <path>?
|
||||
//! (y/N) `. Default no. Pass `-y` / `--yes` to skip the prompt
|
||||
//! (apt-style).
|
||||
//! - Atomic write via `atomic.writeFileAtomic` — a kill mid-write
|
||||
//! leaves the prior file intact.
|
||||
//! - No backup file. Git is the backup. If the file isn't tracked
|
||||
//! by git, the user will see that in `git status` after the run.
|
||||
//! - Hard fail (no fallback) when an export references an
|
||||
//! account_number not present in `accounts.srf`. The user adds
|
||||
//! the mapping and reruns; better that than silently writing
|
||||
//! lots with broken account names.
|
||||
|
||||
const std = @import("std");
|
||||
const zfin = @import("../root.zig");
|
||||
const cli = @import("common.zig");
|
||||
const framework = @import("framework.zig");
|
||||
const Date = @import("../Date.zig");
|
||||
const portfolio_mod = @import("../models/portfolio.zig");
|
||||
const cache = @import("../cache/store.zig");
|
||||
const atomic = @import("../atomic.zig");
|
||||
const fidelity = @import("../brokerage/fidelity.zig");
|
||||
const schwab = @import("../brokerage/schwab.zig");
|
||||
const brokerage_types = @import("../brokerage/types.zig");
|
||||
const analysis = @import("../analytics/analysis.zig");
|
||||
|
||||
const BrokeragePosition = brokerage_types.BrokeragePosition;
|
||||
|
||||
/// Source brokerage for the import. Each variant carries the path
|
||||
/// to the export file. We accept exactly one source per run; the
|
||||
/// parser rejects mixed flags.
|
||||
pub const Source = union(enum) {
|
||||
fidelity: []const u8,
|
||||
schwab: []const u8,
|
||||
|
||||
pub fn label(self: Source) []const u8 {
|
||||
return switch (self) {
|
||||
.fidelity => "fidelity",
|
||||
.schwab => "schwab",
|
||||
};
|
||||
}
|
||||
|
||||
pub fn path(self: Source) []const u8 {
|
||||
return switch (self) {
|
||||
.fidelity => |p| p,
|
||||
.schwab => |p| p,
|
||||
};
|
||||
}
|
||||
};
|
||||
|
||||
pub const ParsedArgs = struct {
|
||||
source: Source,
|
||||
/// Skip the overwrite prompt (apt-style `-y` / `--yes`).
|
||||
yes: bool = false,
|
||||
};
|
||||
|
||||
pub const meta: framework.Meta = .{
|
||||
.name = "import",
|
||||
.group = .hygiene,
|
||||
.synopsis = "Synthesize a portfolio file from a brokerage holdings export",
|
||||
.help =
|
||||
\\Usage: zfin -p <PORTFOLIO> import (--fidelity FILE | --schwab FILE) [-y]
|
||||
\\
|
||||
\\Synthesize a portfolio file from a brokerage positions export.
|
||||
\\Each run REPLACES the target portfolio file with synthetic lots
|
||||
\\drawn from the export — one lot per (account, symbol) at
|
||||
\\open_date=1970-01-01 (sentinel; brokerage CSVs don't carry
|
||||
\\per-buy dates) and open_price=cost-basis-average.
|
||||
\\
|
||||
\\Designed for managed accounts (direct-indexing baskets, accounts
|
||||
\\you don't track at lot granularity). Per-buy history is lost;
|
||||
\\git serves as the file-level history. Re-running import does
|
||||
\\NOT preserve a prior `open_date` (yet — see TODO.md for the
|
||||
\\merge-aware follow-up); the sentinel makes that re-write
|
||||
\\inert rather than misleading.
|
||||
\\
|
||||
\\Required:
|
||||
\\ -p, --portfolio <FILE> Target portfolio file (must be a single
|
||||
\\ concrete path, not a glob). REQUIRED.
|
||||
\\ --fidelity <CSV> Fidelity positions CSV
|
||||
\\ ("All accounts" → Positions tab → Download)
|
||||
\\ --schwab <CSV> Schwab per-account positions CSV
|
||||
\\
|
||||
\\Options:
|
||||
\\ -y, --yes Don't prompt before overwriting an
|
||||
\\ existing file.
|
||||
\\
|
||||
\\Account-name resolution requires `accounts.srf` next to the
|
||||
\\target file with `institution::` + `account_number::` entries
|
||||
\\matching the brokerage export. Import refuses to write when an
|
||||
\\account_number from the export is unmapped.
|
||||
\\
|
||||
,
|
||||
.uppercase_first_arg = false,
|
||||
.user_errors = error{
|
||||
UnexpectedArg,
|
||||
MissingSource,
|
||||
ConflictingSources,
|
||||
MissingPortfolioPath,
|
||||
AmbiguousPortfolioPath,
|
||||
EmptyFile,
|
||||
UnexpectedHeader,
|
||||
CannotReadCsv,
|
||||
CannotReadAccountsFile,
|
||||
UnmappedAccount,
|
||||
UserDeclined,
|
||||
WriteFailed,
|
||||
},
|
||||
};
|
||||
|
||||
pub fn parseArgs(ctx: *framework.RunCtx, cmd_args: []const []const u8) !ParsedArgs {
|
||||
var fidelity_path: ?[]const u8 = null;
|
||||
var schwab_path: ?[]const u8 = null;
|
||||
var yes = false;
|
||||
|
||||
var i: usize = 0;
|
||||
while (i < cmd_args.len) : (i += 1) {
|
||||
const a = cmd_args[i];
|
||||
if (std.mem.eql(u8, a, "--fidelity")) {
|
||||
if (i + 1 >= cmd_args.len) {
|
||||
try cli.stderrPrint(ctx.io, "Error: --fidelity requires a CSV path\n");
|
||||
return error.UnexpectedArg;
|
||||
}
|
||||
i += 1;
|
||||
fidelity_path = cmd_args[i];
|
||||
} else if (std.mem.eql(u8, a, "--schwab")) {
|
||||
if (i + 1 >= cmd_args.len) {
|
||||
try cli.stderrPrint(ctx.io, "Error: --schwab requires a CSV path\n");
|
||||
return error.UnexpectedArg;
|
||||
}
|
||||
i += 1;
|
||||
schwab_path = cmd_args[i];
|
||||
} else if (std.mem.eql(u8, a, "-y") or std.mem.eql(u8, a, "--yes")) {
|
||||
yes = true;
|
||||
} else {
|
||||
try cli.stderrPrint(ctx.io, "Error: unexpected argument to 'import': ");
|
||||
try cli.stderrPrint(ctx.io, a);
|
||||
try cli.stderrPrint(ctx.io, "\n");
|
||||
return error.UnexpectedArg;
|
||||
}
|
||||
}
|
||||
|
||||
if (fidelity_path != null and schwab_path != null) {
|
||||
try cli.stderrPrint(ctx.io, "Error: --fidelity and --schwab are mutually exclusive (one source per import)\n");
|
||||
return error.ConflictingSources;
|
||||
}
|
||||
|
||||
const source: Source = if (fidelity_path) |p|
|
||||
.{ .fidelity = p }
|
||||
else if (schwab_path) |p|
|
||||
.{ .schwab = p }
|
||||
else {
|
||||
try cli.stderrPrint(ctx.io, "Error: import requires a source flag (--fidelity FILE or --schwab FILE)\n");
|
||||
return error.MissingSource;
|
||||
};
|
||||
|
||||
return .{ .source = source, .yes = yes };
|
||||
}
|
||||
|
||||
pub fn run(ctx: *framework.RunCtx, parsed: ParsedArgs) !void {
|
||||
const svc = ctx.svc orelse return error.MissingDataService;
|
||||
const io = ctx.io;
|
||||
const allocator = ctx.allocator;
|
||||
const out = ctx.out;
|
||||
|
||||
// ── Resolve the target portfolio path ─────────────────────
|
||||
//
|
||||
// -p is REQUIRED for import. We never guess which file to
|
||||
// overwrite. We also reject globs and multi-match patterns
|
||||
// here — the user must point us at exactly one file. If they
|
||||
// genuinely mean to import for a portfolio that lives at
|
||||
// multiple paths, they need to pick one explicitly.
|
||||
const target_path = try resolveSingleTarget(ctx);
|
||||
|
||||
// ── Read & parse the brokerage export ─────────────────────
|
||||
const csv_path = parsed.source.path();
|
||||
const csv_data = std.Io.Dir.cwd().readFileAlloc(io, csv_path, allocator, .limited(10 * 1024 * 1024)) catch {
|
||||
var msg_buf: [512]u8 = undefined;
|
||||
const msg = std.fmt.bufPrint(&msg_buf, "Error: Cannot read CSV file: {s}\n", .{csv_path}) catch "Error: Cannot read CSV file\n";
|
||||
try cli.stderrPrint(io, msg);
|
||||
return error.CannotReadCsv;
|
||||
};
|
||||
defer allocator.free(csv_data);
|
||||
|
||||
const positions: []const BrokeragePosition = switch (parsed.source) {
|
||||
.fidelity => try fidelity.parseCsv(allocator, csv_data),
|
||||
.schwab => blk: {
|
||||
const r = try schwab.parseCsv(allocator, csv_data);
|
||||
break :blk r.positions;
|
||||
},
|
||||
};
|
||||
defer allocator.free(positions);
|
||||
|
||||
if (positions.len == 0) {
|
||||
try cli.stderrPrint(io, "Error: brokerage export contained zero positions; refusing to write an empty portfolio.\n");
|
||||
return error.EmptyFile;
|
||||
}
|
||||
|
||||
// ── Load accounts.srf for account-number → name mapping ───
|
||||
//
|
||||
// Sibling file derivation: `DataService.loadAccountMap` walks
|
||||
// up from the portfolio path to find `accounts.srf`, the same
|
||||
// way every other zfin command does. We don't need the
|
||||
// service for anything else (no price fetching), but reusing
|
||||
// its helper keeps sibling-file resolution consistent.
|
||||
var account_map = svc.loadAccountMap(target_path) orelse {
|
||||
try cli.stderrPrint(io, "Error: Cannot read/parse accounts.srf next to the target portfolio.\n");
|
||||
try cli.stderrPrint(io, " Import needs `institution::` + `account_number::` entries to map\n");
|
||||
try cli.stderrPrint(io, " brokerage account numbers to portfolio account names.\n");
|
||||
return error.CannotReadAccountsFile;
|
||||
};
|
||||
defer account_map.deinit();
|
||||
|
||||
// ── Synthesize lots ───────────────────────────────────────
|
||||
const lots = synthesizeLots(io, allocator, positions, account_map, parsed.source, ctx.today) catch |err| switch (err) {
|
||||
error.UnmappedAccount => {
|
||||
// synthesizeLots already printed the offending account
|
||||
// numbers to stderr; just propagate as a user-level error.
|
||||
return err;
|
||||
},
|
||||
else => |e| return e,
|
||||
};
|
||||
defer freeLots(allocator, lots);
|
||||
|
||||
// ── Confirm overwrite when target exists ──────────────────
|
||||
//
|
||||
// Inline access() check: a single call site doesn't earn its
|
||||
// own helper, and the catch-form here matches the pattern in
|
||||
// snapshot.zig / history.zig.
|
||||
const target_exists = blk: {
|
||||
std.Io.Dir.cwd().access(io, target_path, .{}) catch break :blk false;
|
||||
break :blk true;
|
||||
};
|
||||
if (target_exists and !parsed.yes) {
|
||||
if (!try confirmOverwrite(io, target_path)) {
|
||||
try cli.stderrPrint(io, "Aborted; no changes written.\n");
|
||||
return error.UserDeclined;
|
||||
}
|
||||
}
|
||||
|
||||
// ── Serialize + atomic write ──────────────────────────────
|
||||
const serialized = try cache.serializePortfolio(allocator, lots);
|
||||
defer allocator.free(serialized);
|
||||
|
||||
atomic.writeFileAtomic(io, allocator, target_path, serialized) catch |err| {
|
||||
var msg_buf: [512]u8 = undefined;
|
||||
const msg = std.fmt.bufPrint(&msg_buf, "Error: Failed to write portfolio file ({s}): {s}\n", .{ target_path, @errorName(err) }) catch "Error: Failed to write portfolio file\n";
|
||||
try cli.stderrPrint(io, msg);
|
||||
return error.WriteFailed;
|
||||
};
|
||||
|
||||
// ── Summary ───────────────────────────────────────────────
|
||||
var account_count: usize = 0;
|
||||
{
|
||||
// Count distinct accounts referenced by the synthesized
|
||||
// lots. Cheap because lots[].account is already mapped to
|
||||
// portfolio account names; we just dedupe.
|
||||
var seen = std.StringHashMap(void).init(allocator);
|
||||
defer seen.deinit();
|
||||
for (lots) |lot| {
|
||||
if (lot.account) |a| {
|
||||
if (!seen.contains(a)) {
|
||||
seen.put(a, {}) catch {};
|
||||
account_count += 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
try out.print(
|
||||
"Wrote {d} lot{s} across {d} account{s} to {s} (source: {s}).\n",
|
||||
.{
|
||||
lots.len,
|
||||
if (lots.len == 1) "" else "s",
|
||||
account_count,
|
||||
if (account_count == 1) "" else "s",
|
||||
target_path,
|
||||
parsed.source.label(),
|
||||
},
|
||||
);
|
||||
try out.flush();
|
||||
}
|
||||
|
||||
// ── Helpers ──────────────────────────────────────────────────
|
||||
|
||||
/// Resolve `-p`/`--portfolio` to exactly one concrete path. Refuses
|
||||
/// glob patterns and refuses multi-match cases — import is a
|
||||
/// destructive operation, "we'll write to the first match" is not
|
||||
/// an answer.
|
||||
fn resolveSingleTarget(ctx: *framework.RunCtx) ![]const u8 {
|
||||
const patterns = ctx.globals.portfolio_patterns;
|
||||
if (patterns.len == 0) {
|
||||
try cli.stderrPrint(ctx.io, "Error: import requires `-p <FILE>` (the portfolio file to write).\n");
|
||||
return error.MissingPortfolioPath;
|
||||
}
|
||||
if (patterns.len > 1) {
|
||||
try cli.stderrPrint(ctx.io, "Error: import requires exactly one `-p <FILE>` (got multiple).\n");
|
||||
return error.AmbiguousPortfolioPath;
|
||||
}
|
||||
const pat = patterns[0];
|
||||
if (zfin.Config.isGlobPattern(pat)) {
|
||||
try cli.stderrPrint(ctx.io, "Error: import refuses glob patterns for `-p`. Pass an exact filename.\n");
|
||||
return error.AmbiguousPortfolioPath;
|
||||
}
|
||||
// Resolve through cwd → ZFIN_HOME so bare names work the same
|
||||
// as elsewhere in zfin. If the file doesn't exist yet (first
|
||||
// run for a new portfolio), fall back to the literal pattern
|
||||
// so we write to ./<pattern>.
|
||||
if (ctx.config.resolveUserFile(ctx.io, ctx.allocator, pat)) |r| {
|
||||
// Caller doesn't free this; returning the resolved path
|
||||
// string puts the lifetime on the arena allocator.
|
||||
return r.path;
|
||||
}
|
||||
return pat;
|
||||
}
|
||||
|
||||
/// Prompt for y/N confirmation on stderr, read one line from stdin.
|
||||
/// Returns true only on a 'y' or 'Y' answer (with optional whitespace);
|
||||
/// anything else (including EOF / empty) is "no".
|
||||
fn confirmOverwrite(io: std.Io, path: []const u8) !bool {
|
||||
var stderr_buf: [512]u8 = undefined;
|
||||
var stderr_writer = std.Io.File.stderr().writer(io, &stderr_buf);
|
||||
try stderr_writer.interface.print("Overwrite {s}? (y/N) ", .{path});
|
||||
try stderr_writer.interface.flush();
|
||||
|
||||
var stdin_buf: [256]u8 = undefined;
|
||||
var stdin_reader = std.Io.File.stdin().reader(io, &stdin_buf);
|
||||
// Take up to one line; treat EOF as "no".
|
||||
const line = stdin_reader.interface.takeDelimiterExclusive('\n') catch |err| switch (err) {
|
||||
error.EndOfStream => return false,
|
||||
else => return err,
|
||||
};
|
||||
const trimmed = std.mem.trim(u8, line, &std.ascii.whitespace);
|
||||
return trimmed.len > 0 and (trimmed[0] == 'y' or trimmed[0] == 'Y');
|
||||
}
|
||||
|
||||
/// Synthesize a `Lot` per `BrokeragePosition`. Resolves each
|
||||
/// brokerage account_number to a portfolio account name via
|
||||
/// `account_map`; refuses (with a stderr listing of unmapped
|
||||
/// numbers) if any can't be resolved.
|
||||
///
|
||||
/// `today` is currently unused at the lot level (the would-be
|
||||
/// `note::` stamp is suppressed — see the in-body comment for
|
||||
/// rationale). The parameter stays in the signature so the
|
||||
/// merge-aware follow-up PR doesn't have to reshape the call
|
||||
/// site again when the note comes back.
|
||||
///
|
||||
/// Takes `io` so it can print the unmapped-account-number
|
||||
/// enumeration directly to stderr — easier than threading the list
|
||||
/// back to the caller, and keeps the test path simple (tests pass
|
||||
/// `std.testing.io` and observe the error code).
|
||||
///
|
||||
/// Returned lots own their string fields against `allocator` so
|
||||
/// `cache.serializePortfolio` and `freeLots` can be applied
|
||||
/// uniformly. Caller must call `freeLots`.
|
||||
fn synthesizeLots(
|
||||
io: std.Io,
|
||||
allocator: std.mem.Allocator,
|
||||
positions: []const BrokeragePosition,
|
||||
account_map: analysis.AccountMap,
|
||||
source: Source,
|
||||
today: Date,
|
||||
) ![]portfolio_mod.Lot {
|
||||
const institution = source.label();
|
||||
|
||||
// First pass: collect any unmapped account numbers so we can
|
||||
// report all of them at once instead of failing on the first
|
||||
// and making the user re-run for each.
|
||||
var unmapped: std.ArrayList([]const u8) = .empty;
|
||||
defer unmapped.deinit(allocator);
|
||||
{
|
||||
var seen = std.StringHashMap(void).init(allocator);
|
||||
defer seen.deinit();
|
||||
for (positions) |pos| {
|
||||
if (account_map.findByInstitutionAccount(institution, pos.account_number) == null) {
|
||||
if (!seen.contains(pos.account_number)) {
|
||||
seen.put(pos.account_number, {}) catch {};
|
||||
try unmapped.append(allocator, pos.account_number);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
if (unmapped.items.len > 0) {
|
||||
var stderr_buf: [4096]u8 = undefined;
|
||||
var stderr_writer = std.Io.File.stderr().writer(io, &stderr_buf);
|
||||
try stderr_writer.interface.print(
|
||||
"Error: {d} account number{s} from the {s} export {s} not mapped in accounts.srf:\n",
|
||||
.{
|
||||
unmapped.items.len,
|
||||
if (unmapped.items.len == 1) "" else "s",
|
||||
institution,
|
||||
if (unmapped.items.len == 1) "is" else "are",
|
||||
},
|
||||
);
|
||||
for (unmapped.items) |num| {
|
||||
try stderr_writer.interface.print(" - {s}\n", .{num});
|
||||
}
|
||||
try stderr_writer.interface.print(
|
||||
"\nAdd entries to accounts.srf with `institution::{s}` and the matching\n" ++
|
||||
"`account_number::` value, then rerun the import.\n",
|
||||
.{institution},
|
||||
);
|
||||
try stderr_writer.interface.flush();
|
||||
return error.UnmappedAccount;
|
||||
}
|
||||
|
||||
var lots = std.ArrayList(portfolio_mod.Lot).empty;
|
||||
errdefer {
|
||||
for (lots.items) |lot| freeLot(allocator, lot);
|
||||
lots.deinit(allocator);
|
||||
}
|
||||
|
||||
// The synthetic note (`"imported <broker> YYYY-MM-DD"`) is
|
||||
// intentionally omitted for now. It changes on every re-import
|
||||
// (because the date moves), which means held positions show
|
||||
// up as modified in `git diff` even when nothing actually
|
||||
// changed at the brokerage. The merge-aware follow-up PR will
|
||||
// bring the note back together with prior-open_date
|
||||
// preservation, at which point the note can stamp "last
|
||||
// refreshed" without polluting the diff for unchanged
|
||||
// positions. See TODO.md "preserve prior open_date on re-import".
|
||||
_ = today; // keep the parameter wired for the follow-up PR
|
||||
|
||||
for (positions) |pos| {
|
||||
const acct_name = account_map.findByInstitutionAccount(institution, pos.account_number).?;
|
||||
|
||||
const shares = pos.quantity orelse blk: {
|
||||
// Cash positions have null quantity; the convention
|
||||
// elsewhere in zfin (audit, snapshot's cash row) is
|
||||
// shares=current_value, open_price=$1. Mirror that.
|
||||
if (pos.is_cash) break :blk pos.current_value orelse 0;
|
||||
// Non-cash without quantity is malformed export data.
|
||||
// Treat as 0 shares so the lot is harmless; the user
|
||||
// will see it in `git diff` and either fix the export
|
||||
// or hand-edit the lot.
|
||||
break :blk 0;
|
||||
};
|
||||
|
||||
const open_price: f64 = if (pos.is_cash)
|
||||
1.0
|
||||
else if (pos.cost_basis) |cb|
|
||||
if (shares > 0) cb / shares else 0
|
||||
else if (pos.current_value) |cv|
|
||||
if (shares > 0) cv / shares else 0
|
||||
else
|
||||
0;
|
||||
|
||||
const security_type: portfolio_mod.LotType = if (pos.is_cash) .cash else .stock;
|
||||
|
||||
try lots.append(allocator, .{
|
||||
.symbol = try allocator.dupe(u8, pos.symbol),
|
||||
.shares = shares,
|
||||
// `Date.epoch` (1970-01-01) is the "we don't know"
|
||||
// sentinel — see the module doc-block. Once the
|
||||
// merge-aware follow-up PR lands, prior `open_date`
|
||||
// values get preserved across re-imports and a note
|
||||
// can capture the refresh date without diff noise.
|
||||
.open_date = Date.epoch,
|
||||
.open_price = open_price,
|
||||
.account = try allocator.dupe(u8, acct_name),
|
||||
.security_type = security_type,
|
||||
// .note: deliberately unset — see the comment block
|
||||
// above the lots ArrayList init.
|
||||
});
|
||||
}
|
||||
|
||||
return lots.toOwnedSlice(allocator);
|
||||
}
|
||||
|
||||
/// Free per-lot allocator-owned strings + the slice. Mirror of the
|
||||
/// internal cleanup in `Portfolio.deinit` (which we'd use directly
|
||||
/// except we don't construct a Portfolio here — `serializePortfolio`
|
||||
/// takes a bare `[]const Lot`).
|
||||
fn freeLots(allocator: std.mem.Allocator, lots: []const portfolio_mod.Lot) void {
|
||||
for (lots) |lot| freeLot(allocator, lot);
|
||||
allocator.free(lots);
|
||||
}
|
||||
|
||||
fn freeLot(allocator: std.mem.Allocator, lot: portfolio_mod.Lot) void {
|
||||
allocator.free(lot.symbol);
|
||||
if (lot.note) |n| allocator.free(n);
|
||||
if (lot.account) |a| allocator.free(a);
|
||||
if (lot.ticker) |t| allocator.free(t);
|
||||
if (lot.underlying) |u| allocator.free(u);
|
||||
}
|
||||
|
||||
// ── Tests ────────────────────────────────────────────────────
|
||||
|
||||
const testing = std.testing;
|
||||
|
||||
fn testAccountMap(allocator: std.mem.Allocator, entries: []const analysis.AccountTaxEntry) !analysis.AccountMap {
|
||||
var owned = try allocator.alloc(analysis.AccountTaxEntry, entries.len);
|
||||
for (entries, 0..) |e, i| {
|
||||
owned[i] = .{
|
||||
.account = try allocator.dupe(u8, e.account),
|
||||
.tax_type = e.tax_type,
|
||||
.institution = if (e.institution) |s| try allocator.dupe(u8, s) else null,
|
||||
.account_number = if (e.account_number) |s| try allocator.dupe(u8, s) else null,
|
||||
};
|
||||
}
|
||||
return .{ .entries = owned, .allocator = allocator };
|
||||
}
|
||||
|
||||
test "synthesizeLots: stock positions get open_price = cost_basis / quantity" {
|
||||
const allocator = testing.allocator;
|
||||
var account_map = try testAccountMap(allocator, &.{
|
||||
.{ .account = "Mom Brokerage", .tax_type = .taxable, .institution = "fidelity", .account_number = "Z123" },
|
||||
});
|
||||
defer account_map.deinit();
|
||||
|
||||
const positions = [_]BrokeragePosition{
|
||||
.{
|
||||
.account_number = "Z123",
|
||||
.account_name = "Individual",
|
||||
.symbol = "AAPL",
|
||||
.description = "APPLE INC",
|
||||
.quantity = 100,
|
||||
.current_value = 17500,
|
||||
.cost_basis = 12000,
|
||||
.is_cash = false,
|
||||
},
|
||||
};
|
||||
|
||||
const today = Date.fromYmd(2026, 5, 21);
|
||||
const lots = try synthesizeLots(testing.io, allocator, &positions, account_map, .{ .fidelity = "" }, today);
|
||||
defer freeLots(allocator, lots);
|
||||
|
||||
try testing.expectEqual(@as(usize, 1), lots.len);
|
||||
try testing.expectEqualStrings("AAPL", lots[0].symbol);
|
||||
try testing.expectApproxEqAbs(@as(f64, 100), lots[0].shares, 0.01);
|
||||
try testing.expectApproxEqAbs(@as(f64, 120.0), lots[0].open_price, 0.01); // 12000 / 100
|
||||
try testing.expectEqualStrings("Mom Brokerage", lots[0].account.?);
|
||||
// `open_date` is the sentinel (`Date.epoch`, 1970-01-01) — see
|
||||
// module doc-block. No note for now (see synthesizeLots body).
|
||||
try testing.expectEqual(Date.epoch.days, lots[0].open_date.days);
|
||||
try testing.expectEqual(portfolio_mod.LotType.stock, lots[0].security_type);
|
||||
try testing.expect(lots[0].note == null);
|
||||
}
|
||||
|
||||
test "synthesizeLots: missing cost_basis falls back to current_value" {
|
||||
const allocator = testing.allocator;
|
||||
var account_map = try testAccountMap(allocator, &.{
|
||||
.{ .account = "Mom Brokerage", .tax_type = .taxable, .institution = "schwab", .account_number = "716" },
|
||||
});
|
||||
defer account_map.deinit();
|
||||
|
||||
const positions = [_]BrokeragePosition{
|
||||
.{
|
||||
.account_number = "716",
|
||||
.account_name = "Joint trust",
|
||||
.symbol = "MSFT",
|
||||
.description = "MICROSOFT",
|
||||
.quantity = 10,
|
||||
.current_value = 4000,
|
||||
.cost_basis = null,
|
||||
.is_cash = false,
|
||||
},
|
||||
};
|
||||
|
||||
const today = Date.fromYmd(2026, 5, 21);
|
||||
const lots = try synthesizeLots(testing.io, allocator, &positions, account_map, .{ .schwab = "" }, today);
|
||||
defer freeLots(allocator, lots);
|
||||
|
||||
try testing.expectEqual(@as(usize, 1), lots.len);
|
||||
try testing.expectApproxEqAbs(@as(f64, 400.0), lots[0].open_price, 0.01); // 4000 / 10
|
||||
}
|
||||
|
||||
test "synthesizeLots: cash positions become security_type=cash with shares=value, price=1" {
|
||||
const allocator = testing.allocator;
|
||||
var account_map = try testAccountMap(allocator, &.{
|
||||
.{ .account = "Mom Brokerage", .tax_type = .taxable, .institution = "fidelity", .account_number = "Z123" },
|
||||
});
|
||||
defer account_map.deinit();
|
||||
|
||||
const positions = [_]BrokeragePosition{
|
||||
.{
|
||||
.account_number = "Z123",
|
||||
.account_name = "Individual",
|
||||
.symbol = "FZFXX",
|
||||
.description = "MM FUND",
|
||||
.quantity = null,
|
||||
.current_value = 5000,
|
||||
.cost_basis = null,
|
||||
.is_cash = true,
|
||||
},
|
||||
};
|
||||
|
||||
const today = Date.fromYmd(2026, 5, 21);
|
||||
const lots = try synthesizeLots(testing.io, allocator, &positions, account_map, .{ .fidelity = "" }, today);
|
||||
defer freeLots(allocator, lots);
|
||||
|
||||
try testing.expectEqual(@as(usize, 1), lots.len);
|
||||
try testing.expectEqual(portfolio_mod.LotType.cash, lots[0].security_type);
|
||||
try testing.expectApproxEqAbs(@as(f64, 5000), lots[0].shares, 0.01);
|
||||
try testing.expectApproxEqAbs(@as(f64, 1.0), lots[0].open_price, 0.01);
|
||||
}
|
||||
|
||||
test "synthesizeLots: lots are byte-identical regardless of import date" {
|
||||
// Pins the diff-cleanliness property: two imports of the same
|
||||
// brokerage state on different days produce lots whose
|
||||
// serialized form is byte-identical. Without this, held
|
||||
// positions show up as modified in `git diff` even when
|
||||
// nothing changed at the brokerage. The day of the run leaks
|
||||
// into a lot only via the (currently suppressed) note;
|
||||
// reintroducing the note in the merge-aware follow-up will
|
||||
// require an exception here that's gated on (symbol, account)
|
||||
// newness.
|
||||
const allocator = testing.allocator;
|
||||
var account_map = try testAccountMap(allocator, &.{
|
||||
.{ .account = "Mom Brokerage", .tax_type = .taxable, .institution = "fidelity", .account_number = "Z123" },
|
||||
});
|
||||
defer account_map.deinit();
|
||||
|
||||
const positions = [_]BrokeragePosition{
|
||||
.{ .account_number = "Z123", .account_name = "I", .symbol = "AAPL", .description = "", .quantity = 1, .current_value = 100, .cost_basis = 100, .is_cash = false },
|
||||
};
|
||||
|
||||
const lots_a = try synthesizeLots(testing.io, allocator, &positions, account_map, .{ .fidelity = "" }, Date.fromYmd(2026, 5, 21));
|
||||
defer freeLots(allocator, lots_a);
|
||||
const lots_b = try synthesizeLots(testing.io, allocator, &positions, account_map, .{ .fidelity = "" }, Date.fromYmd(2026, 9, 14));
|
||||
defer freeLots(allocator, lots_b);
|
||||
|
||||
// open_date is the sentinel on both runs.
|
||||
try testing.expectEqual(Date.epoch.days, lots_a[0].open_date.days);
|
||||
try testing.expectEqual(Date.epoch.days, lots_b[0].open_date.days);
|
||||
// No note → date-of-run can't leak into the diff.
|
||||
try testing.expect(lots_a[0].note == null);
|
||||
try testing.expect(lots_b[0].note == null);
|
||||
// The serialized bytes match — what `git diff` would actually
|
||||
// see. This is the property we care about for re-imports.
|
||||
const bytes_a = try cache.serializePortfolio(allocator, lots_a);
|
||||
defer allocator.free(bytes_a);
|
||||
const bytes_b = try cache.serializePortfolio(allocator, lots_b);
|
||||
defer allocator.free(bytes_b);
|
||||
try testing.expectEqualStrings(bytes_a, bytes_b);
|
||||
}
|
||||
|
||||
test "synthesizeLots: unmapped account_number fails with UnmappedAccount" {
|
||||
const allocator = testing.allocator;
|
||||
var account_map = try testAccountMap(allocator, &.{
|
||||
.{ .account = "Mom Brokerage", .tax_type = .taxable, .institution = "fidelity", .account_number = "Z123" },
|
||||
});
|
||||
defer account_map.deinit();
|
||||
|
||||
const positions = [_]BrokeragePosition{
|
||||
.{
|
||||
.account_number = "UNKNOWN",
|
||||
.account_name = "?",
|
||||
.symbol = "AAPL",
|
||||
.description = "",
|
||||
.quantity = 1,
|
||||
.current_value = 100,
|
||||
.cost_basis = 100,
|
||||
.is_cash = false,
|
||||
},
|
||||
};
|
||||
|
||||
try testing.expectError(error.UnmappedAccount, synthesizeLots(
|
||||
testing.io,
|
||||
allocator,
|
||||
&positions,
|
||||
account_map,
|
||||
.{ .fidelity = "" },
|
||||
Date.fromYmd(2026, 5, 21),
|
||||
));
|
||||
}
|
||||
|
||||
test "synthesizeLots: institution mismatch counts as unmapped" {
|
||||
// Account number 716 is registered for schwab; an entry coming
|
||||
// in as fidelity must NOT match it. Catches the case where the
|
||||
// user has the same trailing digits at two brokerages.
|
||||
const allocator = testing.allocator;
|
||||
var account_map = try testAccountMap(allocator, &.{
|
||||
.{ .account = "Schwab Trust", .tax_type = .taxable, .institution = "schwab", .account_number = "716" },
|
||||
});
|
||||
defer account_map.deinit();
|
||||
|
||||
const positions = [_]BrokeragePosition{
|
||||
.{
|
||||
.account_number = "716",
|
||||
.account_name = "Some Fidelity Account",
|
||||
.symbol = "AAPL",
|
||||
.description = "",
|
||||
.quantity = 1,
|
||||
.current_value = 100,
|
||||
.cost_basis = 100,
|
||||
.is_cash = false,
|
||||
},
|
||||
};
|
||||
|
||||
try testing.expectError(error.UnmappedAccount, synthesizeLots(
|
||||
testing.io,
|
||||
allocator,
|
||||
&positions,
|
||||
account_map,
|
||||
.{ .fidelity = "" },
|
||||
Date.fromYmd(2026, 5, 21),
|
||||
));
|
||||
}
|
||||
|
||||
test "synthesizeLots: multi-account export fans out per-account" {
|
||||
const allocator = testing.allocator;
|
||||
var account_map = try testAccountMap(allocator, &.{
|
||||
.{ .account = "Mom Roth", .tax_type = .roth, .institution = "fidelity", .account_number = "Z111" },
|
||||
.{ .account = "Mom Brokerage", .tax_type = .taxable, .institution = "fidelity", .account_number = "Z222" },
|
||||
});
|
||||
defer account_map.deinit();
|
||||
|
||||
const positions = [_]BrokeragePosition{
|
||||
.{ .account_number = "Z111", .account_name = "Roth", .symbol = "VTI", .description = "", .quantity = 50, .current_value = 11000, .cost_basis = 10000, .is_cash = false },
|
||||
.{ .account_number = "Z222", .account_name = "Brokerage", .symbol = "VTI", .description = "", .quantity = 100, .current_value = 22000, .cost_basis = 18000, .is_cash = false },
|
||||
};
|
||||
|
||||
const lots = try synthesizeLots(testing.io, allocator, &positions, account_map, .{ .fidelity = "" }, Date.fromYmd(2026, 5, 21));
|
||||
defer freeLots(allocator, lots);
|
||||
|
||||
try testing.expectEqual(@as(usize, 2), lots.len);
|
||||
try testing.expectEqualStrings("Mom Roth", lots[0].account.?);
|
||||
try testing.expectEqualStrings("Mom Brokerage", lots[1].account.?);
|
||||
try testing.expectApproxEqAbs(@as(f64, 200.0), lots[0].open_price, 0.01); // 10000 / 50
|
||||
try testing.expectApproxEqAbs(@as(f64, 180.0), lots[1].open_price, 0.01); // 18000 / 100
|
||||
}
|
||||
|
||||
test "parseArgs: --fidelity captures path" {
|
||||
var ctx: framework.RunCtx = undefined;
|
||||
ctx.io = testing.io;
|
||||
const args = [_][]const u8{ "--fidelity", "/path/to/fid.csv" };
|
||||
const parsed = try parseArgs(&ctx, &args);
|
||||
try testing.expectEqualStrings("/path/to/fid.csv", parsed.source.fidelity);
|
||||
try testing.expect(!parsed.yes);
|
||||
}
|
||||
|
||||
test "parseArgs: --schwab captures path" {
|
||||
var ctx: framework.RunCtx = undefined;
|
||||
ctx.io = testing.io;
|
||||
const args = [_][]const u8{ "--schwab", "/path/to/schwab.csv" };
|
||||
const parsed = try parseArgs(&ctx, &args);
|
||||
try testing.expectEqualStrings("/path/to/schwab.csv", parsed.source.schwab);
|
||||
}
|
||||
|
||||
test "parseArgs: -y sets yes flag" {
|
||||
var ctx: framework.RunCtx = undefined;
|
||||
ctx.io = testing.io;
|
||||
const args = [_][]const u8{ "--fidelity", "f.csv", "-y" };
|
||||
const parsed = try parseArgs(&ctx, &args);
|
||||
try testing.expect(parsed.yes);
|
||||
}
|
||||
|
||||
test "parseArgs: --yes long-form sets yes flag" {
|
||||
var ctx: framework.RunCtx = undefined;
|
||||
ctx.io = testing.io;
|
||||
const args = [_][]const u8{ "--yes", "--schwab", "s.csv" };
|
||||
const parsed = try parseArgs(&ctx, &args);
|
||||
try testing.expect(parsed.yes);
|
||||
}
|
||||
|
||||
test "parseArgs: missing source errors" {
|
||||
var ctx: framework.RunCtx = undefined;
|
||||
ctx.io = testing.io;
|
||||
const args = [_][]const u8{};
|
||||
try testing.expectError(error.MissingSource, parseArgs(&ctx, &args));
|
||||
}
|
||||
|
||||
test "parseArgs: --fidelity + --schwab errors" {
|
||||
var ctx: framework.RunCtx = undefined;
|
||||
ctx.io = testing.io;
|
||||
const args = [_][]const u8{ "--fidelity", "f.csv", "--schwab", "s.csv" };
|
||||
try testing.expectError(error.ConflictingSources, parseArgs(&ctx, &args));
|
||||
}
|
||||
|
||||
test "parseArgs: unknown flag errors" {
|
||||
var ctx: framework.RunCtx = undefined;
|
||||
ctx.io = testing.io;
|
||||
const args = [_][]const u8{"--bogus"};
|
||||
try testing.expectError(error.UnexpectedArg, parseArgs(&ctx, &args));
|
||||
}
|
||||
|
||||
test "parseArgs: --fidelity without value errors" {
|
||||
var ctx: framework.RunCtx = undefined;
|
||||
ctx.io = testing.io;
|
||||
const args = [_][]const u8{"--fidelity"};
|
||||
try testing.expectError(error.UnexpectedArg, parseArgs(&ctx, &args));
|
||||
}
|
||||
|
||||
test "Source.label: returns broker name" {
|
||||
try testing.expectEqualStrings("fidelity", (Source{ .fidelity = "" }).label());
|
||||
try testing.expectEqualStrings("schwab", (Source{ .schwab = "" }).label());
|
||||
}
|
||||
|
||||
test "Source.path: returns the CSV path" {
|
||||
try testing.expectEqualStrings("/x.csv", (Source{ .fidelity = "/x.csv" }).path());
|
||||
try testing.expectEqualStrings("/y.csv", (Source{ .schwab = "/y.csv" }).path());
|
||||
}
|
||||
|
|
@ -36,6 +36,7 @@ const command_modules = .{
|
|||
// Data hygiene
|
||||
.audit = @import("commands/audit.zig"),
|
||||
.enrich = @import("commands/enrich.zig"),
|
||||
.import = @import("commands/import.zig"),
|
||||
.lookup = @import("commands/lookup.zig"),
|
||||
|
||||
// Infrastructure
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue