initial implementation
This commit is contained in:
parent
6ee4f6da88
commit
c605c10010
GPG key ID:
A7B62D657EF764F8
25 changed files with 7296 additions and 0 deletions
5
.gitignore
vendored
Normal file
5
.gitignore
vendored
Normal file
|
|
@ -0,0 +1,5 @@
|
|||
.zig-cache/
|
||||
zig-out/
|
||||
zig-pkg/
|
||||
coverage/
|
||||
.tmp/
|
||||
7
.mise.toml
Normal file
7
.mise.toml
Normal file
|
|
@ -0,0 +1,7 @@
|
|||
[tools]
|
||||
zig = "0.16.0"
|
||||
zls = "0.16.0"
|
||||
"github:j178/prek" = "0.4.1"
|
||||
|
||||
[tools."github:DonIsaac/zlint"]
|
||||
version = "0.9.0"
|
||||
52
.pre-commit-config.yaml
Normal file
52
.pre-commit-config.yaml
Normal file
|
|
@ -0,0 +1,52 @@
|
|||
# See https://pre-commit.com for more information
|
||||
# See https://pre-commit.com/hooks.html for more hooks
|
||||
repos:
|
||||
- repo: https://github.com/pre-commit/pre-commit-hooks
|
||||
rev: v6.0.0
|
||||
hooks:
|
||||
- id: trailing-whitespace
|
||||
- id: end-of-file-fixer
|
||||
- id: check-yaml
|
||||
- id: check-added-large-files
|
||||
- repo: local
|
||||
hooks:
|
||||
- id: forbid-ai-punctuation
|
||||
name: Forbid smart punctuation (en/figure dash, minus, ellipsis, arrows, smart quotes)
|
||||
language: pygrep
|
||||
entry: '(–|‒|―|−|…|→|⇐|⇒|⇔|“|”|‘|’)'
|
||||
files: '\.(zig|zon|md|srf|txt|toml|ya?ml)$'
|
||||
exclude: '^\.pre-commit-config\.yaml$'
|
||||
- id: forbid-prose-em-dash
|
||||
name: Forbid prose em-dash (use ASCII hyphen); no-data sentinel glyphs are exempt
|
||||
language: pygrep
|
||||
entry: ' — '
|
||||
files: '\.(zig|zon|md|srf|txt|toml|ya?ml)$'
|
||||
exclude: '^(\.pre-commit-config\.yaml|src/format\.zig|src/views/projections\.zig|docs/reference/cli/milestones\.md|docs/reference/cli/compare\.md)$'
|
||||
- repo: https://github.com/batmac/pre-commit-zig
|
||||
rev: v0.3.0
|
||||
hooks:
|
||||
- id: zig-fmt
|
||||
- repo: local
|
||||
hooks:
|
||||
- id: zlint
|
||||
name: Run zlint
|
||||
# zlint accepts file paths only via stdin (-S); positional
|
||||
# args are interpreted as directory names and silently
|
||||
# produce no output. Pipe pre-commit's file list through
|
||||
# bash to get the paths to zlint as stdin lines.
|
||||
entry: bash -c 'printf "%s\n" "$@" | zlint --deny-warnings --fix -S' --
|
||||
language: system
|
||||
types: [zig]
|
||||
- repo: https://github.com/batmac/pre-commit-zig
|
||||
rev: v0.3.0
|
||||
hooks:
|
||||
- id: zig-build
|
||||
- repo: local
|
||||
hooks:
|
||||
- id: test
|
||||
name: Run zig build test
|
||||
entry: zig
|
||||
args: ["build", "coverage", "-Dcoverage-threshold=80"]
|
||||
language: system
|
||||
types: [file]
|
||||
pass_filenames: false
|
||||
21
LICENSE
Normal file
21
LICENSE
Normal file
|
|
@ -0,0 +1,21 @@
|
|||
MIT License
|
||||
|
||||
Copyright (c) 2026 Emil Lerch
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
245
README.md
Normal file
245
README.md
Normal file
|
|
@ -0,0 +1,245 @@
|
|||
# zfin-vestwell
|
||||
|
||||
Daily unit-value history for the two Oregon College Savings Plan (Embark)
|
||||
portfolios held in the finance repo: **ORCBI** and **ORC42**.
|
||||
|
||||
```
|
||||
zfin-vestwell reconstruct # rebuild data/ORCBI.srf and data/ORC42.srf
|
||||
zfin-vestwell verify # check them against every evidence source
|
||||
```
|
||||
|
||||
## The problem
|
||||
|
||||
`ORCBI` and `ORC42` are not securities in the ordinary sense. They are
|
||||
**unitized fund-of-funds trust portfolios** inside a 529 plan: no ticker, no
|
||||
CUSIP, no market-data provider coverage. Tiingo, Yahoo and TwelveData all 404
|
||||
them.
|
||||
|
||||
The plan publishes a daily unit value, but only the **current** one. There is no
|
||||
historical endpoint, no downloadable series, and no third-party archive of the
|
||||
numbers. So history has to be reconstructed.
|
||||
|
||||
| Property | ORCBI | ORC42 |
|
||||
|------------------------|----------------|------------------------------|
|
||||
| Plan name | Balanced Index | College Enrollment Year 2042 |
|
||||
| Kind | static | glidepath, steps quarterly |
|
||||
| Feed fund id | `T6GF10080002` | `T6GFRD204202` |
|
||||
| Inception | 2018-09-01 | 2023-07-01 |
|
||||
| Sessions reconstructed | 2006 | 792 |
|
||||
|
||||
## Method
|
||||
|
||||
### 1. Anchors: the only observed values
|
||||
|
||||
Nine exact values per portfolio, in `data/anchors.srf`.
|
||||
|
||||
**Launch.** Every portfolio in the plan launched at exactly `$10.00`. Verified by
|
||||
back-solving the feed's own `inception` field, which is the *annualized*
|
||||
since-inception return: `nav / (1 + inception)^years` lands between 10.0005 and
|
||||
10.0038 for all 37 priced portfolios. The Plan Disclosure Booklet agrees on the
|
||||
date -- "The Portfolios commenced operations in September 2018."
|
||||
|
||||
**Seven Internet Archive captures** of the live feed, spanning 2025-02-13 to
|
||||
2026-07-30, each carrying full 6-decimal values.
|
||||
|
||||
**The current feed value.**
|
||||
|
||||
> Keep `data/anchors.srf`. The wayback rows are not recoverable if the Internet
|
||||
> Archive prunes those captures, and nothing else publishes these numbers.
|
||||
|
||||
### 2. Interpolation between anchors
|
||||
|
||||
Within each gap, walk the trading-day calendar compounding the weighted
|
||||
**total return** of the underlying Vanguard funds, read from zfin's candle cache
|
||||
(`adj_close`). Weights come from `data/model.srf`, per era, so a glidepath step
|
||||
mid-gap needs no special handling.
|
||||
|
||||
Total return, not raw NAV, because the plan's own booklet says these portfolios
|
||||
"reflect changes in value from income and gains and losses on the sale of the
|
||||
Underlying Funds **solely by increasing or decreasing their Unit Value**." All
|
||||
income compounds into the unit value; nothing is distributed. The underlying
|
||||
mutual funds *do* distribute, so their raw NAV drops on every ex-date. A raw-NAV
|
||||
basket under-tracks by roughly the blended yield -- measured at +1.5% to -1.6%
|
||||
drift over four months on ORCBI. That is why `ticker::` + `price_ratio::` is the
|
||||
wrong tool for these symbols.
|
||||
|
||||
### 3. Pinning
|
||||
|
||||
The walk lands near but not exactly on the closing anchor. Take the ratio and
|
||||
distribute it geometrically across the gap, so both endpoints are exact.
|
||||
|
||||
**This step matters more than the model.** No fee term appears anywhere in this
|
||||
project, and none is needed: the asset-based fee, cash drag, trade-date lag and
|
||||
securities-lending income are all roughly constant-rate effects, and a
|
||||
constant-rate misfit is exactly what geometric distribution removes. An unpinned
|
||||
walk across ORCBI's 6.5-year pre-2025 gap is off by -1.29%; pinned, that same gap
|
||||
predicts a held-out interior anchor to **+0.013%**.
|
||||
|
||||
## Accuracy
|
||||
|
||||
`verify` measures it two independent ways.
|
||||
|
||||
**Leave-one-out cross-validation.** Drop an interior anchor, pin its neighbours,
|
||||
predict it. This is the number that matters, because it measures error in the
|
||||
*middle* of an un-anchored gap.
|
||||
|
||||
| Fund | Worst interior error |
|
||||
|-------|----------------------|
|
||||
| ORCBI | 0.079% |
|
||||
| ORC42 | 0.060% |
|
||||
|
||||
**Against hand-typed values.** 16 values per portfolio, recovered from the
|
||||
finance repo's snapshots and **never fed into the model**, so agreement is
|
||||
evidence rather than a fit. Compared at lag=1, since the portal publishes the
|
||||
prior session's value.
|
||||
|
||||
| Fund | n | mean | stdev | max |
|
||||
|-------|-----|---------|--------|----------------|
|
||||
| ORCBI | 16 | +0.0075 | 0.0089 | 0.0287 (0.15%) |
|
||||
| ORC42 | 16 | +0.0035 | 0.0075 | 0.0312 (0.19%) |
|
||||
|
||||
Those values are rounded to 2dp at source, which is already +/-0.005 of noise.
|
||||
|
||||
**Implied drag** is reported for information, not enforced. It should sit near
|
||||
the plan's published cost table (0.234%/yr for ORCBI, 0.239% for ORC42); a fee
|
||||
is not identifiable over a short gap, so per-gap figures are noisy by nature.
|
||||
|
||||
## Limitations
|
||||
|
||||
- **The series is derived, not observed**, except at the nine anchors. Every row
|
||||
carries `source::anchor` or `source::reconstructed` so no consumer can confuse
|
||||
the two.
|
||||
- **It cannot be regenerated from this repo alone.** `reconstruct` needs about ten
|
||||
years of dividend-adjusted daily prices for five institutional-class Vanguard
|
||||
funds, read from zfin's cache -- which lives outside this repo, is not
|
||||
versioned, and is populated through rate-limited provider APIs. A fresh clone on
|
||||
a fresh machine cannot rebuild `data/ORCBI.srf` or `data/ORC42.srf`. That is why
|
||||
they are committed rather than gitignored.
|
||||
- **It is stable under re-basing, but not under a corporate-action correction.**
|
||||
zfin re-bases its adjusted series whenever adjustments are restated, rescaling
|
||||
every historical bar of a fund. The reconstruction consumes only per-day ratios
|
||||
of adjusted closes, so any rescaling cancels exactly -- there is a test for this
|
||||
(`output is invariant to rescaling the input price series`), including
|
||||
independent per-fund factors. What is *not* stable: a provider adding or
|
||||
restating a dividend changes the single daily ratio spanning its ex-date, and
|
||||
scale invariance does not help there.
|
||||
- **ORC42's projected eras are published intent, not fact.** The booklet's
|
||||
glidepath table is a cross-section of every enrollment-year portfolio as of
|
||||
2026-07-01, and because all of them follow one glidepath keyed on
|
||||
years-to-enrollment it maps onto ORC42's future: ORC42 in calendar year `Y`
|
||||
holds what enrollment-year row `E = 4068 - Y` holds today. `Y=2026` gives
|
||||
`E=2042`, matching ORC42's fitted current era, so the mapping is confirmed at
|
||||
the one point where that is possible. Everything past 2027-01-01 is marked
|
||||
`basis::projected` and carries two approximations: the table is annual while
|
||||
allocations actually step quarterly, and era boundaries are placed on 1
|
||||
January. The booklet also reserves the right to "change the asset
|
||||
allocations ... and change the selection of Underlying Funds", so `verify`
|
||||
flags any projected era that has become current.
|
||||
- **ORC42's fitted glidepath boundary is approximate.** `2026-01-01` was chosen
|
||||
by grid search; every candidate from 2025-10-01 to 2026-04-01 gives rms <=
|
||||
0.041%, so the result is insensitive to it.
|
||||
- **ORCBI's pre-2025 weights are an assumption.** That stretch spans a
|
||||
pre-Vestwell (Sumday-era) administrator change and no primary source for the
|
||||
fund lineup that far back was located. Cross-validation says it reconstructs
|
||||
fine, but the weights are unverified there.
|
||||
- **These are instrument prices, not portfolio history.** The finance repo has
|
||||
only held these funds since 2026-02-26. Pre-2026 values are the security's
|
||||
price series and must never be written into `history/*-portfolio.srf`, which
|
||||
records what was actually owned.
|
||||
|
||||
## Files
|
||||
|
||||
| Path | Role |
|
||||
|------------------------------------|---------------------------------------------------------------|
|
||||
| `data/anchors.srf` | The nine observed values, with provenance. **Irreplaceable.** |
|
||||
| `data/model.srf` | Fund weights per era, prefilled to 2047. Generated. |
|
||||
| `data/recorded.srf` | Hand-typed values, used only to verify. |
|
||||
| `data/ORCBI.srf`, `data/ORC42.srf` | Generated. Daily unit values, per-row provenance. |
|
||||
|
||||
## Maintenance
|
||||
|
||||
**Going forward, no reconstruction is needed.** Append each day's published value
|
||||
to `data/anchors.srf` as `source::feed` and the derived portion becomes a
|
||||
progressively smaller fraction of the series. Once anchors are daily the
|
||||
allocation model stops mattering entirely: pinning between two consecutive
|
||||
sessions reproduces both endpoints exactly whatever the weights say, which the
|
||||
test `weights are inert when anchors are one session apart` locks down. The model
|
||||
then matters only for the historical backfill and for filling any gap where
|
||||
forward recording lapsed.
|
||||
|
||||
**ORC42's glidepath is prefilled to 2047**, annually, from the booklet's forward
|
||||
table. Nothing needs doing each quarter. When `verify` reports
|
||||
|
||||
```
|
||||
model era 2027-01-01 is `projected` but is now in force ...
|
||||
```
|
||||
|
||||
confirm that era against the booklet's current allocation table, correct it if the
|
||||
plan changed, and flip it to `basis::fitted`. To regenerate the projected rows
|
||||
after a booklet revision:
|
||||
|
||||
```
|
||||
curl -sLo .tmp/booklet.html \
|
||||
https://marcom.vestwell.com/program-description/oregon-college.html
|
||||
python3 tools/gen_model.py .tmp/booklet.html > data/model.srf
|
||||
```
|
||||
|
||||
`verify` cross-checks the last row against the live feed and fails if the
|
||||
committed series has gone stale.
|
||||
|
||||
## The feed's broken Content-Encoding
|
||||
|
||||
The feed responds with `content-encoding: base64`. The body is **not** base64; it
|
||||
is plain JSON. The header is simply wrong, it is set on every file in that S3
|
||||
prefix, and it survives every request variant.
|
||||
|
||||
Zig 0.16's `std.http.Client` rejects it, correctly per spec, and reports it as
|
||||
`error.HttpHeadersInvalid`. Worse, a head-*parse* failure never marks the
|
||||
connection closing, so with keep-alive on a poisoned connection returns to the
|
||||
pool and teardown can block.
|
||||
|
||||
`src/feed.zig` works around it without a hand-rolled HTTP stack: `receiveHead` is
|
||||
a thin wrapper over two separately-public steps, and only the second fails. So it
|
||||
takes the raw head bytes, drops the offending line, and hands them to the same
|
||||
public parser -- keeping TLS, connection handling and body framing in the standard
|
||||
library.
|
||||
|
||||
## Development
|
||||
|
||||
```
|
||||
zig build # build the exe -- run this, see below
|
||||
zig build test # 55 tests
|
||||
zig build coverage -Dcoverage-threshold=80 # coverage floor
|
||||
zlint --deny-warnings --fix <files>
|
||||
zig fmt <files>
|
||||
```
|
||||
|
||||
Reading the underlying prices requires zfin's cache to be populated for VSMPX,
|
||||
VTPSX, VBMPX, VIPIX and VTIFX. `zfin quote <SYM>` fills it.
|
||||
|
||||
### Coverage, and what is deliberately not covered
|
||||
|
||||
Currently 87.86%. The split is not uniform, on purpose:
|
||||
|
||||
| Module | Coverage |
|
||||
|------------------------------------------------|----------|
|
||||
| recon, verify, data, series, civil | 100% |
|
||||
| candles (`parse` tested, the file read is not) | 95% |
|
||||
| feed (both parsers tested, `fetchBody` is not) | 82% |
|
||||
| main (CLI wiring and file I/O) | 26% |
|
||||
|
||||
Every line of *logic* is tested. What is not tested is the I/O boundary: reading
|
||||
files, the one HTTP call, and CLI plumbing. That end of the program is verified by
|
||||
running it against real data and checking `verify` exits zero, which is stronger
|
||||
evidence than a fixture test would give.
|
||||
|
||||
`src/main.zig` calls `std.testing.refAllDecls` so that `main` and the command
|
||||
functions compile in the test binary. Without it they are dead-code eliminated,
|
||||
and `zig build test` can pass while `zig build` fails on a type error inside
|
||||
`main` -- which happened during development. Keeping it costs about 12 points of
|
||||
coverage but makes the untested surface visible in the number rather than hiding
|
||||
it. **Always run `zig build`, not just `zig build test`.** The pre-commit hook
|
||||
runs both.
|
||||
|
||||
A temp-directory integration harness for `cmdReconstruct` and `cmdVerify
|
||||
--offline` would close most of the remaining gap and is a reasonable follow-up.
|
||||
58
build.zig
Normal file
58
build.zig
Normal file
|
|
@ -0,0 +1,58 @@
|
|||
const std = @import("std");
|
||||
const Coverage = @import("build/Coverage.zig");
|
||||
|
||||
pub fn build(b: *std.Build) void {
|
||||
const target = b.standardTargetOptions(.{});
|
||||
const optimize = b.standardOptimizeOption(.{});
|
||||
|
||||
const srf_dep = b.dependency("srf", .{
|
||||
.target = target,
|
||||
.optimize = optimize,
|
||||
});
|
||||
const srf_mod = srf_dep.module("srf");
|
||||
|
||||
const imports: []const std.Build.Module.Import = &.{
|
||||
.{ .name = "srf", .module = srf_mod },
|
||||
};
|
||||
|
||||
const exe_mod = b.createModule(.{
|
||||
.root_source_file = b.path("src/main.zig"),
|
||||
.target = target,
|
||||
.optimize = optimize,
|
||||
.imports = imports,
|
||||
});
|
||||
|
||||
const exe = b.addExecutable(.{
|
||||
.name = "zfin-vestwell",
|
||||
.root_module = exe_mod,
|
||||
});
|
||||
b.installArtifact(exe);
|
||||
|
||||
const run_cmd = b.addRunArtifact(exe);
|
||||
run_cmd.step.dependOn(b.getInstallStep());
|
||||
if (b.args) |args| run_cmd.addArgs(args);
|
||||
b.step("run", "Build and run the CLI").dependOn(&run_cmd.step);
|
||||
|
||||
const test_mod = b.createModule(.{
|
||||
.root_source_file = b.path("src/main.zig"),
|
||||
.target = target,
|
||||
.optimize = optimize,
|
||||
.imports = imports,
|
||||
});
|
||||
const unit_tests = b.addTest(.{ .root_module = test_mod });
|
||||
const run_tests = b.addRunArtifact(unit_tests);
|
||||
b.step("test", "Run all tests").dependOn(&run_tests.step);
|
||||
|
||||
// Coverage: `zig build coverage -Dcoverage-threshold=N` (kcov, Linux only).
|
||||
// kcov is downloaded into the zig cache on first use.
|
||||
{
|
||||
var cov = Coverage.init(b);
|
||||
const cov_mod = b.createModule(.{
|
||||
.root_source_file = b.path("src/main.zig"),
|
||||
.target = target,
|
||||
.optimize = optimize,
|
||||
.imports = imports,
|
||||
});
|
||||
_ = cov.addModule(cov_mod, "zfin-vestwell");
|
||||
}
|
||||
}
|
||||
22
build.zig.zon
Normal file
22
build.zig.zon
Normal file
|
|
@ -0,0 +1,22 @@
|
|||
.{
|
||||
.name = .zfin_vestwell,
|
||||
.version = "0.1.0",
|
||||
.fingerprint = 0x601dd1243853d32,
|
||||
.minimum_zig_version = "0.16.0",
|
||||
.dependencies = .{
|
||||
.srf = .{
|
||||
.url = "git+https://git.lerch.org/lobo/srf#ea2c35825d652691e6a22526d76e2a06f61d70a5",
|
||||
.hash = "srf-0.0.0-qZj578QeAgCDjih2ii5soqz02fj3g8OhKwUkFh4ReK56",
|
||||
},
|
||||
},
|
||||
.paths = .{
|
||||
"build.zig",
|
||||
"build.zig.zon",
|
||||
"build",
|
||||
"src",
|
||||
"data",
|
||||
"tools",
|
||||
"README.md",
|
||||
"LICENSE",
|
||||
},
|
||||
}
|
||||
239
build/Coverage.zig
Normal file
239
build/Coverage.zig
Normal file
|
|
@ -0,0 +1,239 @@
|
|||
const builtin = @import("builtin");
|
||||
const std = @import("std");
|
||||
const Build = std.Build;
|
||||
|
||||
const Coverage = @This();
|
||||
|
||||
/// Whether the host platform supports kcov-based coverage.
|
||||
/// Only x86_64 and aarch64 Linux are supported (kcov binary availability).
|
||||
/// On unsupported platforms, the coverage step will fail at runtime with
|
||||
/// a clear error from the kcov download or execution step.
|
||||
// pub const supported = builtin.os.tag == .linux and
|
||||
// (builtin.cpu.arch == .x86_64 or builtin.cpu.arch == .aarch64);
|
||||
|
||||
/// Initialize coverage infrastructure. Creates the "coverage" build step,
|
||||
/// registers build options (-Dcoverage-threshold, -Dcoverage-dir),
|
||||
/// and sets up the kcov download step. The kcov binary is downloaded into the
|
||||
/// zig cache on first use and reused thereafter.
|
||||
///
|
||||
/// Use `zig build coverage --verbose` to see per-file coverage breakdown.
|
||||
///
|
||||
/// Call `addModule()` on the returned value to add the test module to the
|
||||
/// coverage run.
|
||||
///
|
||||
/// Because addModule creates a new test executable from the root module provided,
|
||||
/// if there are any linking steps being done to your test executable, those
|
||||
/// must also be done to the test_exe returned by addModule.
|
||||
pub fn init(b: *Build) Coverage {
|
||||
// Add options
|
||||
const coverage_threshold = b.option(u7, "coverage-threshold", "Minimum coverage percentage required") orelse 0;
|
||||
const coverage_dir = b.option([]const u8, "coverage-dir", "Coverage output directory") orelse
|
||||
b.pathJoin(&.{ b.build_root.path orelse ".", "coverage" });
|
||||
const coverage_step = b.step("coverage", "Generate test coverage report");
|
||||
|
||||
// Set up kcov download.
|
||||
// We can't download directly because we are sandboxed during build, but
|
||||
// we can create a helper program and run it. First we need the destination
|
||||
// directory, keyed by architecture.
|
||||
const arch_name = switch (builtin.cpu.arch) {
|
||||
.x86_64 => "x86_64",
|
||||
.aarch64 => "aarch64",
|
||||
else => @tagName(builtin.cpu.arch),
|
||||
};
|
||||
|
||||
const Algo = std.crypto.hash.sha2.Sha256;
|
||||
var hasher = Algo.init(.{});
|
||||
hasher.update("kcov-");
|
||||
hasher.update(arch_name);
|
||||
var cache_hash: [Algo.digest_length]u8 = undefined;
|
||||
hasher.final(&cache_hash);
|
||||
|
||||
const cache_dir = b.pathJoin(&.{
|
||||
b.cache_root.path.?,
|
||||
"o",
|
||||
b.fmt("{s}", .{std.fmt.bytesToHex(cache_hash, .lower)}),
|
||||
});
|
||||
|
||||
const kcov_path = b.pathJoin(&.{ cache_dir, b.fmt("kcov-{s}", .{arch_name}) });
|
||||
|
||||
// Create the download helper executable
|
||||
const download_exe = b.addExecutable(.{
|
||||
.name = "download-kcov",
|
||||
.root_module = b.createModule(.{
|
||||
.root_source_file = b.path("build/download_kcov.zig"),
|
||||
.target = b.resolveTargetQuery(.{}),
|
||||
}),
|
||||
});
|
||||
|
||||
const run_download = b.addRunArtifact(download_exe);
|
||||
run_download.addArg(kcov_path);
|
||||
run_download.addArg(arch_name);
|
||||
|
||||
return .{
|
||||
.b = b,
|
||||
.coverage_step = coverage_step,
|
||||
.coverage_dir = coverage_dir,
|
||||
.coverage_threshold = coverage_threshold,
|
||||
.kcov_path = kcov_path,
|
||||
.run_download = run_download,
|
||||
};
|
||||
}
|
||||
|
||||
/// Add a test module to the coverage run. Runs kcov on the test binary,
|
||||
/// then reads the coverage JSON and prints a summary (with per-file
|
||||
/// breakdown if --verbose). Fails if below -Dcoverage-threshold.
|
||||
///
|
||||
/// Returns the test executable so the caller can add any extra linking steps.
|
||||
pub fn addModule(self: *Coverage, root_module: *Build.Module, name: []const u8) *Build.Step.Compile {
|
||||
const b = self.b;
|
||||
|
||||
// Set up kcov run: filter to src/ only, use custom CSS for HTML report
|
||||
const run_coverage = b.addSystemCommand(&.{self.kcov_path});
|
||||
const include_path = b.pathJoin(&.{ b.build_root.path.?, "src" });
|
||||
run_coverage.addArgs(&.{ "--include-path", include_path });
|
||||
const css_file = b.pathJoin(&.{ b.build_root.path.?, "build", "bcov.css" });
|
||||
run_coverage.addArg(b.fmt("--configure=css-file={s}", .{css_file}));
|
||||
run_coverage.addArg(self.coverage_dir);
|
||||
|
||||
// Create a test executable for this module.
|
||||
// We need to set use_llvm because the self-hosted backend
|
||||
// does not emit the DWARF data that kcov needs.
|
||||
const test_exe = b.addTest(.{
|
||||
.name = name,
|
||||
.root_module = root_module,
|
||||
.use_llvm = true,
|
||||
});
|
||||
run_coverage.addArtifactArg(test_exe);
|
||||
run_coverage.step.dependOn(&test_exe.step);
|
||||
run_coverage.step.dependOn(&self.run_download.step);
|
||||
|
||||
// Wire up the threshold check step after kcov completes
|
||||
const check = b.allocator.create(Check) catch @panic("OOM");
|
||||
check.* = .{
|
||||
.step = Build.Step.init(.{
|
||||
.id = .custom,
|
||||
.name = "check coverage",
|
||||
.owner = b,
|
||||
.makeFn = make,
|
||||
}),
|
||||
.json_path = b.fmt("{s}/{s}/coverage.json", .{ self.coverage_dir, name }),
|
||||
.threshold = self.coverage_threshold,
|
||||
};
|
||||
check.step.dependOn(&run_coverage.step);
|
||||
self.coverage_step.dependOn(&check.step);
|
||||
|
||||
return test_exe;
|
||||
}
|
||||
|
||||
// ── Coverage struct fields ──────────────────────────────────
|
||||
|
||||
// Fields used by init() to configure the shared coverage infrastructure
|
||||
b: *Build,
|
||||
coverage_step: *Build.Step,
|
||||
coverage_dir: []const u8,
|
||||
coverage_threshold: u7,
|
||||
kcov_path: []const u8,
|
||||
run_download: *Build.Step.Run,
|
||||
|
||||
// Per-module threshold-check step. Created in `addModule`; `make`
|
||||
// recovers the instance via `@fieldParentPtr("step", ...)`.
|
||||
const Check = struct {
|
||||
step: Build.Step,
|
||||
json_path: []const u8,
|
||||
threshold: u7,
|
||||
};
|
||||
|
||||
// This must be kept in step with kcov per-binary coverage.json format
|
||||
const CoverageReport = struct {
|
||||
files: []const CoverageFile,
|
||||
};
|
||||
|
||||
const CoverageFile = struct {
|
||||
file: []const u8,
|
||||
covered_lines: usize,
|
||||
total_lines: usize,
|
||||
};
|
||||
|
||||
const File = struct {
|
||||
file: []const u8,
|
||||
percent_covered: f64,
|
||||
covered_lines: usize,
|
||||
total_lines: usize,
|
||||
|
||||
pub fn coverageLessThanDesc(_: void, lhs: File, rhs: File) bool {
|
||||
return lhs.percent_covered > rhs.percent_covered;
|
||||
}
|
||||
};
|
||||
|
||||
/// Build step make function: reads kcov JSON output, prints a summary
|
||||
/// (with per-file breakdown if verbose), and fails if below threshold.
|
||||
fn make(step: *Build.Step, options: Build.Step.MakeOptions) !void {
|
||||
_ = options;
|
||||
const check: *Check = @fieldParentPtr("step", step);
|
||||
const allocator = step.owner.allocator;
|
||||
const io = step.owner.graph.io;
|
||||
|
||||
const file = std.Io.Dir.cwd().openFile(io, check.json_path, .{}) catch |err| {
|
||||
return step.fail("Failed to open coverage report {s}: {}", .{ check.json_path, err });
|
||||
};
|
||||
defer file.close(io);
|
||||
|
||||
var file_reader = file.reader(io, &.{});
|
||||
const content = try file_reader.interface.allocRemaining(allocator, .limited(10 * 1024 * 1024));
|
||||
defer allocator.free(content);
|
||||
|
||||
const json = std.json.parseFromSlice(CoverageReport, allocator, content, .{
|
||||
.ignore_unknown_fields = true,
|
||||
}) catch |err| {
|
||||
return step.fail("Failed to parse coverage JSON: {}", .{err});
|
||||
};
|
||||
defer json.deinit();
|
||||
|
||||
var total_covered: usize = 0;
|
||||
var total_lines: usize = 0;
|
||||
|
||||
var file_list = std.ArrayList(File).empty;
|
||||
defer file_list.deinit(allocator);
|
||||
|
||||
for (json.value.files) |f| {
|
||||
const pct: f64 = if (f.total_lines > 0)
|
||||
@as(f64, @floatFromInt(f.covered_lines)) / @as(f64, @floatFromInt(f.total_lines)) * 100.0
|
||||
else
|
||||
0;
|
||||
try file_list.append(allocator, .{
|
||||
.file = f.file,
|
||||
.covered_lines = f.covered_lines,
|
||||
.total_lines = f.total_lines,
|
||||
.percent_covered = pct,
|
||||
});
|
||||
total_covered += f.covered_lines;
|
||||
total_lines += f.total_lines;
|
||||
}
|
||||
|
||||
std.mem.sort(File, file_list.items, {}, File.coverageLessThanDesc);
|
||||
|
||||
var stdout_buffer: [1024]u8 = undefined;
|
||||
var stdout_writer = std.Io.File.stdout().writer(io, &stdout_buffer);
|
||||
const stdout = &stdout_writer.interface;
|
||||
if (step.owner.verbose) {
|
||||
for (file_list.items) |f| {
|
||||
try stdout.print(
|
||||
"{d: >5.1}% {d: >5}/{d: <5}:{s}\n",
|
||||
.{ f.percent_covered, f.covered_lines, f.total_lines, f.file },
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
const total_pct: f64 = if (total_lines > 0)
|
||||
@as(f64, @floatFromInt(total_covered)) / @as(f64, @floatFromInt(total_lines)) * 100.0
|
||||
else
|
||||
0;
|
||||
try stdout.print(
|
||||
"Total test coverage: {d:.2}% ({d}/{d})\n",
|
||||
.{ total_pct, total_covered, total_lines },
|
||||
);
|
||||
try stdout.flush();
|
||||
|
||||
if (@as(u7, @intFromFloat(@floor(total_pct))) < check.threshold)
|
||||
return step.fail("Coverage {d:.2}% is below threshold {d}%", .{ total_pct, check.threshold });
|
||||
}
|
||||
46
build/bcov.css
Normal file
46
build/bcov.css
Normal file
|
|
@ -0,0 +1,46 @@
|
|||
/* Based upon the lcov CSS style, style files can be reused - Dark Theme */
|
||||
body { color: #e0e0e0; background-color: #1e1e1e; }
|
||||
a:link { color: #6b9aff; text-decoration: underline; }
|
||||
a:visited { color: #4dbb7a; text-decoration: underline; }
|
||||
a:active { color: #ff6b8a; text-decoration: underline; }
|
||||
td.title { text-align: center; padding-bottom: 10px; font-size: 20pt; font-weight: bold; }
|
||||
td.ruler { background-color: #4a6ba8; }
|
||||
td.headerItem { text-align: right; padding-right: 6px; font-family: sans-serif; font-weight: bold; }
|
||||
td.headerValue { text-align: left; color: #6b9aff; font-family: sans-serif; font-weight: bold; }
|
||||
td.versionInfo { text-align: center; padding-top: 2px; }
|
||||
th.headerItem { text-align: right; padding-right: 6px; font-family: sans-serif; font-weight: bold; }
|
||||
th.headerValue { text-align: left; color: #6b9aff; font-family: sans-serif; font-weight: bold; }
|
||||
pre.source { font-family: monospace; white-space: pre; overflow: hidden; text-overflow: ellipsis; }
|
||||
span.lineNum { background-color: #5a5a2a; }
|
||||
span.lineNumLegend { background-color: #5a5a2a; width: 96px; font-weight: bold ;}
|
||||
span.lineCov { background-color: #2d5a2d; }
|
||||
span.linePartCov { background-color: #707000; }
|
||||
span.lineNoCov { background-color: #762c2c; }
|
||||
span.orderNum { background-color: #5a4a2a; float: right; width:5em; text-align: left; }
|
||||
span.orderNumLegend { background-color: #5a4a2a; width: 96px; font-weight: bold ;}
|
||||
span.coverHits { background-color: #4a4a2a; padding-left: 3px; padding-right: 1px; text-align: right; list-style-type: none; display: inline-block; width: 5em; }
|
||||
span.coverHitsLegend { background-color: #4a4a2a; width: 96px; font-weight: bold; margin: 0 auto;}
|
||||
td.tableHead { text-align: center; color: #e0e0e0; background-color: #4a6ba8; font-family: sans-serif; font-size: 120%; font-weight: bold; }
|
||||
td.coverFile { text-align: left; padding-left: 10px; padding-right: 20px; color: #6b9aff; font-family: monospace; background-color: #3a3a3a; }
|
||||
td.coverBar { padding-left: 10px; padding-right: 10px; background-color: #3a3a3a; }
|
||||
td.coverBarOutline { background-color: #4a4a4a; }
|
||||
td.coverPer { text-align: left; padding-left: 10px; padding-right: 10px; font-weight: bold; background-color: #3a3a3a; color: #e0e0e0; }
|
||||
td.coverPerLeftMed { text-align: left; padding-left: 10px; padding-right: 10px; background-color: #5a5a00; font-weight: bold; color: #e0e0e0; }
|
||||
td.coverPerLeftLo { text-align: left; padding-left: 10px; padding-right: 10px; background-color: #5a2d2d; font-weight: bold; color: #e0e0e0; }
|
||||
td.coverPerLeftHi { text-align: left; padding-left: 10px; padding-right: 10px; background-color: #2d5a2d; font-weight: bold; color: #e0e0e0; }
|
||||
td.coverNum { text-align: right; padding-left: 10px; padding-right: 10px; background-color: #3a3a3a; color: #e0e0e0; }
|
||||
|
||||
/* Override tablesorter hover styles for dark theme */
|
||||
.tablesorter-blue tbody > tr:hover > td,
|
||||
.tablesorter-blue tbody > tr:hover + tr.tablesorter-childRow > td,
|
||||
.tablesorter-blue tbody > tr:hover + tr.tablesorter-childRow + tr.tablesorter-childRow > td,
|
||||
.tablesorter-blue tbody > tr.even:hover > td,
|
||||
.tablesorter-blue tbody > tr.even:hover + tr.tablesorter-childRow > td,
|
||||
.tablesorter-blue tbody > tr.even:hover + tr.tablesorter-childRow + tr.tablesorter-childRow > td {
|
||||
background: #4a4a4a;
|
||||
}
|
||||
.tablesorter-blue tbody > tr.odd:hover > td,
|
||||
.tablesorter-blue tbody > tr.odd:hover + tr.tablesorter-childRow > td,
|
||||
.tablesorter-blue tbody > tr.odd:hover + tr.tablesorter-childRow + tr.tablesorter-childRow > td {
|
||||
background: #4a4a4a;
|
||||
}
|
||||
85
build/download_kcov.zig
Normal file
85
build/download_kcov.zig
Normal file
|
|
@ -0,0 +1,85 @@
|
|||
const std = @import("std");
|
||||
|
||||
pub fn main(init: std.process.Init) !void {
|
||||
// Build-time helper: short-lived process that downloads a single
|
||||
// file. Arena lets us skip per-allocation `defer free(...)` and
|
||||
// amortizes the allocation cost across the run via the arena's
|
||||
// exponential block growth. Process exit reclaims everything.
|
||||
const allocator = init.arena.allocator();
|
||||
const io = init.io;
|
||||
|
||||
const args = try init.minimal.args.toSlice(allocator);
|
||||
|
||||
if (args.len != 3) return error.InvalidArgs;
|
||||
|
||||
const kcov_path = args[1];
|
||||
const arch_name = args[2];
|
||||
|
||||
// Check to see if file exists. If it does, we have nothing more to do
|
||||
const stat = std.Io.Dir.cwd().statFile(io, kcov_path, .{}) catch |err| blk: {
|
||||
if (err == error.FileNotFound) break :blk null else return err;
|
||||
};
|
||||
// This might be better checking whether it's executable and >= 7MB, but
|
||||
// for now, we'll do a simple exists check
|
||||
if (stat != null) return;
|
||||
var stdout_buffer: [1024]u8 = undefined;
|
||||
var stdout_writer = std.Io.File.stdout().writer(io, &stdout_buffer);
|
||||
const stdout = &stdout_writer.interface;
|
||||
|
||||
try stdout.writeAll("Determining latest kcov version\n");
|
||||
try stdout.flush();
|
||||
|
||||
var client = std.http.Client{ .allocator = allocator, .io = io };
|
||||
defer client.deinit();
|
||||
|
||||
// Get redirect to find latest version
|
||||
const list_uri = try std.Uri.parse("https://git.lerch.org/lobo/-/packages/generic/kcov/");
|
||||
var req = try client.request(.GET, list_uri, .{ .redirect_behavior = .unhandled });
|
||||
defer req.deinit();
|
||||
|
||||
try req.sendBodiless();
|
||||
var redirect_buf: [1024]u8 = undefined;
|
||||
const response = try req.receiveHead(&redirect_buf);
|
||||
|
||||
if (response.head.status != .see_other) return error.UnexpectedResponse;
|
||||
|
||||
const location = response.head.location orelse return error.NoLocation;
|
||||
const version_start = std.mem.lastIndexOfScalar(u8, location, '/') orelse return error.InvalidLocation;
|
||||
const version = location[version_start + 1 ..];
|
||||
|
||||
try stdout.print(
|
||||
"Downloading kcov version {s} for {s} to {s}...",
|
||||
.{ version, arch_name, kcov_path },
|
||||
);
|
||||
try stdout.flush();
|
||||
|
||||
const binary_url = try std.fmt.allocPrint(
|
||||
allocator,
|
||||
"https://git.lerch.org/api/packages/lobo/generic/kcov/{s}/kcov-{s}",
|
||||
.{ version, arch_name },
|
||||
);
|
||||
|
||||
const cache_dir = std.fs.path.dirname(kcov_path) orelse return error.InvalidPath;
|
||||
std.Io.Dir.cwd().createDir(io, cache_dir, std.Io.File.Permissions.default_dir) catch |e| switch (e) {
|
||||
error.PathAlreadyExists => {},
|
||||
else => return e,
|
||||
};
|
||||
|
||||
const uri = try std.Uri.parse(binary_url);
|
||||
const file = try std.Io.Dir.cwd().createFile(io, kcov_path, .{});
|
||||
defer file.close(io);
|
||||
try file.setPermissions(io, @enumFromInt(0o755));
|
||||
|
||||
var buffer: [8192]u8 = undefined;
|
||||
var writer = file.writer(io, &buffer);
|
||||
const result = try client.fetch(.{
|
||||
.location = .{ .uri = uri },
|
||||
.response_writer = &writer.interface,
|
||||
});
|
||||
|
||||
if (result.status != .ok) return error.DownloadFailed;
|
||||
try writer.interface.flush();
|
||||
|
||||
try stdout.writeAll("done\n");
|
||||
try stdout.flush();
|
||||
}
|
||||
807
data/ORC42.srf
Normal file
807
data/ORC42.srf
Normal file
|
|
@ -0,0 +1,807 @@
|
|||
#!srfv1
|
||||
# Daily unit-value series for ORC42, Oregon College Savings Plan (Embark).
|
||||
#
|
||||
# GENERATED FILE -- produced by `zfin-vestwell reconstruct`. Do not hand
|
||||
# edit. To change the output, change data/model.srf or data/anchors.srf
|
||||
# and regenerate.
|
||||
#
|
||||
# source::anchor observed. Reproduced exactly. See data/anchors.srf
|
||||
# for where each observation came from.
|
||||
# source::reconstructed derived from the underlying funds' total return,
|
||||
# pinned to the surrounding anchors. Cross-validated
|
||||
# to under 0.08%; accurate, but NOT observed.
|
||||
#
|
||||
# These are unit values of a unitized trust, not mutual-fund NAVs: no
|
||||
# distributions, all income compounds into the value.
|
||||
date::2023-07-03,unit_value:num:10.000000,source::anchor
|
||||
date::2023-07-05,unit_value:num:9.953713,source::reconstructed
|
||||
date::2023-07-06,unit_value:num:9.840878,source::reconstructed
|
||||
date::2023-07-07,unit_value:num:9.856379,source::reconstructed
|
||||
date::2023-07-10,unit_value:num:9.887285,source::reconstructed
|
||||
date::2023-07-11,unit_value:num:9.966819,source::reconstructed
|
||||
date::2023-07-12,unit_value:num:10.074488,source::reconstructed
|
||||
date::2023-07-13,unit_value:num:10.185994,source::reconstructed
|
||||
date::2023-07-14,unit_value:num:10.153835,source::reconstructed
|
||||
date::2023-07-17,unit_value:num:10.177897,source::reconstructed
|
||||
date::2023-07-18,unit_value:num:10.233003,source::reconstructed
|
||||
date::2023-07-19,unit_value:num:10.250867,source::reconstructed
|
||||
date::2023-07-20,unit_value:num:10.186303,source::reconstructed
|
||||
date::2023-07-21,unit_value:num:10.188080,source::reconstructed
|
||||
date::2023-07-24,unit_value:num:10.212709,source::reconstructed
|
||||
date::2023-07-25,unit_value:num:10.245691,source::reconstructed
|
||||
date::2023-07-26,unit_value:num:10.255803,source::reconstructed
|
||||
date::2023-07-27,unit_value:num:10.193295,source::reconstructed
|
||||
date::2023-07-28,unit_value:num:10.288951,source::reconstructed
|
||||
date::2023-07-31,unit_value:num:10.312106,source::reconstructed
|
||||
date::2023-08-01,unit_value:num:10.253981,source::reconstructed
|
||||
date::2023-08-02,unit_value:num:10.099676,source::reconstructed
|
||||
date::2023-08-03,unit_value:num:10.072179,source::reconstructed
|
||||
date::2023-08-04,unit_value:num:10.060656,source::reconstructed
|
||||
date::2023-08-07,unit_value:num:10.123593,source::reconstructed
|
||||
date::2023-08-08,unit_value:num:10.070195,source::reconstructed
|
||||
date::2023-08-09,unit_value:num:10.034978,source::reconstructed
|
||||
date::2023-08-10,unit_value:num:10.042134,source::reconstructed
|
||||
date::2023-08-11,unit_value:num:10.005567,source::reconstructed
|
||||
date::2023-08-14,unit_value:num:10.013580,source::reconstructed
|
||||
date::2023-08-15,unit_value:num:9.908177,source::reconstructed
|
||||
date::2023-08-16,unit_value:num:9.838017,source::reconstructed
|
||||
date::2023-08-17,unit_value:num:9.777358,source::reconstructed
|
||||
date::2023-08-18,unit_value:num:9.772140,source::reconstructed
|
||||
date::2023-08-21,unit_value:num:9.808668,source::reconstructed
|
||||
date::2023-08-22,unit_value:num:9.792638,source::reconstructed
|
||||
date::2023-08-23,unit_value:num:9.896667,source::reconstructed
|
||||
date::2023-08-24,unit_value:num:9.787599,source::reconstructed
|
||||
date::2023-08-25,unit_value:num:9.833799,source::reconstructed
|
||||
date::2023-08-28,unit_value:num:9.903091,source::reconstructed
|
||||
date::2023-08-29,unit_value:num:10.037445,source::reconstructed
|
||||
date::2023-08-30,unit_value:num:10.056521,source::reconstructed
|
||||
date::2023-08-31,unit_value:num:10.035036,source::reconstructed
|
||||
date::2023-09-01,unit_value:num:10.057662,source::reconstructed
|
||||
date::2023-09-05,unit_value:num:9.999076,source::reconstructed
|
||||
date::2023-09-06,unit_value:num:9.950545,source::reconstructed
|
||||
date::2023-09-07,unit_value:num:9.912895,source::reconstructed
|
||||
date::2023-09-08,unit_value:num:9.918246,source::reconstructed
|
||||
date::2023-09-11,unit_value:num:9.984675,source::reconstructed
|
||||
date::2023-09-12,unit_value:num:9.947912,source::reconstructed
|
||||
date::2023-09-13,unit_value:num:9.942848,source::reconstructed
|
||||
date::2023-09-14,unit_value:num:10.028493,source::reconstructed
|
||||
date::2023-09-15,unit_value:num:9.955023,source::reconstructed
|
||||
date::2023-09-18,unit_value:num:9.941458,source::reconstructed
|
||||
date::2023-09-19,unit_value:num:9.925549,source::reconstructed
|
||||
date::2023-09-20,unit_value:num:9.864908,source::reconstructed
|
||||
date::2023-09-21,unit_value:num:9.716130,source::reconstructed
|
||||
date::2023-09-22,unit_value:num:9.714991,source::reconstructed
|
||||
date::2023-09-25,unit_value:num:9.717211,source::reconstructed
|
||||
date::2023-09-26,unit_value:num:9.594912,source::reconstructed
|
||||
date::2023-09-27,unit_value:num:9.595055,source::reconstructed
|
||||
date::2023-09-28,unit_value:num:9.649148,source::reconstructed
|
||||
date::2023-09-29,unit_value:num:9.632904,source::reconstructed
|
||||
date::2023-10-02,unit_value:num:9.578216,source::reconstructed
|
||||
date::2023-10-03,unit_value:num:9.452274,source::reconstructed
|
||||
date::2023-10-04,unit_value:num:9.494693,source::reconstructed
|
||||
date::2023-10-05,unit_value:num:9.510880,source::reconstructed
|
||||
date::2023-10-06,unit_value:num:9.605401,source::reconstructed
|
||||
date::2023-10-09,unit_value:num:9.640685,source::reconstructed
|
||||
date::2023-10-10,unit_value:num:9.720556,source::reconstructed
|
||||
date::2023-10-11,unit_value:num:9.760902,source::reconstructed
|
||||
date::2023-10-12,unit_value:num:9.687186,source::reconstructed
|
||||
date::2023-10-13,unit_value:num:9.632496,source::reconstructed
|
||||
date::2023-10-16,unit_value:num:9.706193,source::reconstructed
|
||||
date::2023-10-17,unit_value:num:9.708181,source::reconstructed
|
||||
date::2023-10-18,unit_value:num:9.575483,source::reconstructed
|
||||
date::2023-10-19,unit_value:num:9.498855,source::reconstructed
|
||||
date::2023-10-20,unit_value:num:9.402141,source::reconstructed
|
||||
date::2023-10-23,unit_value:num:9.389234,source::reconstructed
|
||||
date::2023-10-24,unit_value:num:9.447017,source::reconstructed
|
||||
date::2023-10-25,unit_value:num:9.337003,source::reconstructed
|
||||
date::2023-10-26,unit_value:num:9.269560,source::reconstructed
|
||||
date::2023-10-27,unit_value:num:9.238071,source::reconstructed
|
||||
date::2023-10-30,unit_value:num:9.327833,source::reconstructed
|
||||
date::2023-10-31,unit_value:num:9.361910,source::reconstructed
|
||||
date::2023-11-01,unit_value:num:9.451499,source::reconstructed
|
||||
date::2023-11-02,unit_value:num:9.622482,source::reconstructed
|
||||
date::2023-11-03,unit_value:num:9.732418,source::reconstructed
|
||||
date::2023-11-06,unit_value:num:9.734369,source::reconstructed
|
||||
date::2023-11-07,unit_value:num:9.733922,source::reconstructed
|
||||
date::2023-11-08,unit_value:num:9.729043,source::reconstructed
|
||||
date::2023-11-09,unit_value:num:9.668693,source::reconstructed
|
||||
date::2023-11-10,unit_value:num:9.764861,source::reconstructed
|
||||
date::2023-11-13,unit_value:num:9.765099,source::reconstructed
|
||||
date::2023-11-14,unit_value:num:9.976927,source::reconstructed
|
||||
date::2023-11-15,unit_value:num:9.994777,source::reconstructed
|
||||
date::2023-11-16,unit_value:num:9.984701,source::reconstructed
|
||||
date::2023-11-17,unit_value:num:10.035404,source::reconstructed
|
||||
date::2023-11-20,unit_value:num:10.099876,source::reconstructed
|
||||
date::2023-11-21,unit_value:num:10.069078,source::reconstructed
|
||||
date::2023-11-22,unit_value:num:10.095197,source::reconstructed
|
||||
date::2023-11-24,unit_value:num:10.118359,source::reconstructed
|
||||
date::2023-11-27,unit_value:num:10.104077,source::reconstructed
|
||||
date::2023-11-28,unit_value:num:10.122093,source::reconstructed
|
||||
date::2023-11-29,unit_value:num:10.131516,source::reconstructed
|
||||
date::2023-11-30,unit_value:num:10.157366,source::reconstructed
|
||||
date::2023-12-01,unit_value:num:10.238552,source::reconstructed
|
||||
date::2023-12-04,unit_value:num:10.187272,source::reconstructed
|
||||
date::2023-12-05,unit_value:num:10.169136,source::reconstructed
|
||||
date::2023-12-06,unit_value:num:10.157153,source::reconstructed
|
||||
date::2023-12-07,unit_value:num:10.214280,source::reconstructed
|
||||
date::2023-12-08,unit_value:num:10.241185,source::reconstructed
|
||||
date::2023-12-11,unit_value:num:10.270654,source::reconstructed
|
||||
date::2023-12-12,unit_value:num:10.295204,source::reconstructed
|
||||
date::2023-12-13,unit_value:num:10.437212,source::reconstructed
|
||||
date::2023-12-14,unit_value:num:10.523260,source::reconstructed
|
||||
date::2023-12-15,unit_value:num:10.493902,source::reconstructed
|
||||
date::2023-12-18,unit_value:num:10.517991,source::reconstructed
|
||||
date::2023-12-19,unit_value:num:10.591111,source::reconstructed
|
||||
date::2023-12-20,unit_value:num:10.465138,source::reconstructed
|
||||
date::2023-12-21,unit_value:num:10.581471,source::reconstructed
|
||||
date::2023-12-22,unit_value:num:10.593854,source::reconstructed
|
||||
date::2023-12-26,unit_value:num:10.636567,source::reconstructed
|
||||
date::2023-12-27,unit_value:num:10.683951,source::reconstructed
|
||||
date::2023-12-28,unit_value:num:10.684952,source::reconstructed
|
||||
date::2023-12-29,unit_value:num:10.686239,source::reconstructed
|
||||
date::2024-01-02,unit_value:num:10.585886,source::reconstructed
|
||||
date::2024-01-03,unit_value:num:10.498668,source::reconstructed
|
||||
date::2024-01-04,unit_value:num:10.484828,source::reconstructed
|
||||
date::2024-01-05,unit_value:num:10.494419,source::reconstructed
|
||||
date::2024-01-08,unit_value:num:10.608699,source::reconstructed
|
||||
date::2024-01-09,unit_value:num:10.564849,source::reconstructed
|
||||
date::2024-01-10,unit_value:num:10.603448,source::reconstructed
|
||||
date::2024-01-11,unit_value:num:10.603242,source::reconstructed
|
||||
date::2024-01-12,unit_value:num:10.621422,source::reconstructed
|
||||
date::2024-01-16,unit_value:num:10.530304,source::reconstructed
|
||||
date::2024-01-17,unit_value:num:10.444346,source::reconstructed
|
||||
date::2024-01-18,unit_value:num:10.519507,source::reconstructed
|
||||
date::2024-01-19,unit_value:num:10.607190,source::reconstructed
|
||||
date::2024-01-22,unit_value:num:10.628550,source::reconstructed
|
||||
date::2024-01-23,unit_value:num:10.642239,source::reconstructed
|
||||
date::2024-01-24,unit_value:num:10.664651,source::reconstructed
|
||||
date::2024-01-25,unit_value:num:10.709835,source::reconstructed
|
||||
date::2024-01-26,unit_value:num:10.719210,source::reconstructed
|
||||
date::2024-01-29,unit_value:num:10.792225,source::reconstructed
|
||||
date::2024-01-30,unit_value:num:10.772709,source::reconstructed
|
||||
date::2024-01-31,unit_value:num:10.662634,source::reconstructed
|
||||
date::2024-02-01,unit_value:num:10.776091,source::reconstructed
|
||||
date::2024-02-02,unit_value:num:10.798780,source::reconstructed
|
||||
date::2024-02-05,unit_value:num:10.745744,source::reconstructed
|
||||
date::2024-02-06,unit_value:num:10.804577,source::reconstructed
|
||||
date::2024-02-07,unit_value:num:10.852895,source::reconstructed
|
||||
date::2024-02-08,unit_value:num:10.860248,source::reconstructed
|
||||
date::2024-02-09,unit_value:num:10.902126,source::reconstructed
|
||||
date::2024-02-12,unit_value:num:10.916178,source::reconstructed
|
||||
date::2024-02-13,unit_value:num:10.759607,source::reconstructed
|
||||
date::2024-02-14,unit_value:num:10.865906,source::reconstructed
|
||||
date::2024-02-15,unit_value:num:10.948025,source::reconstructed
|
||||
date::2024-02-16,unit_value:num:10.925689,source::reconstructed
|
||||
date::2024-02-20,unit_value:num:10.898188,source::reconstructed
|
||||
date::2024-02-21,unit_value:num:10.903969,source::reconstructed
|
||||
date::2024-02-22,unit_value:num:11.061622,source::reconstructed
|
||||
date::2024-02-23,unit_value:num:11.071530,source::reconstructed
|
||||
date::2024-02-26,unit_value:num:11.045190,source::reconstructed
|
||||
date::2024-02-27,unit_value:num:11.066298,source::reconstructed
|
||||
date::2024-02-28,unit_value:num:11.029582,source::reconstructed
|
||||
date::2024-02-29,unit_value:num:11.078383,source::reconstructed
|
||||
date::2024-03-01,unit_value:num:11.171726,source::reconstructed
|
||||
date::2024-03-04,unit_value:num:11.155599,source::reconstructed
|
||||
date::2024-03-05,unit_value:num:11.078975,source::reconstructed
|
||||
date::2024-03-06,unit_value:num:11.159730,source::reconstructed
|
||||
date::2024-03-07,unit_value:num:11.261901,source::reconstructed
|
||||
date::2024-03-08,unit_value:num:11.214247,source::reconstructed
|
||||
date::2024-03-11,unit_value:num:11.190306,source::reconstructed
|
||||
date::2024-03-12,unit_value:num:11.282364,source::reconstructed
|
||||
date::2024-03-13,unit_value:num:11.265064,source::reconstructed
|
||||
date::2024-03-14,unit_value:num:11.212233,source::reconstructed
|
||||
date::2024-03-15,unit_value:num:11.167900,source::reconstructed
|
||||
date::2024-03-18,unit_value:num:11.203160,source::reconstructed
|
||||
date::2024-03-19,unit_value:num:11.239598,source::reconstructed
|
||||
date::2024-03-20,unit_value:num:11.341223,source::reconstructed
|
||||
date::2024-03-21,unit_value:num:11.375033,source::reconstructed
|
||||
date::2024-03-22,unit_value:num:11.347378,source::reconstructed
|
||||
date::2024-03-25,unit_value:num:11.323586,source::reconstructed
|
||||
date::2024-03-26,unit_value:num:11.312533,source::reconstructed
|
||||
date::2024-03-27,unit_value:num:11.394926,source::reconstructed
|
||||
date::2024-03-28,unit_value:num:11.401004,source::reconstructed
|
||||
date::2024-04-01,unit_value:num:11.363879,source::reconstructed
|
||||
date::2024-04-02,unit_value:num:11.301221,source::reconstructed
|
||||
date::2024-04-03,unit_value:num:11.331387,source::reconstructed
|
||||
date::2024-04-04,unit_value:num:11.237320,source::reconstructed
|
||||
date::2024-04-05,unit_value:num:11.310321,source::reconstructed
|
||||
date::2024-04-08,unit_value:num:11.329488,source::reconstructed
|
||||
date::2024-04-09,unit_value:num:11.349860,source::reconstructed
|
||||
date::2024-04-10,unit_value:num:11.229296,source::reconstructed
|
||||
date::2024-04-11,unit_value:num:11.278713,source::reconstructed
|
||||
date::2024-04-12,unit_value:num:11.122512,source::reconstructed
|
||||
date::2024-04-15,unit_value:num:11.021025,source::reconstructed
|
||||
date::2024-04-16,unit_value:num:10.959367,source::reconstructed
|
||||
date::2024-04-17,unit_value:num:10.925545,source::reconstructed
|
||||
date::2024-04-18,unit_value:num:10.913168,source::reconstructed
|
||||
date::2024-04-19,unit_value:num:10.857095,source::reconstructed
|
||||
date::2024-04-22,unit_value:num:10.956715,source::reconstructed
|
||||
date::2024-04-23,unit_value:num:11.070816,source::reconstructed
|
||||
date::2024-04-24,unit_value:num:11.068023,source::reconstructed
|
||||
date::2024-04-25,unit_value:num:11.025504,source::reconstructed
|
||||
date::2024-04-26,unit_value:num:11.114017,source::reconstructed
|
||||
date::2024-04-29,unit_value:num:11.164497,source::reconstructed
|
||||
date::2024-04-30,unit_value:num:11.013188,source::reconstructed
|
||||
date::2024-05-01,unit_value:num:10.997104,source::reconstructed
|
||||
date::2024-05-02,unit_value:num:11.124775,source::reconstructed
|
||||
date::2024-05-03,unit_value:num:11.236171,source::reconstructed
|
||||
date::2024-05-06,unit_value:num:11.327691,source::reconstructed
|
||||
date::2024-05-07,unit_value:num:11.333447,source::reconstructed
|
||||
date::2024-05-08,unit_value:num:11.323254,source::reconstructed
|
||||
date::2024-05-09,unit_value:num:11.381607,source::reconstructed
|
||||
date::2024-05-10,unit_value:num:11.396558,source::reconstructed
|
||||
date::2024-05-13,unit_value:num:11.402186,source::reconstructed
|
||||
date::2024-05-14,unit_value:num:11.463614,source::reconstructed
|
||||
date::2024-05-15,unit_value:num:11.578184,source::reconstructed
|
||||
date::2024-05-16,unit_value:num:11.554843,source::reconstructed
|
||||
date::2024-05-17,unit_value:num:11.576712,source::reconstructed
|
||||
date::2024-05-20,unit_value:num:11.586744,source::reconstructed
|
||||
date::2024-05-21,unit_value:num:11.583759,source::reconstructed
|
||||
date::2024-05-22,unit_value:num:11.534331,source::reconstructed
|
||||
date::2024-05-23,unit_value:num:11.454857,source::reconstructed
|
||||
date::2024-05-24,unit_value:num:11.523538,source::reconstructed
|
||||
date::2024-05-28,unit_value:num:11.517806,source::reconstructed
|
||||
date::2024-05-29,unit_value:num:11.398375,source::reconstructed
|
||||
date::2024-05-30,unit_value:num:11.393349,source::reconstructed
|
||||
date::2024-05-31,unit_value:num:11.468890,source::reconstructed
|
||||
date::2024-06-03,unit_value:num:11.497005,source::reconstructed
|
||||
date::2024-06-04,unit_value:num:11.473188,source::reconstructed
|
||||
date::2024-06-05,unit_value:num:11.588493,source::reconstructed
|
||||
date::2024-06-06,unit_value:num:11.594182,source::reconstructed
|
||||
date::2024-06-07,unit_value:num:11.533233,source::reconstructed
|
||||
date::2024-06-10,unit_value:num:11.560788,source::reconstructed
|
||||
date::2024-06-11,unit_value:num:11.538833,source::reconstructed
|
||||
date::2024-06-12,unit_value:num:11.639509,source::reconstructed
|
||||
date::2024-06-13,unit_value:num:11.611524,source::reconstructed
|
||||
date::2024-06-14,unit_value:num:11.575795,source::reconstructed
|
||||
date::2024-06-17,unit_value:num:11.633179,source::reconstructed
|
||||
date::2024-06-18,unit_value:num:11.672341,source::reconstructed
|
||||
date::2024-06-20,unit_value:num:11.655449,source::reconstructed
|
||||
date::2024-06-21,unit_value:num:11.623633,source::reconstructed
|
||||
date::2024-06-24,unit_value:num:11.634685,source::reconstructed
|
||||
date::2024-06-25,unit_value:num:11.662071,source::reconstructed
|
||||
date::2024-06-26,unit_value:num:11.640418,source::reconstructed
|
||||
date::2024-06-27,unit_value:num:11.657940,source::reconstructed
|
||||
date::2024-06-28,unit_value:num:11.635240,source::reconstructed
|
||||
date::2024-07-01,unit_value:num:11.649592,source::reconstructed
|
||||
date::2024-07-02,unit_value:num:11.702588,source::reconstructed
|
||||
date::2024-07-03,unit_value:num:11.781456,source::reconstructed
|
||||
date::2024-07-05,unit_value:num:11.845500,source::reconstructed
|
||||
date::2024-07-08,unit_value:num:11.838758,source::reconstructed
|
||||
date::2024-07-09,unit_value:num:11.832957,source::reconstructed
|
||||
date::2024-07-10,unit_value:num:11.943243,source::reconstructed
|
||||
date::2024-07-11,unit_value:num:11.936662,source::reconstructed
|
||||
date::2024-07-12,unit_value:num:12.015842,source::reconstructed
|
||||
date::2024-07-15,unit_value:num:12.007871,source::reconstructed
|
||||
date::2024-07-16,unit_value:num:12.089671,source::reconstructed
|
||||
date::2024-07-17,unit_value:num:11.963997,source::reconstructed
|
||||
date::2024-07-18,unit_value:num:11.872735,source::reconstructed
|
||||
date::2024-07-19,unit_value:num:11.795893,source::reconstructed
|
||||
date::2024-07-22,unit_value:num:11.900341,source::reconstructed
|
||||
date::2024-07-23,unit_value:num:11.877349,source::reconstructed
|
||||
date::2024-07-24,unit_value:num:11.674091,source::reconstructed
|
||||
date::2024-07-25,unit_value:num:11.634273,source::reconstructed
|
||||
date::2024-07-26,unit_value:num:11.756352,source::reconstructed
|
||||
date::2024-07-29,unit_value:num:11.746583,source::reconstructed
|
||||
date::2024-07-30,unit_value:num:11.719814,source::reconstructed
|
||||
date::2024-07-31,unit_value:num:11.893559,source::reconstructed
|
||||
date::2024-08-01,unit_value:num:11.709120,source::reconstructed
|
||||
date::2024-08-02,unit_value:num:11.519842,source::reconstructed
|
||||
date::2024-08-05,unit_value:num:11.228692,source::reconstructed
|
||||
date::2024-08-06,unit_value:num:11.311107,source::reconstructed
|
||||
date::2024-08-07,unit_value:num:11.275620,source::reconstructed
|
||||
date::2024-08-08,unit_value:num:11.486943,source::reconstructed
|
||||
date::2024-08-09,unit_value:num:11.531456,source::reconstructed
|
||||
date::2024-08-12,unit_value:num:11.531049,source::reconstructed
|
||||
date::2024-08-13,unit_value:num:11.697819,source::reconstructed
|
||||
date::2024-08-14,unit_value:num:11.726208,source::reconstructed
|
||||
date::2024-08-15,unit_value:num:11.879301,source::reconstructed
|
||||
date::2024-08-16,unit_value:num:11.924807,source::reconstructed
|
||||
date::2024-08-19,unit_value:num:12.035467,source::reconstructed
|
||||
date::2024-08-20,unit_value:num:12.000856,source::reconstructed
|
||||
date::2024-08-21,unit_value:num:12.069503,source::reconstructed
|
||||
date::2024-08-22,unit_value:num:11.978167,source::reconstructed
|
||||
date::2024-08-23,unit_value:num:12.133806,source::reconstructed
|
||||
date::2024-08-26,unit_value:num:12.096291,source::reconstructed
|
||||
date::2024-08-27,unit_value:num:12.119541,source::reconstructed
|
||||
date::2024-08-28,unit_value:num:12.054552,source::reconstructed
|
||||
date::2024-08-29,unit_value:num:12.075580,source::reconstructed
|
||||
date::2024-08-30,unit_value:num:12.153477,source::reconstructed
|
||||
date::2024-09-03,unit_value:num:11.935508,source::reconstructed
|
||||
date::2024-09-04,unit_value:num:11.917011,source::reconstructed
|
||||
date::2024-09-05,unit_value:num:11.906275,source::reconstructed
|
||||
date::2024-09-06,unit_value:num:11.722842,source::reconstructed
|
||||
date::2024-09-09,unit_value:num:11.830887,source::reconstructed
|
||||
date::2024-09-10,unit_value:num:11.844588,source::reconstructed
|
||||
date::2024-09-11,unit_value:num:11.932384,source::reconstructed
|
||||
date::2024-09-12,unit_value:num:12.021373,source::reconstructed
|
||||
date::2024-09-13,unit_value:num:12.081377,source::reconstructed
|
||||
date::2024-09-16,unit_value:num:12.122487,source::reconstructed
|
||||
date::2024-09-17,unit_value:num:12.116732,source::reconstructed
|
||||
date::2024-09-18,unit_value:num:12.081914,source::reconstructed
|
||||
date::2024-09-19,unit_value:num:12.284296,source::reconstructed
|
||||
date::2024-09-20,unit_value:num:12.235471,source::reconstructed
|
||||
date::2024-09-23,unit_value:num:12.271674,source::reconstructed
|
||||
date::2024-09-24,unit_value:num:12.338835,source::reconstructed
|
||||
date::2024-09-25,unit_value:num:12.293837,source::reconstructed
|
||||
date::2024-09-26,unit_value:num:12.427839,source::reconstructed
|
||||
date::2024-09-27,unit_value:num:12.404368,source::reconstructed
|
||||
date::2024-09-30,unit_value:num:12.418017,source::reconstructed
|
||||
date::2024-10-01,unit_value:num:12.349151,source::reconstructed
|
||||
date::2024-10-02,unit_value:num:12.359903,source::reconstructed
|
||||
date::2024-10-03,unit_value:num:12.297420,source::reconstructed
|
||||
date::2024-10-04,unit_value:num:12.385326,source::reconstructed
|
||||
date::2024-10-07,unit_value:num:12.303120,source::reconstructed
|
||||
date::2024-10-08,unit_value:num:12.330761,source::reconstructed
|
||||
date::2024-10-09,unit_value:num:12.367738,source::reconstructed
|
||||
date::2024-10-10,unit_value:num:12.352308,source::reconstructed
|
||||
date::2024-10-11,unit_value:num:12.424744,source::reconstructed
|
||||
date::2024-10-14,unit_value:num:12.477902,source::reconstructed
|
||||
date::2024-10-15,unit_value:num:12.363262,source::reconstructed
|
||||
date::2024-10-16,unit_value:num:12.423593,source::reconstructed
|
||||
date::2024-10-17,unit_value:num:12.412666,source::reconstructed
|
||||
date::2024-10-18,unit_value:num:12.471323,source::reconstructed
|
||||
date::2024-10-21,unit_value:num:12.402167,source::reconstructed
|
||||
date::2024-10-22,unit_value:num:12.371289,source::reconstructed
|
||||
date::2024-10-23,unit_value:num:12.273166,source::reconstructed
|
||||
date::2024-10-24,unit_value:num:12.297376,source::reconstructed
|
||||
date::2024-10-25,unit_value:num:12.277428,source::reconstructed
|
||||
date::2024-10-28,unit_value:num:12.327908,source::reconstructed
|
||||
date::2024-10-29,unit_value:num:12.329056,source::reconstructed
|
||||
date::2024-10-30,unit_value:num:12.282598,source::reconstructed
|
||||
date::2024-10-31,unit_value:num:12.130305,source::reconstructed
|
||||
date::2024-11-01,unit_value:num:12.166772,source::reconstructed
|
||||
date::2024-11-04,unit_value:num:12.166173,source::reconstructed
|
||||
date::2024-11-05,unit_value:num:12.300999,source::reconstructed
|
||||
date::2024-11-06,unit_value:num:12.435771,source::reconstructed
|
||||
date::2024-11-07,unit_value:num:12.553326,source::reconstructed
|
||||
date::2024-11-08,unit_value:num:12.526310,source::reconstructed
|
||||
date::2024-11-11,unit_value:num:12.536238,source::reconstructed
|
||||
date::2024-11-12,unit_value:num:12.431737,source::reconstructed
|
||||
date::2024-11-13,unit_value:num:12.394629,source::reconstructed
|
||||
date::2024-11-14,unit_value:num:12.346042,source::reconstructed
|
||||
date::2024-11-15,unit_value:num:12.238037,source::reconstructed
|
||||
date::2024-11-18,unit_value:num:12.303075,source::reconstructed
|
||||
date::2024-11-19,unit_value:num:12.338827,source::reconstructed
|
||||
date::2024-11-20,unit_value:num:12.327378,source::reconstructed
|
||||
date::2024-11-21,unit_value:num:12.377600,source::reconstructed
|
||||
date::2024-11-22,unit_value:num:12.423821,source::reconstructed
|
||||
date::2024-11-25,unit_value:num:12.478552,source::reconstructed
|
||||
date::2024-11-26,unit_value:num:12.483474,source::reconstructed
|
||||
date::2024-11-27,unit_value:num:12.478401,source::reconstructed
|
||||
date::2024-11-29,unit_value:num:12.564043,source::reconstructed
|
||||
date::2024-12-02,unit_value:num:12.594778,source::reconstructed
|
||||
date::2024-12-03,unit_value:num:12.617050,source::reconstructed
|
||||
date::2024-12-04,unit_value:num:12.671497,source::reconstructed
|
||||
date::2024-12-05,unit_value:num:12.669669,source::reconstructed
|
||||
date::2024-12-06,unit_value:num:12.690695,source::reconstructed
|
||||
date::2024-12-09,unit_value:num:12.654479,source::reconstructed
|
||||
date::2024-12-10,unit_value:num:12.588062,source::reconstructed
|
||||
date::2024-12-11,unit_value:num:12.666468,source::reconstructed
|
||||
date::2024-12-12,unit_value:num:12.584685,source::reconstructed
|
||||
date::2024-12-13,unit_value:num:12.568755,source::reconstructed
|
||||
date::2024-12-16,unit_value:num:12.581979,source::reconstructed
|
||||
date::2024-12-17,unit_value:num:12.524863,source::reconstructed
|
||||
date::2024-12-18,unit_value:num:12.213035,source::reconstructed
|
||||
date::2024-12-19,unit_value:num:12.190385,source::reconstructed
|
||||
date::2024-12-20,unit_value:num:12.267888,source::reconstructed
|
||||
date::2024-12-23,unit_value:num:12.333793,source::reconstructed
|
||||
date::2024-12-24,unit_value:num:12.416300,source::reconstructed
|
||||
date::2024-12-26,unit_value:num:12.423462,source::reconstructed
|
||||
date::2024-12-27,unit_value:num:12.341823,source::reconstructed
|
||||
date::2024-12-30,unit_value:num:12.253525,source::reconstructed
|
||||
date::2024-12-31,unit_value:num:12.222951,source::reconstructed
|
||||
date::2025-01-02,unit_value:num:12.205282,source::reconstructed
|
||||
date::2025-01-03,unit_value:num:12.312851,source::reconstructed
|
||||
date::2025-01-06,unit_value:num:12.374166,source::reconstructed
|
||||
date::2025-01-07,unit_value:num:12.285522,source::reconstructed
|
||||
date::2025-01-08,unit_value:num:12.284625,source::reconstructed
|
||||
date::2025-01-10,unit_value:num:12.101311,source::reconstructed
|
||||
date::2025-01-13,unit_value:num:12.087202,source::reconstructed
|
||||
date::2025-01-14,unit_value:num:12.134129,source::reconstructed
|
||||
date::2025-01-15,unit_value:num:12.311973,source::reconstructed
|
||||
date::2025-01-16,unit_value:num:12.327068,source::reconstructed
|
||||
date::2025-01-17,unit_value:num:12.409698,source::reconstructed
|
||||
date::2025-01-21,unit_value:num:12.547982,source::reconstructed
|
||||
date::2025-01-22,unit_value:num:12.578163,source::reconstructed
|
||||
date::2025-01-23,unit_value:num:12.632877,source::reconstructed
|
||||
date::2025-01-24,unit_value:num:12.641064,source::reconstructed
|
||||
date::2025-01-27,unit_value:num:12.521483,source::reconstructed
|
||||
date::2025-01-28,unit_value:num:12.589188,source::reconstructed
|
||||
date::2025-01-29,unit_value:num:12.560423,source::reconstructed
|
||||
date::2025-01-30,unit_value:num:12.653243,source::reconstructed
|
||||
date::2025-01-31,unit_value:num:12.583014,source::reconstructed
|
||||
date::2025-02-03,unit_value:num:12.483404,source::reconstructed
|
||||
date::2025-02-04,unit_value:num:12.589758,source::reconstructed
|
||||
date::2025-02-05,unit_value:num:12.658150,source::reconstructed
|
||||
date::2025-02-06,unit_value:num:12.694559,source::reconstructed
|
||||
date::2025-02-07,unit_value:num:12.603619,source::reconstructed
|
||||
date::2025-02-10,unit_value:num:12.674005,source::reconstructed
|
||||
date::2025-02-11,unit_value:num:12.676437,source::reconstructed
|
||||
date::2025-02-12,unit_value:num:12.656098,source::reconstructed
|
||||
date::2025-02-13,unit_value:num:12.787304,source::anchor
|
||||
date::2025-02-14,unit_value:num:12.796567,source::reconstructed
|
||||
date::2025-02-18,unit_value:num:12.842923,source::reconstructed
|
||||
date::2025-02-19,unit_value:num:12.822710,source::reconstructed
|
||||
date::2025-02-20,unit_value:num:12.808087,source::reconstructed
|
||||
date::2025-02-21,unit_value:num:12.657725,source::reconstructed
|
||||
date::2025-02-24,unit_value:num:12.601368,source::reconstructed
|
||||
date::2025-02-25,unit_value:num:12.599258,source::reconstructed
|
||||
date::2025-02-26,unit_value:num:12.621301,source::reconstructed
|
||||
date::2025-02-27,unit_value:num:12.456858,source::reconstructed
|
||||
date::2025-02-28,unit_value:num:12.557580,source::reconstructed
|
||||
date::2025-03-03,unit_value:num:12.436605,source::reconstructed
|
||||
date::2025-03-04,unit_value:num:12.357796,source::reconstructed
|
||||
date::2025-03-05,unit_value:num:12.541447,source::reconstructed
|
||||
date::2025-03-06,unit_value:num:12.380093,source::reconstructed
|
||||
date::2025-03-07,unit_value:num:12.449454,source::reconstructed
|
||||
date::2025-03-10,unit_value:num:12.165024,source::reconstructed
|
||||
date::2025-03-11,unit_value:num:12.118093,source::reconstructed
|
||||
date::2025-03-12,unit_value:num:12.175403,source::reconstructed
|
||||
date::2025-03-13,unit_value:num:12.050715,source::reconstructed
|
||||
date::2025-03-14,unit_value:num:12.269339,source::reconstructed
|
||||
date::2025-03-17,unit_value:num:12.374254,source::reconstructed
|
||||
date::2025-03-18,unit_value:num:12.304433,source::reconstructed
|
||||
date::2025-03-19,unit_value:num:12.398291,source::reconstructed
|
||||
date::2025-03-20,unit_value:num:12.352744,source::reconstructed
|
||||
date::2025-03-21,unit_value:num:12.330061,source::reconstructed
|
||||
date::2025-03-24,unit_value:num:12.455764,source::reconstructed
|
||||
date::2025-03-25,unit_value:num:12.473824,source::reconstructed
|
||||
date::2025-03-26,unit_value:num:12.353753,source::reconstructed
|
||||
date::2025-03-27,unit_value:num:12.338607,source::reconstructed
|
||||
date::2025-03-28,unit_value:num:12.161648,source::reconstructed
|
||||
date::2025-03-31,unit_value:num:12.163220,source::reconstructed
|
||||
date::2025-04-01,unit_value:num:12.206264,source::reconstructed
|
||||
date::2025-04-02,unit_value:num:12.270447,source::reconstructed
|
||||
date::2025-04-03,unit_value:num:11.850448,source::reconstructed
|
||||
date::2025-04-04,unit_value:num:11.232947,source::reconstructed
|
||||
date::2025-04-07,unit_value:num:11.097672,source::reconstructed
|
||||
date::2025-04-08,unit_value:num:10.964863,source::reconstructed
|
||||
date::2025-04-09,unit_value:num:11.748701,source::reconstructed
|
||||
date::2025-04-10,unit_value:num:11.478211,source::reconstructed
|
||||
date::2025-04-11,unit_value:num:11.689763,source::reconstructed
|
||||
date::2025-04-14,unit_value:num:11.802612,source::reconstructed
|
||||
date::2025-04-15,unit_value:num:11.819235,source::reconstructed
|
||||
date::2025-04-16,unit_value:num:11.666866,source::reconstructed
|
||||
date::2025-04-17,unit_value:num:11.723166,source::reconstructed
|
||||
date::2025-04-21,unit_value:num:11.570209,source::anchor
|
||||
date::2025-04-22,unit_value:num:11.788315,source::reconstructed
|
||||
date::2025-04-23,unit_value:num:11.921036,source::reconstructed
|
||||
date::2025-04-24,unit_value:num:12.117537,source::reconstructed
|
||||
date::2025-04-25,unit_value:num:12.164607,source::reconstructed
|
||||
date::2025-04-28,unit_value:num:12.202833,source::reconstructed
|
||||
date::2025-04-29,unit_value:num:12.252683,source::reconstructed
|
||||
date::2025-04-30,unit_value:num:12.271626,source::reconstructed
|
||||
date::2025-05-01,unit_value:num:12.295300,source::reconstructed
|
||||
date::2025-05-02,unit_value:num:12.470939,source::reconstructed
|
||||
date::2025-05-05,unit_value:num:12.432352,source::reconstructed
|
||||
date::2025-05-06,unit_value:num:12.381645,source::reconstructed
|
||||
date::2025-05-07,unit_value:num:12.400991,source::reconstructed
|
||||
date::2025-05-08,unit_value:num:12.426537,source::reconstructed
|
||||
date::2025-05-09,unit_value:num:12.450835,source::reconstructed
|
||||
date::2025-05-12,unit_value:num:12.714235,source::reconstructed
|
||||
date::2025-05-13,unit_value:num:12.777372,source::reconstructed
|
||||
date::2025-05-14,unit_value:num:12.769515,source::reconstructed
|
||||
date::2025-05-15,unit_value:num:12.835671,source::reconstructed
|
||||
date::2025-05-16,unit_value:num:12.896630,source::reconstructed
|
||||
date::2025-05-19,unit_value:num:12.925511,source::reconstructed
|
||||
date::2025-05-20,unit_value:num:12.909078,source::reconstructed
|
||||
date::2025-05-21,unit_value:num:12.760445,source::reconstructed
|
||||
date::2025-05-22,unit_value:num:12.761744,source::reconstructed
|
||||
date::2025-05-23,unit_value:num:12.734421,source::reconstructed
|
||||
date::2025-05-27,unit_value:num:12.920235,source::reconstructed
|
||||
date::2025-05-28,unit_value:num:12.836867,source::reconstructed
|
||||
date::2025-05-29,unit_value:num:12.887486,source::reconstructed
|
||||
date::2025-05-30,unit_value:num:12.887722,source::reconstructed
|
||||
date::2025-06-02,unit_value:num:12.956196,source::reconstructed
|
||||
date::2025-06-03,unit_value:num:12.981267,source::reconstructed
|
||||
date::2025-06-04,unit_value:num:13.016670,source::reconstructed
|
||||
date::2025-06-05,unit_value:num:12.986747,source::reconstructed
|
||||
date::2025-06-06,unit_value:num:13.070636,source::reconstructed
|
||||
date::2025-06-09,unit_value:num:13.094226,source::reconstructed
|
||||
date::2025-06-10,unit_value:num:13.144315,source::reconstructed
|
||||
date::2025-06-11,unit_value:num:13.131311,source::reconstructed
|
||||
date::2025-06-12,unit_value:num:13.185274,source::reconstructed
|
||||
date::2025-06-13,unit_value:num:13.034010,source::reconstructed
|
||||
date::2025-06-16,unit_value:num:13.129234,source::reconstructed
|
||||
date::2025-06-17,unit_value:num:13.020044,source::reconstructed
|
||||
date::2025-06-18,unit_value:num:13.026102,source::reconstructed
|
||||
date::2025-06-20,unit_value:num:12.978210,source::reconstructed
|
||||
date::2025-06-23,unit_value:num:13.073354,source::reconstructed
|
||||
date::2025-06-24,unit_value:num:13.232069,source::reconstructed
|
||||
date::2025-06-25,unit_value:num:13.212529,source::reconstructed
|
||||
date::2025-06-26,unit_value:num:13.326675,source::reconstructed
|
||||
date::2025-06-27,unit_value:num:13.380561,source::reconstructed
|
||||
date::2025-06-30,unit_value:num:13.436085,source::reconstructed
|
||||
date::2025-07-01,unit_value:num:13.432535,source::reconstructed
|
||||
date::2025-07-02,unit_value:num:13.487534,source::reconstructed
|
||||
date::2025-07-03,unit_value:num:13.556602,source::reconstructed
|
||||
date::2025-07-07,unit_value:num:13.436331,source::reconstructed
|
||||
date::2025-07-08,unit_value:num:13.460886,source::reconstructed
|
||||
date::2025-07-09,unit_value:num:13.532542,source::reconstructed
|
||||
date::2025-07-10,unit_value:num:13.559650,source::reconstructed
|
||||
date::2025-07-11,unit_value:num:13.491769,source::reconstructed
|
||||
date::2025-07-14,unit_value:num:13.509285,source::reconstructed
|
||||
date::2025-07-15,unit_value:num:13.444577,source::reconstructed
|
||||
date::2025-07-16,unit_value:num:13.487884,source::reconstructed
|
||||
date::2025-07-17,unit_value:num:13.556887,source::reconstructed
|
||||
date::2025-07-18,unit_value:num:13.550092,source::reconstructed
|
||||
date::2025-07-21,unit_value:num:13.581015,source::reconstructed
|
||||
date::2025-07-22,unit_value:num:13.613795,source::reconstructed
|
||||
date::2025-07-23,unit_value:num:13.758377,source::reconstructed
|
||||
date::2025-07-24,unit_value:num:13.727808,source::reconstructed
|
||||
date::2025-07-25,unit_value:num:13.752395,source::reconstructed
|
||||
date::2025-07-28,unit_value:num:13.689051,source::reconstructed
|
||||
date::2025-07-29,unit_value:num:13.672850,source::reconstructed
|
||||
date::2025-07-30,unit_value:num:13.617346,source::reconstructed
|
||||
date::2025-07-31,unit_value:num:13.550725,source::reconstructed
|
||||
date::2025-08-01,unit_value:num:13.422512,source::reconstructed
|
||||
date::2025-08-04,unit_value:num:13.599595,source::reconstructed
|
||||
date::2025-08-05,unit_value:num:13.576751,source::reconstructed
|
||||
date::2025-08-06,unit_value:num:13.654453,source::reconstructed
|
||||
date::2025-08-07,unit_value:num:13.688138,source::reconstructed
|
||||
date::2025-08-08,unit_value:num:13.745569,source::reconstructed
|
||||
date::2025-08-11,unit_value:num:13.713480,source::reconstructed
|
||||
date::2025-08-12,unit_value:num:13.856984,source::reconstructed
|
||||
date::2025-08-13,unit_value:num:13.929284,source::reconstructed
|
||||
date::2025-08-14,unit_value:num:13.901342,source::reconstructed
|
||||
date::2025-08-15,unit_value:num:13.897063,source::reconstructed
|
||||
date::2025-08-18,unit_value:num:13.903258,source::reconstructed
|
||||
date::2025-08-19,unit_value:num:13.843964,source::reconstructed
|
||||
date::2025-08-20,unit_value:num:13.832378,source::reconstructed
|
||||
date::2025-08-21,unit_value:num:13.790919,source::reconstructed
|
||||
date::2025-08-22,unit_value:num:13.997673,source::reconstructed
|
||||
date::2025-08-25,unit_value:num:13.915473,source::reconstructed
|
||||
date::2025-08-26,unit_value:num:13.952843,source::reconstructed
|
||||
date::2025-08-27,unit_value:num:13.958343,source::anchor
|
||||
date::2025-08-28,unit_value:num:14.006720,source::reconstructed
|
||||
date::2025-08-29,unit_value:num:13.933310,source::reconstructed
|
||||
date::2025-09-02,unit_value:num:13.848912,source::reconstructed
|
||||
date::2025-09-03,unit_value:num:13.896271,source::reconstructed
|
||||
date::2025-09-04,unit_value:num:13.986506,source::reconstructed
|
||||
date::2025-09-05,unit_value:num:14.007099,source::reconstructed
|
||||
date::2025-09-08,unit_value:num:14.075240,source::reconstructed
|
||||
date::2025-09-09,unit_value:num:14.086797,source::reconstructed
|
||||
date::2025-09-10,unit_value:num:14.121083,source::reconstructed
|
||||
date::2025-09-11,unit_value:num:14.244415,source::reconstructed
|
||||
date::2025-09-12,unit_value:num:14.219436,source::reconstructed
|
||||
date::2025-09-15,unit_value:num:14.289871,source::reconstructed
|
||||
date::2025-09-16,unit_value:num:14.294772,source::reconstructed
|
||||
date::2025-09-17,unit_value:num:14.273346,source::reconstructed
|
||||
date::2025-09-18,unit_value:num:14.330228,source::reconstructed
|
||||
date::2025-09-19,unit_value:num:14.343644,source::reconstructed
|
||||
date::2025-09-22,unit_value:num:14.393404,source::reconstructed
|
||||
date::2025-09-23,unit_value:num:14.344019,source::reconstructed
|
||||
date::2025-09-24,unit_value:num:14.288225,source::reconstructed
|
||||
date::2025-09-25,unit_value:num:14.213470,source::reconstructed
|
||||
date::2025-09-26,unit_value:num:14.274187,source::reconstructed
|
||||
date::2025-09-29,unit_value:num:14.324456,source::reconstructed
|
||||
date::2025-09-30,unit_value:num:14.386956,source::reconstructed
|
||||
date::2025-10-01,unit_value:num:14.446641,source::reconstructed
|
||||
date::2025-10-02,unit_value:num:14.470308,source::reconstructed
|
||||
date::2025-10-03,unit_value:num:14.504962,source::reconstructed
|
||||
date::2025-10-06,unit_value:num:14.550410,source::reconstructed
|
||||
date::2025-10-07,unit_value:num:14.471277,source::reconstructed
|
||||
date::2025-10-08,unit_value:num:14.545357,source::reconstructed
|
||||
date::2025-10-09,unit_value:num:14.477839,source::reconstructed
|
||||
date::2025-10-10,unit_value:num:14.146409,source::reconstructed
|
||||
date::2025-10-13,unit_value:num:14.346777,source::reconstructed
|
||||
date::2025-10-14,unit_value:num:14.343092,source::reconstructed
|
||||
date::2025-10-15,unit_value:num:14.420741,source::reconstructed
|
||||
date::2025-10-16,unit_value:num:14.388088,source::reconstructed
|
||||
date::2025-10-17,unit_value:num:14.416659,source::reconstructed
|
||||
date::2025-10-20,unit_value:num:14.555014,source::reconstructed
|
||||
date::2025-10-21,unit_value:num:14.518406,source::reconstructed
|
||||
date::2025-10-22,unit_value:num:14.462816,source::reconstructed
|
||||
date::2025-10-23,unit_value:num:14.546789,source::reconstructed
|
||||
date::2025-10-24,unit_value:num:14.628242,source::reconstructed
|
||||
date::2025-10-27,unit_value:num:14.760399,source::reconstructed
|
||||
date::2025-10-28,unit_value:num:14.765746,source::reconstructed
|
||||
date::2025-10-29,unit_value:num:14.724087,source::reconstructed
|
||||
date::2025-10-30,unit_value:num:14.620311,source::reconstructed
|
||||
date::2025-10-31,unit_value:num:14.643334,source::reconstructed
|
||||
date::2025-11-03,unit_value:num:14.668746,source::reconstructed
|
||||
date::2025-11-04,unit_value:num:14.499908,source::reconstructed
|
||||
date::2025-11-05,unit_value:num:14.559941,source::reconstructed
|
||||
date::2025-11-06,unit_value:num:14.454720,source::reconstructed
|
||||
date::2025-11-07,unit_value:num:14.477632,source::reconstructed
|
||||
date::2025-11-10,unit_value:num:14.659409,source::reconstructed
|
||||
date::2025-11-11,unit_value:num:14.701518,source::reconstructed
|
||||
date::2025-11-12,unit_value:num:14.732192,source::reconstructed
|
||||
date::2025-11-13,unit_value:num:14.535632,source::reconstructed
|
||||
date::2025-11-14,unit_value:num:14.517073,source::reconstructed
|
||||
date::2025-11-17,unit_value:num:14.382078,source::reconstructed
|
||||
date::2025-11-18,unit_value:num:14.276407,source::reconstructed
|
||||
date::2025-11-19,unit_value:num:14.287377,source::reconstructed
|
||||
date::2025-11-20,unit_value:num:14.098396,source::reconstructed
|
||||
date::2025-11-21,unit_value:num:14.236234,source::reconstructed
|
||||
date::2025-11-24,unit_value:num:14.382828,source::reconstructed
|
||||
date::2025-11-25,unit_value:num:14.511125,source::reconstructed
|
||||
date::2025-11-26,unit_value:num:14.622756,source::reconstructed
|
||||
date::2025-11-28,unit_value:num:14.687002,source::reconstructed
|
||||
date::2025-12-01,unit_value:num:14.617508,source::reconstructed
|
||||
date::2025-12-02,unit_value:num:14.651172,source::reconstructed
|
||||
date::2025-12-03,unit_value:num:14.702234,source::reconstructed
|
||||
date::2025-12-04,unit_value:num:14.723319,source::reconstructed
|
||||
date::2025-12-05,unit_value:num:14.743820,source::reconstructed
|
||||
date::2025-12-08,unit_value:num:14.705750,source::reconstructed
|
||||
date::2025-12-09,unit_value:num:14.685344,source::reconstructed
|
||||
date::2025-12-10,unit_value:num:14.809365,source::reconstructed
|
||||
date::2025-12-11,unit_value:num:14.843659,source::reconstructed
|
||||
date::2025-12-12,unit_value:num:14.719397,source::reconstructed
|
||||
date::2025-12-15,unit_value:num:14.718653,source::reconstructed
|
||||
date::2025-12-16,unit_value:num:14.665433,source::reconstructed
|
||||
date::2025-12-17,unit_value:num:14.544934,source::reconstructed
|
||||
date::2025-12-18,unit_value:num:14.650613,source::reconstructed
|
||||
date::2025-12-19,unit_value:num:14.750028,source::reconstructed
|
||||
date::2025-12-22,unit_value:num:14.837297,source::reconstructed
|
||||
date::2025-12-23,unit_value:num:14.901234,source::reconstructed
|
||||
date::2025-12-24,unit_value:num:14.930142,source::reconstructed
|
||||
date::2025-12-26,unit_value:num:14.939776,source::reconstructed
|
||||
date::2025-12-29,unit_value:num:14.898242,source::reconstructed
|
||||
date::2025-12-30,unit_value:num:14.889493,source::reconstructed
|
||||
date::2025-12-31,unit_value:num:14.817161,source::reconstructed
|
||||
date::2026-01-02,unit_value:num:14.908520,source::reconstructed
|
||||
date::2026-01-05,unit_value:num:15.024251,source::reconstructed
|
||||
date::2026-01-06,unit_value:num:15.103740,source::reconstructed
|
||||
date::2026-01-07,unit_value:num:15.053100,source::reconstructed
|
||||
date::2026-01-08,unit_value:num:15.055209,source::reconstructed
|
||||
date::2026-01-09,unit_value:num:15.149296,source::reconstructed
|
||||
date::2026-01-12,unit_value:num:15.206728,source::reconstructed
|
||||
date::2026-01-13,unit_value:num:15.171925,source::anchor
|
||||
date::2026-01-14,unit_value:num:15.153757,source::reconstructed
|
||||
date::2026-01-15,unit_value:num:15.194516,source::reconstructed
|
||||
date::2026-01-16,unit_value:num:15.187513,source::reconstructed
|
||||
date::2026-01-20,unit_value:num:14.957905,source::reconstructed
|
||||
date::2026-01-21,unit_value:num:15.103138,source::reconstructed
|
||||
date::2026-01-22,unit_value:num:15.193451,source::reconstructed
|
||||
date::2026-01-23,unit_value:num:15.213820,source::reconstructed
|
||||
date::2026-01-26,unit_value:num:15.271352,source::reconstructed
|
||||
date::2026-01-27,unit_value:num:15.381674,source::reconstructed
|
||||
date::2026-01-28,unit_value:num:15.365652,source::reconstructed
|
||||
date::2026-01-29,unit_value:num:15.359475,source::reconstructed
|
||||
date::2026-01-30,unit_value:num:15.239369,source::reconstructed
|
||||
date::2026-02-02,unit_value:num:15.295365,source::reconstructed
|
||||
date::2026-02-03,unit_value:num:15.261150,source::reconstructed
|
||||
date::2026-02-04,unit_value:num:15.214489,source::reconstructed
|
||||
date::2026-02-05,unit_value:num:15.055064,source::reconstructed
|
||||
date::2026-02-06,unit_value:num:15.337182,source::reconstructed
|
||||
date::2026-02-09,unit_value:num:15.456531,source::reconstructed
|
||||
date::2026-02-10,unit_value:num:15.445643,source::reconstructed
|
||||
date::2026-02-11,unit_value:num:15.477171,source::reconstructed
|
||||
date::2026-02-12,unit_value:num:15.299676,source::reconstructed
|
||||
date::2026-02-13,unit_value:num:15.332604,source::reconstructed
|
||||
date::2026-02-17,unit_value:num:15.336162,source::reconstructed
|
||||
date::2026-02-18,unit_value:num:15.399612,source::reconstructed
|
||||
date::2026-02-19,unit_value:num:15.375956,source::reconstructed
|
||||
date::2026-02-20,unit_value:num:15.480397,source::reconstructed
|
||||
date::2026-02-23,unit_value:num:15.360501,source::reconstructed
|
||||
date::2026-02-24,unit_value:num:15.463422,source::reconstructed
|
||||
date::2026-02-25,unit_value:num:15.574644,source::reconstructed
|
||||
date::2026-02-26,unit_value:num:15.534871,source::reconstructed
|
||||
date::2026-02-27,unit_value:num:15.500558,source::reconstructed
|
||||
date::2026-03-02,unit_value:num:15.423861,source::reconstructed
|
||||
date::2026-03-03,unit_value:num:15.144422,source::reconstructed
|
||||
date::2026-03-04,unit_value:num:15.224831,source::reconstructed
|
||||
date::2026-03-05,unit_value:num:15.095671,source::reconstructed
|
||||
date::2026-03-06,unit_value:num:14.934456,source::reconstructed
|
||||
date::2026-03-09,unit_value:num:15.038139,source::reconstructed
|
||||
date::2026-03-10,unit_value:num:15.046633,source::reconstructed
|
||||
date::2026-03-11,unit_value:num:15.025473,source::reconstructed
|
||||
date::2026-03-12,unit_value:num:14.783717,source::reconstructed
|
||||
date::2026-03-13,unit_value:num:14.690459,source::reconstructed
|
||||
date::2026-03-16,unit_value:num:14.869770,source::reconstructed
|
||||
date::2026-03-17,unit_value:num:14.919104,source::reconstructed
|
||||
date::2026-03-18,unit_value:num:14.726944,source::reconstructed
|
||||
date::2026-03-19,unit_value:num:14.694405,source::reconstructed
|
||||
date::2026-03-20,unit_value:num:14.413198,source::reconstructed
|
||||
date::2026-03-23,unit_value:num:14.606317,source::reconstructed
|
||||
date::2026-03-24,unit_value:num:14.557789,source::reconstructed
|
||||
date::2026-03-25,unit_value:num:14.689219,source::reconstructed
|
||||
date::2026-03-26,unit_value:num:14.422602,source::reconstructed
|
||||
date::2026-03-27,unit_value:num:14.251436,source::reconstructed
|
||||
date::2026-03-30,unit_value:num:14.215311,source::anchor
|
||||
date::2026-03-31,unit_value:num:14.585971,source::reconstructed
|
||||
date::2026-04-01,unit_value:num:14.726733,source::reconstructed
|
||||
date::2026-04-02,unit_value:num:14.707277,source::reconstructed
|
||||
date::2026-04-06,unit_value:num:14.764362,source::reconstructed
|
||||
date::2026-04-07,unit_value:num:14.776531,source::reconstructed
|
||||
date::2026-04-08,unit_value:num:15.202160,source::reconstructed
|
||||
date::2026-04-09,unit_value:num:15.241825,source::reconstructed
|
||||
date::2026-04-10,unit_value:num:15.234611,source::reconstructed
|
||||
date::2026-04-13,unit_value:num:15.366422,source::reconstructed
|
||||
date::2026-04-14,unit_value:num:15.507634,source::reconstructed
|
||||
date::2026-04-15,unit_value:num:15.571996,source::reconstructed
|
||||
date::2026-04-16,unit_value:num:15.594577,source::reconstructed
|
||||
date::2026-04-17,unit_value:num:15.770150,source::reconstructed
|
||||
date::2026-04-20,unit_value:num:15.739085,source::reconstructed
|
||||
date::2026-04-21,unit_value:num:15.600610,source::reconstructed
|
||||
date::2026-04-22,unit_value:num:15.711620,source::reconstructed
|
||||
date::2026-04-23,unit_value:num:15.626630,source::reconstructed
|
||||
date::2026-04-24,unit_value:num:15.727205,source::reconstructed
|
||||
date::2026-04-27,unit_value:num:15.723471,source::reconstructed
|
||||
date::2026-04-28,unit_value:num:15.649459,source::reconstructed
|
||||
date::2026-04-29,unit_value:num:15.608473,source::reconstructed
|
||||
date::2026-04-30,unit_value:num:15.808101,source::reconstructed
|
||||
date::2026-05-01,unit_value:num:15.820850,source::reconstructed
|
||||
date::2026-05-04,unit_value:num:15.762539,source::reconstructed
|
||||
date::2026-05-05,unit_value:num:15.897593,source::reconstructed
|
||||
date::2026-05-06,unit_value:num:16.168536,source::reconstructed
|
||||
date::2026-05-07,unit_value:num:16.060920,source::reconstructed
|
||||
date::2026-05-08,unit_value:num:16.169179,source::reconstructed
|
||||
date::2026-05-11,unit_value:num:16.188926,source::anchor
|
||||
date::2026-05-12,unit_value:num:16.103611,source::reconstructed
|
||||
date::2026-05-13,unit_value:num:16.194894,source::reconstructed
|
||||
date::2026-05-14,unit_value:num:16.258216,source::reconstructed
|
||||
date::2026-05-15,unit_value:num:16.005294,source::reconstructed
|
||||
date::2026-05-18,unit_value:num:16.020747,source::reconstructed
|
||||
date::2026-05-19,unit_value:num:15.912003,source::reconstructed
|
||||
date::2026-05-20,unit_value:num:16.095253,source::reconstructed
|
||||
date::2026-05-21,unit_value:num:16.159888,source::reconstructed
|
||||
date::2026-05-22,unit_value:num:16.199847,source::reconstructed
|
||||
date::2026-05-26,unit_value:num:16.365372,source::reconstructed
|
||||
date::2026-05-27,unit_value:num:16.352154,source::reconstructed
|
||||
date::2026-05-28,unit_value:num:16.420148,source::reconstructed
|
||||
date::2026-05-29,unit_value:num:16.454750,source::reconstructed
|
||||
date::2026-06-01,unit_value:num:16.501828,source::reconstructed
|
||||
date::2026-06-02,unit_value:num:16.558631,source::reconstructed
|
||||
date::2026-06-03,unit_value:num:16.440935,source::reconstructed
|
||||
date::2026-06-04,unit_value:num:16.488210,source::reconstructed
|
||||
date::2026-06-05,unit_value:num:16.039356,source::reconstructed
|
||||
date::2026-06-08,unit_value:num:16.087422,source::reconstructed
|
||||
date::2026-06-09,unit_value:num:16.092485,source::reconstructed
|
||||
date::2026-06-10,unit_value:num:15.862736,source::reconstructed
|
||||
date::2026-06-11,unit_value:num:16.203277,source::reconstructed
|
||||
date::2026-06-12,unit_value:num:16.285155,source::reconstructed
|
||||
date::2026-06-15,unit_value:num:16.508209,source::reconstructed
|
||||
date::2026-06-16,unit_value:num:16.431629,source::reconstructed
|
||||
date::2026-06-17,unit_value:num:16.291137,source::reconstructed
|
||||
date::2026-06-18,unit_value:num:16.470192,source::reconstructed
|
||||
date::2026-06-22,unit_value:num:16.444424,source::reconstructed
|
||||
date::2026-06-23,unit_value:num:16.155012,source::reconstructed
|
||||
date::2026-06-24,unit_value:num:16.157727,source::reconstructed
|
||||
date::2026-06-25,unit_value:num:16.207671,source::reconstructed
|
||||
date::2026-06-26,unit_value:num:16.170886,source::reconstructed
|
||||
date::2026-06-29,unit_value:num:16.320983,source::reconstructed
|
||||
date::2026-06-30,unit_value:num:16.419623,source::reconstructed
|
||||
date::2026-07-01,unit_value:num:16.313538,source::reconstructed
|
||||
date::2026-07-02,unit_value:num:16.322848,source::reconstructed
|
||||
date::2026-07-06,unit_value:num:16.491716,source::reconstructed
|
||||
date::2026-07-07,unit_value:num:16.338366,source::reconstructed
|
||||
date::2026-07-08,unit_value:num:16.286293,source::reconstructed
|
||||
date::2026-07-09,unit_value:num:16.394467,source::reconstructed
|
||||
date::2026-07-10,unit_value:num:16.441663,source::reconstructed
|
||||
date::2026-07-13,unit_value:num:16.268134,source::reconstructed
|
||||
date::2026-07-14,unit_value:num:16.356302,source::reconstructed
|
||||
date::2026-07-15,unit_value:num:16.420538,source::reconstructed
|
||||
date::2026-07-16,unit_value:num:16.320064,source::reconstructed
|
||||
date::2026-07-17,unit_value:num:16.194060,source::reconstructed
|
||||
date::2026-07-20,unit_value:num:16.132974,source::reconstructed
|
||||
date::2026-07-21,unit_value:num:16.313828,source::reconstructed
|
||||
date::2026-07-22,unit_value:num:16.292595,source::reconstructed
|
||||
date::2026-07-23,unit_value:num:16.141155,source::reconstructed
|
||||
date::2026-07-24,unit_value:num:16.115249,source::reconstructed
|
||||
date::2026-07-27,unit_value:num:16.155028,source::reconstructed
|
||||
date::2026-07-28,unit_value:num:16.136337,source::reconstructed
|
||||
date::2026-07-29,unit_value:num:15.950959,source::reconstructed
|
||||
date::2026-07-30,unit_value:num:16.241833,source::anchor
|
||||
date::2026-07-31,unit_value:num:16.296212,source::reconstructed
|
||||
date::2026-08-03,unit_value:num:16.468808,source::reconstructed
|
||||
date::2026-08-04,unit_value:num:16.715958,source::reconstructed
|
||||
date::2026-08-05,unit_value:num:16.695028,source::reconstructed
|
||||
date::2026-08-06,unit_value:num:16.655095,source::reconstructed
|
||||
date::2026-08-07,unit_value:num:16.781024,source::reconstructed
|
||||
date::2026-08-10,unit_value:num:16.748694,source::reconstructed
|
||||
date::2026-08-11,unit_value:num:16.729451,source::reconstructed
|
||||
date::2026-08-12,unit_value:num:16.798012,source::reconstructed
|
||||
date::2026-08-13,unit_value:num:16.878408,source::reconstructed
|
||||
date::2026-08-14,unit_value:num:16.860847,source::reconstructed
|
||||
date::2026-08-17,unit_value:num:16.824359,source::reconstructed
|
||||
date::2026-08-18,unit_value:num:16.665155,source::reconstructed
|
||||
date::2026-08-19,unit_value:num:16.715407,source::reconstructed
|
||||
date::2026-08-20,unit_value:num:16.633839,source::reconstructed
|
||||
date::2026-08-21,unit_value:num:16.732537,source::reconstructed
|
||||
date::2026-08-24,unit_value:num:16.667659,source::reconstructed
|
||||
date::2026-08-25,unit_value:num:16.757465,source::reconstructed
|
||||
date::2026-08-26,unit_value:num:16.743025,source::reconstructed
|
||||
date::2026-08-27,unit_value:num:16.808106,source::anchor
|
||||
2021
data/ORCBI.srf
Normal file
2021
data/ORCBI.srf
Normal file
File diff suppressed because it is too large
Load diff
49
data/anchors.srf
Normal file
49
data/anchors.srf
Normal file
|
|
@ -0,0 +1,49 @@
|
|||
#!srfv1
|
||||
# Exact, independently-sourced unit values for the Oregon College Savings Plan
|
||||
# (Embark) portfolios. These are the ONLY observed values in this project;
|
||||
# everything in data/ORCBI.srf and data/ORC42.srf between them is derived.
|
||||
#
|
||||
# Keep this file. The wayback-sourced rows are not otherwise recoverable if the
|
||||
# Internet Archive prunes those captures, and there is no public historical NAV
|
||||
# endpoint for this plan.
|
||||
#
|
||||
# source::launch
|
||||
# Every portfolio in the plan launched at exactly $10.00. Verified by
|
||||
# back-solving the live feed's `inception` field, which is the ANNUALIZED
|
||||
# since-inception return: nav / (1 + inception)^years lands between 10.0005
|
||||
# and 10.0038 for all 37 priced portfolios in the feed. The Plan Disclosure
|
||||
# Booklet corroborates the date: "The Portfolios commenced operations in
|
||||
# September 2018."
|
||||
#
|
||||
# source::wayback
|
||||
# Captures of https://marcom.vestwell.com/data/fund-performance/OR-CSP.json
|
||||
# The `evidence` field is the Internet Archive timestamp; fetch with
|
||||
# https://web.archive.org/web/<evidence>id_/<url>
|
||||
# `date` is the feed's own navDate, not the capture date. The feed publishes
|
||||
# the prior business day's value, so navDate lags the capture by ~1 day.
|
||||
#
|
||||
# source::feed
|
||||
# Read live from the same URL.
|
||||
#
|
||||
# ORCBI is feed fund id T6GF10080002 ("Balanced Index", a static portfolio).
|
||||
# ORC42 is feed fund id T6GFRD204202 ("Enrollment Year 2042", a glidepath
|
||||
# portfolio).
|
||||
symbol::ORCBI,date::2018-09-01,unit_value:num:10.0,source::launch,evidence::plan inception - every portfolio launched at 10.00
|
||||
symbol::ORCBI,date::2025-02-13,unit_value:num:15.529444,source::wayback,evidence::20250215060344
|
||||
symbol::ORCBI,date::2025-04-21,unit_value:num:14.584891,source::wayback,evidence::20250422234815
|
||||
symbol::ORCBI,date::2025-08-27,unit_value:num:16.694905,source::wayback,evidence::20250828135705
|
||||
symbol::ORCBI,date::2026-01-13,unit_value:num:17.796687,source::wayback,evidence::20260115021958
|
||||
symbol::ORCBI,date::2026-03-30,unit_value:num:17.010366,source::wayback,evidence::20260331154656
|
||||
symbol::ORCBI,date::2026-05-11,unit_value:num:18.615628,source::wayback,evidence::20260512235157
|
||||
symbol::ORCBI,date::2026-07-30,unit_value:num:18.616344,source::wayback,evidence::20260802074514
|
||||
symbol::ORCBI,date::2026-08-27,unit_value:num:19.092721,source::feed,evidence::read 2026-08-28
|
||||
|
||||
symbol::ORC42,date::2023-07-01,unit_value:num:10.0,source::launch,evidence::portfolio inception - launched at 10.00
|
||||
symbol::ORC42,date::2025-02-13,unit_value:num:12.787304,source::wayback,evidence::20250215060344
|
||||
symbol::ORC42,date::2025-04-21,unit_value:num:11.570209,source::wayback,evidence::20250422234815
|
||||
symbol::ORC42,date::2025-08-27,unit_value:num:13.958343,source::wayback,evidence::20250828135705
|
||||
symbol::ORC42,date::2026-01-13,unit_value:num:15.171925,source::wayback,evidence::20260115021958
|
||||
symbol::ORC42,date::2026-03-30,unit_value:num:14.215311,source::wayback,evidence::20260331154656
|
||||
symbol::ORC42,date::2026-05-11,unit_value:num:16.188926,source::wayback,evidence::20260512235157
|
||||
symbol::ORC42,date::2026-07-30,unit_value:num:16.241833,source::wayback,evidence::20260802074514
|
||||
symbol::ORC42,date::2026-08-27,unit_value:num:16.808106,source::feed,evidence::read 2026-08-28
|
||||
232
data/model.srf
Normal file
232
data/model.srf
Normal file
|
|
@ -0,0 +1,232 @@
|
|||
#!srfv1
|
||||
# Underlying-fund weights for each plan portfolio, by era.
|
||||
#
|
||||
# GENERATED by tools/gen_model.py from the Plan Disclosure Booklet. Regenerate
|
||||
# rather than hand-editing the projected rows; 21 eras is too many to retype
|
||||
# reliably. Hand edits to the `fitted` rows are fine, but keep them in sync.
|
||||
#
|
||||
# These portfolios are unitized fund-of-funds trusts, not mutual funds. The Plan
|
||||
# Disclosure Booklet is explicit that they "reflect changes in value from income
|
||||
# and gains and losses on the sale of the Underlying Funds solely by increasing
|
||||
# or decreasing their Unit Value" -- all income compounds into the unit value and
|
||||
# nothing is distributed. That is why the reconstruction uses each underlying
|
||||
# fund's dividend-adjusted (total-return) close, not its raw NAV.
|
||||
#
|
||||
# Source of the weights: the Plan Disclosure Booklet allocation tables at
|
||||
# https://marcom.vestwell.com/program-description/oregon-college.html
|
||||
# Corroborated for the current era by two live JSON endpoints that agree exactly:
|
||||
# https://vss-api.vestwell.com/plans/oregon-college/portfolios (array form)
|
||||
# https://vss-api.vestwell.com/plans/oregon-college (object form)
|
||||
#
|
||||
# `era_start` is inclusive: a weight row applies to every trading day >= its
|
||||
# era_start, until superseded by a later era for the same symbol.
|
||||
#
|
||||
# `basis` records how much a row is worth trusting:
|
||||
# fitted In effect now or in the past, and validated -- the reconstruction
|
||||
# reproduces every observed anchor to within 0.08% using it.
|
||||
# projected Not yet in effect. Read off the booklet forward table, so it is
|
||||
# the plan's published intent, NOT an observed fact. The booklet
|
||||
# reserves the right to "change the asset allocations ... and change
|
||||
# the selection of Underlying Funds", so confirm before relying on
|
||||
# one. `verify` flags any projected era that has become current.
|
||||
#
|
||||
# NOTE ON FEES: no fee or drag term appears here, and none is needed. The
|
||||
# reconstruction pins the series at both ends of every anchor gap and distributes
|
||||
# the residual geometrically, which absorbs the asset-based fee, cash drag,
|
||||
# trade-date lag and securities-lending income together. `verify` reports the
|
||||
# drag each gap implies, as a cross-check against the booklet's published cost
|
||||
# table ($23.40 per $10,000/yr = 0.234% for ORCBI, $23.91 = 0.239% for ORC42).
|
||||
|
||||
# ORCBI -- "Balanced Index", a STATIC portfolio. The booklet: "Static Portfolio
|
||||
# investments remain fixed, subject to periodic rebalancing". No glidepath, so
|
||||
# one era covers its whole life. These weights change only by Board action.
|
||||
symbol::ORCBI,era_start::2018-09-01,basis::fitted,ticker::VSMPX,weight:num:36
|
||||
symbol::ORCBI,era_start::2018-09-01,basis::fitted,ticker::VTPSX,weight:num:24
|
||||
symbol::ORCBI,era_start::2018-09-01,basis::fitted,ticker::VBMPX,weight:num:40
|
||||
|
||||
# ORC42 -- "College Enrollment Year 2042", a GLIDEPATH portfolio.
|
||||
#
|
||||
# The booklet publishes a full cross-section of every enrollment-year portfolio
|
||||
# as of 2026-07-01. Because all of them follow one glidepath keyed on
|
||||
# years-to-enrollment, that cross-section IS ORC42's forward path shifted in
|
||||
# time: ORC42 in calendar year Y holds what enrollment-year row E = 4068 - Y
|
||||
# holds today. Y=2026 gives E=2042, which matches ORC42's fitted current era, so
|
||||
# the mapping checks out against observed data at the one point where it can.
|
||||
#
|
||||
# Two approximations in the projected rows:
|
||||
# 1. The booklet table is annual, but the booklet also says allocations step
|
||||
# QUARTERLY. So each projected era is up to three quarters coarse, and the
|
||||
# intermediate quarterly steps are not published anywhere.
|
||||
# 2. Era boundaries sit on 1 January. The current boundary was fitted to
|
||||
# 2026-01-01 by grid search over quarter boundaries; every candidate from
|
||||
# 2025-10-01 to 2026-04-01 gives rms <= 0.041%, so the fit cannot resolve
|
||||
# it better than that.
|
||||
#
|
||||
# Neither approximation matters much. Future eras cannot affect a reconstruction
|
||||
# that stops at today, and once daily feed values are appended to anchors.srf
|
||||
# every gap is one session, where pinning makes the weights literally inert (see
|
||||
# the test "weights are inert when anchors are one session apart"). They are
|
||||
# prefilled so the model degrades gracefully instead of silently using 2026
|
||||
# weights in 2035 if forward recording ever lapses.
|
||||
#
|
||||
# Era 1 is the booklet's max-equity plateau (its 2043/2044/2045 rows). ORC42
|
||||
# launched 19 years out from enrollment, so it sat on the plateau from
|
||||
# inception.
|
||||
symbol::ORC42,era_start::2023-07-01,basis::fitted,ticker::VSMPX,weight:num:54
|
||||
symbol::ORC42,era_start::2023-07-01,basis::fitted,ticker::VTPSX,weight:num:36
|
||||
symbol::ORC42,era_start::2023-07-01,basis::fitted,ticker::VIPIX,weight:num:1.67
|
||||
symbol::ORC42,era_start::2023-07-01,basis::fitted,ticker::VBMPX,weight:num:6.66
|
||||
symbol::ORC42,era_start::2023-07-01,basis::fitted,ticker::VTIFX,weight:num:1.67
|
||||
|
||||
# Era 2 is the booklet current 2042 row. Validated: the reconstruction
|
||||
# reproduces all nine ORC42 anchors to within 0.06% using it.
|
||||
symbol::ORC42,era_start::2026-01-01,basis::fitted,ticker::VSMPX,weight:num:52.8
|
||||
symbol::ORC42,era_start::2026-01-01,basis::fitted,ticker::VTPSX,weight:num:35.2
|
||||
symbol::ORC42,era_start::2026-01-01,basis::fitted,ticker::VIPIX,weight:num:2
|
||||
symbol::ORC42,era_start::2026-01-01,basis::fitted,ticker::VBMPX,weight:num:8
|
||||
symbol::ORC42,era_start::2026-01-01,basis::fitted,ticker::VTIFX,weight:num:2
|
||||
|
||||
# Projected: calendar 2027 takes the booklet enrollment-year 2041 row.
|
||||
symbol::ORC42,era_start::2027-01-01,basis::projected,ticker::VSMPX,weight:num:51.6
|
||||
symbol::ORC42,era_start::2027-01-01,basis::projected,ticker::VTPSX,weight:num:34.4
|
||||
symbol::ORC42,era_start::2027-01-01,basis::projected,ticker::VIPIX,weight:num:2.33
|
||||
symbol::ORC42,era_start::2027-01-01,basis::projected,ticker::VBMPX,weight:num:9.34
|
||||
symbol::ORC42,era_start::2027-01-01,basis::projected,ticker::VTIFX,weight:num:2.33
|
||||
|
||||
# Projected: calendar 2028 takes the booklet enrollment-year 2040 row.
|
||||
symbol::ORC42,era_start::2028-01-01,basis::projected,ticker::VSMPX,weight:num:49.8
|
||||
symbol::ORC42,era_start::2028-01-01,basis::projected,ticker::VTPSX,weight:num:33.2
|
||||
symbol::ORC42,era_start::2028-01-01,basis::projected,ticker::VIPIX,weight:num:2.83
|
||||
symbol::ORC42,era_start::2028-01-01,basis::projected,ticker::VBMPX,weight:num:11.34
|
||||
symbol::ORC42,era_start::2028-01-01,basis::projected,ticker::VTIFX,weight:num:2.83
|
||||
|
||||
# Projected: calendar 2029 takes the booklet enrollment-year 2039 row.
|
||||
symbol::ORC42,era_start::2029-01-01,basis::projected,ticker::VSMPX,weight:num:48
|
||||
symbol::ORC42,era_start::2029-01-01,basis::projected,ticker::VTPSX,weight:num:32
|
||||
symbol::ORC42,era_start::2029-01-01,basis::projected,ticker::VIPIX,weight:num:3.33
|
||||
symbol::ORC42,era_start::2029-01-01,basis::projected,ticker::VBMPX,weight:num:13.34
|
||||
symbol::ORC42,era_start::2029-01-01,basis::projected,ticker::VTIFX,weight:num:3.33
|
||||
|
||||
# Projected: calendar 2030 takes the booklet enrollment-year 2038 row.
|
||||
symbol::ORC42,era_start::2030-01-01,basis::projected,ticker::VSMPX,weight:num:45.6
|
||||
symbol::ORC42,era_start::2030-01-01,basis::projected,ticker::VTPSX,weight:num:30.4
|
||||
symbol::ORC42,era_start::2030-01-01,basis::projected,ticker::VIPIX,weight:num:4
|
||||
symbol::ORC42,era_start::2030-01-01,basis::projected,ticker::VBMPX,weight:num:16
|
||||
symbol::ORC42,era_start::2030-01-01,basis::projected,ticker::VTIFX,weight:num:4
|
||||
|
||||
# Projected: calendar 2031 takes the booklet enrollment-year 2037 row.
|
||||
symbol::ORC42,era_start::2031-01-01,basis::projected,ticker::VSMPX,weight:num:43.2
|
||||
symbol::ORC42,era_start::2031-01-01,basis::projected,ticker::VTPSX,weight:num:28.8
|
||||
symbol::ORC42,era_start::2031-01-01,basis::projected,ticker::VIPIX,weight:num:4.67
|
||||
symbol::ORC42,era_start::2031-01-01,basis::projected,ticker::VBMPX,weight:num:18.66
|
||||
symbol::ORC42,era_start::2031-01-01,basis::projected,ticker::VTIFX,weight:num:4.67
|
||||
|
||||
# Projected: calendar 2032 takes the booklet enrollment-year 2036 row.
|
||||
symbol::ORC42,era_start::2032-01-01,basis::projected,ticker::VSMPX,weight:num:41.16
|
||||
symbol::ORC42,era_start::2032-01-01,basis::projected,ticker::VTPSX,weight:num:27.44
|
||||
symbol::ORC42,era_start::2032-01-01,basis::projected,ticker::VIPIX,weight:num:5.23
|
||||
symbol::ORC42,era_start::2032-01-01,basis::projected,ticker::VBMPX,weight:num:20.94
|
||||
symbol::ORC42,era_start::2032-01-01,basis::projected,ticker::VTIFX,weight:num:5.23
|
||||
|
||||
# Projected: calendar 2033 takes the booklet enrollment-year 2035 row.
|
||||
symbol::ORC42,era_start::2033-01-01,basis::projected,ticker::VSMPX,weight:num:39.12
|
||||
symbol::ORC42,era_start::2033-01-01,basis::projected,ticker::VTPSX,weight:num:26.08
|
||||
symbol::ORC42,era_start::2033-01-01,basis::projected,ticker::VIPIX,weight:num:5.8
|
||||
symbol::ORC42,era_start::2033-01-01,basis::projected,ticker::VBMPX,weight:num:23.2
|
||||
symbol::ORC42,era_start::2033-01-01,basis::projected,ticker::VTIFX,weight:num:5.8
|
||||
|
||||
# Projected: calendar 2034 takes the booklet enrollment-year 2034 row.
|
||||
symbol::ORC42,era_start::2034-01-01,basis::projected,ticker::VSMPX,weight:num:37.2
|
||||
symbol::ORC42,era_start::2034-01-01,basis::projected,ticker::VTPSX,weight:num:24.8
|
||||
symbol::ORC42,era_start::2034-01-01,basis::projected,ticker::VIPIX,weight:num:6.33
|
||||
symbol::ORC42,era_start::2034-01-01,basis::projected,ticker::VBMPX,weight:num:25.34
|
||||
symbol::ORC42,era_start::2034-01-01,basis::projected,ticker::VTIFX,weight:num:6.33
|
||||
|
||||
# Projected: calendar 2035 takes the booklet enrollment-year 2033 row.
|
||||
symbol::ORC42,era_start::2035-01-01,basis::projected,ticker::VSMPX,weight:num:34.2
|
||||
symbol::ORC42,era_start::2035-01-01,basis::projected,ticker::VTPSX,weight:num:22.8
|
||||
symbol::ORC42,era_start::2035-01-01,basis::projected,ticker::VIPIX,weight:num:7.17
|
||||
symbol::ORC42,era_start::2035-01-01,basis::projected,ticker::VBMPX,weight:num:28.66
|
||||
symbol::ORC42,era_start::2035-01-01,basis::projected,ticker::VTIFX,weight:num:7.17
|
||||
|
||||
# Projected: calendar 2036 takes the booklet enrollment-year 2032 row.
|
||||
symbol::ORC42,era_start::2036-01-01,basis::projected,ticker::VSMPX,weight:num:31.2
|
||||
symbol::ORC42,era_start::2036-01-01,basis::projected,ticker::VTPSX,weight:num:20.8
|
||||
symbol::ORC42,era_start::2036-01-01,basis::projected,ticker::VIPIX,weight:num:8
|
||||
symbol::ORC42,era_start::2036-01-01,basis::projected,ticker::VBMPX,weight:num:32
|
||||
symbol::ORC42,era_start::2036-01-01,basis::projected,ticker::VTIFX,weight:num:8
|
||||
|
||||
# Projected: calendar 2037 takes the booklet enrollment-year 2031 row.
|
||||
symbol::ORC42,era_start::2037-01-01,basis::projected,ticker::VSMPX,weight:num:28.2
|
||||
symbol::ORC42,era_start::2037-01-01,basis::projected,ticker::VTPSX,weight:num:18.8
|
||||
symbol::ORC42,era_start::2037-01-01,basis::projected,ticker::VIPIX,weight:num:8.83
|
||||
symbol::ORC42,era_start::2037-01-01,basis::projected,ticker::VBMPX,weight:num:35.34
|
||||
symbol::ORC42,era_start::2037-01-01,basis::projected,ticker::VTIFX,weight:num:8.83
|
||||
|
||||
# Projected: calendar 2038 takes the booklet enrollment-year 2030 row.
|
||||
symbol::ORC42,era_start::2038-01-01,basis::projected,ticker::VSMPX,weight:num:24.8
|
||||
symbol::ORC42,era_start::2038-01-01,basis::projected,ticker::VTPSX,weight:num:16.53
|
||||
symbol::ORC42,era_start::2038-01-01,basis::projected,ticker::VIPIX,weight:num:7.61
|
||||
symbol::ORC42,era_start::2038-01-01,basis::projected,ticker::VBMPX,weight:num:30.45
|
||||
symbol::ORC42,era_start::2038-01-01,basis::projected,ticker::VTIFX,weight:num:7.61
|
||||
symbol::ORC42,era_start::2038-01-01,basis::projected,ticker::VBIPX,weight:num:7.5
|
||||
symbol::ORC42,era_start::2038-01-01,basis::projected,ticker::VTSPX,weight:num:5.5
|
||||
|
||||
# Projected: calendar 2039 takes the booklet enrollment-year 2029 row.
|
||||
symbol::ORC42,era_start::2039-01-01,basis::projected,ticker::VSMPX,weight:num:20.3
|
||||
symbol::ORC42,era_start::2039-01-01,basis::projected,ticker::VTPSX,weight:num:13.53
|
||||
symbol::ORC42,era_start::2039-01-01,basis::projected,ticker::VIPIX,weight:num:5.78
|
||||
symbol::ORC42,era_start::2039-01-01,basis::projected,ticker::VBMPX,weight:num:23.11
|
||||
symbol::ORC42,era_start::2039-01-01,basis::projected,ticker::VTIFX,weight:num:5.78
|
||||
symbol::ORC42,era_start::2039-01-01,basis::projected,ticker::VBIPX,weight:num:20.5
|
||||
symbol::ORC42,era_start::2039-01-01,basis::projected,ticker::VTSPX,weight:num:11
|
||||
|
||||
# Projected: calendar 2040 takes the booklet enrollment-year 2028 row.
|
||||
symbol::ORC42,era_start::2040-01-01,basis::projected,ticker::VSMPX,weight:num:15
|
||||
symbol::ORC42,era_start::2040-01-01,basis::projected,ticker::VTPSX,weight:num:10
|
||||
symbol::ORC42,era_start::2040-01-01,basis::projected,ticker::VIPIX,weight:num:3.33
|
||||
symbol::ORC42,era_start::2040-01-01,basis::projected,ticker::VBMPX,weight:num:13.34
|
||||
symbol::ORC42,era_start::2040-01-01,basis::projected,ticker::VTIFX,weight:num:3.33
|
||||
symbol::ORC42,era_start::2040-01-01,basis::projected,ticker::VBIPX,weight:num:35
|
||||
symbol::ORC42,era_start::2040-01-01,basis::projected,ticker::VTSPX,weight:num:20
|
||||
|
||||
# Projected: calendar 2041 takes the booklet enrollment-year 2027 row.
|
||||
symbol::ORC42,era_start::2041-01-01,basis::projected,ticker::VSMPX,weight:num:10.5
|
||||
symbol::ORC42,era_start::2041-01-01,basis::projected,ticker::VTPSX,weight:num:7
|
||||
symbol::ORC42,era_start::2041-01-01,basis::projected,ticker::VIPIX,weight:num:0.33
|
||||
symbol::ORC42,era_start::2041-01-01,basis::projected,ticker::VBMPX,weight:num:1.34
|
||||
symbol::ORC42,era_start::2041-01-01,basis::projected,ticker::VTIFX,weight:num:0.33
|
||||
symbol::ORC42,era_start::2041-01-01,basis::projected,ticker::VBIPX,weight:num:50
|
||||
symbol::ORC42,era_start::2041-01-01,basis::projected,ticker::VTSPX,weight:num:30.5
|
||||
|
||||
# Projected: calendar 2042 takes the booklet enrollment-year 2026 row.
|
||||
symbol::ORC42,era_start::2042-01-01,basis::projected,ticker::VSMPX,weight:num:6
|
||||
symbol::ORC42,era_start::2042-01-01,basis::projected,ticker::VTPSX,weight:num:4
|
||||
symbol::ORC42,era_start::2042-01-01,basis::projected,ticker::VBIPX,weight:num:30
|
||||
symbol::ORC42,era_start::2042-01-01,basis::projected,ticker::VTSPX,weight:num:30
|
||||
symbol::ORC42,era_start::2042-01-01,basis::projected,ticker::VUSXX,weight:num:30
|
||||
|
||||
# Projected: calendar 2043 takes the booklet enrollment-year 2025 row.
|
||||
symbol::ORC42,era_start::2043-01-01,basis::projected,ticker::VSMPX,weight:num:3
|
||||
symbol::ORC42,era_start::2043-01-01,basis::projected,ticker::VTPSX,weight:num:2
|
||||
symbol::ORC42,era_start::2043-01-01,basis::projected,ticker::VBIPX,weight:num:23
|
||||
symbol::ORC42,era_start::2043-01-01,basis::projected,ticker::VTSPX,weight:num:23
|
||||
symbol::ORC42,era_start::2043-01-01,basis::projected,ticker::VUSXX,weight:num:49
|
||||
|
||||
# Projected: calendar 2044 takes the booklet enrollment-year 2024 row.
|
||||
symbol::ORC42,era_start::2044-01-01,basis::projected,ticker::VBIPX,weight:num:14.5
|
||||
symbol::ORC42,era_start::2044-01-01,basis::projected,ticker::VTSPX,weight:num:14.5
|
||||
symbol::ORC42,era_start::2044-01-01,basis::projected,ticker::VUSXX,weight:num:71
|
||||
|
||||
# Projected: calendar 2045 takes the booklet enrollment-year 2023 row.
|
||||
symbol::ORC42,era_start::2045-01-01,basis::projected,ticker::VBIPX,weight:num:5
|
||||
symbol::ORC42,era_start::2045-01-01,basis::projected,ticker::VTSPX,weight:num:5
|
||||
symbol::ORC42,era_start::2045-01-01,basis::projected,ticker::VUSXX,weight:num:90
|
||||
|
||||
# Projected: calendar 2046 takes the booklet enrollment-year 2022 row.
|
||||
symbol::ORC42,era_start::2046-01-01,basis::projected,ticker::VBIPX,weight:num:5
|
||||
symbol::ORC42,era_start::2046-01-01,basis::projected,ticker::VTSPX,weight:num:5
|
||||
symbol::ORC42,era_start::2046-01-01,basis::projected,ticker::VUSXX,weight:num:90
|
||||
|
||||
# Projected: calendar 2047 takes the booklet enrollment-year 2021 row.
|
||||
symbol::ORC42,era_start::2047-01-01,basis::projected,ticker::VUSXX,weight:num:100
|
||||
54
data/recorded.srf
Normal file
54
data/recorded.srf
Normal file
|
|
@ -0,0 +1,54 @@
|
|||
#!srfv1
|
||||
# Hand-typed unit values recovered from the finance repo's portfolio snapshots
|
||||
# (history/*-portfolio.srf). Used by `verify` as an INDEPENDENT check: these
|
||||
# values are never fed into the reconstruction, so agreement with them is
|
||||
# evidence the model is right rather than evidence it was fitted.
|
||||
#
|
||||
# `snapshot_date` is the date the value was recorded, NOT the date the value is
|
||||
# for. The operator read the number off the Vestwell portal, which publishes the
|
||||
# prior business day's unit value, so each row is compared against the
|
||||
# reconstruction at the last trading day STRICTLY BEFORE snapshot_date. That
|
||||
# lag=1 alignment was established empirically: it minimises residual variance
|
||||
# across the whole series, and it matches the live feed's own navDate lag.
|
||||
#
|
||||
# Values are rounded to 2dp at the source, which is +/-0.005 of unavoidable
|
||||
# noise (about 0.03% at these levels). Expect agreement at roughly that scale,
|
||||
# not better.
|
||||
#
|
||||
# DELIBERATELY EXCLUDED: the 2026-04-06 snapshot, which carried 18.15 / 15.50.
|
||||
# Those are the lots' open_price from 2026-02-26, still sitting in the price
|
||||
# field because no manual refresh had happened yet. They are not observations of
|
||||
# anything and including them corrupts the comparison.
|
||||
symbol::ORCBI,snapshot_date::2026-04-27,unit_value:num:18.27
|
||||
symbol::ORCBI,snapshot_date::2026-05-04,unit_value:num:18.32
|
||||
symbol::ORCBI,snapshot_date::2026-05-11,unit_value:num:18.61
|
||||
symbol::ORCBI,snapshot_date::2026-05-26,unit_value:num:18.59
|
||||
symbol::ORCBI,snapshot_date::2026-06-01,unit_value:num:18.84
|
||||
symbol::ORCBI,snapshot_date::2026-06-08,unit_value:num:18.47
|
||||
symbol::ORCBI,snapshot_date::2026-06-15,unit_value:num:18.70
|
||||
symbol::ORCBI,snapshot_date::2026-06-22,unit_value:num:18.86
|
||||
symbol::ORCBI,snapshot_date::2026-06-29,unit_value:num:18.64
|
||||
symbol::ORCBI,snapshot_date::2026-07-06,unit_value:num:18.74
|
||||
symbol::ORCBI,snapshot_date::2026-07-20,unit_value:num:18.62
|
||||
symbol::ORCBI,snapshot_date::2026-07-24,unit_value:num:18.52
|
||||
symbol::ORCBI,snapshot_date::2026-08-03,unit_value:num:18.64
|
||||
symbol::ORCBI,snapshot_date::2026-08-10,unit_value:num:19.06
|
||||
symbol::ORCBI,snapshot_date::2026-08-17,unit_value:num:19.12
|
||||
symbol::ORCBI,snapshot_date::2026-08-24,unit_value:num:19.01
|
||||
|
||||
symbol::ORC42,snapshot_date::2026-04-27,unit_value:num:15.73
|
||||
symbol::ORC42,snapshot_date::2026-05-04,unit_value:num:15.82
|
||||
symbol::ORC42,snapshot_date::2026-05-11,unit_value:num:16.17
|
||||
symbol::ORC42,snapshot_date::2026-05-26,unit_value:num:16.20
|
||||
symbol::ORC42,snapshot_date::2026-06-01,unit_value:num:16.45
|
||||
symbol::ORC42,snapshot_date::2026-06-08,unit_value:num:16.04
|
||||
symbol::ORC42,snapshot_date::2026-06-15,unit_value:num:16.28
|
||||
symbol::ORC42,snapshot_date::2026-06-22,unit_value:num:16.47
|
||||
symbol::ORC42,snapshot_date::2026-06-29,unit_value:num:16.17
|
||||
symbol::ORC42,snapshot_date::2026-07-06,unit_value:num:16.32
|
||||
symbol::ORC42,snapshot_date::2026-07-20,unit_value:num:16.19
|
||||
symbol::ORC42,snapshot_date::2026-07-24,unit_value:num:16.11
|
||||
symbol::ORC42,snapshot_date::2026-08-03,unit_value:num:16.29
|
||||
symbol::ORC42,snapshot_date::2026-08-10,unit_value:num:16.78
|
||||
symbol::ORC42,snapshot_date::2026-08-17,unit_value:num:16.86
|
||||
symbol::ORC42,snapshot_date::2026-08-24,unit_value:num:16.73
|
||||
277
src/candles.zig
Normal file
277
src/candles.zig
Normal file
|
|
@ -0,0 +1,277 @@
|
|||
//! Reader for zfin's candle cache.
|
||||
//!
|
||||
//! The reconstruction needs one number per underlying fund per trading day: the
|
||||
//! dividend-adjusted close, which is a total-return series. zfin already
|
||||
//! maintains exactly that, so this module reads it rather than re-fetching from
|
||||
//! a provider.
|
||||
//!
|
||||
//! ## Why adj_close and not close
|
||||
//!
|
||||
//! The plan portfolios are unitized trusts: all income from the underlying funds
|
||||
//! compounds into the unit value and nothing is distributed. The underlying
|
||||
//! funds, being mutual funds, DO distribute -- their raw NAV drops on every
|
||||
//! ex-date. So a raw-NAV basket systematically under-tracks the unit value by
|
||||
//! roughly the blended yield. `adj_close` removes exactly that gap.
|
||||
//!
|
||||
//! ## What re-basing does and does not affect
|
||||
//!
|
||||
//! zfin re-bases its adjusted series whenever it restates adjustments (the
|
||||
//! `adj_basis::` field in `candles_meta.srf`), which rescales every historical
|
||||
//! bar of a fund. That is harmless here: the reconstruction consumes only
|
||||
//! per-day RATIOS of adjusted closes, so a rescaling cancels exactly, even when
|
||||
//! each fund is rescaled by a different factor. `recon.zig` has a test for this
|
||||
//! (`output is invariant to rescaling the input price series`).
|
||||
//!
|
||||
//! What does move the output is a provider adding or restating a dividend, which
|
||||
//! changes the single daily ratio spanning that ex-date.
|
||||
//!
|
||||
//! Either way the series cannot be rebuilt from the repo alone -- this cache is
|
||||
//! external, unversioned, and filled by rate-limited API calls -- which is why
|
||||
//! the generated series is committed rather than regenerated on demand.
|
||||
|
||||
const std = @import("std");
|
||||
const srf = @import("srf");
|
||||
const srf_opts = @import("srf_opts.zig");
|
||||
const civil = @import("civil.zig");
|
||||
|
||||
/// A row of `candles_daily.srf`. Only `date` and `adj_close` are consumed; the
|
||||
/// rest are declared with defaults so a record carrying them coerces cleanly and
|
||||
/// so a future column addition upstream cannot break this reader.
|
||||
const Row = struct {
|
||||
date: []const u8,
|
||||
open: f64 = 0,
|
||||
high: f64 = 0,
|
||||
low: f64 = 0,
|
||||
close: f64 = 0,
|
||||
adj_close: f64 = 0,
|
||||
volume: f64 = 0,
|
||||
};
|
||||
|
||||
/// One underlying fund's total-return series, indexed by ISO date.
|
||||
pub const Ticker = struct {
|
||||
name: []const u8,
|
||||
/// date -> adj_close. Keys are arena-owned copies.
|
||||
adj: std.StringHashMapUnmanaged(f64),
|
||||
/// Ascending, arena-owned. Same keys as `adj`.
|
||||
dates: []const []const u8,
|
||||
|
||||
pub fn at(self: Ticker, date: []const u8) ?f64 {
|
||||
return self.adj.get(date);
|
||||
}
|
||||
};
|
||||
|
||||
/// Resolve zfin's cache directory the same way zfin does:
|
||||
/// `$ZFIN_CACHE_DIR`, else `$XDG_CACHE_HOME/zfin`, else `$HOME/.cache/zfin`.
|
||||
pub fn cacheDir(arena: std.mem.Allocator, env: *const std.process.Environ.Map) ![]const u8 {
|
||||
if (env.get("ZFIN_CACHE_DIR")) |d| {
|
||||
if (d.len > 0) return arena.dupe(u8, d);
|
||||
}
|
||||
if (env.get("XDG_CACHE_HOME")) |d| {
|
||||
if (d.len > 0) return std.fs.path.join(arena, &.{ d, "zfin" });
|
||||
}
|
||||
const home = env.get("HOME") orelse return error.NoHomeDirectory;
|
||||
return std.fs.path.join(arena, &.{ home, ".cache", "zfin" });
|
||||
}
|
||||
|
||||
/// Parse the bytes of a `candles_daily.srf` into a `Ticker`.
|
||||
///
|
||||
/// Public although `load` is the only caller in this module: it is the pure core
|
||||
/// that `load` wraps, and other modules' tests build `Ticker` fixtures from SRF
|
||||
/// text with it rather than touching the filesystem. Keeping the parser
|
||||
/// separable from the I/O is the reason those tests can be hermetic at all.
|
||||
///
|
||||
/// Rows with a malformed date are skipped rather than fatal: an individual bad
|
||||
/// bar should not cost the whole series, and the reconstruction will fail loudly
|
||||
/// later if a date it needs is absent. A duplicate date keeps the last value,
|
||||
/// matching the append-only way the file is written.
|
||||
pub fn parse(arena: std.mem.Allocator, name: []const u8, data: []const u8) !Ticker {
|
||||
var reader = std.Io.Reader.fixed(data);
|
||||
var it = srf.iterator(&reader, arena, .{ .parse_allocator = .none }) catch
|
||||
return error.InvalidCandleFile;
|
||||
defer it.deinit();
|
||||
|
||||
var adj: std.StringHashMapUnmanaged(f64) = .empty;
|
||||
var dates: std.ArrayList([]const u8) = .empty;
|
||||
|
||||
while (try it.next()) |fields| {
|
||||
const row = fields.to(Row, srf_opts.machine_written) catch continue;
|
||||
if (!civil.isValidIso(row.date)) continue;
|
||||
const key = try arena.dupe(u8, row.date);
|
||||
const gop = try adj.getOrPut(arena, key);
|
||||
if (!gop.found_existing) try dates.append(arena, key);
|
||||
gop.value_ptr.* = row.adj_close;
|
||||
}
|
||||
|
||||
const owned = try dates.toOwnedSlice(arena);
|
||||
std.mem.sort([]const u8, owned, {}, struct {
|
||||
fn lt(_: void, a: []const u8, b: []const u8) bool {
|
||||
return civil.lessThan(a, b);
|
||||
}
|
||||
}.lt);
|
||||
|
||||
return .{ .name = name, .adj = adj, .dates = owned };
|
||||
}
|
||||
|
||||
/// Read and parse `<cache_dir>/<name>/candles_daily.srf`.
|
||||
pub fn load(
|
||||
arena: std.mem.Allocator,
|
||||
io: std.Io,
|
||||
cache_root: []const u8,
|
||||
name: []const u8,
|
||||
) !Ticker {
|
||||
const path = try std.fs.path.join(arena, &.{ cache_root, name, "candles_daily.srf" });
|
||||
const data = std.Io.Dir.cwd().readFileAlloc(io, path, arena, .limited(64 * 1024 * 1024)) catch |err| switch (err) {
|
||||
error.FileNotFound => return error.TickerNotCached,
|
||||
else => return err,
|
||||
};
|
||||
return parse(arena, name, data);
|
||||
}
|
||||
|
||||
/// Index of the first calendar entry >= `date`, or null when none is.
|
||||
pub fn firstAtOrAfter(calendar: []const []const u8, date: []const u8) ?usize {
|
||||
for (calendar, 0..) |d, i| {
|
||||
if (!civil.lessThan(d, date)) return i;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
/// Index of the last calendar entry <= `date`, or null when none is.
|
||||
pub fn lastAtOrBefore(calendar: []const []const u8, date: []const u8) ?usize {
|
||||
var i = calendar.len;
|
||||
while (i > 0) {
|
||||
i -= 1;
|
||||
if (!civil.lessThan(date, calendar[i])) return i;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
const testing = std.testing;
|
||||
|
||||
test "parse reads dates and adjusted closes" {
|
||||
var arena_state: std.heap.ArenaAllocator = .init(testing.allocator);
|
||||
defer arena_state.deinit();
|
||||
const arena = arena_state.allocator();
|
||||
|
||||
const data =
|
||||
"#!srfv1\n" ++
|
||||
"date::2026-08-25,open:num:9.55,high:num:9.55,low:num:9.55,close:num:9.55,adj_close:num:9.55,volume:num:0\n" ++
|
||||
"date::2026-08-26,open:num:9.54,high:num:9.54,low:num:9.54,close:num:9.54,adj_close:num:9.54,volume:num:0\n" ++
|
||||
"date::2026-08-27,open:num:9.53,high:num:9.53,low:num:9.53,close:num:9.53,adj_close:num:9.53,volume:num:0\n";
|
||||
|
||||
const t = try parse(arena, "VBMPX", data);
|
||||
try testing.expectEqual(@as(usize, 3), t.dates.len);
|
||||
try testing.expectEqualStrings("VBMPX", t.name);
|
||||
try testing.expectEqualStrings("2026-08-25", t.dates[0]);
|
||||
try testing.expectEqualStrings("2026-08-27", t.dates[2]);
|
||||
try testing.expectEqual(@as(f64, 9.53), t.at("2026-08-27").?);
|
||||
try testing.expectEqual(@as(?f64, null), t.at("2026-08-28"));
|
||||
}
|
||||
|
||||
test "parse prefers adj_close over close" {
|
||||
var arena_state: std.heap.ArenaAllocator = .init(testing.allocator);
|
||||
defer arena_state.deinit();
|
||||
const arena = arena_state.allocator();
|
||||
|
||||
// A back-adjusted bar: close and adj_close differ. Taking `close` here
|
||||
// would reintroduce the distribution gap this whole project exists to
|
||||
// avoid, so assert the adjusted value is the one kept.
|
||||
const data =
|
||||
"#!srfv1\n" ++
|
||||
"date::2016-01-04,open:num:93.79,high:num:93.79,low:num:93.79,close:num:93.79,adj_close:num:79.0001306117,volume:num:0\n";
|
||||
|
||||
const t = try parse(arena, "VSMPX", data);
|
||||
try testing.expectApproxEqAbs(@as(f64, 79.0001306117), t.at("2016-01-04").?, 1e-9);
|
||||
}
|
||||
|
||||
test "parse skips malformed dates but keeps good rows" {
|
||||
var arena_state: std.heap.ArenaAllocator = .init(testing.allocator);
|
||||
defer arena_state.deinit();
|
||||
const arena = arena_state.allocator();
|
||||
|
||||
const data =
|
||||
"#!srfv1\n" ++
|
||||
"date::not-a-date,adj_close:num:1.0\n" ++
|
||||
"date::2026-08-27,adj_close:num:16.5\n";
|
||||
|
||||
const t = try parse(arena, "X", data);
|
||||
try testing.expectEqual(@as(usize, 1), t.dates.len);
|
||||
try testing.expectEqual(@as(f64, 16.5), t.at("2026-08-27").?);
|
||||
}
|
||||
|
||||
test "parse sorts out-of-order rows and dedupes keeping the last value" {
|
||||
var arena_state: std.heap.ArenaAllocator = .init(testing.allocator);
|
||||
defer arena_state.deinit();
|
||||
const arena = arena_state.allocator();
|
||||
|
||||
const data =
|
||||
"#!srfv1\n" ++
|
||||
"date::2026-08-27,adj_close:num:3.0\n" ++
|
||||
"date::2026-08-25,adj_close:num:1.0\n" ++
|
||||
"date::2026-08-26,adj_close:num:2.0\n" ++
|
||||
"date::2026-08-25,adj_close:num:1.5\n";
|
||||
|
||||
const t = try parse(arena, "X", data);
|
||||
try testing.expectEqual(@as(usize, 3), t.dates.len);
|
||||
try testing.expectEqualStrings("2026-08-25", t.dates[0]);
|
||||
try testing.expectEqualStrings("2026-08-26", t.dates[1]);
|
||||
try testing.expectEqualStrings("2026-08-27", t.dates[2]);
|
||||
try testing.expectEqual(@as(f64, 1.5), t.at("2026-08-25").?);
|
||||
}
|
||||
|
||||
test "calendar snapping brackets a non-trading-day date" {
|
||||
const cal: []const []const u8 = &.{ "2018-08-31", "2018-09-04", "2018-09-05" };
|
||||
|
||||
// 2018-09-01 is a Saturday: no bar. Forward-snap reaches the next session.
|
||||
try testing.expectEqual(@as(?usize, 1), firstAtOrAfter(cal, "2018-09-01"));
|
||||
// Backward-snap from the same date reaches the previous session.
|
||||
try testing.expectEqual(@as(?usize, 0), lastAtOrBefore(cal, "2018-09-01"));
|
||||
|
||||
// Exact hits return themselves.
|
||||
try testing.expectEqual(@as(?usize, 1), firstAtOrAfter(cal, "2018-09-04"));
|
||||
try testing.expectEqual(@as(?usize, 1), lastAtOrBefore(cal, "2018-09-04"));
|
||||
|
||||
// Out of range in both directions.
|
||||
try testing.expectEqual(@as(?usize, null), firstAtOrAfter(cal, "2019-01-01"));
|
||||
try testing.expectEqual(@as(?usize, null), lastAtOrBefore(cal, "2000-01-01"));
|
||||
}
|
||||
|
||||
test "cacheDir honours ZFIN_CACHE_DIR, then XDG_CACHE_HOME, then HOME" {
|
||||
var arena_state: std.heap.ArenaAllocator = .init(testing.allocator);
|
||||
defer arena_state.deinit();
|
||||
const arena = arena_state.allocator();
|
||||
|
||||
{
|
||||
var env: std.process.Environ.Map = .init(testing.allocator);
|
||||
defer env.deinit();
|
||||
try env.put("ZFIN_CACHE_DIR", "/explicit");
|
||||
try env.put("XDG_CACHE_HOME", "/xdg");
|
||||
try env.put("HOME", "/home/x");
|
||||
try testing.expectEqualStrings("/explicit", try cacheDir(arena, &env));
|
||||
}
|
||||
{
|
||||
var env: std.process.Environ.Map = .init(testing.allocator);
|
||||
defer env.deinit();
|
||||
try env.put("XDG_CACHE_HOME", "/xdg");
|
||||
try env.put("HOME", "/home/x");
|
||||
try testing.expectEqualStrings("/xdg/zfin", try cacheDir(arena, &env));
|
||||
}
|
||||
{
|
||||
var env: std.process.Environ.Map = .init(testing.allocator);
|
||||
defer env.deinit();
|
||||
try env.put("HOME", "/home/x");
|
||||
try testing.expectEqualStrings("/home/x/.cache/zfin", try cacheDir(arena, &env));
|
||||
}
|
||||
{
|
||||
// An empty value must not win over the next candidate.
|
||||
var env: std.process.Environ.Map = .init(testing.allocator);
|
||||
defer env.deinit();
|
||||
try env.put("ZFIN_CACHE_DIR", "");
|
||||
try env.put("HOME", "/home/x");
|
||||
try testing.expectEqualStrings("/home/x/.cache/zfin", try cacheDir(arena, &env));
|
||||
}
|
||||
{
|
||||
var env: std.process.Environ.Map = .init(testing.allocator);
|
||||
defer env.deinit();
|
||||
try testing.expectError(error.NoHomeDirectory, cacheDir(arena, &env));
|
||||
}
|
||||
}
|
||||
177
src/civil.zig
Normal file
177
src/civil.zig
Normal file
|
|
@ -0,0 +1,177 @@
|
|||
//! Calendar helpers for ISO-8601 date strings.
|
||||
//!
|
||||
//! Dates in this project are carried as `YYYY-MM-DD` strings rather than a
|
||||
//! parsed date type. That is a deliberate simplification: every date that
|
||||
//! matters here comes from an SRF file or a JSON feed already in that form,
|
||||
//! ISO-8601 sorts correctly under plain byte comparison, and the reconstruction
|
||||
//! itself never does calendar arithmetic -- it walks a trading-day calendar
|
||||
//! supplied by the underlying price data.
|
||||
//!
|
||||
//! `daysBetween` exists only for the implied-drag diagnostic in `verify`, which
|
||||
//! annualises a residual and therefore needs real elapsed days.
|
||||
|
||||
const std = @import("std");
|
||||
|
||||
/// True when `s` is exactly `YYYY-MM-DD` with digits in the right places.
|
||||
/// Does not validate that the date exists (no month-length or leap check);
|
||||
/// callers get that for free by looking the date up in a real calendar.
|
||||
pub fn isValidIso(s: []const u8) bool {
|
||||
if (s.len != 10) return false;
|
||||
if (s[4] != '-' or s[7] != '-') return false;
|
||||
for ([_]usize{ 0, 1, 2, 3, 5, 6, 8, 9 }) |i| {
|
||||
if (!std.ascii.isDigit(s[i])) return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
/// Order two ISO dates. ISO-8601 is designed so lexicographic order equals
|
||||
/// chronological order, so this is byte comparison with an intention-revealing
|
||||
/// name.
|
||||
fn order(a: []const u8, b: []const u8) std.math.Order {
|
||||
return std.mem.order(u8, a, b);
|
||||
}
|
||||
|
||||
pub fn lessThan(a: []const u8, b: []const u8) bool {
|
||||
return order(a, b) == .lt;
|
||||
}
|
||||
|
||||
/// Days from the civil epoch (1970-01-01) for an ISO date.
|
||||
///
|
||||
/// Howard Hinnant's `days_from_civil`, which is exact for the proleptic
|
||||
/// Gregorian calendar over the range we care about and needs no lookup tables.
|
||||
fn daysFromIso(s: []const u8) !i64 {
|
||||
if (!isValidIso(s)) return error.InvalidDate;
|
||||
const y = try std.fmt.parseInt(i64, s[0..4], 10);
|
||||
const m = try std.fmt.parseInt(i64, s[5..7], 10);
|
||||
const d = try std.fmt.parseInt(i64, s[8..10], 10);
|
||||
if (m < 1 or m > 12 or d < 1 or d > 31) return error.InvalidDate;
|
||||
|
||||
const yy = y - @as(i64, if (m <= 2) 1 else 0);
|
||||
const era = @divFloor(if (yy >= 0) yy else yy - 399, 400);
|
||||
const yoe = yy - era * 400; // [0, 399]
|
||||
const doy = @divFloor(153 * (m + (if (m > 2) @as(i64, -3) else 9)) + 2, 5) + d - 1;
|
||||
const doe = yoe * 365 + @divFloor(yoe, 4) - @divFloor(yoe, 100) + doy;
|
||||
return era * 146097 + doe - 719468;
|
||||
}
|
||||
|
||||
/// Calendar days from `a` to `b`. Negative when `b` precedes `a`.
|
||||
pub fn daysBetween(a: []const u8, b: []const u8) !i64 {
|
||||
return (try daysFromIso(b)) - (try daysFromIso(a));
|
||||
}
|
||||
|
||||
/// Inverse of `daysFromIso`: render days-since-epoch as `YYYY-MM-DD` into `buf`.
|
||||
///
|
||||
/// Hinnant's `civil_from_days`. Exact over the same range as its inverse.
|
||||
fn isoFromDays(days: i64, buf: *[10]u8) ![]const u8 {
|
||||
const z = days + 719468;
|
||||
const era = @divFloor(if (z >= 0) z else z - 146096, 146097);
|
||||
const doe = z - era * 146097; // [0, 146096]
|
||||
const yoe = @divFloor(doe - @divFloor(doe, 1460) + @divFloor(doe, 36524) - @divFloor(doe, 146096), 365);
|
||||
const y = yoe + era * 400;
|
||||
const doy = doe - (365 * yoe + @divFloor(yoe, 4) - @divFloor(yoe, 100)); // [0, 365]
|
||||
const mp = @divFloor(5 * doy + 2, 153); // [0, 11]
|
||||
const d = doy - @divFloor(153 * mp + 2, 5) + 1; // [1, 31]
|
||||
const m = mp + (if (mp < 10) @as(i64, 3) else -9); // [1, 12]
|
||||
const year = y + @as(i64, if (m <= 2) 1 else 0);
|
||||
if (year < 0 or year > 9999 or m < 1 or m > 12 or d < 1 or d > 31) return error.DateOutOfRange;
|
||||
// Cast to unsigned before formatting: with an explicit fill/width, Zig's
|
||||
// `{d}` emits a sign for signed integers, so an i64 year renders as "+1970"
|
||||
// and the whole date overflows a 10-byte buffer as "+1970-+1-+1".
|
||||
return std.fmt.bufPrint(buf, "{d:0>4}-{d:0>2}-{d:0>2}", .{
|
||||
@as(u16, @intCast(year)),
|
||||
@as(u8, @intCast(m)),
|
||||
@as(u8, @intCast(d)),
|
||||
});
|
||||
}
|
||||
|
||||
/// Today's date in UTC, rendered into `buf`.
|
||||
///
|
||||
/// UTC rather than local time: every date this program handles comes from a feed
|
||||
/// or a price series that is already UTC-dated, and the only consumer is a
|
||||
/// coarse "has calendar time crossed into a projected era" check where a few
|
||||
/// hours cannot matter.
|
||||
pub fn today(io: std.Io, buf: *[10]u8) ![]const u8 {
|
||||
const secs = std.Io.Timestamp.now(io, .real).toSeconds();
|
||||
return isoFromDays(@divFloor(secs, std.time.s_per_day), buf);
|
||||
}
|
||||
|
||||
test "isValidIso accepts well-formed dates and rejects malformed ones" {
|
||||
try std.testing.expect(isValidIso("2026-08-27"));
|
||||
try std.testing.expect(isValidIso("1970-01-01"));
|
||||
|
||||
try std.testing.expect(!isValidIso("2026-8-27")); // not zero-padded
|
||||
try std.testing.expect(!isValidIso("2026/08/27")); // wrong separator
|
||||
try std.testing.expect(!isValidIso("2026-08-2")); // too short
|
||||
try std.testing.expect(!isValidIso("2026-08-270")); // too long
|
||||
try std.testing.expect(!isValidIso("20x6-08-27")); // non-digit
|
||||
try std.testing.expect(!isValidIso(""));
|
||||
}
|
||||
|
||||
test "order and lessThan follow chronology" {
|
||||
try std.testing.expectEqual(std.math.Order.lt, order("2025-01-01", "2026-01-01"));
|
||||
try std.testing.expectEqual(std.math.Order.gt, order("2026-02-01", "2026-01-31"));
|
||||
try std.testing.expectEqual(std.math.Order.eq, order("2026-01-01", "2026-01-01"));
|
||||
try std.testing.expect(lessThan("2018-09-01", "2018-09-04"));
|
||||
try std.testing.expect(!lessThan("2018-09-04", "2018-09-01"));
|
||||
}
|
||||
|
||||
test "daysFromIso matches known epoch offsets" {
|
||||
try std.testing.expectEqual(@as(i64, 0), try daysFromIso("1970-01-01"));
|
||||
try std.testing.expectEqual(@as(i64, 1), try daysFromIso("1970-01-02"));
|
||||
try std.testing.expectEqual(@as(i64, 31), try daysFromIso("1970-02-01"));
|
||||
// 2000-01-01 is 10957 days after the epoch.
|
||||
try std.testing.expectEqual(@as(i64, 10957), try daysFromIso("2000-01-01"));
|
||||
// Pre-epoch dates go negative.
|
||||
try std.testing.expectEqual(@as(i64, -1), try daysFromIso("1969-12-31"));
|
||||
}
|
||||
|
||||
test "daysFromIso handles leap years" {
|
||||
// 2024 is a leap year: Feb has 29 days.
|
||||
try std.testing.expectEqual(@as(i64, 1), try daysBetween("2024-02-28", "2024-02-29"));
|
||||
try std.testing.expectEqual(@as(i64, 2), try daysBetween("2024-02-28", "2024-03-01"));
|
||||
// 2100 is NOT a leap year (century rule).
|
||||
try std.testing.expectEqual(@as(i64, 1), try daysBetween("2100-02-28", "2100-03-01"));
|
||||
// 2000 IS a leap year (400 rule).
|
||||
try std.testing.expectEqual(@as(i64, 2), try daysBetween("2000-02-28", "2000-03-01"));
|
||||
}
|
||||
|
||||
test "daysBetween is signed and matches the spans used in verification" {
|
||||
// The anchor spans the reconstruction actually uses.
|
||||
try std.testing.expectEqual(@as(i64, 2357), try daysBetween("2018-09-01", "2025-02-13"));
|
||||
try std.testing.expectEqual(@as(i64, 593), try daysBetween("2023-07-01", "2025-02-13"));
|
||||
try std.testing.expectEqual(@as(i64, 28), try daysBetween("2026-07-30", "2026-08-27"));
|
||||
try std.testing.expectEqual(@as(i64, -28), try daysBetween("2026-08-27", "2026-07-30"));
|
||||
try std.testing.expectEqual(@as(i64, 0), try daysBetween("2026-08-27", "2026-08-27"));
|
||||
}
|
||||
|
||||
test "daysFromIso rejects malformed input" {
|
||||
try std.testing.expectError(error.InvalidDate, daysFromIso("2026-13-01"));
|
||||
try std.testing.expectError(error.InvalidDate, daysFromIso("2026-00-01"));
|
||||
try std.testing.expectError(error.InvalidDate, daysFromIso("2026-01-00"));
|
||||
try std.testing.expectError(error.InvalidDate, daysFromIso("2026-01-32"));
|
||||
try std.testing.expectError(error.InvalidDate, daysFromIso("nonsense"));
|
||||
}
|
||||
|
||||
test "isoFromDays inverts daysFromIso" {
|
||||
var buf: [10]u8 = undefined;
|
||||
try std.testing.expectEqualStrings("1970-01-01", try isoFromDays(0, &buf));
|
||||
try std.testing.expectEqualStrings("1969-12-31", try isoFromDays(-1, &buf));
|
||||
try std.testing.expectEqualStrings("2000-01-01", try isoFromDays(10957, &buf));
|
||||
try std.testing.expectEqualStrings("2024-02-29", try isoFromDays(try daysFromIso("2024-02-29"), &buf));
|
||||
try std.testing.expectEqualStrings("2100-03-01", try isoFromDays(try daysFromIso("2100-03-01"), &buf));
|
||||
|
||||
// Round-trip a long stretch spanning leap years and century rules.
|
||||
var d = try daysFromIso("2018-01-01");
|
||||
const end = try daysFromIso("2048-01-01");
|
||||
while (d <= end) : (d += 1) {
|
||||
const iso = try isoFromDays(d, &buf);
|
||||
try std.testing.expect(isValidIso(iso));
|
||||
try std.testing.expectEqual(d, try daysFromIso(iso));
|
||||
}
|
||||
}
|
||||
|
||||
test "isoFromDays refuses dates it cannot render in four digits" {
|
||||
var buf: [10]u8 = undefined;
|
||||
try std.testing.expectError(error.DateOutOfRange, isoFromDays(-800000, &buf));
|
||||
try std.testing.expectError(error.DateOutOfRange, isoFromDays(3000000, &buf));
|
||||
}
|
||||
506
src/data.zig
Normal file
506
src/data.zig
Normal file
|
|
@ -0,0 +1,506 @@
|
|||
//! Readers for this project's own SRF inputs: the allocation model, the observed
|
||||
//! anchors, and the hand-typed values used for independent verification.
|
||||
//!
|
||||
//! All three are small, human-edited files. They are parsed strictly -- a
|
||||
//! malformed row is fatal rather than skipped -- because each row materially
|
||||
//! changes the output and a silently dropped weight or anchor would corrupt the
|
||||
//! series without any visible symptom.
|
||||
|
||||
const std = @import("std");
|
||||
const srf = @import("srf");
|
||||
const srf_opts = @import("srf_opts.zig");
|
||||
const civil = @import("civil.zig");
|
||||
|
||||
// ── model.srf ────────────────────────────────────────────────────────
|
||||
|
||||
const ModelRow = struct {
|
||||
symbol: []const u8,
|
||||
era_start: []const u8,
|
||||
ticker: []const u8,
|
||||
weight: f64,
|
||||
basis: []const u8 = "fitted",
|
||||
};
|
||||
|
||||
/// How far a set of weights can be trusted.
|
||||
pub const Basis = enum {
|
||||
/// In effect now or in the past, and validated against the observed anchors.
|
||||
fitted,
|
||||
/// Not yet in effect. Read from the booklet's forward glidepath table, so it
|
||||
/// is the plan's published intent rather than an observed fact -- and the
|
||||
/// booklet reserves the right to change allocations and fund selection.
|
||||
projected,
|
||||
};
|
||||
|
||||
const Weight = struct {
|
||||
ticker: []const u8,
|
||||
/// Fraction of the portfolio, i.e. the file's percentage divided by 100.
|
||||
fraction: f64,
|
||||
};
|
||||
|
||||
const Era = struct {
|
||||
start: []const u8,
|
||||
basis: Basis,
|
||||
weights: []const Weight,
|
||||
};
|
||||
|
||||
pub const Model = struct {
|
||||
symbol: []const u8,
|
||||
/// Ascending by `start`.
|
||||
eras: []const Era,
|
||||
|
||||
/// The era in force on `date`: the last one whose `start` is <= `date`.
|
||||
/// Null when `date` precedes every era, which means the portfolio did not
|
||||
/// exist yet.
|
||||
pub fn eraOn(self: Model, date: []const u8) ?Era {
|
||||
var found: ?Era = null;
|
||||
for (self.eras) |e| {
|
||||
if (civil.lessThan(date, e.start)) break;
|
||||
found = e;
|
||||
}
|
||||
return found;
|
||||
}
|
||||
|
||||
/// Every distinct ticker across every era, so callers can load exactly the
|
||||
/// price series this portfolio needs.
|
||||
pub fn tickers(self: Model, arena: std.mem.Allocator) ![]const []const u8 {
|
||||
return self.tickersThrough(arena, null);
|
||||
}
|
||||
|
||||
/// Tickers referenced by eras in force on or before `date`, or all of them
|
||||
/// when `date` is null.
|
||||
///
|
||||
/// The distinction matters because projected eras name funds the portfolio
|
||||
/// will not hold for years (VBIPX, VTSPX, VUSXX). Those are not worth
|
||||
/// warning about when absent from a price cache; a fund an era already in
|
||||
/// force depends on very much is.
|
||||
pub fn tickersThrough(
|
||||
self: Model,
|
||||
arena: std.mem.Allocator,
|
||||
date: ?[]const u8,
|
||||
) ![]const []const u8 {
|
||||
var seen: std.StringHashMapUnmanaged(void) = .empty;
|
||||
var out: std.ArrayList([]const u8) = .empty;
|
||||
for (self.eras) |e| {
|
||||
if (date) |d| {
|
||||
if (civil.lessThan(d, e.start)) continue;
|
||||
}
|
||||
for (e.weights) |w| {
|
||||
const gop = try seen.getOrPut(arena, w.ticker);
|
||||
if (!gop.found_existing) try out.append(arena, w.ticker);
|
||||
}
|
||||
}
|
||||
return out.toOwnedSlice(arena);
|
||||
}
|
||||
};
|
||||
|
||||
pub fn parseModel(arena: std.mem.Allocator, data: []const u8) ![]const Model {
|
||||
var reader = std.Io.Reader.fixed(data);
|
||||
var it = srf.iterator(&reader, arena, .{ .parse_allocator = .none }) catch
|
||||
return error.InvalidModelFile;
|
||||
defer it.deinit();
|
||||
|
||||
// symbol -> era_start -> weights, preserving first-seen order at both levels.
|
||||
var symbols: std.ArrayList([]const u8) = .empty;
|
||||
var by_symbol: std.StringHashMapUnmanaged(std.ArrayList(Era)) = .empty;
|
||||
|
||||
while (try it.next()) |fields| {
|
||||
const row = try fields.to(ModelRow, srf_opts.user_edited);
|
||||
if (!civil.isValidIso(row.era_start)) return error.InvalidModelEraStart;
|
||||
if (row.ticker.len == 0) return error.InvalidModelTicker;
|
||||
const basis = std.meta.stringToEnum(Basis, row.basis) orelse
|
||||
return error.UnknownModelBasis;
|
||||
|
||||
const sym = try arena.dupe(u8, row.symbol);
|
||||
const sym_gop = try by_symbol.getOrPut(arena, sym);
|
||||
if (!sym_gop.found_existing) {
|
||||
sym_gop.value_ptr.* = .empty;
|
||||
try symbols.append(arena, sym);
|
||||
}
|
||||
const eras = sym_gop.value_ptr;
|
||||
|
||||
const start = try arena.dupe(u8, row.era_start);
|
||||
const weight: Weight = .{
|
||||
.ticker = try arena.dupe(u8, row.ticker),
|
||||
.fraction = row.weight / 100.0,
|
||||
};
|
||||
|
||||
// Append into the matching era, or start a new one.
|
||||
var placed = false;
|
||||
for (eras.items) |*e| {
|
||||
if (std.mem.eql(u8, e.start, start)) {
|
||||
// Every row of one era must agree about how trustworthy it is;
|
||||
// a mixed era would make the `verify` warning meaningless.
|
||||
if (e.basis != basis) return error.MixedModelBasis;
|
||||
var ws: std.ArrayList(Weight) = .empty;
|
||||
try ws.appendSlice(arena, e.weights);
|
||||
try ws.append(arena, weight);
|
||||
e.weights = try ws.toOwnedSlice(arena);
|
||||
placed = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
if (!placed) {
|
||||
const ws = try arena.dupe(Weight, &.{weight});
|
||||
try eras.append(arena, .{ .start = start, .basis = basis, .weights = ws });
|
||||
}
|
||||
}
|
||||
|
||||
var out: std.ArrayList(Model) = .empty;
|
||||
for (symbols.items) |sym| {
|
||||
const eras = by_symbol.getPtr(sym).?;
|
||||
const owned = try eras.toOwnedSlice(arena);
|
||||
std.mem.sort(Era, owned, {}, struct {
|
||||
fn lt(_: void, a: Era, b: Era) bool {
|
||||
return civil.lessThan(a.start, b.start);
|
||||
}
|
||||
}.lt);
|
||||
try out.append(arena, .{ .symbol = sym, .eras = owned });
|
||||
}
|
||||
return out.toOwnedSlice(arena);
|
||||
}
|
||||
|
||||
/// Look up one portfolio by symbol.
|
||||
pub fn findModel(models: []const Model, symbol: []const u8) ?Model {
|
||||
for (models) |m| {
|
||||
if (std.mem.eql(u8, m.symbol, symbol)) return m;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
// ── anchors.srf ──────────────────────────────────────────────────────
|
||||
|
||||
const Source = enum { launch, wayback, feed };
|
||||
|
||||
const AnchorRow = struct {
|
||||
symbol: []const u8,
|
||||
date: []const u8,
|
||||
unit_value: f64,
|
||||
source: []const u8,
|
||||
evidence: []const u8 = "",
|
||||
};
|
||||
|
||||
pub const Anchor = struct {
|
||||
date: []const u8,
|
||||
unit_value: f64,
|
||||
source: Source,
|
||||
evidence: []const u8,
|
||||
};
|
||||
|
||||
pub const AnchorSet = struct {
|
||||
symbol: []const u8,
|
||||
/// Ascending by `date`, at least two entries.
|
||||
items: []const Anchor,
|
||||
};
|
||||
|
||||
pub fn parseAnchors(arena: std.mem.Allocator, data: []const u8) ![]const AnchorSet {
|
||||
var reader = std.Io.Reader.fixed(data);
|
||||
var it = srf.iterator(&reader, arena, .{ .parse_allocator = .none }) catch
|
||||
return error.InvalidAnchorFile;
|
||||
defer it.deinit();
|
||||
|
||||
var symbols: std.ArrayList([]const u8) = .empty;
|
||||
var by_symbol: std.StringHashMapUnmanaged(std.ArrayList(Anchor)) = .empty;
|
||||
|
||||
while (try it.next()) |fields| {
|
||||
const row = try fields.to(AnchorRow, srf_opts.user_edited);
|
||||
if (!civil.isValidIso(row.date)) return error.InvalidAnchorDate;
|
||||
if (!(row.unit_value > 0)) return error.InvalidAnchorValue;
|
||||
const source = std.meta.stringToEnum(Source, row.source) orelse
|
||||
return error.UnknownAnchorSource;
|
||||
|
||||
const sym = try arena.dupe(u8, row.symbol);
|
||||
const gop = try by_symbol.getOrPut(arena, sym);
|
||||
if (!gop.found_existing) {
|
||||
gop.value_ptr.* = .empty;
|
||||
try symbols.append(arena, sym);
|
||||
}
|
||||
try gop.value_ptr.append(arena, .{
|
||||
.date = try arena.dupe(u8, row.date),
|
||||
.unit_value = row.unit_value,
|
||||
.source = source,
|
||||
.evidence = try arena.dupe(u8, row.evidence),
|
||||
});
|
||||
}
|
||||
|
||||
var out: std.ArrayList(AnchorSet) = .empty;
|
||||
for (symbols.items) |sym| {
|
||||
const list = by_symbol.getPtr(sym).?;
|
||||
const owned = try list.toOwnedSlice(arena);
|
||||
std.mem.sort(Anchor, owned, {}, struct {
|
||||
fn lt(_: void, a: Anchor, b: Anchor) bool {
|
||||
return civil.lessThan(a.date, b.date);
|
||||
}
|
||||
}.lt);
|
||||
// Two anchors is the minimum that defines a pinned gap.
|
||||
if (owned.len < 2) return error.TooFewAnchors;
|
||||
for (owned[1..], 0..) |a, i| {
|
||||
if (!civil.lessThan(owned[i].date, a.date)) return error.DuplicateAnchorDate;
|
||||
}
|
||||
try out.append(arena, .{ .symbol = sym, .items = owned });
|
||||
}
|
||||
return out.toOwnedSlice(arena);
|
||||
}
|
||||
|
||||
pub fn findAnchors(sets: []const AnchorSet, symbol: []const u8) ?AnchorSet {
|
||||
for (sets) |s| {
|
||||
if (std.mem.eql(u8, s.symbol, symbol)) return s;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
// ── recorded.srf ─────────────────────────────────────────────────────
|
||||
|
||||
const RecordedRow = struct {
|
||||
symbol: []const u8,
|
||||
snapshot_date: []const u8,
|
||||
unit_value: f64,
|
||||
};
|
||||
|
||||
pub const Recorded = struct {
|
||||
symbol: []const u8,
|
||||
/// The date the value was WRITTEN DOWN, not the date it is for.
|
||||
snapshot_date: []const u8,
|
||||
unit_value: f64,
|
||||
};
|
||||
|
||||
pub fn parseRecorded(arena: std.mem.Allocator, data: []const u8) ![]const Recorded {
|
||||
var reader = std.Io.Reader.fixed(data);
|
||||
var it = srf.iterator(&reader, arena, .{ .parse_allocator = .none }) catch
|
||||
return error.InvalidRecordedFile;
|
||||
defer it.deinit();
|
||||
|
||||
var out: std.ArrayList(Recorded) = .empty;
|
||||
while (try it.next()) |fields| {
|
||||
const row = try fields.to(RecordedRow, srf_opts.user_edited);
|
||||
if (!civil.isValidIso(row.snapshot_date)) return error.InvalidRecordedDate;
|
||||
try out.append(arena, .{
|
||||
.symbol = try arena.dupe(u8, row.symbol),
|
||||
.snapshot_date = try arena.dupe(u8, row.snapshot_date),
|
||||
.unit_value = row.unit_value,
|
||||
});
|
||||
}
|
||||
return out.toOwnedSlice(arena);
|
||||
}
|
||||
|
||||
const testing = std.testing;
|
||||
|
||||
test "parseModel groups rows into eras and scales percentages to fractions" {
|
||||
var arena_state: std.heap.ArenaAllocator = .init(testing.allocator);
|
||||
defer arena_state.deinit();
|
||||
const arena = arena_state.allocator();
|
||||
|
||||
const models = try parseModel(arena,
|
||||
\\#!srfv1
|
||||
\\symbol::ORCBI,era_start::2018-09-01,ticker::VSMPX,weight:num:36
|
||||
\\symbol::ORCBI,era_start::2018-09-01,ticker::VTPSX,weight:num:24
|
||||
\\symbol::ORCBI,era_start::2018-09-01,ticker::VBMPX,weight:num:40
|
||||
\\
|
||||
);
|
||||
|
||||
try testing.expectEqual(@as(usize, 1), models.len);
|
||||
const m = findModel(models, "ORCBI").?;
|
||||
try testing.expectEqual(@as(usize, 1), m.eras.len);
|
||||
try testing.expectEqual(@as(usize, 3), m.eras[0].weights.len);
|
||||
try testing.expectApproxEqAbs(@as(f64, 0.36), m.eras[0].weights[0].fraction, 1e-12);
|
||||
try testing.expectApproxEqAbs(@as(f64, 0.40), m.eras[0].weights[2].fraction, 1e-12);
|
||||
|
||||
// Weights must sum to 1.
|
||||
var sum: f64 = 0;
|
||||
for (m.eras[0].weights) |w| sum += w.fraction;
|
||||
try testing.expectApproxEqAbs(@as(f64, 1.0), sum, 1e-12);
|
||||
}
|
||||
|
||||
test "parseModel sorts eras and eraOn selects the one in force" {
|
||||
var arena_state: std.heap.ArenaAllocator = .init(testing.allocator);
|
||||
defer arena_state.deinit();
|
||||
const arena = arena_state.allocator();
|
||||
|
||||
// Deliberately out of order in the file.
|
||||
const models = try parseModel(arena,
|
||||
\\#!srfv1
|
||||
\\symbol::ORC42,era_start::2026-01-01,ticker::VSMPX,weight:num:52.8
|
||||
\\symbol::ORC42,era_start::2023-07-01,ticker::VSMPX,weight:num:54
|
||||
\\
|
||||
);
|
||||
const m = findModel(models, "ORC42").?;
|
||||
try testing.expectEqual(@as(usize, 2), m.eras.len);
|
||||
try testing.expectEqualStrings("2023-07-01", m.eras[0].start);
|
||||
try testing.expectEqualStrings("2026-01-01", m.eras[1].start);
|
||||
|
||||
// Before any era: the portfolio did not exist.
|
||||
try testing.expectEqual(@as(?Era, null), m.eraOn("2023-06-30"));
|
||||
// Boundaries are inclusive.
|
||||
try testing.expectEqualStrings("2023-07-01", m.eraOn("2023-07-01").?.start);
|
||||
try testing.expectEqualStrings("2023-07-01", m.eraOn("2025-12-31").?.start);
|
||||
try testing.expectEqualStrings("2026-01-01", m.eraOn("2026-01-01").?.start);
|
||||
try testing.expectEqualStrings("2026-01-01", m.eraOn("2030-01-01").?.start);
|
||||
}
|
||||
|
||||
test "Model.tickers dedupes across eras" {
|
||||
var arena_state: std.heap.ArenaAllocator = .init(testing.allocator);
|
||||
defer arena_state.deinit();
|
||||
const arena = arena_state.allocator();
|
||||
|
||||
const models = try parseModel(arena,
|
||||
\\#!srfv1
|
||||
\\symbol::ORC42,era_start::2023-07-01,ticker::VSMPX,weight:num:54
|
||||
\\symbol::ORC42,era_start::2023-07-01,ticker::VTPSX,weight:num:46
|
||||
\\symbol::ORC42,era_start::2026-01-01,ticker::VSMPX,weight:num:52.8
|
||||
\\symbol::ORC42,era_start::2026-01-01,ticker::VBMPX,weight:num:47.2
|
||||
\\
|
||||
);
|
||||
const t = try findModel(models, "ORC42").?.tickers(arena);
|
||||
try testing.expectEqual(@as(usize, 3), t.len);
|
||||
try testing.expectEqualStrings("VSMPX", t[0]);
|
||||
try testing.expectEqualStrings("VTPSX", t[1]);
|
||||
try testing.expectEqualStrings("VBMPX", t[2]);
|
||||
}
|
||||
|
||||
test "parseModel rejects a malformed era_start" {
|
||||
var arena_state: std.heap.ArenaAllocator = .init(testing.allocator);
|
||||
defer arena_state.deinit();
|
||||
try testing.expectError(error.InvalidModelEraStart, parseModel(arena_state.allocator(),
|
||||
\\#!srfv1
|
||||
\\symbol::X,era_start::2026-1-1,ticker::A,weight:num:100
|
||||
\\
|
||||
));
|
||||
}
|
||||
|
||||
test "parseAnchors sorts, groups and decodes sources" {
|
||||
var arena_state: std.heap.ArenaAllocator = .init(testing.allocator);
|
||||
defer arena_state.deinit();
|
||||
const arena = arena_state.allocator();
|
||||
|
||||
const sets = try parseAnchors(arena,
|
||||
\\#!srfv1
|
||||
\\symbol::ORCBI,date::2026-08-27,unit_value:num:19.092721,source::feed,evidence::live
|
||||
\\symbol::ORCBI,date::2018-09-01,unit_value:num:10.0,source::launch,evidence::inception
|
||||
\\symbol::ORCBI,date::2025-02-13,unit_value:num:15.529444,source::wayback,evidence::20250215060344
|
||||
\\
|
||||
);
|
||||
try testing.expectEqual(@as(usize, 1), sets.len);
|
||||
const a = findAnchors(sets, "ORCBI").?;
|
||||
try testing.expectEqual(@as(usize, 3), a.items.len);
|
||||
try testing.expectEqualStrings("2018-09-01", a.items[0].date);
|
||||
try testing.expectEqual(Source.launch, a.items[0].source);
|
||||
try testing.expectEqual(Source.wayback, a.items[1].source);
|
||||
try testing.expectEqualStrings("20250215060344", a.items[1].evidence);
|
||||
try testing.expectEqual(Source.feed, a.items[2].source);
|
||||
try testing.expectApproxEqAbs(@as(f64, 19.092721), a.items[2].unit_value, 1e-12);
|
||||
}
|
||||
|
||||
test "parseAnchors rejects unusable anchor sets" {
|
||||
var arena_state: std.heap.ArenaAllocator = .init(testing.allocator);
|
||||
defer arena_state.deinit();
|
||||
const arena = arena_state.allocator();
|
||||
|
||||
// A single anchor defines no gap.
|
||||
try testing.expectError(error.TooFewAnchors, parseAnchors(arena,
|
||||
\\#!srfv1
|
||||
\\symbol::X,date::2026-08-27,unit_value:num:10,source::feed
|
||||
\\
|
||||
));
|
||||
// Two values for one date is ambiguous.
|
||||
try testing.expectError(error.DuplicateAnchorDate, parseAnchors(arena,
|
||||
\\#!srfv1
|
||||
\\symbol::X,date::2026-08-27,unit_value:num:10,source::feed
|
||||
\\symbol::X,date::2026-08-27,unit_value:num:11,source::feed
|
||||
\\
|
||||
));
|
||||
try testing.expectError(error.UnknownAnchorSource, parseAnchors(arena,
|
||||
\\#!srfv1
|
||||
\\symbol::X,date::2026-08-26,unit_value:num:10,source::guess
|
||||
\\symbol::X,date::2026-08-27,unit_value:num:11,source::feed
|
||||
\\
|
||||
));
|
||||
try testing.expectError(error.InvalidAnchorValue, parseAnchors(arena,
|
||||
\\#!srfv1
|
||||
\\symbol::X,date::2026-08-26,unit_value:num:0,source::feed
|
||||
\\symbol::X,date::2026-08-27,unit_value:num:11,source::feed
|
||||
\\
|
||||
));
|
||||
try testing.expectError(error.InvalidAnchorDate, parseAnchors(arena,
|
||||
\\#!srfv1
|
||||
\\symbol::X,date::26-08-26,unit_value:num:10,source::feed
|
||||
\\symbol::X,date::2026-08-27,unit_value:num:11,source::feed
|
||||
\\
|
||||
));
|
||||
}
|
||||
|
||||
test "parseRecorded reads snapshot rows" {
|
||||
var arena_state: std.heap.ArenaAllocator = .init(testing.allocator);
|
||||
defer arena_state.deinit();
|
||||
const arena = arena_state.allocator();
|
||||
|
||||
const rows = try parseRecorded(arena,
|
||||
\\#!srfv1
|
||||
\\symbol::ORCBI,snapshot_date::2026-04-27,unit_value:num:18.27
|
||||
\\symbol::ORC42,snapshot_date::2026-04-27,unit_value:num:15.73
|
||||
\\
|
||||
);
|
||||
try testing.expectEqual(@as(usize, 2), rows.len);
|
||||
try testing.expectEqualStrings("ORCBI", rows[0].symbol);
|
||||
try testing.expectEqualStrings("2026-04-27", rows[0].snapshot_date);
|
||||
try testing.expectApproxEqAbs(@as(f64, 15.73), rows[1].unit_value, 1e-12);
|
||||
|
||||
try testing.expectError(error.InvalidRecordedDate, parseRecorded(arena,
|
||||
\\#!srfv1
|
||||
\\symbol::X,snapshot_date::nope,unit_value:num:1
|
||||
\\
|
||||
));
|
||||
}
|
||||
|
||||
test "findModel and findAnchors miss cleanly" {
|
||||
var arena_state: std.heap.ArenaAllocator = .init(testing.allocator);
|
||||
defer arena_state.deinit();
|
||||
const arena = arena_state.allocator();
|
||||
|
||||
const models = try parseModel(arena,
|
||||
\\#!srfv1
|
||||
\\symbol::A,era_start::2020-01-01,ticker::T,weight:num:100
|
||||
\\
|
||||
);
|
||||
try testing.expect(findModel(models, "A") != null);
|
||||
try testing.expectEqual(@as(?Model, null), findModel(models, "MISSING"));
|
||||
|
||||
const sets = try parseAnchors(arena,
|
||||
\\#!srfv1
|
||||
\\symbol::A,date::2020-01-01,unit_value:num:10,source::launch
|
||||
\\symbol::A,date::2020-06-01,unit_value:num:11,source::feed
|
||||
\\
|
||||
);
|
||||
try testing.expect(findAnchors(sets, "A") != null);
|
||||
try testing.expectEqual(@as(?AnchorSet, null), findAnchors(sets, "MISSING"));
|
||||
}
|
||||
|
||||
test "tickersThrough excludes funds only a future era needs" {
|
||||
var arena_state: std.heap.ArenaAllocator = .init(testing.allocator);
|
||||
defer arena_state.deinit();
|
||||
const arena = arena_state.allocator();
|
||||
|
||||
// This is the real ORC42 shape: the current era holds five funds, and
|
||||
// projections a decade out introduce three more.
|
||||
const models = try parseModel(arena,
|
||||
\\#!srfv1
|
||||
\\symbol::ORC42,era_start::2026-01-01,basis::fitted,ticker::VSMPX,weight:num:52.8
|
||||
\\symbol::ORC42,era_start::2026-01-01,basis::fitted,ticker::VBMPX,weight:num:47.2
|
||||
\\symbol::ORC42,era_start::2038-01-01,basis::projected,ticker::VSMPX,weight:num:24.8
|
||||
\\symbol::ORC42,era_start::2038-01-01,basis::projected,ticker::VBIPX,weight:num:75.2
|
||||
\\
|
||||
);
|
||||
const m = findModel(models, "ORC42").?;
|
||||
|
||||
// Everything, for deciding what to try loading.
|
||||
try testing.expectEqual(@as(usize, 3), (try m.tickers(arena)).len);
|
||||
|
||||
// Only what is actually held today, for deciding what to complain about.
|
||||
const now = try m.tickersThrough(arena, "2026-08-28");
|
||||
try testing.expectEqual(@as(usize, 2), now.len);
|
||||
for (now) |t| try testing.expect(!std.mem.eql(u8, t, "VBIPX"));
|
||||
|
||||
// Once the projection is in force, the new fund does count.
|
||||
try testing.expectEqual(@as(usize, 3), (try m.tickersThrough(arena, "2038-06-01")).len);
|
||||
|
||||
// Before any era, nothing is needed.
|
||||
try testing.expectEqual(@as(usize, 0), (try m.tickersThrough(arena, "2020-01-01")).len);
|
||||
}
|
||||
248
src/feed.zig
Normal file
248
src/feed.zig
Normal file
|
|
@ -0,0 +1,248 @@
|
|||
//! Live reader for the plan's daily unit-value feed.
|
||||
//!
|
||||
//! ## The bogus Content-Encoding, and why this is not just `Client.fetch`
|
||||
//!
|
||||
//! The feed is a static file on S3 behind CloudFront, and its object metadata
|
||||
//! carries `content-encoding: base64`. The body is NOT base64 -- it is plain
|
||||
//! JSON. The header is simply wrong, it is present on every file in that prefix,
|
||||
//! it survives every request variant (HTTP/1.1, Range, any Accept-Encoding), and
|
||||
//! since the file is regenerated daily by the same pipeline it keeps coming back.
|
||||
//!
|
||||
//! Zig 0.16's `std.http.Client` refuses it, correctly by the spec:
|
||||
//!
|
||||
//! std/http.zig ContentEncoding.fromString("base64") -> null
|
||||
//! std/http/Client.zig -> error.HttpContentEncodingUnsupported
|
||||
//! std/http/Client.zig receiveHead collapses that to error.HttpHeadersInvalid
|
||||
//!
|
||||
//! Worse, a head-PARSE failure never marks the connection closing, so with
|
||||
//! keep-alive on, a poisoned connection goes back to the pool and teardown can
|
||||
//! block.
|
||||
//!
|
||||
//! The fix does not need a hand-rolled HTTP stack. `receiveHead` is a thin
|
||||
//! wrapper over two separately-public steps, and only the second one fails:
|
||||
//!
|
||||
//! const head_bytes = req.reader.receiveHead(); // public, works
|
||||
//! Response.Head.parse(head_bytes) // public, fails here
|
||||
//!
|
||||
//! So we take the raw head, drop the offending line, and hand it to the same
|
||||
//! public parser. TLS, connection handling and body framing all stay in the
|
||||
//! standard library.
|
||||
|
||||
const std = @import("std");
|
||||
const Response = std.http.Client.Response;
|
||||
|
||||
pub const url = "https://marcom.vestwell.com/data/fund-performance/OR-CSP.json";
|
||||
|
||||
/// Feed fund ids for the two portfolios tracked here. These are the plan's
|
||||
/// internal reporting-account numbers, and they are the only stable join key
|
||||
/// between our symbols and the feed.
|
||||
const fund_id_orcbi = "T6GF10080002";
|
||||
const fund_id_orc42 = "T6GFRD204202";
|
||||
|
||||
pub fn fundIdFor(symbol: []const u8) ?[]const u8 {
|
||||
if (std.mem.eql(u8, symbol, "ORCBI")) return fund_id_orcbi;
|
||||
if (std.mem.eql(u8, symbol, "ORC42")) return fund_id_orc42;
|
||||
return null;
|
||||
}
|
||||
|
||||
const Quote = struct {
|
||||
/// The feed's own navDate, which is the prior business day.
|
||||
date: []const u8,
|
||||
unit_value: f64,
|
||||
};
|
||||
|
||||
/// Copy an HTTP head, dropping any `content-encoding` header line.
|
||||
///
|
||||
/// Dropping rather than rewriting avoids all length bookkeeping, and an absent
|
||||
/// header means `identity`, which is what this body actually is. Returns
|
||||
/// `error.NothingStripped` when there was no such header, so a silent change in
|
||||
/// the server's behaviour surfaces as a failure rather than as a mystery later.
|
||||
fn stripContentEncoding(arena: std.mem.Allocator, head: []const u8) ![]u8 {
|
||||
var out: std.ArrayList(u8) = .empty;
|
||||
var stripped: usize = 0;
|
||||
var it = std.mem.splitSequence(u8, head, "\r\n");
|
||||
while (it.next()) |line| {
|
||||
if (std.ascii.startsWithIgnoreCase(line, "content-encoding:")) {
|
||||
stripped += 1;
|
||||
continue;
|
||||
}
|
||||
try out.appendSlice(arena, line);
|
||||
try out.appendSlice(arena, "\r\n");
|
||||
}
|
||||
if (stripped == 0) return error.NothingStripped;
|
||||
return out.toOwnedSlice(arena);
|
||||
}
|
||||
|
||||
/// Fetch the feed body, tolerating the bogus Content-Encoding.
|
||||
pub fn fetchBody(arena: std.mem.Allocator, io: std.Io, feed_url: []const u8) ![]const u8 {
|
||||
var client = std.http.Client{ .allocator = arena, .io = io };
|
||||
defer client.deinit();
|
||||
|
||||
const uri = try std.Uri.parse(feed_url);
|
||||
// keep_alive = false: a head-parse failure would otherwise return a
|
||||
// connection with an undrained body to the pool and hang at teardown.
|
||||
var req = try client.request(.GET, uri, .{ .keep_alive = false });
|
||||
defer req.deinit();
|
||||
try req.sendBodiless();
|
||||
|
||||
const raw_head = try req.reader.receiveHead();
|
||||
const clean_head = try stripContentEncoding(arena, raw_head);
|
||||
const head = try Response.Head.parse(clean_head);
|
||||
if (head.status != .ok) return error.FeedHttpStatus;
|
||||
|
||||
// Copy the scalars out before the body stream invalidates head strings.
|
||||
const te = head.transfer_encoding;
|
||||
const cl = head.content_length;
|
||||
|
||||
var transfer_buffer: [4096]u8 = undefined;
|
||||
const body_reader = req.reader.bodyReader(&transfer_buffer, te, cl);
|
||||
|
||||
var aw: std.Io.Writer.Allocating = .init(arena);
|
||||
_ = try body_reader.streamRemaining(&aw.writer);
|
||||
return aw.written();
|
||||
}
|
||||
|
||||
/// Pull one fund's unit value out of a feed body.
|
||||
pub fn quoteFrom(arena: std.mem.Allocator, body: []const u8, fund_id: []const u8) !Quote {
|
||||
const parsed = try std.json.parseFromSlice(std.json.Value, arena, body, .{});
|
||||
defer parsed.deinit();
|
||||
|
||||
const root = switch (parsed.value) {
|
||||
.object => |o| o,
|
||||
else => return error.FeedShapeUnexpected,
|
||||
};
|
||||
const funds = switch (root.get("funds") orelse return error.FeedShapeUnexpected) {
|
||||
.array => |a| a,
|
||||
else => return error.FeedShapeUnexpected,
|
||||
};
|
||||
|
||||
for (funds.items) |f| {
|
||||
const o = switch (f) {
|
||||
.object => |o| o,
|
||||
else => continue,
|
||||
};
|
||||
const id = switch (o.get("id") orelse continue) {
|
||||
.string => |s| s,
|
||||
else => continue,
|
||||
};
|
||||
if (!std.mem.eql(u8, id, fund_id)) continue;
|
||||
|
||||
const nav: f64 = switch (o.get("nav") orelse return error.FeedMissingNav) {
|
||||
.float => |x| x,
|
||||
.integer => |i| @floatFromInt(i),
|
||||
else => return error.FeedMissingNav,
|
||||
};
|
||||
// A zero here means the portfolio is closed to new money and no longer
|
||||
// priced. Treating it as a price would be worse than failing.
|
||||
if (!(nav > 0)) return error.FeedNavNotPositive;
|
||||
|
||||
const nav_date = switch (o.get("navDate") orelse return error.FeedMissingNavDate) {
|
||||
.string => |s| s,
|
||||
else => return error.FeedMissingNavDate,
|
||||
};
|
||||
if (nav_date.len < 10) return error.FeedMissingNavDate;
|
||||
|
||||
return .{ .date = try arena.dupe(u8, nav_date[0..10]), .unit_value = nav };
|
||||
}
|
||||
return error.FeedFundNotFound;
|
||||
}
|
||||
|
||||
const testing = std.testing;
|
||||
|
||||
test "fundIdFor maps the tracked symbols and rejects others" {
|
||||
try testing.expectEqualStrings(fund_id_orcbi, fundIdFor("ORCBI").?);
|
||||
try testing.expectEqualStrings(fund_id_orc42, fundIdFor("ORC42").?);
|
||||
try testing.expectEqual(@as(?[]const u8, null), fundIdFor("QQQ"));
|
||||
}
|
||||
|
||||
test "stripContentEncoding removes the bogus header and keeps everything else" {
|
||||
var arena_state: std.heap.ArenaAllocator = .init(testing.allocator);
|
||||
defer arena_state.deinit();
|
||||
const arena = arena_state.allocator();
|
||||
|
||||
const head =
|
||||
"HTTP/1.1 200 OK\r\n" ++
|
||||
"content-type: application/json\r\n" ++
|
||||
"content-length: 42\r\n" ++
|
||||
"Content-Encoding: base64\r\n" ++
|
||||
"etag: \"abc\"\r\n" ++
|
||||
"\r\n";
|
||||
|
||||
const clean = try stripContentEncoding(arena, head);
|
||||
try testing.expect(std.mem.indexOf(u8, clean, "base64") == null);
|
||||
try testing.expect(std.mem.indexOf(u8, clean, "content-length: 42") != null);
|
||||
try testing.expect(std.mem.indexOf(u8, clean, "etag: \"abc\"") != null);
|
||||
try testing.expect(std.mem.startsWith(u8, clean, "HTTP/1.1 200 OK\r\n"));
|
||||
|
||||
// And the cleaned head must still satisfy the real parser.
|
||||
const parsed = try Response.Head.parse(clean);
|
||||
try testing.expectEqual(std.http.Status.ok, parsed.status);
|
||||
try testing.expectEqual(@as(?u64, 42), parsed.content_length);
|
||||
try testing.expectEqual(std.http.ContentEncoding.identity, parsed.content_encoding);
|
||||
}
|
||||
|
||||
test "stripContentEncoding reports when there was nothing to strip" {
|
||||
var arena_state: std.heap.ArenaAllocator = .init(testing.allocator);
|
||||
defer arena_state.deinit();
|
||||
const head = "HTTP/1.1 200 OK\r\ncontent-length: 1\r\n\r\n";
|
||||
try testing.expectError(error.NothingStripped, stripContentEncoding(arena_state.allocator(), head));
|
||||
}
|
||||
|
||||
const sample_body =
|
||||
\\{
|
||||
\\ "state": "OR",
|
||||
\\ "frequency": "daily",
|
||||
\\ "navDate": "2026-08-27T00:00:00.000Z",
|
||||
\\ "funds": [
|
||||
\\ {"id":"T6GF10010002","name":"Target Risk - Conservative","nav":13.90205,"navDate":"2026-08-27T00:00:00.000Z"},
|
||||
\\ {"id":"T6GF10080002","name":"Balanced Index","nav":19.092721,"navDate":"2026-08-27T00:00:00.000Z"},
|
||||
\\ {"id":"T6GFRD201702","name":"Enrollment Year 2017","nav":0,"navDate":"2026-08-27T00:00:00.000Z"},
|
||||
\\ {"id":"T6GFRD204202","name":"Enrollment Year 2042","nav":16.808106,"navDate":"2026-08-27T00:00:00.000Z"}
|
||||
\\ ]
|
||||
\\}
|
||||
;
|
||||
|
||||
test "quoteFrom extracts the tracked funds" {
|
||||
var arena_state: std.heap.ArenaAllocator = .init(testing.allocator);
|
||||
defer arena_state.deinit();
|
||||
const arena = arena_state.allocator();
|
||||
|
||||
const bi = try quoteFrom(arena, sample_body, fund_id_orcbi);
|
||||
try testing.expectEqualStrings("2026-08-27", bi.date);
|
||||
try testing.expectApproxEqAbs(@as(f64, 19.092721), bi.unit_value, 1e-9);
|
||||
|
||||
const c4 = try quoteFrom(arena, sample_body, fund_id_orc42);
|
||||
try testing.expectApproxEqAbs(@as(f64, 16.808106), c4.unit_value, 1e-9);
|
||||
}
|
||||
|
||||
test "quoteFrom refuses a zero nav rather than reporting it as a price" {
|
||||
var arena_state: std.heap.ArenaAllocator = .init(testing.allocator);
|
||||
defer arena_state.deinit();
|
||||
// Closed portfolios are published with nav 0. Absent must never become $0.00.
|
||||
try testing.expectError(error.FeedNavNotPositive, quoteFrom(
|
||||
arena_state.allocator(),
|
||||
sample_body,
|
||||
"T6GFRD201702",
|
||||
));
|
||||
}
|
||||
|
||||
test "quoteFrom surfaces missing funds and bad shapes distinctly" {
|
||||
var arena_state: std.heap.ArenaAllocator = .init(testing.allocator);
|
||||
defer arena_state.deinit();
|
||||
const arena = arena_state.allocator();
|
||||
|
||||
try testing.expectError(error.FeedFundNotFound, quoteFrom(arena, sample_body, "NOPE"));
|
||||
try testing.expectError(error.FeedShapeUnexpected, quoteFrom(arena, "{\"funds\":{}}", "X"));
|
||||
try testing.expectError(error.FeedShapeUnexpected, quoteFrom(arena, "[]", "X"));
|
||||
try testing.expectError(error.FeedShapeUnexpected, quoteFrom(arena, "{}", "X"));
|
||||
try testing.expectError(error.FeedMissingNav, quoteFrom(
|
||||
arena,
|
||||
"{\"funds\":[{\"id\":\"X\",\"navDate\":\"2026-08-27T00:00:00.000Z\"}]}",
|
||||
"X",
|
||||
));
|
||||
try testing.expectError(error.FeedMissingNavDate, quoteFrom(
|
||||
arena,
|
||||
"{\"funds\":[{\"id\":\"X\",\"nav\":1.0}]}",
|
||||
"X",
|
||||
));
|
||||
}
|
||||
396
src/main.zig
Normal file
396
src/main.zig
Normal file
|
|
@ -0,0 +1,396 @@
|
|||
//! zfin-vestwell: unit-value history for the Oregon College Savings Plan
|
||||
//! (Embark) portfolios held in the finance repo, ORCBI and ORC42.
|
||||
//!
|
||||
//! These are unitized fund-of-funds trusts with no ticker, no CUSIP and no
|
||||
//! market-data coverage. The plan publishes only a single current value per
|
||||
//! portfolio, so historical prices have to be reconstructed. See README.md.
|
||||
//!
|
||||
//! reconstruct rebuild data/<SYMBOL>.srf from data/model.srf + data/anchors.srf
|
||||
//! verify check the series on disk against every available evidence source
|
||||
|
||||
const std = @import("std");
|
||||
|
||||
const candles = @import("candles.zig");
|
||||
const civil = @import("civil.zig");
|
||||
const data = @import("data.zig");
|
||||
const feed = @import("feed.zig");
|
||||
const recon = @import("recon.zig");
|
||||
const series = @import("series.zig");
|
||||
const verify = @import("verify.zig");
|
||||
|
||||
const symbols = [_][]const u8{ "ORCBI", "ORC42" };
|
||||
|
||||
const usage =
|
||||
\\zfin-vestwell - unit-value history for Oregon College Savings Plan portfolios
|
||||
\\
|
||||
\\Usage:
|
||||
\\ zfin-vestwell reconstruct [--data-dir DIR]
|
||||
\\ zfin-vestwell verify [--data-dir DIR] [--offline]
|
||||
\\
|
||||
\\Commands:
|
||||
\\ reconstruct Rebuild data/<SYMBOL>.srf from data/model.srf and
|
||||
\\ data/anchors.srf, using the underlying funds' adjusted
|
||||
\\ closes from zfin's candle cache.
|
||||
\\ verify Re-read the series from disk and check it: anchor exactness,
|
||||
\\ leave-one-out cross-validation, agreement with hand-typed
|
||||
\\ values, structural sanity, and a live feed cross-check.
|
||||
\\ Exits non-zero on any failure.
|
||||
\\
|
||||
\\Options:
|
||||
\\ --data-dir DIR Where the SRF inputs and outputs live (default: ./data)
|
||||
\\ --offline Skip the live feed cross-check in `verify`
|
||||
\\ -h, --help Show this message
|
||||
\\
|
||||
;
|
||||
|
||||
const Command = enum { reconstruct, verify, help };
|
||||
|
||||
const Options = struct {
|
||||
cmd: Command,
|
||||
data_dir: []const u8 = "data",
|
||||
offline: bool = false,
|
||||
};
|
||||
|
||||
fn parseArgs(args: []const []const u8) !Options {
|
||||
if (args.len < 2) return .{ .cmd = .help };
|
||||
|
||||
const cmd: Command = if (std.mem.eql(u8, args[1], "reconstruct"))
|
||||
.reconstruct
|
||||
else if (std.mem.eql(u8, args[1], "verify"))
|
||||
.verify
|
||||
else if (std.mem.eql(u8, args[1], "-h") or
|
||||
std.mem.eql(u8, args[1], "--help") or
|
||||
std.mem.eql(u8, args[1], "help"))
|
||||
return .{ .cmd = .help }
|
||||
else
|
||||
return error.UnknownCommand;
|
||||
|
||||
var o: Options = .{ .cmd = cmd };
|
||||
var i: usize = 2;
|
||||
while (i < args.len) : (i += 1) {
|
||||
const a = args[i];
|
||||
if (std.mem.eql(u8, a, "--offline")) {
|
||||
o.offline = true;
|
||||
} else if (std.mem.eql(u8, a, "--data-dir")) {
|
||||
i += 1;
|
||||
if (i >= args.len) return error.MissingArgument;
|
||||
o.data_dir = args[i];
|
||||
} else {
|
||||
return error.UnknownFlag;
|
||||
}
|
||||
}
|
||||
return o;
|
||||
}
|
||||
|
||||
/// Everything both commands need: the model, the anchors and the price series
|
||||
/// for every underlying fund referenced by either symbol.
|
||||
const Loaded = struct {
|
||||
models: []const data.Model,
|
||||
anchor_sets: []const data.AnchorSet,
|
||||
recorded: []const data.Recorded,
|
||||
tickers: []const candles.Ticker,
|
||||
|
||||
fn inputsFor(self: Loaded, symbol: []const u8) !recon.Inputs {
|
||||
return .{
|
||||
.model = data.findModel(self.models, symbol) orelse return error.SymbolMissingFromModel,
|
||||
.anchors = data.findAnchors(self.anchor_sets, symbol) orelse return error.SymbolMissingFromAnchors,
|
||||
.tickers = self.tickers,
|
||||
};
|
||||
}
|
||||
};
|
||||
|
||||
fn readDataFile(
|
||||
arena: std.mem.Allocator,
|
||||
io: std.Io,
|
||||
dir: []const u8,
|
||||
name: []const u8,
|
||||
) ![]const u8 {
|
||||
const path = try std.fs.path.join(arena, &.{ dir, name });
|
||||
return std.Io.Dir.cwd().readFileAlloc(io, path, arena, .limited(32 * 1024 * 1024)) catch |err| {
|
||||
std.log.err("cannot read {s}: {s}", .{ path, @errorName(err) });
|
||||
return err;
|
||||
};
|
||||
}
|
||||
|
||||
fn load(
|
||||
arena: std.mem.Allocator,
|
||||
io: std.Io,
|
||||
env: *const std.process.Environ.Map,
|
||||
dir: []const u8,
|
||||
) !Loaded {
|
||||
const models = try data.parseModel(arena, try readDataFile(arena, io, dir, "model.srf"));
|
||||
const anchor_sets = try data.parseAnchors(arena, try readDataFile(arena, io, dir, "anchors.srf"));
|
||||
const recorded = try data.parseRecorded(arena, try readDataFile(arena, io, dir, "recorded.srf"));
|
||||
|
||||
// Union of every underlying fund across every symbol, loaded once.
|
||||
//
|
||||
// A fund that is not in zfin's cache is skipped rather than being fatal.
|
||||
// Projected glidepath eras reference funds ORC42 will not hold for another
|
||||
// decade (VBIPX, VTSPX, VUSXX), and requiring them now would make the tool
|
||||
// unusable for no benefit. This is safe because `recon.calendar` drops any
|
||||
// day whose era references a fund it cannot price, and `verify` fails if that
|
||||
// shortens the series -- it asserts the series still ends on the last anchor.
|
||||
//
|
||||
// The skip is only *warned* about when an era already in force needs the
|
||||
// fund. Warning about funds a projection will need in 2038 would be noise on
|
||||
// every single run, and noise is how real warnings get ignored.
|
||||
const cache_root = try candles.cacheDir(arena, env);
|
||||
var today_buf: [10]u8 = undefined;
|
||||
const today = try civil.today(io, &today_buf);
|
||||
|
||||
var seen: std.StringHashMapUnmanaged(void) = .empty;
|
||||
var loaded: std.ArrayList(candles.Ticker) = .empty;
|
||||
for (models) |m| {
|
||||
const needed_now = try m.tickersThrough(arena, today);
|
||||
for (try m.tickers(arena)) |name| {
|
||||
const gop = try seen.getOrPut(arena, name);
|
||||
if (gop.found_existing) continue;
|
||||
const t = candles.load(arena, io, cache_root, name) catch |err| {
|
||||
for (needed_now) |n| {
|
||||
if (!std.mem.eql(u8, n, name)) continue;
|
||||
std.log.warn(
|
||||
"{s}: no candles in {s} ({s}), but {s} holds it today. " ++
|
||||
"Days needing it are excluded; run `zfin quote {s}` and re-run.",
|
||||
.{ name, cache_root, @errorName(err), m.symbol, name },
|
||||
);
|
||||
break;
|
||||
}
|
||||
continue;
|
||||
};
|
||||
try loaded.append(arena, t);
|
||||
}
|
||||
}
|
||||
|
||||
return .{
|
||||
.models = models,
|
||||
.anchor_sets = anchor_sets,
|
||||
.recorded = recorded,
|
||||
.tickers = try loaded.toOwnedSlice(arena),
|
||||
};
|
||||
}
|
||||
|
||||
fn cmdReconstruct(
|
||||
arena: std.mem.Allocator,
|
||||
w: *std.Io.Writer,
|
||||
io: std.Io,
|
||||
env: *const std.process.Environ.Map,
|
||||
o: Options,
|
||||
) !u8 {
|
||||
const l = try load(arena, io, env, o.data_dir);
|
||||
|
||||
for (symbols) |sym| {
|
||||
const in = try l.inputsFor(sym);
|
||||
const points = try recon.build(arena, in);
|
||||
|
||||
var aw: std.Io.Writer.Allocating = .init(arena);
|
||||
try series.write(&aw.writer, sym, points);
|
||||
|
||||
const path = try std.fs.path.join(arena, &.{ o.data_dir, try std.fmt.allocPrint(arena, "{s}.srf", .{sym}) });
|
||||
try std.Io.Dir.cwd().writeFile(io, .{ .sub_path = path, .data = aw.written() });
|
||||
|
||||
var anchors: usize = 0;
|
||||
for (points) |p| {
|
||||
if (p.source == .anchor) anchors += 1;
|
||||
}
|
||||
try w.print("{s:<6} {d:>5} sessions {s} .. {s} ({d} anchors, {d} reconstructed) -> {s}\n", .{
|
||||
sym,
|
||||
points.len,
|
||||
points[0].date,
|
||||
points[points.len - 1].date,
|
||||
anchors,
|
||||
points.len - anchors,
|
||||
path,
|
||||
});
|
||||
}
|
||||
try w.print("\nRun `zfin-vestwell verify` to check the result.\n", .{});
|
||||
return 0;
|
||||
}
|
||||
|
||||
fn cmdVerify(
|
||||
arena: std.mem.Allocator,
|
||||
w: *std.Io.Writer,
|
||||
io: std.Io,
|
||||
env: *const std.process.Environ.Map,
|
||||
o: Options,
|
||||
) !u8 {
|
||||
const l = try load(arena, io, env, o.data_dir);
|
||||
var failed = false;
|
||||
|
||||
var verify_today_buf: [10]u8 = undefined;
|
||||
const today = try civil.today(io, &verify_today_buf);
|
||||
|
||||
// Live feed cross-check first: if the committed series no longer ends at the
|
||||
// published value, everything below is being checked against stale evidence.
|
||||
var feed_body: ?[]const u8 = null;
|
||||
if (!o.offline) {
|
||||
feed_body = feed.fetchBody(arena, io, feed.url) catch |err| blk: {
|
||||
std.log.warn("live feed unreachable ({s}); continuing without it", .{@errorName(err)});
|
||||
break :blk null;
|
||||
};
|
||||
}
|
||||
|
||||
for (symbols) |sym| {
|
||||
const in = try l.inputsFor(sym);
|
||||
const raw = try readDataFile(arena, io, o.data_dir, try std.fmt.allocPrint(arena, "{s}.srf", .{sym}));
|
||||
const points = try series.read(arena, raw);
|
||||
|
||||
const r = try verify.run(arena, in, points, l.recorded, today, .{});
|
||||
|
||||
try w.print("== {s} ==\n", .{r.symbol});
|
||||
try w.print(" sessions {d} ({s} .. {s})\n", .{ r.points, r.first_date, r.last_date });
|
||||
try w.print(" anchors exact {d}/{d}, worst {e}\n", .{
|
||||
r.anchors_checked, in.anchors.items.len, r.anchor_max_abs,
|
||||
});
|
||||
try w.print(" leave-one-out {d} interior anchors, worst {d:.3}%\n", .{
|
||||
r.loo_checked, r.loo_worst_pct,
|
||||
});
|
||||
try w.print(" vs hand-typed n={d}, mean {d:.4}, stdev {d:.4}, max {d:.4} ({d:.3}%)\n", .{
|
||||
r.recorded.n, r.recorded.mean, r.recorded.stdev, r.recorded.max_abs, r.recorded_worst_pct,
|
||||
});
|
||||
|
||||
// Informational: what fee each long gap implies, vs the published table.
|
||||
try w.print(" implied drag ", .{});
|
||||
var first = true;
|
||||
for (0..in.anchors.items.len - 1) |g| {
|
||||
const days = try civil.daysBetween(in.anchors.items[g].date, in.anchors.items[g + 1].date);
|
||||
if (days < 90) continue; // shorter spans cannot identify a fee
|
||||
const drag = (try verify.impliedDrag(arena, in, g)) orelse continue;
|
||||
if (!first) try w.print(", ", .{});
|
||||
try w.print("{d:.3}%/yr over {d}d", .{ drag * 100, days });
|
||||
first = false;
|
||||
}
|
||||
if (first) try w.print("(no gap long enough)", .{});
|
||||
try w.print("\n", .{});
|
||||
|
||||
if (feed_body) |body| {
|
||||
const fund_id = feed.fundIdFor(sym) orelse return error.NoFundIdForSymbol;
|
||||
const q = try feed.quoteFrom(arena, body, fund_id);
|
||||
const last = points[points.len - 1];
|
||||
const matches_date = std.mem.eql(u8, last.date, q.date);
|
||||
const err = @abs(last.unit_value - q.unit_value) / q.unit_value;
|
||||
if (!matches_date or err > 1e-9) {
|
||||
try w.print(" live feed MISMATCH: feed {d:.6} @ {s}, series ends {d:.6} @ {s}\n", .{
|
||||
q.unit_value, q.date, last.unit_value, last.date,
|
||||
});
|
||||
try w.print(" the series is stale; add the current value to anchors.srf and re-run reconstruct\n", .{});
|
||||
failed = true;
|
||||
} else {
|
||||
try w.print(" live feed matches ({d:.6} @ {s})\n", .{ q.unit_value, q.date });
|
||||
}
|
||||
} else {
|
||||
try w.print(" live feed skipped\n", .{});
|
||||
}
|
||||
|
||||
if (r.ok()) {
|
||||
try w.print(" RESULT pass\n\n", .{});
|
||||
} else {
|
||||
failed = true;
|
||||
try w.print(" RESULT FAIL ({d} problem(s))\n", .{r.failures.len});
|
||||
for (r.failures) |msg| try w.print(" - {s}\n", .{msg});
|
||||
try w.print("\n", .{});
|
||||
}
|
||||
}
|
||||
|
||||
if (failed) {
|
||||
try w.print("verification FAILED\n", .{});
|
||||
return 1;
|
||||
}
|
||||
try w.print("verification passed\n", .{});
|
||||
return 0;
|
||||
}
|
||||
|
||||
pub fn main(init: std.process.Init) !u8 {
|
||||
var arena_state: std.heap.ArenaAllocator = .init(init.gpa);
|
||||
defer arena_state.deinit();
|
||||
const arena = arena_state.allocator();
|
||||
const io = init.io;
|
||||
|
||||
var out_buf: [16 * 1024]u8 = undefined;
|
||||
var stdout = std.Io.File.stdout().writer(io, &out_buf);
|
||||
const w = &stdout.interface;
|
||||
|
||||
const code = try run(arena, w, io, init);
|
||||
|
||||
// Flush explicitly rather than in a `defer ... catch {}`: a failed flush
|
||||
// means output was lost, which is not something to discard silently.
|
||||
//
|
||||
// `WriteFailed` is the only error `flush` can produce, and it collapses a
|
||||
// benign closed pipe (`| head`) together with a genuine I/O failure. Since
|
||||
// the two cannot be told apart here, and the closed pipe is overwhelmingly
|
||||
// the common case, return the command's own exit code -- but log the error
|
||||
// so the fact of it is not thrown away.
|
||||
w.flush() catch |err| {
|
||||
std.log.debug("stdout flush failed: {s}", .{@errorName(err)});
|
||||
};
|
||||
return code;
|
||||
}
|
||||
|
||||
fn run(arena: std.mem.Allocator, w: *std.Io.Writer, io: std.Io, init: std.process.Init) !u8 {
|
||||
const args = try init.minimal.args.toSlice(arena);
|
||||
const o = parseArgs(args) catch |err| {
|
||||
try w.print("{s}\nerror: {s}\n", .{ usage, @errorName(err) });
|
||||
return 2;
|
||||
};
|
||||
|
||||
return switch (o.cmd) {
|
||||
.help => blk: {
|
||||
try w.print("{s}", .{usage});
|
||||
break :blk 0;
|
||||
},
|
||||
.reconstruct => cmdReconstruct(arena, w, io, init.environ_map, o),
|
||||
.verify => cmdVerify(arena, w, io, init.environ_map, o),
|
||||
};
|
||||
}
|
||||
|
||||
const testing = std.testing;
|
||||
|
||||
test "parseArgs recognises both commands and their flags" {
|
||||
{
|
||||
const o = try parseArgs(&.{ "zfin-vestwell", "reconstruct" });
|
||||
try testing.expectEqual(Command.reconstruct, o.cmd);
|
||||
try testing.expectEqualStrings("data", o.data_dir);
|
||||
try testing.expect(!o.offline);
|
||||
}
|
||||
{
|
||||
const o = try parseArgs(&.{ "zfin-vestwell", "verify", "--offline" });
|
||||
try testing.expectEqual(Command.verify, o.cmd);
|
||||
try testing.expect(o.offline);
|
||||
}
|
||||
{
|
||||
const o = try parseArgs(&.{ "zfin-vestwell", "verify", "--data-dir", "/tmp/x", "--offline" });
|
||||
try testing.expectEqualStrings("/tmp/x", o.data_dir);
|
||||
try testing.expect(o.offline);
|
||||
}
|
||||
}
|
||||
|
||||
test "parseArgs defaults to help and rejects bad input" {
|
||||
try testing.expectEqual(Command.help, (try parseArgs(&.{"prog"})).cmd);
|
||||
try testing.expectEqual(Command.help, (try parseArgs(&.{ "prog", "--help" })).cmd);
|
||||
try testing.expectEqual(Command.help, (try parseArgs(&.{ "prog", "-h" })).cmd);
|
||||
try testing.expectEqual(Command.help, (try parseArgs(&.{ "prog", "help" })).cmd);
|
||||
try testing.expectError(error.UnknownCommand, parseArgs(&.{ "prog", "frobnicate" }));
|
||||
try testing.expectError(error.UnknownFlag, parseArgs(&.{ "prog", "verify", "--nope" }));
|
||||
try testing.expectError(error.MissingArgument, parseArgs(&.{ "prog", "verify", "--data-dir" }));
|
||||
}
|
||||
|
||||
test "the tracked symbol list matches the feed's known fund ids" {
|
||||
// A symbol added here without a fund id would make `verify` fail at runtime
|
||||
// rather than at compile time, so assert the two stay in step.
|
||||
for (symbols) |s| try testing.expect(feed.fundIdFor(s) != null);
|
||||
}
|
||||
|
||||
test {
|
||||
// Force the CLI entry points to compile in the test binary. Without this
|
||||
// they are dead-code eliminated, and `zig build test` can pass while
|
||||
// `zig build` fails on a type error inside `main`.
|
||||
std.testing.refAllDecls(@This());
|
||||
|
||||
// Pull every module into the test binary.
|
||||
_ = candles;
|
||||
_ = civil;
|
||||
_ = data;
|
||||
_ = feed;
|
||||
_ = recon;
|
||||
_ = series;
|
||||
_ = verify;
|
||||
}
|
||||
622
src/recon.zig
Normal file
622
src/recon.zig
Normal file
|
|
@ -0,0 +1,622 @@
|
|||
//! The reconstruction.
|
||||
//!
|
||||
//! ## What it computes
|
||||
//!
|
||||
//! A daily unit-value series for a plan portfolio, exact at every observed
|
||||
//! anchor and interpolated between them.
|
||||
//!
|
||||
//! Within each anchor gap:
|
||||
//!
|
||||
//! 1. Walk the trading-day calendar, compounding the weighted total return of
|
||||
//! the underlying funds. Weights come from the era in force on each day, so
|
||||
//! a glidepath step mid-gap is handled without splitting the gap.
|
||||
//! 2. That walk lands somewhere near, but not exactly on, the closing anchor.
|
||||
//! Take the ratio of actual to walked value and distribute it geometrically
|
||||
//! across the gap, so both endpoints are exact and the correction is spread
|
||||
//! evenly in log space.
|
||||
//!
|
||||
//! ## Why the pinning matters more than the model
|
||||
//!
|
||||
//! Step 2 is what makes this accurate, and it is why no fee term appears
|
||||
//! anywhere. The asset-based fee, cash drag, trade-date lag and
|
||||
//! securities-lending income are all roughly constant-rate effects, and a
|
||||
//! constant-rate misfit is exactly what geometric distribution removes. An
|
||||
//! unpinned walk across ORCBI's 6.5-year pre-2025 gap is off by -1.29%; pinned,
|
||||
//! the same gap predicts a held-out interior anchor to +0.013%.
|
||||
//!
|
||||
//! Leave-one-out cross-validation over the real anchors bounds interior error at
|
||||
//! 0.079% (ORCBI) and 0.060% (ORC42). `verify` re-runs that on demand.
|
||||
|
||||
const std = @import("std");
|
||||
const data = @import("data.zig");
|
||||
const candles = @import("candles.zig");
|
||||
const civil = @import("civil.zig");
|
||||
|
||||
pub const PointSource = enum {
|
||||
/// An observed value, reproduced exactly.
|
||||
anchor,
|
||||
/// Derived. Accurate to well under 0.1%, but not observed.
|
||||
reconstructed,
|
||||
};
|
||||
|
||||
pub const Point = struct {
|
||||
date: []const u8,
|
||||
unit_value: f64,
|
||||
source: PointSource,
|
||||
};
|
||||
|
||||
/// Everything one symbol's reconstruction needs.
|
||||
pub const Inputs = struct {
|
||||
model: data.Model,
|
||||
anchors: data.AnchorSet,
|
||||
tickers: []const candles.Ticker,
|
||||
};
|
||||
|
||||
fn findTicker(tickers: []const candles.Ticker, name: []const u8) ?candles.Ticker {
|
||||
for (tickers) |t| {
|
||||
if (std.mem.eql(u8, t.name, name)) return t;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
/// The trading-day calendar for a portfolio: every date on which the model can
|
||||
/// actually be evaluated.
|
||||
///
|
||||
/// Built as the union of all tickers' dates, then filtered to days where an era
|
||||
/// is in force AND every fund that era references has a bar. Union-then-filter
|
||||
/// rather than a flat intersection because a future glidepath era will introduce
|
||||
/// funds (VBIPX, VTSPX, VUSXX) with shorter histories -- intersecting across all
|
||||
/// eras would truncate the whole series back to the newest fund's inception.
|
||||
pub fn calendar(
|
||||
arena: std.mem.Allocator,
|
||||
model: data.Model,
|
||||
tickers: []const candles.Ticker,
|
||||
) ![]const []const u8 {
|
||||
var seen: std.StringHashMapUnmanaged(void) = .empty;
|
||||
var all: std.ArrayList([]const u8) = .empty;
|
||||
for (tickers) |t| {
|
||||
for (t.dates) |d| {
|
||||
const gop = try seen.getOrPut(arena, d);
|
||||
if (!gop.found_existing) try all.append(arena, d);
|
||||
}
|
||||
}
|
||||
const merged = try all.toOwnedSlice(arena);
|
||||
std.mem.sort([]const u8, merged, {}, struct {
|
||||
fn lt(_: void, a: []const u8, b: []const u8) bool {
|
||||
return civil.lessThan(a, b);
|
||||
}
|
||||
}.lt);
|
||||
|
||||
var out: std.ArrayList([]const u8) = .empty;
|
||||
next: for (merged) |d| {
|
||||
const era = model.eraOn(d) orelse continue;
|
||||
for (era.weights) |w| {
|
||||
const t = findTicker(tickers, w.ticker) orelse continue :next;
|
||||
if (!t.adj.contains(d)) continue :next;
|
||||
}
|
||||
try out.append(arena, d);
|
||||
}
|
||||
return out.toOwnedSlice(arena);
|
||||
}
|
||||
|
||||
/// One day's weighted total return, as a fraction.
|
||||
///
|
||||
/// Sum of weight times each fund's own return, i.e. a daily-rebalanced basket.
|
||||
/// That matches the plan's actual mechanics closely enough: the booklet says
|
||||
/// daily cash flows are allocated toward target weights with a full rebalance
|
||||
/// quarterly.
|
||||
fn dayReturn(
|
||||
model: data.Model,
|
||||
tickers: []const candles.Ticker,
|
||||
prev: []const u8,
|
||||
cur: []const u8,
|
||||
) !f64 {
|
||||
const era = model.eraOn(cur) orelse return error.NoEraInForce;
|
||||
var r: f64 = 0;
|
||||
for (era.weights) |w| {
|
||||
const t = findTicker(tickers, w.ticker) orelse return error.MissingTicker;
|
||||
const a = t.at(prev) orelse return error.MissingBar;
|
||||
const b = t.at(cur) orelse return error.MissingBar;
|
||||
if (!(a > 0) or !(b > 0)) return error.NonPositivePrice;
|
||||
r += w.fraction * (b / a - 1.0);
|
||||
}
|
||||
return r;
|
||||
}
|
||||
|
||||
/// Reconstruct one anchor gap, pinned exactly at both ends.
|
||||
///
|
||||
/// Returns a value for every calendar day in `cal[from..to]` inclusive, so the
|
||||
/// result has `to - from + 1` entries. `out[0] == v0` and the last entry is `v1`
|
||||
/// to within floating-point rounding.
|
||||
///
|
||||
/// Exposed because leave-one-out validation needs to pin a gap across a
|
||||
/// deliberately-omitted anchor.
|
||||
pub fn pinGap(
|
||||
arena: std.mem.Allocator,
|
||||
model: data.Model,
|
||||
tickers: []const candles.Ticker,
|
||||
cal: []const []const u8,
|
||||
from: usize,
|
||||
v0: f64,
|
||||
to: usize,
|
||||
v1: f64,
|
||||
) ![]f64 {
|
||||
if (to <= from) return error.EmptyGap;
|
||||
if (to >= cal.len) return error.GapOutOfRange;
|
||||
|
||||
const n = to - from;
|
||||
const raw = try arena.alloc(f64, n + 1);
|
||||
raw[0] = v0;
|
||||
for (1..n + 1) |k| {
|
||||
const r = try dayReturn(model, tickers, cal[from + k - 1], cal[from + k]);
|
||||
raw[k] = raw[k - 1] * (1.0 + r);
|
||||
}
|
||||
if (!(raw[n] > 0)) return error.WalkCollapsed;
|
||||
|
||||
// Geometric residual distribution: multiply day k by resid^(k/n), so the
|
||||
// correction compounds smoothly and both endpoints land exactly.
|
||||
const resid = v1 / raw[n];
|
||||
const nf: f64 = @floatFromInt(n);
|
||||
for (0..n + 1) |k| {
|
||||
const kf: f64 = @floatFromInt(k);
|
||||
raw[k] *= std.math.pow(f64, resid, kf / nf);
|
||||
}
|
||||
// Pin the tail exactly rather than trusting pow(resid, 1.0) to round-trip.
|
||||
raw[n] = v1;
|
||||
return raw;
|
||||
}
|
||||
|
||||
/// Snap each anchor onto the trading-day calendar.
|
||||
///
|
||||
/// The first anchor snaps FORWARD and the rest snap BACKWARD. That asymmetry is
|
||||
/// deliberate: both funds launched on a Saturday, and the launch value belongs to
|
||||
/// the first session the fund actually traded, not to the Friday before it
|
||||
/// existed. Every other anchor is a published value for a date that already is a
|
||||
/// session, so backward-snapping is a no-op in practice and merely defensive.
|
||||
fn snapAnchors(
|
||||
arena: std.mem.Allocator,
|
||||
anchors: []const data.Anchor,
|
||||
cal: []const []const u8,
|
||||
) ![]usize {
|
||||
const idx = try arena.alloc(usize, anchors.len);
|
||||
for (anchors, 0..) |a, i| {
|
||||
idx[i] = if (i == 0)
|
||||
candles.firstAtOrAfter(cal, a.date) orelse return error.AnchorBeforeCalendar
|
||||
else
|
||||
candles.lastAtOrBefore(cal, a.date) orelse return error.AnchorBeforeCalendar;
|
||||
}
|
||||
for (idx[1..], 0..) |v, i| {
|
||||
if (v <= idx[i]) return error.AnchorsCollideOnCalendar;
|
||||
}
|
||||
return idx;
|
||||
}
|
||||
|
||||
/// Build the full daily series for one symbol.
|
||||
pub fn build(arena: std.mem.Allocator, in: Inputs) ![]const Point {
|
||||
const cal = try calendar(arena, in.model, in.tickers);
|
||||
if (cal.len == 0) return error.EmptyCalendar;
|
||||
|
||||
const idx = try snapAnchors(arena, in.anchors.items, cal);
|
||||
|
||||
var out: std.ArrayList(Point) = .empty;
|
||||
for (in.anchors.items[1..], 0..) |a1, seg| {
|
||||
const a0 = in.anchors.items[seg];
|
||||
const gap_start = idx[seg];
|
||||
const gap_end = idx[seg + 1];
|
||||
const vals = try pinGap(arena, in.model, in.tickers, cal, gap_start, a0.unit_value, gap_end, a1.unit_value);
|
||||
|
||||
// The shared boundary day was already emitted as the previous segment's
|
||||
// closing anchor, so only the first segment contributes its own start.
|
||||
const skip: usize = if (seg == 0) 0 else 1;
|
||||
for (vals[skip..], skip..) |v, k| {
|
||||
try out.append(arena, .{
|
||||
.date = cal[gap_start + k],
|
||||
.unit_value = v,
|
||||
.source = if (k == 0 or k == vals.len - 1) .anchor else .reconstructed,
|
||||
});
|
||||
}
|
||||
}
|
||||
return out.toOwnedSlice(arena);
|
||||
}
|
||||
|
||||
/// Value at an exact date, or null when the series has no entry for it.
|
||||
pub fn valueAt(points: []const Point, date: []const u8) ?f64 {
|
||||
for (points) |p| {
|
||||
if (std.mem.eql(u8, p.date, date)) return p.unit_value;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
/// Value on the last day strictly before `date`, or null when there is none.
|
||||
///
|
||||
/// This is the lag=1 alignment the hand-typed values need: the portal publishes
|
||||
/// the prior session's unit value, so a number written down on day D reports
|
||||
/// day D-1.
|
||||
pub fn valueBefore(points: []const Point, date: []const u8) ?f64 {
|
||||
var found: ?f64 = null;
|
||||
for (points) |p| {
|
||||
if (!civil.lessThan(p.date, date)) break;
|
||||
found = p.unit_value;
|
||||
}
|
||||
return found;
|
||||
}
|
||||
|
||||
const testing = std.testing;
|
||||
|
||||
/// A two-fund model and two flat price series, for algebra tests where the
|
||||
/// expected answer can be computed by hand.
|
||||
fn fixture(arena: std.mem.Allocator) !struct {
|
||||
model: data.Model,
|
||||
tickers: []const candles.Ticker,
|
||||
} {
|
||||
const models = try data.parseModel(arena,
|
||||
\\#!srfv1
|
||||
\\symbol::T,era_start::2020-01-01,ticker::AA,weight:num:50
|
||||
\\symbol::T,era_start::2020-01-01,ticker::BB,weight:num:50
|
||||
\\
|
||||
);
|
||||
const aa = try candles.parse(arena, "AA", "#!srfv1\n" ++
|
||||
"date::2020-01-01,adj_close:num:100\n" ++
|
||||
"date::2020-01-02,adj_close:num:110\n" ++
|
||||
"date::2020-01-03,adj_close:num:121\n");
|
||||
const bb = try candles.parse(arena, "BB", "#!srfv1\n" ++
|
||||
"date::2020-01-01,adj_close:num:100\n" ++
|
||||
"date::2020-01-02,adj_close:num:100\n" ++
|
||||
"date::2020-01-03,adj_close:num:100\n");
|
||||
const tickers = try arena.dupe(candles.Ticker, &.{ aa, bb });
|
||||
return .{ .model = data.findModel(models, "T").?, .tickers = tickers };
|
||||
}
|
||||
|
||||
test "calendar keeps only days where the era in force has every fund" {
|
||||
var arena_state: std.heap.ArenaAllocator = .init(testing.allocator);
|
||||
defer arena_state.deinit();
|
||||
const arena = arena_state.allocator();
|
||||
|
||||
const models = try data.parseModel(arena,
|
||||
\\#!srfv1
|
||||
\\symbol::T,era_start::2020-01-02,ticker::AA,weight:num:100
|
||||
\\
|
||||
);
|
||||
const aa = try candles.parse(arena, "AA", "#!srfv1\n" ++
|
||||
"date::2020-01-01,adj_close:num:1\n" ++
|
||||
"date::2020-01-02,adj_close:num:1\n" ++
|
||||
"date::2020-01-03,adj_close:num:1\n");
|
||||
const cal = try calendar(arena, data.findModel(models, "T").?, &.{aa});
|
||||
|
||||
// 2020-01-01 predates the only era, so it is dropped.
|
||||
try testing.expectEqual(@as(usize, 2), cal.len);
|
||||
try testing.expectEqualStrings("2020-01-02", cal[0]);
|
||||
try testing.expectEqualStrings("2020-01-03", cal[1]);
|
||||
}
|
||||
|
||||
test "calendar is not truncated by a fund introduced in a later era" {
|
||||
var arena_state: std.heap.ArenaAllocator = .init(testing.allocator);
|
||||
defer arena_state.deinit();
|
||||
const arena = arena_state.allocator();
|
||||
|
||||
// BB only exists from 2020-01-03, and only era 2 references it. The early
|
||||
// days must survive, driven by era 1's AA-only weights.
|
||||
const models = try data.parseModel(arena,
|
||||
\\#!srfv1
|
||||
\\symbol::T,era_start::2020-01-01,ticker::AA,weight:num:100
|
||||
\\symbol::T,era_start::2020-01-03,ticker::AA,weight:num:50
|
||||
\\symbol::T,era_start::2020-01-03,ticker::BB,weight:num:50
|
||||
\\
|
||||
);
|
||||
const aa = try candles.parse(arena, "AA", "#!srfv1\n" ++
|
||||
"date::2020-01-01,adj_close:num:1\n" ++
|
||||
"date::2020-01-02,adj_close:num:1\n" ++
|
||||
"date::2020-01-03,adj_close:num:1\n");
|
||||
const bb = try candles.parse(arena, "BB", "#!srfv1\n" ++
|
||||
"date::2020-01-03,adj_close:num:1\n");
|
||||
|
||||
const cal = try calendar(arena, data.findModel(models, "T").?, &.{ aa, bb });
|
||||
try testing.expectEqual(@as(usize, 3), cal.len);
|
||||
try testing.expectEqualStrings("2020-01-01", cal[0]);
|
||||
try testing.expectEqualStrings("2020-01-03", cal[2]);
|
||||
}
|
||||
|
||||
test "pinGap hits both endpoints exactly" {
|
||||
var arena_state: std.heap.ArenaAllocator = .init(testing.allocator);
|
||||
defer arena_state.deinit();
|
||||
const arena = arena_state.allocator();
|
||||
const f = try fixture(arena);
|
||||
const cal = try calendar(arena, f.model, f.tickers);
|
||||
|
||||
// Deliberately ask for a closing value the basket walk will NOT produce, so
|
||||
// the residual distribution has real work to do.
|
||||
const vals = try pinGap(arena, f.model, f.tickers, cal, 0, 10.0, 2, 12.0);
|
||||
try testing.expectEqual(@as(usize, 3), vals.len);
|
||||
try testing.expectApproxEqAbs(@as(f64, 10.0), vals[0], 1e-12);
|
||||
try testing.expectApproxEqAbs(@as(f64, 12.0), vals[2], 1e-12);
|
||||
// Interior is bracketed and monotone for a monotone basket.
|
||||
try testing.expect(vals[1] > vals[0]);
|
||||
try testing.expect(vals[1] < vals[2]);
|
||||
}
|
||||
|
||||
test "pinGap reproduces the basket walk when the endpoint already agrees" {
|
||||
var arena_state: std.heap.ArenaAllocator = .init(testing.allocator);
|
||||
defer arena_state.deinit();
|
||||
const arena = arena_state.allocator();
|
||||
const f = try fixture(arena);
|
||||
const cal = try calendar(arena, f.model, f.tickers);
|
||||
|
||||
// AA doubles-ish, BB flat, 50/50 daily-rebalanced:
|
||||
// day 1: 0.5*(110/100 - 1) + 0.5*0 = +5%
|
||||
// day 2: 0.5*(121/110 - 1) + 0.5*0 = +5%
|
||||
// So 10.00 -> 10.50 -> 11.025. Pin to that and nothing should move.
|
||||
const vals = try pinGap(arena, f.model, f.tickers, cal, 0, 10.0, 2, 11.025);
|
||||
try testing.expectApproxEqAbs(@as(f64, 10.0), vals[0], 1e-12);
|
||||
try testing.expectApproxEqAbs(@as(f64, 10.5), vals[1], 1e-9);
|
||||
try testing.expectApproxEqAbs(@as(f64, 11.025), vals[2], 1e-12);
|
||||
}
|
||||
|
||||
test "pinGap rejects degenerate ranges" {
|
||||
var arena_state: std.heap.ArenaAllocator = .init(testing.allocator);
|
||||
defer arena_state.deinit();
|
||||
const arena = arena_state.allocator();
|
||||
const f = try fixture(arena);
|
||||
const cal = try calendar(arena, f.model, f.tickers);
|
||||
|
||||
try testing.expectError(error.EmptyGap, pinGap(arena, f.model, f.tickers, cal, 1, 10, 1, 11));
|
||||
try testing.expectError(error.EmptyGap, pinGap(arena, f.model, f.tickers, cal, 2, 10, 1, 11));
|
||||
try testing.expectError(error.GapOutOfRange, pinGap(arena, f.model, f.tickers, cal, 0, 10, 99, 11));
|
||||
}
|
||||
|
||||
test "build labels anchors and reconstructed days and spans every session" {
|
||||
var arena_state: std.heap.ArenaAllocator = .init(testing.allocator);
|
||||
defer arena_state.deinit();
|
||||
const arena = arena_state.allocator();
|
||||
const f = try fixture(arena);
|
||||
|
||||
const sets = try data.parseAnchors(arena,
|
||||
\\#!srfv1
|
||||
\\symbol::T,date::2020-01-01,unit_value:num:10.0,source::launch
|
||||
\\symbol::T,date::2020-01-03,unit_value:num:12.0,source::feed
|
||||
\\
|
||||
);
|
||||
const points = try build(arena, .{
|
||||
.model = f.model,
|
||||
.anchors = data.findAnchors(sets, "T").?,
|
||||
.tickers = f.tickers,
|
||||
});
|
||||
|
||||
try testing.expectEqual(@as(usize, 3), points.len);
|
||||
try testing.expectEqual(PointSource.anchor, points[0].source);
|
||||
try testing.expectEqual(PointSource.reconstructed, points[1].source);
|
||||
try testing.expectEqual(PointSource.anchor, points[2].source);
|
||||
try testing.expectApproxEqAbs(@as(f64, 10.0), points[0].unit_value, 1e-12);
|
||||
try testing.expectApproxEqAbs(@as(f64, 12.0), points[2].unit_value, 1e-12);
|
||||
|
||||
// Dates strictly ascending, no duplicate at a segment boundary.
|
||||
for (points[1..], 0..) |p, i| {
|
||||
try testing.expect(civil.lessThan(points[i].date, p.date));
|
||||
}
|
||||
}
|
||||
|
||||
test "build emits each boundary day once across adjacent segments" {
|
||||
var arena_state: std.heap.ArenaAllocator = .init(testing.allocator);
|
||||
defer arena_state.deinit();
|
||||
const arena = arena_state.allocator();
|
||||
const f = try fixture(arena);
|
||||
|
||||
// Three anchors means two segments sharing 2020-01-02.
|
||||
const sets = try data.parseAnchors(arena,
|
||||
\\#!srfv1
|
||||
\\symbol::T,date::2020-01-01,unit_value:num:10.0,source::launch
|
||||
\\symbol::T,date::2020-01-02,unit_value:num:10.4,source::wayback
|
||||
\\symbol::T,date::2020-01-03,unit_value:num:12.0,source::feed
|
||||
\\
|
||||
);
|
||||
const points = try build(arena, .{
|
||||
.model = f.model,
|
||||
.anchors = data.findAnchors(sets, "T").?,
|
||||
.tickers = f.tickers,
|
||||
});
|
||||
|
||||
try testing.expectEqual(@as(usize, 3), points.len);
|
||||
// The shared day carries its observed value, exactly, and only once.
|
||||
try testing.expectEqualStrings("2020-01-02", points[1].date);
|
||||
try testing.expectApproxEqAbs(@as(f64, 10.4), points[1].unit_value, 1e-12);
|
||||
try testing.expectEqual(PointSource.anchor, points[1].source);
|
||||
}
|
||||
|
||||
test "build snaps a weekend launch anchor forward to the first session" {
|
||||
var arena_state: std.heap.ArenaAllocator = .init(testing.allocator);
|
||||
defer arena_state.deinit();
|
||||
const arena = arena_state.allocator();
|
||||
|
||||
const models = try data.parseModel(arena,
|
||||
\\#!srfv1
|
||||
\\symbol::T,era_start::2018-01-01,ticker::AA,weight:num:100
|
||||
\\
|
||||
);
|
||||
// 2018-09-01 and 09-02 are a weekend, 09-03 Labor Day: first session is 09-04.
|
||||
const aa = try candles.parse(arena, "AA", "#!srfv1\n" ++
|
||||
"date::2018-08-31,adj_close:num:100\n" ++
|
||||
"date::2018-09-04,adj_close:num:101\n" ++
|
||||
"date::2018-09-05,adj_close:num:102\n");
|
||||
const sets = try data.parseAnchors(arena,
|
||||
\\#!srfv1
|
||||
\\symbol::T,date::2018-09-01,unit_value:num:10.0,source::launch
|
||||
\\symbol::T,date::2018-09-05,unit_value:num:10.5,source::feed
|
||||
\\
|
||||
);
|
||||
const points = try build(arena, .{
|
||||
.model = data.findModel(models, "T").?,
|
||||
.anchors = data.findAnchors(sets, "T").?,
|
||||
.tickers = &.{aa},
|
||||
});
|
||||
|
||||
// 2018-08-31 must NOT appear: the fund did not exist before launch.
|
||||
try testing.expectEqualStrings("2018-09-04", points[0].date);
|
||||
try testing.expectApproxEqAbs(@as(f64, 10.0), points[0].unit_value, 1e-12);
|
||||
try testing.expectEqualStrings("2018-09-05", points[points.len - 1].date);
|
||||
}
|
||||
|
||||
test "valueAt and valueBefore implement exact and lag=1 lookup" {
|
||||
const points: []const Point = &.{
|
||||
.{ .date = "2026-08-25", .unit_value = 1.0, .source = .anchor },
|
||||
.{ .date = "2026-08-26", .unit_value = 2.0, .source = .reconstructed },
|
||||
.{ .date = "2026-08-27", .unit_value = 3.0, .source = .anchor },
|
||||
};
|
||||
|
||||
try testing.expectEqual(@as(?f64, 2.0), valueAt(points, "2026-08-26"));
|
||||
try testing.expectEqual(@as(?f64, null), valueAt(points, "2026-08-28"));
|
||||
|
||||
// Written down on the 27th means the 26th's value.
|
||||
try testing.expectEqual(@as(?f64, 2.0), valueBefore(points, "2026-08-27"));
|
||||
// A weekend snapshot reaches back to the last session.
|
||||
try testing.expectEqual(@as(?f64, 3.0), valueBefore(points, "2026-08-29"));
|
||||
// Nothing precedes the first point.
|
||||
try testing.expectEqual(@as(?f64, null), valueBefore(points, "2026-08-25"));
|
||||
}
|
||||
|
||||
test "dayReturn errors rather than guessing when a bar is missing" {
|
||||
var arena_state: std.heap.ArenaAllocator = .init(testing.allocator);
|
||||
defer arena_state.deinit();
|
||||
const arena = arena_state.allocator();
|
||||
|
||||
const models = try data.parseModel(arena,
|
||||
\\#!srfv1
|
||||
\\symbol::T,era_start::2020-01-01,ticker::AA,weight:num:100
|
||||
\\
|
||||
);
|
||||
const model = data.findModel(models, "T").?;
|
||||
const aa = try candles.parse(arena, "AA", "#!srfv1\n" ++
|
||||
"date::2020-01-01,adj_close:num:100\n" ++
|
||||
"date::2020-01-02,adj_close:num:110\n");
|
||||
|
||||
try testing.expectApproxEqAbs(@as(f64, 0.10), try dayReturn(model, &.{aa}, "2020-01-01", "2020-01-02"), 1e-12);
|
||||
try testing.expectError(error.MissingBar, dayReturn(model, &.{aa}, "2020-01-01", "2020-01-03"));
|
||||
try testing.expectError(error.NoEraInForce, dayReturn(model, &.{aa}, "2019-12-30", "2019-12-31"));
|
||||
|
||||
const bb = try candles.parse(arena, "BB", "#!srfv1\ndate::2020-01-02,adj_close:num:1\n");
|
||||
try testing.expectError(error.MissingTicker, dayReturn(model, &.{bb}, "2020-01-01", "2020-01-02"));
|
||||
}
|
||||
|
||||
test "dayReturn rejects a non-positive price rather than producing nonsense" {
|
||||
var arena_state: std.heap.ArenaAllocator = .init(testing.allocator);
|
||||
defer arena_state.deinit();
|
||||
const arena = arena_state.allocator();
|
||||
|
||||
const models = try data.parseModel(arena,
|
||||
\\#!srfv1
|
||||
\\symbol::T,era_start::2020-01-01,ticker::AA,weight:num:100
|
||||
\\
|
||||
);
|
||||
// A zero adj_close is zfin's "unusable" sentinel, not a real price.
|
||||
const aa = try candles.parse(arena, "AA", "#!srfv1\n" ++
|
||||
"date::2020-01-01,adj_close:num:0\n" ++
|
||||
"date::2020-01-02,adj_close:num:110\n");
|
||||
try testing.expectError(error.NonPositivePrice, dayReturn(
|
||||
data.findModel(models, "T").?,
|
||||
&.{aa},
|
||||
"2020-01-01",
|
||||
"2020-01-02",
|
||||
));
|
||||
}
|
||||
|
||||
test "output is invariant to rescaling the input price series" {
|
||||
// This is the property that decides whether the generated series can be
|
||||
// treated as a reproducible build artifact.
|
||||
//
|
||||
// zfin back-adjusts its `adj_close` series and re-bases it (`adj_basis::`)
|
||||
// whenever adjustments are restated, which rescales every historical bar of
|
||||
// a fund by some factor. The reconstruction consumes only per-day RATIOS of
|
||||
// adjusted closes, so any such rescaling must cancel exactly -- including
|
||||
// when each fund is rescaled by a DIFFERENT factor, which is what actually
|
||||
// happens since funds re-base independently.
|
||||
//
|
||||
// What this does NOT protect against: a provider adding or restating a
|
||||
// dividend. That changes the single daily ratio spanning its ex-date, and no
|
||||
// amount of scale invariance helps. So the output is stable under re-basing
|
||||
// but not under a corporate-action correction.
|
||||
var arena_state: std.heap.ArenaAllocator = .init(testing.allocator);
|
||||
defer arena_state.deinit();
|
||||
const arena = arena_state.allocator();
|
||||
|
||||
const models = try data.parseModel(arena,
|
||||
\\#!srfv1
|
||||
\\symbol::T,era_start::2020-01-01,ticker::AA,weight:num:60
|
||||
\\symbol::T,era_start::2020-01-01,ticker::BB,weight:num:40
|
||||
\\
|
||||
);
|
||||
const sets = try data.parseAnchors(arena,
|
||||
\\#!srfv1
|
||||
\\symbol::T,date::2020-01-01,unit_value:num:10.0,source::launch
|
||||
\\symbol::T,date::2020-01-08,unit_value:num:10.4,source::wayback
|
||||
\\symbol::T,date::2020-01-15,unit_value:num:11.1,source::feed
|
||||
\\
|
||||
);
|
||||
|
||||
// Same shape twice, second time with each fund scaled by its own factor.
|
||||
const build_series = struct {
|
||||
fn go(a: std.mem.Allocator, m: data.Model, an: data.AnchorSet, ka: f64, kb: f64) ![]const Point {
|
||||
var ta: std.ArrayList(u8) = .empty;
|
||||
var tb: std.ArrayList(u8) = .empty;
|
||||
try ta.appendSlice(a, "#!srfv1\n");
|
||||
try tb.appendSlice(a, "#!srfv1\n");
|
||||
for (1..16) |d| {
|
||||
const fd: f64 = @floatFromInt(d);
|
||||
try ta.print(a, "date::2020-01-{d:0>2},adj_close:num:{d}\n", .{ d, (100.0 + fd * 1.5) * ka });
|
||||
try tb.print(a, "date::2020-01-{d:0>2},adj_close:num:{d}\n", .{ d, (50.0 - fd * 0.25) * kb });
|
||||
}
|
||||
const aa = try candles.parse(a, "AA", ta.items);
|
||||
const bb = try candles.parse(a, "BB", tb.items);
|
||||
return build(a, .{ .model = m, .anchors = an, .tickers = try a.dupe(candles.Ticker, &.{ aa, bb }) });
|
||||
}
|
||||
}.go;
|
||||
|
||||
const model = data.findModel(models, "T").?;
|
||||
const anchors = data.findAnchors(sets, "T").?;
|
||||
|
||||
const base = try build_series(arena, model, anchors, 1.0, 1.0);
|
||||
// Wildly different, independent rescalings of each fund.
|
||||
const scaled = try build_series(arena, model, anchors, 0.8123456789, 41.7);
|
||||
|
||||
try testing.expectEqual(base.len, scaled.len);
|
||||
try testing.expect(base.len > 10);
|
||||
for (base, scaled) |b, s| {
|
||||
try testing.expectEqualStrings(b.date, s.date);
|
||||
try testing.expectEqual(b.source, s.source);
|
||||
// Ratios cancel, so this is equality up to floating-point noise only.
|
||||
try testing.expectApproxEqRel(b.unit_value, s.unit_value, 1e-12);
|
||||
}
|
||||
}
|
||||
|
||||
test "weights are inert when anchors are one session apart" {
|
||||
// Decides how much the model's forward-looking eras are actually worth.
|
||||
//
|
||||
// `pinGap` pins its terminal value to the caller's `v1`. With n == 1 there is
|
||||
// no interior day, so the output is exactly {v0, v1} whatever the weights
|
||||
// say. Once daily feed values are being appended to anchors.srf, every gap is
|
||||
// one session and the allocation model stops influencing the series at all.
|
||||
// It matters only for the historical backfill and for filling gaps if
|
||||
// forward recording lapses.
|
||||
var arena_state: std.heap.ArenaAllocator = .init(testing.allocator);
|
||||
defer arena_state.deinit();
|
||||
const arena = arena_state.allocator();
|
||||
|
||||
const cal: []const []const u8 = &.{ "2020-01-01", "2020-01-02" };
|
||||
const aa = try candles.parse(arena, "AA", "#!srfv1\n" ++
|
||||
"date::2020-01-01,adj_close:num:100\n" ++
|
||||
"date::2020-01-02,adj_close:num:180\n"); // a violent +80% day
|
||||
|
||||
for ([_]f64{ 100, 50, 1 }) |pct| {
|
||||
const src = try std.fmt.allocPrint(
|
||||
arena,
|
||||
"#!srfv1\nsymbol::T,era_start::2020-01-01,ticker::AA,weight:num:{d}\nsymbol::T,era_start::2020-01-01,ticker::BB,weight:num:{d}\n",
|
||||
.{ pct, 100 - pct },
|
||||
);
|
||||
const models = try data.parseModel(arena, src);
|
||||
const bb = try candles.parse(arena, "BB", "#!srfv1\n" ++
|
||||
"date::2020-01-01,adj_close:num:10\n" ++
|
||||
"date::2020-01-02,adj_close:num:10\n");
|
||||
|
||||
const vals = try pinGap(arena, data.findModel(models, "T").?, &.{ aa, bb }, cal, 0, 10.0, 1, 10.25);
|
||||
try testing.expectEqual(@as(usize, 2), vals.len);
|
||||
try testing.expectApproxEqAbs(@as(f64, 10.0), vals[0], 1e-12);
|
||||
try testing.expectApproxEqAbs(@as(f64, 10.25), vals[1], 1e-12);
|
||||
}
|
||||
}
|
||||
145
src/series.zig
Normal file
145
src/series.zig
Normal file
|
|
@ -0,0 +1,145 @@
|
|||
//! Emit and read back the reconstructed unit-value series.
|
||||
//!
|
||||
//! The series is stored as a unit-value series, NOT in zfin's candle shape.
|
||||
//! That separation is deliberate: the unit value is the semantic truth, and
|
||||
//! projecting it into `open`/`high`/`low`/`close`/`adj_close`/`volume` is the
|
||||
//! job of whatever populates a cache. Keeping the durable artifact free of
|
||||
//! zfin's cache format means a change to that format cannot invalidate it.
|
||||
//!
|
||||
//! Every row carries its own `source`, so no consumer can mistake a derived
|
||||
//! value for an observed one.
|
||||
|
||||
const std = @import("std");
|
||||
const srf = @import("srf");
|
||||
const srf_opts = @import("srf_opts.zig");
|
||||
const recon = @import("recon.zig");
|
||||
const civil = @import("civil.zig");
|
||||
|
||||
/// On-disk row. Field order here is the column order in the file.
|
||||
const Row = struct {
|
||||
date: []const u8,
|
||||
unit_value: f64,
|
||||
source: []const u8,
|
||||
};
|
||||
|
||||
/// Number of decimals used for `unit_value` on disk.
|
||||
///
|
||||
/// The plan reports unit values to 4 decimals (`navDecimals: 4` in its own
|
||||
/// config) and the feed publishes 6. Six preserves the feed's values verbatim,
|
||||
/// so anchors round-trip exactly, and keeps reconstructed values well inside
|
||||
/// their own error bars -- the sixth decimal is worth about a hundredth of a
|
||||
/// cent on the largest position.
|
||||
///
|
||||
/// Rows are formatted explicitly rather than through `srf.fmt` because SRF has
|
||||
/// no decimal-precision option, and its default shortest-round-trip float
|
||||
/// formatting would emit up to 17 significant digits. That is exact but makes a
|
||||
/// 2000-row committed file noisy, and makes its diffs jump on the last digit
|
||||
/// whenever an upstream adjusted price shifts imperceptibly.
|
||||
const decimals = 6;
|
||||
|
||||
pub fn write(
|
||||
w: *std.Io.Writer,
|
||||
symbol: []const u8,
|
||||
points: []const recon.Point,
|
||||
) !void {
|
||||
try w.print(
|
||||
\\#!srfv1
|
||||
\\# Daily unit-value series for {s}, Oregon College Savings Plan (Embark).
|
||||
\\#
|
||||
\\# GENERATED FILE -- produced by `zfin-vestwell reconstruct`. Do not hand
|
||||
\\# edit. To change the output, change data/model.srf or data/anchors.srf
|
||||
\\# and regenerate.
|
||||
\\#
|
||||
\\# source::anchor observed. Reproduced exactly. See data/anchors.srf
|
||||
\\# for where each observation came from.
|
||||
\\# source::reconstructed derived from the underlying funds' total return,
|
||||
\\# pinned to the surrounding anchors. Cross-validated
|
||||
\\# to under 0.08%; accurate, but NOT observed.
|
||||
\\#
|
||||
\\# These are unit values of a unitized trust, not mutual-fund NAVs: no
|
||||
\\# distributions, all income compounds into the value.
|
||||
\\
|
||||
, .{symbol});
|
||||
|
||||
// Built at comptime so `decimals` stays the single source of truth for the
|
||||
// on-disk precision, rather than being duplicated in a literal.
|
||||
const row_fmt = std.fmt.comptimePrint(
|
||||
"date::{{s}},unit_value:num:{{d:.{d}}},source::{{s}}\n",
|
||||
.{decimals},
|
||||
);
|
||||
for (points) |p| {
|
||||
try w.print(row_fmt, .{ p.date, p.unit_value, @tagName(p.source) });
|
||||
}
|
||||
}
|
||||
|
||||
/// Read a series file back. Used by `verify` so it checks the artifact on disk
|
||||
/// rather than an in-memory value it just computed.
|
||||
pub fn read(arena: std.mem.Allocator, data: []const u8) ![]const recon.Point {
|
||||
var reader = std.Io.Reader.fixed(data);
|
||||
var it = srf.iterator(&reader, arena, .{ .parse_allocator = .none }) catch
|
||||
return error.InvalidSeriesFile;
|
||||
defer it.deinit();
|
||||
|
||||
var out: std.ArrayList(recon.Point) = .empty;
|
||||
while (try it.next()) |fields| {
|
||||
const row = try fields.to(Row, srf_opts.machine_written);
|
||||
if (!civil.isValidIso(row.date)) return error.InvalidSeriesDate;
|
||||
const source = std.meta.stringToEnum(recon.PointSource, row.source) orelse
|
||||
return error.UnknownSeriesSource;
|
||||
try out.append(arena, .{
|
||||
.date = try arena.dupe(u8, row.date),
|
||||
.unit_value = row.unit_value,
|
||||
.source = source,
|
||||
});
|
||||
}
|
||||
return out.toOwnedSlice(arena);
|
||||
}
|
||||
|
||||
const testing = std.testing;
|
||||
|
||||
test "write then read round-trips values, dates and provenance" {
|
||||
var arena_state: std.heap.ArenaAllocator = .init(testing.allocator);
|
||||
defer arena_state.deinit();
|
||||
const arena = arena_state.allocator();
|
||||
|
||||
const points: []const recon.Point = &.{
|
||||
.{ .date = "2026-08-25", .unit_value = 19.012345, .source = .anchor },
|
||||
.{ .date = "2026-08-26", .unit_value = 19.05, .source = .reconstructed },
|
||||
.{ .date = "2026-08-27", .unit_value = 19.092721, .source = .anchor },
|
||||
};
|
||||
|
||||
var aw: std.Io.Writer.Allocating = .init(testing.allocator);
|
||||
defer aw.deinit();
|
||||
try write(&aw.writer, "ORCBI", points);
|
||||
const bytes = aw.written();
|
||||
|
||||
try testing.expect(std.mem.startsWith(u8, bytes, "#!srfv1\n"));
|
||||
try testing.expect(std.mem.endsWith(u8, bytes, "\n"));
|
||||
try testing.expect(std.mem.indexOf(u8, bytes, "GENERATED FILE") != null);
|
||||
|
||||
const back = try read(arena, bytes);
|
||||
try testing.expectEqual(@as(usize, 3), back.len);
|
||||
try testing.expectEqualStrings("2026-08-25", back[0].date);
|
||||
try testing.expectEqual(recon.PointSource.anchor, back[0].source);
|
||||
try testing.expectEqual(recon.PointSource.reconstructed, back[1].source);
|
||||
// Six decimals must survive the round trip: anchors are exact values and
|
||||
// silently rounding them would break the exactness check in `verify`.
|
||||
try testing.expectApproxEqAbs(@as(f64, 19.012345), back[0].unit_value, 1e-9);
|
||||
try testing.expectApproxEqAbs(@as(f64, 19.092721), back[2].unit_value, 1e-9);
|
||||
}
|
||||
|
||||
test "read rejects a malformed date or unknown source" {
|
||||
var arena_state: std.heap.ArenaAllocator = .init(testing.allocator);
|
||||
defer arena_state.deinit();
|
||||
const arena = arena_state.allocator();
|
||||
|
||||
try testing.expectError(error.InvalidSeriesDate, read(arena, "#!srfv1\ndate::2026-8-27,unit_value:num:1.0,source::anchor\n"));
|
||||
try testing.expectError(error.UnknownSeriesSource, read(arena, "#!srfv1\ndate::2026-08-27,unit_value:num:1.0,source::invented\n"));
|
||||
}
|
||||
|
||||
test "read accepts an empty series body" {
|
||||
var arena_state: std.heap.ArenaAllocator = .init(testing.allocator);
|
||||
defer arena_state.deinit();
|
||||
const back = try read(arena_state.allocator(), "#!srfv1\n");
|
||||
try testing.expectEqual(@as(usize, 0), back.len);
|
||||
}
|
||||
56
src/srf_opts.zig
Normal file
56
src/srf_opts.zig
Normal file
|
|
@ -0,0 +1,56 @@
|
|||
//! SRF coercion policy, in one place.
|
||||
//!
|
||||
//! SRF encodes a field's type in its separator: `key::v` is a string,
|
||||
//! `key:num:v` a number, `key:bool:v` a boolean. Coercion into a typed
|
||||
//! struct therefore depends on the writer having picked the right one.
|
||||
//!
|
||||
//! That assumption holds for files zfin writes and breaks for files
|
||||
//! people write, so the two get different options - and which one a
|
||||
//! parser wants is a property of where the file came from, not of the
|
||||
//! struct being parsed. Naming the two policies keeps that decision
|
||||
//! visible at the call site instead of buried in whichever comment
|
||||
//! happened to explain it first.
|
||||
|
||||
const srf = @import("srf");
|
||||
|
||||
/// For files a HUMAN edits: `portfolio.srf`, `accounts.srf`,
|
||||
/// `metadata.srf`, `watchlist.srf`, `transaction_log.srf`,
|
||||
/// `projections.srf`, `imported_values.srf`, `acknowledgments.srf`, and
|
||||
/// the keybind config.
|
||||
///
|
||||
/// Accepts a string where a number was declared, because a hand-typed
|
||||
/// `close_price::200.00` instead of `close_price:num:200.00` is a
|
||||
/// slip, not a different intent. SRF's own doc says as much: strict
|
||||
/// coercion is "intended for performant access for cache use cases...
|
||||
/// if you want to use this for human-edited files, turn this on".
|
||||
///
|
||||
/// It is also a safety measure, which is the part worth not
|
||||
/// forgetting. Under strict coercion a string reaching a numeric field
|
||||
/// falls through to an unchecked `val.?.number` inside SRF - a panic
|
||||
/// in Debug/ReleaseSafe and undefined behaviour in ReleaseFast, which
|
||||
/// is how zfin is built. One `close_price::200.00` once took down
|
||||
/// every `zfin portfolio` run; the fix was to turn this on for
|
||||
/// `portfolio.srf` alone, which left every other hand-edited file
|
||||
/// exposed to the same typo.
|
||||
///
|
||||
/// This is mitigation, not a cure. Two holes remain, both needing an
|
||||
/// upstream fix in SRF's `coerce`:
|
||||
///
|
||||
/// - a non-string, non-number value in a numeric field (say
|
||||
/// `harvested:bool:true`) still reaches the unchecked access;
|
||||
/// - enum fields ignore this option entirely, so a typo like
|
||||
/// `security_type::stok` still hits `stringToEnum(...).?`.
|
||||
pub const user_edited: srf.CoercionOptions = .{ .strings_to_numbers = true };
|
||||
|
||||
/// For files ZFIN writes: the candle, quote, and options caches,
|
||||
/// `cusip_tickers.srf` under `cache_dir`, server responses, and the
|
||||
/// `history/<date>-portfolio.srf` snapshots produced by
|
||||
/// `zfin snapshot`.
|
||||
///
|
||||
/// Keeps SRF's strict default. The writer is a machine that always
|
||||
/// emits `:num:` for numbers, so accepting a string instead would only
|
||||
/// mask a serializer bug rather than tolerate a human slip - these
|
||||
/// files are not edited after the fact. Strictness is also free
|
||||
/// performance on the hot paths, where a cached candle file is
|
||||
/// millions of records.
|
||||
pub const machine_written: srf.CoercionOptions = .{};
|
||||
747
src/verify.zig
Normal file
747
src/verify.zig
Normal file
|
|
@ -0,0 +1,747 @@
|
|||
//! Verification of a reconstructed series.
|
||||
//!
|
||||
//! The reconstruction is derived data, so it needs to be checkable rather than
|
||||
//! trusted. Five independent checks:
|
||||
//!
|
||||
//! 1. Structure -- ascending dates, no duplicates, finite positive values,
|
||||
//! endpoints where they belong.
|
||||
//! 2. Anchors -- every observed value reproduced exactly.
|
||||
//! 3. Cross-check -- agreement with hand-typed values that were never fed
|
||||
//! into the model, so this is evidence rather than a fit.
|
||||
//! 4. Leave-one-out -- drop an interior anchor, pin its neighbours, predict it.
|
||||
//! This measures error in the MIDDLE of an un-anchored gap,
|
||||
//! which is the accuracy question that actually matters.
|
||||
//! 5. Implied drag -- the annualised fee each gap implies, as a sanity check
|
||||
//! against the plan's published cost table.
|
||||
//!
|
||||
//! Checks 1 and 2 are pass/fail. Checks 3 and 4 are pass/fail against explicit
|
||||
//! tolerances. Check 5 is informational: fees are not identifiable over a short
|
||||
//! gap, so a per-gap number is noisy by nature and is reported, not enforced.
|
||||
|
||||
const std = @import("std");
|
||||
const data = @import("data.zig");
|
||||
const candles = @import("candles.zig");
|
||||
const recon = @import("recon.zig");
|
||||
const civil = @import("civil.zig");
|
||||
|
||||
/// Tolerances, as fractions.
|
||||
///
|
||||
/// Deliberately not exported: callers pass an anonymous literal (`.{}`, or
|
||||
/// `.{ .loo = 0.01 }`) to `run`, so nothing outside this file needs to name the
|
||||
/// type. Same for `Report` and `Stats`, which callers reach by inference from
|
||||
/// `run`'s return value rather than by name.
|
||||
const Tolerance = struct {
|
||||
/// Anchors are exact values; this allows only float round-trip noise.
|
||||
anchor: f64 = 1e-9,
|
||||
/// Leave-one-out interior error. Measured worst case is 0.079%, so 0.25%
|
||||
/// leaves headroom for a future glidepath era without hiding a real break.
|
||||
loo: f64 = 0.0025,
|
||||
/// Agreement with hand-typed values. Those are rounded to 2dp at source
|
||||
/// (about 0.03% here) and measured worst case is 0.185%, so 0.5% catches a
|
||||
/// misalignment or a wrong era without tripping on rounding.
|
||||
recorded: f64 = 0.005,
|
||||
};
|
||||
|
||||
const Stats = struct {
|
||||
n: usize = 0,
|
||||
mean: f64 = 0,
|
||||
stdev: f64 = 0,
|
||||
max_abs: f64 = 0,
|
||||
|
||||
fn from(xs: []const f64) Stats {
|
||||
if (xs.len == 0) return .{};
|
||||
var sum: f64 = 0;
|
||||
var max_abs: f64 = 0;
|
||||
for (xs) |x| {
|
||||
sum += x;
|
||||
max_abs = @max(max_abs, @abs(x));
|
||||
}
|
||||
const mean = sum / @as(f64, @floatFromInt(xs.len));
|
||||
var sq: f64 = 0;
|
||||
for (xs) |x| sq += (x - mean) * (x - mean);
|
||||
return .{
|
||||
.n = xs.len,
|
||||
.mean = mean,
|
||||
.stdev = @sqrt(sq / @as(f64, @floatFromInt(xs.len))),
|
||||
.max_abs = max_abs,
|
||||
};
|
||||
}
|
||||
};
|
||||
|
||||
const Report = struct {
|
||||
symbol: []const u8,
|
||||
points: usize,
|
||||
first_date: []const u8,
|
||||
last_date: []const u8,
|
||||
anchors_checked: usize = 0,
|
||||
anchor_max_abs: f64 = 0,
|
||||
loo_checked: usize = 0,
|
||||
loo_worst_pct: f64 = 0,
|
||||
recorded: Stats = .{},
|
||||
recorded_worst_pct: f64 = 0,
|
||||
failures: []const []const u8 = &.{},
|
||||
|
||||
pub fn ok(self: Report) bool {
|
||||
return self.failures.len == 0;
|
||||
}
|
||||
};
|
||||
|
||||
const Failures = struct {
|
||||
arena: std.mem.Allocator,
|
||||
list: std.ArrayList([]const u8) = .empty,
|
||||
|
||||
fn add(self: *Failures, comptime f: []const u8, args: anytype) !void {
|
||||
try self.list.append(self.arena, try std.fmt.allocPrint(self.arena, f, args));
|
||||
}
|
||||
};
|
||||
|
||||
/// Check 1: the series is well-formed and spans what it claims to.
|
||||
fn checkStructure(f: *Failures, points: []const recon.Point, anchors: data.AnchorSet) !void {
|
||||
if (points.len == 0) {
|
||||
try f.add("series is empty", .{});
|
||||
return;
|
||||
}
|
||||
for (points, 0..) |p, i| {
|
||||
if (!civil.isValidIso(p.date)) {
|
||||
try f.add("row {d}: malformed date {s}", .{ i, p.date });
|
||||
}
|
||||
if (!std.math.isFinite(p.unit_value) or !(p.unit_value > 0)) {
|
||||
try f.add("row {d} ({s}): unit_value not finite and positive: {d}", .{ i, p.date, p.unit_value });
|
||||
}
|
||||
if (i > 0 and !civil.lessThan(points[i - 1].date, p.date)) {
|
||||
try f.add("row {d}: date {s} does not follow {s}", .{ i, p.date, points[i - 1].date });
|
||||
}
|
||||
}
|
||||
// The series must start at or after the launch anchor: a value predating the
|
||||
// portfolio's existence would be fabricated.
|
||||
const first_anchor = anchors.items[0];
|
||||
if (civil.lessThan(points[0].date, first_anchor.date)) {
|
||||
try f.add("series starts {s}, before the first anchor {s}", .{ points[0].date, first_anchor.date });
|
||||
}
|
||||
// And it must end on the last anchor, which is the live feed value.
|
||||
const last_anchor = anchors.items[anchors.items.len - 1];
|
||||
if (!std.mem.eql(u8, points[points.len - 1].date, last_anchor.date)) {
|
||||
try f.add("series ends {s}, expected the last anchor {s}", .{ points[points.len - 1].date, last_anchor.date });
|
||||
}
|
||||
}
|
||||
|
||||
/// Check 2: every anchor is reproduced exactly.
|
||||
///
|
||||
/// An anchor whose date is not a session is looked up at the adjacent session
|
||||
/// instead, matching how the reconstruction snaps them.
|
||||
fn checkAnchors(
|
||||
f: *Failures,
|
||||
r: *Report,
|
||||
points: []const recon.Point,
|
||||
anchors: data.AnchorSet,
|
||||
tol: Tolerance,
|
||||
) !void {
|
||||
for (anchors.items, 0..) |a, i| {
|
||||
const got = recon.valueAt(points, a.date) orelse blk: {
|
||||
// Not a session. The first anchor was snapped forward, the rest back.
|
||||
if (i == 0) {
|
||||
for (points) |p| {
|
||||
if (!civil.lessThan(p.date, a.date)) break :blk p.unit_value;
|
||||
}
|
||||
} else {
|
||||
var last: ?f64 = null;
|
||||
for (points) |p| {
|
||||
if (civil.lessThan(a.date, p.date)) break;
|
||||
last = p.unit_value;
|
||||
}
|
||||
if (last) |v| break :blk v;
|
||||
}
|
||||
try f.add("anchor {s} ({s}) has no corresponding series row", .{ a.date, @tagName(a.source) });
|
||||
continue;
|
||||
};
|
||||
const err = @abs(got - a.unit_value) / a.unit_value;
|
||||
r.anchors_checked += 1;
|
||||
r.anchor_max_abs = @max(r.anchor_max_abs, err);
|
||||
if (err > tol.anchor) {
|
||||
try f.add(
|
||||
"anchor {s} ({s}): expected {d:.6}, series has {d:.6} ({d:.4}% off)",
|
||||
.{ a.date, @tagName(a.source), a.unit_value, got, err * 100 },
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Check 3: agreement with the independently recorded hand-typed values.
|
||||
fn checkRecorded(
|
||||
f: *Failures,
|
||||
r: *Report,
|
||||
arena: std.mem.Allocator,
|
||||
points: []const recon.Point,
|
||||
recorded: []const data.Recorded,
|
||||
symbol: []const u8,
|
||||
tol: Tolerance,
|
||||
) !void {
|
||||
var diffs: std.ArrayList(f64) = .empty;
|
||||
for (recorded) |rec| {
|
||||
if (!std.mem.eql(u8, rec.symbol, symbol)) continue;
|
||||
// lag=1: a value written on day D reports day D-1.
|
||||
const got = recon.valueBefore(points, rec.snapshot_date) orelse continue;
|
||||
try diffs.append(arena, got - rec.unit_value);
|
||||
const err = @abs(got - rec.unit_value) / rec.unit_value;
|
||||
r.recorded_worst_pct = @max(r.recorded_worst_pct, err * 100);
|
||||
if (err > tol.recorded) {
|
||||
try f.add(
|
||||
"recorded {s}: typed {d:.2}, series has {d:.4} ({d:.3}% off)",
|
||||
.{ rec.snapshot_date, rec.unit_value, got, err * 100 },
|
||||
);
|
||||
}
|
||||
}
|
||||
r.recorded = Stats.from(diffs.items);
|
||||
if (r.recorded.n == 0) {
|
||||
try f.add("no recorded values matched {s}; the cross-check did not run", .{symbol});
|
||||
}
|
||||
}
|
||||
|
||||
/// Check 4: leave-one-out cross-validation over the interior anchors.
|
||||
fn checkLeaveOneOut(
|
||||
f: *Failures,
|
||||
r: *Report,
|
||||
arena: std.mem.Allocator,
|
||||
in: recon.Inputs,
|
||||
tol: Tolerance,
|
||||
) !void {
|
||||
const anchors = in.anchors.items;
|
||||
if (anchors.len < 3) return; // no interior anchor to hold out
|
||||
|
||||
const cal = try recon.calendar(arena, in.model, in.tickers);
|
||||
for (1..anchors.len - 1) |k| {
|
||||
const a0 = anchors[k - 1];
|
||||
const held = anchors[k];
|
||||
const a1 = anchors[k + 1];
|
||||
|
||||
const from = candles.firstAtOrAfter(cal, a0.date) orelse continue;
|
||||
const to = candles.lastAtOrBefore(cal, a1.date) orelse continue;
|
||||
const ih = candles.lastAtOrBefore(cal, held.date) orelse continue;
|
||||
if (!(from < ih and ih < to)) continue;
|
||||
|
||||
const vals = try recon.pinGap(arena, in.model, in.tickers, cal, from, a0.unit_value, to, a1.unit_value);
|
||||
const pred = vals[ih - from];
|
||||
const err = @abs(pred - held.unit_value) / held.unit_value;
|
||||
|
||||
r.loo_checked += 1;
|
||||
r.loo_worst_pct = @max(r.loo_worst_pct, err * 100);
|
||||
if (err > tol.loo) {
|
||||
try f.add(
|
||||
"leave-one-out {s}: actual {d:.6}, predicted {d:.6} ({d:.3}% off, tolerance {d:.3}%)",
|
||||
.{ held.date, held.unit_value, pred, err * 100, tol.loo * 100 },
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Check 5 (informational): the annualised drag each anchor gap implies.
|
||||
///
|
||||
/// Solved by bisection on the drag that makes an unpinned walk land exactly on
|
||||
/// the closing anchor. Compare against the plan's published cost table: 0.234%
|
||||
/// for ORCBI, 0.239% for ORC42. Only spans of a few hundred days say anything
|
||||
/// useful; short gaps are dominated by noise.
|
||||
pub fn impliedDrag(
|
||||
arena: std.mem.Allocator,
|
||||
in: recon.Inputs,
|
||||
gap: usize,
|
||||
) !?f64 {
|
||||
const anchors = in.anchors.items;
|
||||
if (gap + 1 >= anchors.len) return null;
|
||||
const a0 = anchors[gap];
|
||||
const a1 = anchors[gap + 1];
|
||||
|
||||
const cal = try recon.calendar(arena, in.model, in.tickers);
|
||||
const from = candles.firstAtOrAfter(cal, a0.date) orelse return null;
|
||||
const to = candles.lastAtOrBefore(cal, a1.date) orelse return null;
|
||||
if (to <= from) return null;
|
||||
|
||||
const days = try civil.daysBetween(a0.date, a1.date);
|
||||
if (days <= 0) return null;
|
||||
const years = @as(f64, @floatFromInt(days)) / 365.0;
|
||||
|
||||
// Terminal value of the undragged walk, i.e. what the basket alone predicts.
|
||||
const raw = try rawTerminal(in, cal, from, a0.unit_value, to);
|
||||
if (!(raw > 0)) return null;
|
||||
|
||||
// raw * (1 - drag)^years == actual => drag = 1 - (actual/raw)^(1/years)
|
||||
const ratio = a1.unit_value / raw;
|
||||
return 1.0 - std.math.pow(f64, ratio, 1.0 / years);
|
||||
}
|
||||
|
||||
/// Compound the basket forward with no residual distribution and no drag.
|
||||
///
|
||||
/// `recon.pinGap` cannot serve here: it always pins its terminal value to the
|
||||
/// caller's `v1`, which is exactly the number this function needs to discover.
|
||||
fn rawTerminal(
|
||||
in: recon.Inputs,
|
||||
cal: []const []const u8,
|
||||
from: usize,
|
||||
v0: f64,
|
||||
to: usize,
|
||||
) !f64 {
|
||||
var v = v0;
|
||||
for (from + 1..to + 1) |k| {
|
||||
const era = in.model.eraOn(cal[k]) orelse return error.NoEraInForce;
|
||||
var r: f64 = 0;
|
||||
for (era.weights) |w| {
|
||||
var found: ?candles.Ticker = null;
|
||||
for (in.tickers) |t| {
|
||||
if (std.mem.eql(u8, t.name, w.ticker)) found = t;
|
||||
}
|
||||
const t = found orelse return error.MissingTicker;
|
||||
const a = t.at(cal[k - 1]) orelse return error.MissingBar;
|
||||
const b = t.at(cal[k]) orelse return error.MissingBar;
|
||||
r += w.fraction * (b / a - 1.0);
|
||||
}
|
||||
v *= (1.0 + r);
|
||||
}
|
||||
return v;
|
||||
}
|
||||
|
||||
/// Check 6: no projected era is silently in force.
|
||||
///
|
||||
/// Projected weights come from the booklet's forward glidepath table, which the
|
||||
/// plan explicitly reserves the right to change. Once calendar time crosses into
|
||||
/// one, the model is running on published intent rather than confirmed fact --
|
||||
/// and it would do so silently forever. Turn that into a visible prompt.
|
||||
///
|
||||
/// `today` is passed in rather than read from the clock so this is testable.
|
||||
fn checkModelBasis(
|
||||
f: *Failures,
|
||||
model: data.Model,
|
||||
today: []const u8,
|
||||
) !void {
|
||||
const era = model.eraOn(today) orelse return;
|
||||
if (era.basis != .projected) return;
|
||||
try f.add(
|
||||
"model era {s} is `projected` but is now in force as of {s}: confirm the " ++
|
||||
"weights against the booklet allocation table and mark it `fitted`",
|
||||
.{ era.start, today },
|
||||
);
|
||||
}
|
||||
|
||||
/// Run every check for one symbol.
|
||||
pub fn run(
|
||||
arena: std.mem.Allocator,
|
||||
in: recon.Inputs,
|
||||
points: []const recon.Point,
|
||||
recorded: []const data.Recorded,
|
||||
today: []const u8,
|
||||
tol: Tolerance,
|
||||
) !Report {
|
||||
var f: Failures = .{ .arena = arena };
|
||||
var r: Report = .{
|
||||
.symbol = in.anchors.symbol,
|
||||
.points = points.len,
|
||||
.first_date = if (points.len > 0) points[0].date else "",
|
||||
.last_date = if (points.len > 0) points[points.len - 1].date else "",
|
||||
};
|
||||
|
||||
try checkStructure(&f, points, in.anchors);
|
||||
if (points.len > 0) {
|
||||
try checkAnchors(&f, &r, points, in.anchors, tol);
|
||||
try checkRecorded(&f, &r, arena, points, recorded, in.anchors.symbol, tol);
|
||||
try checkLeaveOneOut(&f, &r, arena, in, tol);
|
||||
}
|
||||
try checkModelBasis(&f, in.model, today);
|
||||
|
||||
r.failures = try f.list.toOwnedSlice(arena);
|
||||
return r;
|
||||
}
|
||||
|
||||
const testing = std.testing;
|
||||
|
||||
test "Stats computes mean, population stdev and max magnitude" {
|
||||
const s = Stats.from(&.{ 1.0, -1.0, 3.0 });
|
||||
try testing.expectEqual(@as(usize, 3), s.n);
|
||||
try testing.expectApproxEqAbs(@as(f64, 1.0), s.mean, 1e-12);
|
||||
try testing.expectApproxEqAbs(@as(f64, 3.0), s.max_abs, 1e-12);
|
||||
// population stdev of {1,-1,3} about mean 1 is sqrt((0+4+4)/3)
|
||||
try testing.expectApproxEqAbs(@sqrt(8.0 / 3.0), s.stdev, 1e-12);
|
||||
|
||||
const empty = Stats.from(&.{});
|
||||
try testing.expectEqual(@as(usize, 0), empty.n);
|
||||
}
|
||||
|
||||
/// Build a self-consistent fixture: a model, prices, anchors taken FROM the
|
||||
/// reconstruction itself, so a correct implementation must pass every check.
|
||||
fn selfConsistent(arena: std.mem.Allocator) !struct {
|
||||
in: recon.Inputs,
|
||||
points: []const recon.Point,
|
||||
} {
|
||||
const models = try data.parseModel(arena,
|
||||
\\#!srfv1
|
||||
\\symbol::T,era_start::2020-01-01,ticker::AA,weight:num:100
|
||||
\\
|
||||
);
|
||||
// A smooth ramp, so a pinned walk is essentially exact everywhere.
|
||||
var buf: std.ArrayList(u8) = .empty;
|
||||
try buf.appendSlice(arena, "#!srfv1\n");
|
||||
for (1..21) |d| {
|
||||
try buf.print(arena, "date::2020-01-{d:0>2},adj_close:num:{d}\n", .{ d, 100 + d });
|
||||
}
|
||||
const aa = try candles.parse(arena, "AA", buf.items);
|
||||
|
||||
const sets = try data.parseAnchors(arena,
|
||||
\\#!srfv1
|
||||
\\symbol::T,date::2020-01-01,unit_value:num:10.0,source::launch
|
||||
\\symbol::T,date::2020-01-10,unit_value:num:10.8910891089,source::wayback
|
||||
\\symbol::T,date::2020-01-20,unit_value:num:11.8811881188,source::feed
|
||||
\\
|
||||
);
|
||||
const in: recon.Inputs = .{
|
||||
.model = data.findModel(models, "T").?,
|
||||
.anchors = data.findAnchors(sets, "T").?,
|
||||
.tickers = try arena.dupe(candles.Ticker, &.{aa}),
|
||||
};
|
||||
return .{ .in = in, .points = try recon.build(arena, in) };
|
||||
}
|
||||
|
||||
test "run passes a self-consistent series" {
|
||||
var arena_state: std.heap.ArenaAllocator = .init(testing.allocator);
|
||||
defer arena_state.deinit();
|
||||
const arena = arena_state.allocator();
|
||||
|
||||
const f = try selfConsistent(arena);
|
||||
const recorded: []const data.Recorded = &.{
|
||||
// Written on the 11th, so it reports the 10th: the middle anchor.
|
||||
.{ .symbol = "T", .snapshot_date = "2020-01-11", .unit_value = 10.89 },
|
||||
};
|
||||
const r = try run(arena, f.in, f.points, recorded, "2020-01-20", .{});
|
||||
|
||||
for (r.failures) |msg| std.debug.print("unexpected failure: {s}\n", .{msg});
|
||||
try testing.expect(r.ok());
|
||||
try testing.expectEqual(@as(usize, 3), r.anchors_checked);
|
||||
try testing.expectEqual(@as(usize, 1), r.loo_checked);
|
||||
try testing.expectEqual(@as(usize, 1), r.recorded.n);
|
||||
try testing.expect(r.anchor_max_abs < 1e-9);
|
||||
}
|
||||
|
||||
test "run reports a corrupted anchor" {
|
||||
var arena_state: std.heap.ArenaAllocator = .init(testing.allocator);
|
||||
defer arena_state.deinit();
|
||||
const arena = arena_state.allocator();
|
||||
|
||||
const f = try selfConsistent(arena);
|
||||
const mutated = try arena.dupe(recon.Point, f.points);
|
||||
mutated[0].unit_value *= 1.05; // 5% off the launch anchor
|
||||
|
||||
const r = try run(arena, f.in, mutated, &.{}, "2020-01-20", .{});
|
||||
try testing.expect(!r.ok());
|
||||
var saw = false;
|
||||
for (r.failures) |msg| {
|
||||
if (std.mem.indexOf(u8, msg, "anchor 2020-01-01") != null) saw = true;
|
||||
}
|
||||
try testing.expect(saw);
|
||||
}
|
||||
|
||||
test "run reports non-monotonic dates and non-finite values" {
|
||||
var arena_state: std.heap.ArenaAllocator = .init(testing.allocator);
|
||||
defer arena_state.deinit();
|
||||
const arena = arena_state.allocator();
|
||||
|
||||
const f = try selfConsistent(arena);
|
||||
const mutated = try arena.dupe(recon.Point, f.points);
|
||||
mutated[3].date = mutated[2].date; // duplicate
|
||||
mutated[4].unit_value = std.math.nan(f64);
|
||||
|
||||
const r = try run(arena, f.in, mutated, &.{}, "2020-01-20", .{});
|
||||
try testing.expect(!r.ok());
|
||||
var saw_order = false;
|
||||
var saw_nan = false;
|
||||
for (r.failures) |msg| {
|
||||
if (std.mem.indexOf(u8, msg, "does not follow") != null) saw_order = true;
|
||||
if (std.mem.indexOf(u8, msg, "not finite") != null) saw_nan = true;
|
||||
}
|
||||
try testing.expect(saw_order);
|
||||
try testing.expect(saw_nan);
|
||||
}
|
||||
|
||||
test "run reports an empty series without crashing" {
|
||||
var arena_state: std.heap.ArenaAllocator = .init(testing.allocator);
|
||||
defer arena_state.deinit();
|
||||
const arena = arena_state.allocator();
|
||||
|
||||
const f = try selfConsistent(arena);
|
||||
const r = try run(arena, f.in, &.{}, &.{}, "2020-01-20", .{});
|
||||
try testing.expect(!r.ok());
|
||||
try testing.expectEqual(@as(usize, 0), r.points);
|
||||
}
|
||||
|
||||
test "run flags a recorded value that disagrees badly" {
|
||||
var arena_state: std.heap.ArenaAllocator = .init(testing.allocator);
|
||||
defer arena_state.deinit();
|
||||
const arena = arena_state.allocator();
|
||||
|
||||
const f = try selfConsistent(arena);
|
||||
const recorded: []const data.Recorded = &.{
|
||||
.{ .symbol = "T", .snapshot_date = "2020-01-11", .unit_value = 99.0 },
|
||||
};
|
||||
const r = try run(arena, f.in, f.points, recorded, "2020-01-20", .{});
|
||||
try testing.expect(!r.ok());
|
||||
var saw = false;
|
||||
for (r.failures) |msg| {
|
||||
if (std.mem.indexOf(u8, msg, "recorded 2020-01-11") != null) saw = true;
|
||||
}
|
||||
try testing.expect(saw);
|
||||
}
|
||||
|
||||
test "run notes when no recorded value matched the symbol" {
|
||||
var arena_state: std.heap.ArenaAllocator = .init(testing.allocator);
|
||||
defer arena_state.deinit();
|
||||
const arena = arena_state.allocator();
|
||||
|
||||
const f = try selfConsistent(arena);
|
||||
const recorded: []const data.Recorded = &.{
|
||||
.{ .symbol = "OTHER", .snapshot_date = "2020-01-11", .unit_value = 10.89 },
|
||||
};
|
||||
const r = try run(arena, f.in, f.points, recorded, "2020-01-20", .{});
|
||||
try testing.expect(!r.ok());
|
||||
var saw = false;
|
||||
for (r.failures) |msg| {
|
||||
if (std.mem.indexOf(u8, msg, "cross-check did not run") != null) saw = true;
|
||||
}
|
||||
try testing.expect(saw);
|
||||
}
|
||||
|
||||
test "impliedDrag recovers a drag that was applied on purpose" {
|
||||
var arena_state: std.heap.ArenaAllocator = .init(testing.allocator);
|
||||
defer arena_state.deinit();
|
||||
const arena = arena_state.allocator();
|
||||
|
||||
const models = try data.parseModel(arena,
|
||||
\\#!srfv1
|
||||
\\symbol::T,era_start::2020-01-01,ticker::AA,weight:num:100
|
||||
\\
|
||||
);
|
||||
// Flat prices over exactly 365 days, so the whole gap return IS the drag.
|
||||
const aa = try candles.parse(arena, "AA", "#!srfv1\n" ++
|
||||
"date::2020-01-01,adj_close:num:100\n" ++
|
||||
"date::2020-07-01,adj_close:num:100\n" ++
|
||||
"date::2020-12-31,adj_close:num:100\n");
|
||||
// 1% lower after 365 days with a flat basket implies ~1%/yr drag.
|
||||
const sets = try data.parseAnchors(arena,
|
||||
\\#!srfv1
|
||||
\\symbol::T,date::2020-01-01,unit_value:num:10.0,source::launch
|
||||
\\symbol::T,date::2020-12-31,unit_value:num:9.9,source::feed
|
||||
\\
|
||||
);
|
||||
const in: recon.Inputs = .{
|
||||
.model = data.findModel(models, "T").?,
|
||||
.anchors = data.findAnchors(sets, "T").?,
|
||||
.tickers = try arena.dupe(candles.Ticker, &.{aa}),
|
||||
};
|
||||
|
||||
const drag = (try impliedDrag(arena, in, 0)).?;
|
||||
try testing.expectApproxEqAbs(@as(f64, 0.01), drag, 5e-4);
|
||||
// Out of range gaps return null rather than erroring.
|
||||
try testing.expectEqual(@as(?f64, null), try impliedDrag(arena, in, 1));
|
||||
}
|
||||
|
||||
test "checkAnchors resolves anchors that fall on non-trading days" {
|
||||
var arena_state: std.heap.ArenaAllocator = .init(testing.allocator);
|
||||
defer arena_state.deinit();
|
||||
const arena = arena_state.allocator();
|
||||
|
||||
// This is the real ORCBI/ORC42 shape: both launched on a Saturday, so the
|
||||
// launch anchor's own date is never a session and has to be resolved to the
|
||||
// adjacent one. An interior anchor on a weekend exercises the other branch.
|
||||
const models = try data.parseModel(arena,
|
||||
\\#!srfv1
|
||||
\\symbol::T,era_start::2018-01-01,ticker::AA,weight:num:100
|
||||
\\
|
||||
);
|
||||
const aa = try candles.parse(arena, "AA", "#!srfv1\n" ++
|
||||
"date::2018-09-04,adj_close:num:100\n" ++ // Tue after Labor Day
|
||||
"date::2018-09-05,adj_close:num:101\n" ++
|
||||
"date::2018-09-06,adj_close:num:102\n" ++
|
||||
"date::2018-09-07,adj_close:num:103\n" ++ // Fri
|
||||
"date::2018-09-10,adj_close:num:104\n"); // Mon
|
||||
const sets = try data.parseAnchors(arena,
|
||||
\\#!srfv1
|
||||
\\symbol::T,date::2018-09-01,unit_value:num:10.0,source::launch
|
||||
\\symbol::T,date::2018-09-08,unit_value:num:10.3,source::wayback
|
||||
\\symbol::T,date::2018-09-10,unit_value:num:10.4,source::feed
|
||||
\\
|
||||
);
|
||||
const in: recon.Inputs = .{
|
||||
.model = data.findModel(models, "T").?,
|
||||
.anchors = data.findAnchors(sets, "T").?,
|
||||
.tickers = try arena.dupe(candles.Ticker, &.{aa}),
|
||||
};
|
||||
const points = try recon.build(arena, in);
|
||||
|
||||
// 2018-09-01 (Sat) resolved forward to 09-04; 2018-09-08 (Sat) resolved
|
||||
// back to 09-07. Neither date appears in the series itself.
|
||||
try testing.expectEqual(@as(?f64, null), recon.valueAt(points, "2018-09-01"));
|
||||
try testing.expectEqual(@as(?f64, null), recon.valueAt(points, "2018-09-08"));
|
||||
|
||||
var f: Failures = .{ .arena = arena };
|
||||
var r: Report = .{ .symbol = "T", .points = points.len, .first_date = points[0].date, .last_date = points[points.len - 1].date };
|
||||
try checkAnchors(&f, &r, points, in.anchors, .{});
|
||||
|
||||
for (f.list.items) |msg| std.debug.print("unexpected: {s}\n", .{msg});
|
||||
try testing.expectEqual(@as(usize, 0), f.list.items.len);
|
||||
try testing.expectEqual(@as(usize, 3), r.anchors_checked);
|
||||
try testing.expect(r.anchor_max_abs < 1e-9);
|
||||
}
|
||||
|
||||
test "checkAnchors reports an anchor it cannot place at all" {
|
||||
var arena_state: std.heap.ArenaAllocator = .init(testing.allocator);
|
||||
defer arena_state.deinit();
|
||||
const arena = arena_state.allocator();
|
||||
|
||||
const sets = try data.parseAnchors(arena,
|
||||
\\#!srfv1
|
||||
\\symbol::T,date::2020-01-01,unit_value:num:10.0,source::launch
|
||||
\\symbol::T,date::2030-01-01,unit_value:num:99.0,source::feed
|
||||
\\
|
||||
);
|
||||
const anchors = data.findAnchors(sets, "T").?;
|
||||
|
||||
{
|
||||
// A series that stops long before the closing anchor: the backward scan
|
||||
// resolves 2030 to the only row, which holds 10.0 rather than 99.0, so
|
||||
// this is a mismatch rather than a silent pass.
|
||||
const points: []const recon.Point = &.{
|
||||
.{ .date = "2020-01-01", .unit_value = 10.0, .source = .anchor },
|
||||
};
|
||||
var f: Failures = .{ .arena = arena };
|
||||
var r: Report = .{ .symbol = "T", .points = 1, .first_date = "2020-01-01", .last_date = "2020-01-01" };
|
||||
try checkAnchors(&f, &r, points, anchors, .{});
|
||||
try testing.expect(f.list.items.len > 0);
|
||||
}
|
||||
{
|
||||
// A series entirely BEFORE the launch anchor. The forward scan finds
|
||||
// nothing to resolve it to, so the anchor is unplaceable and must be
|
||||
// named rather than skipped.
|
||||
const points: []const recon.Point = &.{
|
||||
.{ .date = "2019-01-01", .unit_value = 5.0, .source = .reconstructed },
|
||||
};
|
||||
var f: Failures = .{ .arena = arena };
|
||||
var r: Report = .{ .symbol = "T", .points = 1, .first_date = "2019-01-01", .last_date = "2019-01-01" };
|
||||
try checkAnchors(&f, &r, points, anchors, .{});
|
||||
|
||||
var saw = false;
|
||||
for (f.list.items) |msg| {
|
||||
if (std.mem.indexOf(u8, msg, "no corresponding series row") != null) saw = true;
|
||||
}
|
||||
try testing.expect(saw);
|
||||
// And the unplaceable anchor is not counted as checked.
|
||||
try testing.expect(r.anchors_checked < anchors.items.len);
|
||||
}
|
||||
}
|
||||
|
||||
test "checkLeaveOneOut reports a held-out anchor the model cannot predict" {
|
||||
var arena_state: std.heap.ArenaAllocator = .init(testing.allocator);
|
||||
defer arena_state.deinit();
|
||||
const arena = arena_state.allocator();
|
||||
|
||||
const models = try data.parseModel(arena,
|
||||
\\#!srfv1
|
||||
\\symbol::T,era_start::2020-01-01,ticker::AA,weight:num:100
|
||||
\\
|
||||
);
|
||||
// Flat prices, so a pinned gap from 10.0 to 10.0 predicts 10.0 throughout.
|
||||
var buf: std.ArrayList(u8) = .empty;
|
||||
try buf.appendSlice(arena, "#!srfv1\n");
|
||||
for (1..11) |d| try buf.print(arena, "date::2020-01-{d:0>2},adj_close:num:100\n", .{d});
|
||||
const aa = try candles.parse(arena, "AA", buf.items);
|
||||
|
||||
// The middle anchor claims a 50% spike that the basket says never happened.
|
||||
const sets = try data.parseAnchors(arena,
|
||||
\\#!srfv1
|
||||
\\symbol::T,date::2020-01-01,unit_value:num:10.0,source::launch
|
||||
\\symbol::T,date::2020-01-05,unit_value:num:15.0,source::wayback
|
||||
\\symbol::T,date::2020-01-10,unit_value:num:10.0,source::feed
|
||||
\\
|
||||
);
|
||||
const in: recon.Inputs = .{
|
||||
.model = data.findModel(models, "T").?,
|
||||
.anchors = data.findAnchors(sets, "T").?,
|
||||
.tickers = try arena.dupe(candles.Ticker, &.{aa}),
|
||||
};
|
||||
|
||||
var f: Failures = .{ .arena = arena };
|
||||
var r: Report = .{ .symbol = "T", .points = 0, .first_date = "", .last_date = "" };
|
||||
try checkLeaveOneOut(&f, &r, arena, in, .{});
|
||||
|
||||
try testing.expectEqual(@as(usize, 1), r.loo_checked);
|
||||
try testing.expect(r.loo_worst_pct > 30); // predicted 10, actual 15
|
||||
var saw = false;
|
||||
for (f.list.items) |msg| {
|
||||
if (std.mem.indexOf(u8, msg, "leave-one-out 2020-01-05") != null) saw = true;
|
||||
}
|
||||
try testing.expect(saw);
|
||||
}
|
||||
|
||||
test "checkModelBasis stays quiet on a fitted era and speaks up on a projected one" {
|
||||
var arena_state: std.heap.ArenaAllocator = .init(testing.allocator);
|
||||
defer arena_state.deinit();
|
||||
const arena = arena_state.allocator();
|
||||
|
||||
const models = try data.parseModel(arena,
|
||||
\\#!srfv1
|
||||
\\symbol::T,era_start::2023-07-01,basis::fitted,ticker::AA,weight:num:100
|
||||
\\symbol::T,era_start::2026-01-01,basis::fitted,ticker::AA,weight:num:100
|
||||
\\symbol::T,era_start::2027-01-01,basis::projected,ticker::AA,weight:num:100
|
||||
\\symbol::T,era_start::2028-01-01,basis::projected,ticker::AA,weight:num:100
|
||||
\\
|
||||
);
|
||||
const model = data.findModel(models, "T").?;
|
||||
|
||||
// Before any era, and inside the fitted eras: nothing to say.
|
||||
for ([_][]const u8{ "2020-01-01", "2023-07-01", "2025-12-31", "2026-06-15" }) |day| {
|
||||
var f: Failures = .{ .arena = arena };
|
||||
try checkModelBasis(&f, model, day);
|
||||
try testing.expectEqual(@as(usize, 0), f.list.items.len);
|
||||
}
|
||||
|
||||
// Once time crosses into a projection, say so and name the era.
|
||||
{
|
||||
var f: Failures = .{ .arena = arena };
|
||||
try checkModelBasis(&f, model, "2027-03-01");
|
||||
try testing.expectEqual(@as(usize, 1), f.list.items.len);
|
||||
try testing.expect(std.mem.indexOf(u8, f.list.items[0], "2027-01-01") != null);
|
||||
try testing.expect(std.mem.indexOf(u8, f.list.items[0], "projected") != null);
|
||||
}
|
||||
// The era named is the one actually in force, not merely the first projection.
|
||||
{
|
||||
var f: Failures = .{ .arena = arena };
|
||||
try checkModelBasis(&f, model, "2029-01-01");
|
||||
try testing.expectEqual(@as(usize, 1), f.list.items.len);
|
||||
try testing.expect(std.mem.indexOf(u8, f.list.items[0], "2028-01-01") != null);
|
||||
}
|
||||
}
|
||||
|
||||
test "parseModel rejects an era whose rows disagree about basis" {
|
||||
var arena_state: std.heap.ArenaAllocator = .init(testing.allocator);
|
||||
defer arena_state.deinit();
|
||||
// A half-fitted era would make the projected warning meaningless.
|
||||
try testing.expectError(error.MixedModelBasis, data.parseModel(arena_state.allocator(),
|
||||
\\#!srfv1
|
||||
\\symbol::T,era_start::2027-01-01,basis::projected,ticker::AA,weight:num:60
|
||||
\\symbol::T,era_start::2027-01-01,basis::fitted,ticker::BB,weight:num:40
|
||||
\\
|
||||
));
|
||||
}
|
||||
|
||||
test "parseModel rejects an unknown basis and defaults an absent one to fitted" {
|
||||
var arena_state: std.heap.ArenaAllocator = .init(testing.allocator);
|
||||
defer arena_state.deinit();
|
||||
const arena = arena_state.allocator();
|
||||
|
||||
try testing.expectError(error.UnknownModelBasis, data.parseModel(arena,
|
||||
\\#!srfv1
|
||||
\\symbol::T,era_start::2027-01-01,basis::guessed,ticker::AA,weight:num:100
|
||||
\\
|
||||
));
|
||||
|
||||
// Older files without the column keep parsing, as `fitted`.
|
||||
const models = try data.parseModel(arena,
|
||||
\\#!srfv1
|
||||
\\symbol::T,era_start::2020-01-01,ticker::AA,weight:num:100
|
||||
\\
|
||||
);
|
||||
try testing.expectEqual(data.Basis.fitted, data.findModel(models, "T").?.eras[0].basis);
|
||||
}
|
||||
179
tools/gen_model.py
Normal file
179
tools/gen_model.py
Normal file
|
|
@ -0,0 +1,179 @@
|
|||
#!/usr/bin/env python3
|
||||
"""Regenerate data/model.srf from the Plan Disclosure Booklet glidepath table.
|
||||
|
||||
The weights are transcribed by machine rather than by hand: 21 projected eras of
|
||||
five to eight funds each is far too much to retype reliably.
|
||||
|
||||
Usage:
|
||||
curl -sLo .tmp/booklet.html \
|
||||
https://marcom.vestwell.com/program-description/oregon-college.html
|
||||
python3 tools/gen_model.py .tmp/booklet.html > data/model.srf
|
||||
"""
|
||||
import html
|
||||
import re
|
||||
import sys
|
||||
|
||||
DASH = "[-\u2010\u2011\u2012\u2013\u2014\u2015\u2212]"
|
||||
FUNDS = ["VSMPX", "VTPSX", "VIPIX", "VBMPX", "VTIFX", "VBIPX", "VTSPX", "VUSXX"]
|
||||
|
||||
|
||||
def parse_glidepath(raw):
|
||||
txt = re.sub(r"\s+", " ", html.unescape(re.sub(r"<[^>]+>", " ", raw)))
|
||||
i = txt.find(" ".join(FUNDS))
|
||||
if i < 0:
|
||||
sys.exit("glidepath table header not found; booklet layout changed")
|
||||
rows = re.findall(
|
||||
r"(20\d\d)((?:\s+(?:[\d.]+%|" + DASH + r")){8})", txt[i : i + 2600]
|
||||
)
|
||||
table = {}
|
||||
for year, cells in rows:
|
||||
vals = [
|
||||
0.0 if re.fullmatch(DASH, c) else float(c.rstrip("%"))
|
||||
for c in re.split(r"\s+", cells.strip())
|
||||
]
|
||||
table[int(year)] = vals
|
||||
if 2042 not in table or len(table) < 20:
|
||||
sys.exit(f"glidepath table parsed but looks wrong: {sorted(table)}")
|
||||
for year, vals in table.items():
|
||||
if abs(sum(vals) - 100.0) > 0.005:
|
||||
sys.exit(f"enrollment-year {year} weights sum to {sum(vals)}, not 100")
|
||||
return table
|
||||
|
||||
|
||||
def num(v):
|
||||
return f"{v:g}"
|
||||
|
||||
|
||||
def rows_for(symbol, era_start, basis, pairs):
|
||||
return [
|
||||
f"symbol::{symbol},era_start::{era_start},basis::{basis},"
|
||||
f"ticker::{t},weight:num:{num(w)}"
|
||||
for t, w in pairs
|
||||
if w > 0
|
||||
]
|
||||
|
||||
|
||||
def main():
|
||||
table = parse_glidepath(open(sys.argv[1], encoding="utf-8", errors="replace").read())
|
||||
out = []
|
||||
|
||||
out.append(HEADER.rstrip("\n"))
|
||||
out.append("")
|
||||
out.append(ORCBI_NOTE.rstrip("\n"))
|
||||
out += rows_for(
|
||||
"ORCBI", "2018-09-01", "fitted",
|
||||
[("VSMPX", 36), ("VTPSX", 24), ("VBMPX", 40)],
|
||||
)
|
||||
out.append("")
|
||||
out.append(ORC42_NOTE.rstrip("\n"))
|
||||
out += rows_for(
|
||||
"ORC42", "2023-07-01", "fitted",
|
||||
[("VSMPX", 54), ("VTPSX", 36), ("VIPIX", 1.67), ("VBMPX", 6.66), ("VTIFX", 1.67)],
|
||||
)
|
||||
out.append("")
|
||||
out.append(
|
||||
"# Era 2 is the booklet current 2042 row. Validated: the reconstruction\n"
|
||||
"# reproduces all nine ORC42 anchors to within 0.06% using it."
|
||||
)
|
||||
out += rows_for(
|
||||
"ORC42", "2026-01-01", "fitted",
|
||||
[("VSMPX", 52.8), ("VTPSX", 35.2), ("VIPIX", 2), ("VBMPX", 8), ("VTIFX", 2)],
|
||||
)
|
||||
|
||||
# ORC42 in calendar year Y holds what enrollment-year row E = 4068 - Y holds
|
||||
# in the 2026-07-01 cross-section. Y=2026 -> E=2042 reproduces the fitted
|
||||
# current era, which is the one point where the mapping can be checked.
|
||||
for year in range(2027, 2048):
|
||||
enrollment = 4068 - year
|
||||
weights = table.get(enrollment)
|
||||
if weights is None:
|
||||
continue
|
||||
out.append("")
|
||||
out.append(
|
||||
f"# Projected: calendar {year} takes the booklet enrollment-year "
|
||||
f"{enrollment} row."
|
||||
)
|
||||
out += rows_for(
|
||||
"ORC42", f"{year}-01-01", "projected", list(zip(FUNDS, weights))
|
||||
)
|
||||
|
||||
sys.stdout.write("\n".join(out) + "\n")
|
||||
|
||||
|
||||
HEADER = """\
|
||||
#!srfv1
|
||||
# Underlying-fund weights for each plan portfolio, by era.
|
||||
#
|
||||
# GENERATED by tools/gen_model.py from the Plan Disclosure Booklet. Regenerate
|
||||
# rather than hand-editing the projected rows; 21 eras is too many to retype
|
||||
# reliably. Hand edits to the `fitted` rows are fine, but keep them in sync.
|
||||
#
|
||||
# These portfolios are unitized fund-of-funds trusts, not mutual funds. The Plan
|
||||
# Disclosure Booklet is explicit that they "reflect changes in value from income
|
||||
# and gains and losses on the sale of the Underlying Funds solely by increasing
|
||||
# or decreasing their Unit Value" -- all income compounds into the unit value and
|
||||
# nothing is distributed. That is why the reconstruction uses each underlying
|
||||
# fund's dividend-adjusted (total-return) close, not its raw NAV.
|
||||
#
|
||||
# Source of the weights: the Plan Disclosure Booklet allocation tables at
|
||||
# https://marcom.vestwell.com/program-description/oregon-college.html
|
||||
# Corroborated for the current era by two live JSON endpoints that agree exactly:
|
||||
# https://vss-api.vestwell.com/plans/oregon-college/portfolios (array form)
|
||||
# https://vss-api.vestwell.com/plans/oregon-college (object form)
|
||||
#
|
||||
# `era_start` is inclusive: a weight row applies to every trading day >= its
|
||||
# era_start, until superseded by a later era for the same symbol.
|
||||
#
|
||||
# `basis` records how much a row is worth trusting:
|
||||
# fitted In effect now or in the past, and validated -- the reconstruction
|
||||
# reproduces every observed anchor to within 0.08% using it.
|
||||
# projected Not yet in effect. Read off the booklet forward table, so it is
|
||||
# the plan's published intent, NOT an observed fact. The booklet
|
||||
# reserves the right to "change the asset allocations ... and change
|
||||
# the selection of Underlying Funds", so confirm before relying on
|
||||
# one. `verify` flags any projected era that has become current.
|
||||
#
|
||||
# NOTE ON FEES: no fee or drag term appears here, and none is needed. The
|
||||
# reconstruction pins the series at both ends of every anchor gap and distributes
|
||||
# the residual geometrically, which absorbs the asset-based fee, cash drag,
|
||||
# trade-date lag and securities-lending income together. `verify` reports the
|
||||
# drag each gap implies, as a cross-check against the booklet's published cost
|
||||
# table ($23.40 per $10,000/yr = 0.234% for ORCBI, $23.91 = 0.239% for ORC42)."""
|
||||
|
||||
ORCBI_NOTE = """\
|
||||
# ORCBI -- "Balanced Index", a STATIC portfolio. The booklet: "Static Portfolio
|
||||
# investments remain fixed, subject to periodic rebalancing". No glidepath, so
|
||||
# one era covers its whole life. These weights change only by Board action."""
|
||||
|
||||
ORC42_NOTE = """\
|
||||
# ORC42 -- "College Enrollment Year 2042", a GLIDEPATH portfolio.
|
||||
#
|
||||
# The booklet publishes a full cross-section of every enrollment-year portfolio
|
||||
# as of 2026-07-01. Because all of them follow one glidepath keyed on
|
||||
# years-to-enrollment, that cross-section IS ORC42's forward path shifted in
|
||||
# time: ORC42 in calendar year Y holds what enrollment-year row E = 4068 - Y
|
||||
# holds today. Y=2026 gives E=2042, which matches ORC42's fitted current era, so
|
||||
# the mapping checks out against observed data at the one point where it can.
|
||||
#
|
||||
# Two approximations in the projected rows:
|
||||
# 1. The booklet table is annual, but the booklet also says allocations step
|
||||
# QUARTERLY. So each projected era is up to three quarters coarse, and the
|
||||
# intermediate quarterly steps are not published anywhere.
|
||||
# 2. Era boundaries sit on 1 January. The current boundary was fitted to
|
||||
# 2026-01-01 by grid search over quarter boundaries; every candidate from
|
||||
# 2025-10-01 to 2026-04-01 gives rms <= 0.041%, so the fit cannot resolve
|
||||
# it better than that.
|
||||
#
|
||||
# Neither approximation matters much. Future eras cannot affect a reconstruction
|
||||
# that stops at today, and once daily feed values are appended to anchors.srf
|
||||
# every gap is one session, where pinning makes the weights literally inert (see
|
||||
# the test "weights are inert when anchors are one session apart"). They are
|
||||
# prefilled so the model degrades gracefully instead of silently using 2026
|
||||
# weights in 2035 if forward recording ever lapses.
|
||||
#
|
||||
# Era 1 is the booklet's max-equity plateau (its 2043/2044/2045 rows). ORC42
|
||||
# launched 19 years out from enrollment, so it sat on the plateau from
|
||||
# inception."""
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Loading…
Add table
Reference in a new issue