341 lines
19 KiB
Markdown
341 lines
19 KiB
Markdown
# Tally
|
|
|
|
A calculator with one engine and three frontends: a command line, a terminal UI, and an
|
|
Android app. The engine is Zig, and every frontend asks it for answers rather than
|
|
computing its own, so `2^100 + 1` has the same last digit everywhere and an error is
|
|
worded the same way everywhere.
|
|
|
|
```
|
|
$ tally '2^100 + 1'
|
|
1,267,650,600,228,229,401,496,703,205,377
|
|
$ tally '1/3 + 1/6'
|
|
0.5
|
|
$ tally 100 km to mi
|
|
100 km = 62.13711922373339696174 mi
|
|
$ tally -p '0xDEADBEEF and 0xFFFF'
|
|
dec(signed): 48,879
|
|
dec(unsigned): 48,879
|
|
hex: 00 00 00 00 00 00 BE EF
|
|
oct: 0 000 000 000 000 000 137 357
|
|
bin: 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 1011 1110 1110 1111
|
|
```
|
|
|
|
## What it does
|
|
|
|
- **Exact arithmetic.** Rationals of any size wherever the operations allow it, so
|
|
`1/3 + 1/6` is exactly `0.5`, `factorial(500)` has every digit, and a conversion like
|
|
`12 in to ft` is exactly `1`. Where exactness is impossible (`sqrt(2)`, `sin`), values
|
|
fall back to f64. The engine reports which tier every result is in, and the frontends
|
|
use it: the Android converter marks approximate rows.
|
|
- **Programmer mode.** 8, 16, 32, 64 and 128-bit integers, signed or unsigned, in every
|
|
base at once, with `and or xor not << >> >>> rol ror`. `^` is always power; XOR is the
|
|
`xor` keyword, so an operator means the same thing in every mode. Byte order is a
|
|
display choice for the hex and ASCII rows. The TUI adds an IEEE 754 float view.
|
|
- **Unit conversion.** 118 units in 12 categories (length, mass, temperature, volume,
|
|
speed, time, digital storage, area, energy, pressure, data rate, angle). A conversion is
|
|
part of the expression language, `2*3 kg to lb`, and is exact where the factors are.
|
|
- **Financial functions.** CAGR, compound interest, time value of money and amortization,
|
|
as ordinary functions: `cagr(10000, 25000, 5) * 100`. `tally --help` lists them all.
|
|
- **Variables and `Ans`.** `x = 21`, then `x * 2`, then `Ans + 1`.
|
|
|
|
The requirements and design, including what is planned and not built (a struct layout
|
|
visualizer, a financial screen on Android), are in [.kiro/specs/calculator](.kiro/specs/calculator).
|
|
|
|
## The three frontends
|
|
|
|
**Command line.** `tally EXPRESSION` evaluates and prints. `-p` is programmer mode;
|
|
`--bits N`, `--unsigned` and `--endian little` set it up and imply `-p`. Subcommands:
|
|
`tally convert 100 km to mi` and `tally amort 200000 6 360 --monthly` (a schedule). Run
|
|
`tally --help`, or `tally SUBCOMMAND --help`.
|
|
|
|
**Terminal UI.** `tally` with no arguments. Standard, Programmer, Financial and Convert
|
|
modes, switched with Tab; `?` shows the keys for the mode you are in. The mouse works
|
|
too: click a mode tab, a bit to flip it, a setting to change it.
|
|
|
|
**Android.** A calculator in the style of NCalc (NCalcLibre on F-Droid): a keypad, a
|
|
live answer before `=`, history on its own screen. A programmer screen with a tappable
|
|
bit grid. A converter that shows a value in every unit of its category and reopens
|
|
exactly where you left it. Everything reopens as it was left, including variables. See
|
|
[android/README.md](android/README.md).
|
|
|
|
## Install
|
|
|
|
Builds of `master` are published to the
|
|
[package registry](https://git.lerch.org/lobo/-/packages/generic/tally/latest), each
|
|
file with a detached signature:
|
|
|
|
| File | What |
|
|
|-----------------------|----------------------------------------------|
|
|
| `tally-x86_64-linux` | CLI and TUI, static (musl), any distribution |
|
|
| `tally-aarch64-linux` | the same for 64-bit ARM |
|
|
| `tally-aarch64-macos` | Apple silicon |
|
|
| `tally.apk` | the Android app, Android 8.0 and later |
|
|
|
|
```
|
|
curl -fLO https://git.lerch.org/api/packages/lobo/generic/tally/latest/tally-x86_64-linux
|
|
chmod +x tally-x86_64-linux
|
|
```
|
|
|
|
`latest` moves with every push to master. Each build is also published under its short
|
|
commit SHA, which never changes.
|
|
|
|
### Verifying a download
|
|
|
|
Every file is signed by a key held in a hardware security module, and every signature is
|
|
recorded in [sigstore's public transparency log](https://search.sigstore.dev); the build
|
|
log links each entry. To check a file against the published public key:
|
|
|
|
```
|
|
curl -fsSLO https://emil.lerch.org/serverpublic.pem
|
|
curl -fLO https://git.lerch.org/api/packages/lobo/generic/tally/latest/tally-x86_64-linux.sig
|
|
openssl dgst -sha256 -verify serverpublic.pem -signature tally-x86_64-linux.sig tally-x86_64-linux
|
|
```
|
|
|
|
The APK is signed twice: the detached signature above, and Android's own APK signature
|
|
(schemes v2 and v3), made with the same HSM key. To confirm the app is signed by that
|
|
key, compare the two digests below; they should be equal.
|
|
|
|
```
|
|
apksigner verify --verbose --print-certs tally.apk | grep 'public key SHA-256'
|
|
openssl pkey -pubin -in serverpublic.pem -outform DER | sha256sum
|
|
```
|
|
|
|
Android ties an app to its signing key permanently, so a published APK cannot update a
|
|
copy you built yourself (which is signed with your debug key), and the reverse. Uninstall
|
|
one before installing the other; the app's saved state goes with it.
|
|
|
|
### Install with F-Droid
|
|
|
|
Every master build is also published to an F-Droid repository, so the phone keeps itself
|
|
up to date. In F-Droid (or Droid-ify, or Obtainium's "F-Droid third-party repo" source),
|
|
add this repository; the fingerprint in the URL is what lets the client check the index
|
|
it downloads:
|
|
|
|
```
|
|
https://fdroid.lerch.org/repo?fingerprint=99387004A55002CF54F291D7ECA6A83AD7C7945E8E633160CC76790AD12E2D6E
|
|
```
|
|
|
|
Then install Tally from it. Each build's versionCode is the CI run number (its version
|
|
name is `0.1.0+<run>`), so every master build is an update to the one before, and the
|
|
repo keeps the newest three. The APK is the same one the registry publishes, signed by
|
|
the HSM key with the certificate in
|
|
[android/signing-cert.pem](android/signing-cert.pem).
|
|
|
|
The repository itself is not in this repo. It is `/data/fdroid` on the HSM runner's host,
|
|
served by nginx (only its `repo/`), with its `config.yml`, the app metadata, and the
|
|
keystore that signs the index. The keystore's password is kept off that host: CI has it
|
|
as the `FDROID_KEYSTORE_PASS` repository secret. The `fdroid` job in the workflow adds
|
|
each build and runs `fdroid update` there, in fdroidserver's container (pinned in the
|
|
workflow). To regenerate the index by hand on that host, after editing the metadata say:
|
|
|
|
```
|
|
read -rs FDROID_KEY_STORE_PASS && export FDROID_KEY_STORE_PASS FDROID_KEY_PASS="$FDROID_KEY_STORE_PASS"
|
|
docker run --rm -u 1000:1000 -e HOME=/tmp -e FDROID_KEY_STORE_PASS -e FDROID_KEY_PASS \
|
|
-e GIT_CONFIG_COUNT=1 -e GIT_CONFIG_KEY_0=safe.directory \
|
|
-e GIT_CONFIG_VALUE_0=/home/vagrant/fdroidserver \
|
|
--mount type=bind,source=/data/fdroid,target=/repo \
|
|
--entrypoint sh "<FDROID_IMAGE from the workflow>" -euc '
|
|
. /etc/profile.d/bsenv.sh
|
|
"$fdroidserver/fdroid" update'
|
|
```
|
|
|
|
### With nix
|
|
|
|
This repository is a flake, for the CLI and TUI (the Android app stays with gradle). To
|
|
add it to a `buildEnv`-style home profile:
|
|
|
|
```nix
|
|
inputs = {
|
|
tally.url = "git+https://git.lerch.org/lobo/tally.git";
|
|
# Optional: reuse your own nixpkgs instead of instantiating another one.
|
|
tally.inputs.nixpkgs.follows = "nixpkgs";
|
|
};
|
|
```
|
|
|
|
then add `tally.packages.${system}.default` to your package list. Or run it directly:
|
|
|
|
```
|
|
nix run git+https://git.lerch.org/lobo/tally.git -- '2^100 + 1'
|
|
```
|
|
|
|
The package is the `tally` binary, plus the engine as `lib/libtally-engine.a`,
|
|
`lib/libtally.so` and `include/tally.h`. `nix flake update tally` picks up a new version.
|
|
The build runs `zig build test` in the sandbox, in ReleaseSafe like the binary it
|
|
installs, so a version that fails its tests will not install. `nix develop` gives a shell
|
|
with Zig 0.16 and zls.
|
|
|
|
Changing the dependencies in `build.zig.zon` invalidates the `zigDeps` hash in
|
|
`nix/package.nix`. To refresh it: set the hash to `lib.fakeHash`, build, and paste the
|
|
hash nix reports.
|
|
|
|
## Building
|
|
|
|
Every tool comes from [mise](https://mise.jdx.dev), pinned in [.mise.toml](.mise.toml):
|
|
Zig 0.16.0, zls, zlint, and for Android a JDK, Gradle and the Android SDK command-line
|
|
tools. Nothing is installed system-wide.
|
|
|
|
```
|
|
mise install # the toolchain
|
|
zig build # zig-out/bin/tally, plus libtally (C ABI) and tally.h
|
|
zig build run -- '2 + 2' # or with no arguments, the TUI
|
|
zig build test # every test that needs only Zig
|
|
```
|
|
|
|
Cross-compiling is a flag: `zig build -Dtarget=aarch64-macos -Doptimize=ReleaseSafe`.
|
|
|
|
The Android app is two steps, because Gradle never invokes Zig: `zig build android`
|
|
writes one `libtally.so` per ABI, then Gradle packages it. Setup, the emulator and
|
|
deploying to a phone (including over Wi-Fi) are in [android/SETUP.md](android/SETUP.md).
|
|
|
|
```
|
|
mise run android-sdk # the SDK packages, once
|
|
zig build android # the native library
|
|
cd android && gradle assembleDebug # the APK
|
|
mise run android-install # all of the above, then install and launch
|
|
```
|
|
|
|
### Build steps
|
|
|
|
| Step | What it does |
|
|
|----------------------|-------------------------------------------------------------------------------------------|
|
|
| `zig build` | the CLI/TUI binary, the static engine, the shared C ABI library and its header |
|
|
| `zig build run` | build and run; arguments after `--` go to `tally` |
|
|
| `zig build test` | engine, CLI, TUI, C ABI and JNI tests, and a C program linked through `tally.h` |
|
|
| `zig build coverage` | the same tests under kcov, one report per tree; `-Dcoverage-threshold=80` fails below 80% |
|
|
| `zig build jvm-test` | the JNI layer loaded into a real JVM (needs a JDK; mise provides one) |
|
|
| `zig build android` | `zig-out/android/{arm64-v8a,x86_64,armeabi-v7a}/libtally.so`, ReleaseSmall |
|
|
|
|
### mise tasks
|
|
|
|
| Task | What it does |
|
|
|-----------------------------------|------------------------------------------------------------------------------------------------|
|
|
| `android-sdk` | install the SDK packages the app builds against |
|
|
| `android-emulator`, `android-avd` | install the emulator and a 16 KB-page image, create the `tally35` AVD in `.tmp/avd` |
|
|
| `android-run` | boot that AVD |
|
|
| `android-install` | build the libraries and APK, install on the attached device, launch it |
|
|
| `android-audit` | check the Android libraries define every symbol they use (no libc, no TLS, no `DT_NEEDED`) |
|
|
| `apk-signer-test` | build a release APK and sign it with CI's HSM signer, against SoftHSM (needs docker or podman) |
|
|
|
|
## How it is put together
|
|
|
|
```
|
|
engine/src/ the engine: tokenizer, Parser, evaluator, Rational, Integer, units,
|
|
financial, programmer, bitwise, float_interp
|
|
engine/src/c_api.zig the C ABI every non-Zig frontend uses: sessions, JSON results
|
|
engine/src/jni.zig JNI entry points for Android, written against the JNI function
|
|
table by index - no NDK, no jni.h
|
|
include/tally.h the C header, checked against the library by engine/test/c_abi_test.c
|
|
src/main.zig, src/cli/ the command line
|
|
src/tui.zig, src/tui/ the terminal UI (libvaxis)
|
|
android/ the Compose app; Kotlin parses no expressions and formats no numbers
|
|
build/ coverage (kcov) support for build.zig
|
|
.kiro/specs/ requirements, design and task history
|
|
```
|
|
|
|
A few decisions shape everything else, and the design document explains each:
|
|
|
|
- **One engine, no reimplementation.** Frontends format nothing and phrase no errors
|
|
themselves. A wrong answer is wrong in all three places, which is the point.
|
|
- **The C ABI is the seam.** A session owns its memory; results are JSON borrowed until
|
|
the next call; one session per thread. Android reaches it through a hand-written JNI
|
|
layer whose function-table indices are proved against the running JVM in `JNI_OnLoad`,
|
|
and against a desktop JVM by `zig build jvm-test`.
|
|
- **The Android library links against nothing.** No libc, no NDK. That is why
|
|
`android-audit` exists: an undefined symbol links fine and fails in `dlopen` on a phone.
|
|
- **Display budgets belong to the frontend.** The engine renders to whatever digit budget
|
|
it is given; a phone, a terminal and a clipboard want different ones.
|
|
|
|
## Tests
|
|
|
|
- `zig build test`: about a thousand Zig tests across the engine, CLI, TUI (rendering
|
|
real frames and reading the cells back), the C ABI, and the JNI layer against a
|
|
synthetic function table; plus a C program that links the shared library through
|
|
`tally.h`, which catches a header that disagrees with the library. CI also runs it with
|
|
`-Doptimize=ReleaseSafe`, `ReleaseSmall` and `ReleaseFast`: what ships is not Debug,
|
|
and Debug's 0xaa fill of undefined memory can hide a read of something never written
|
|
(it hid one in the JNI tests).
|
|
- `zig build jvm-test`: the JNI entry points from Java, which is the only way to check
|
|
the function-table indices without a device. CI runs it in Debug and in ReleaseSmall,
|
|
the optimization the Android library ships with.
|
|
- `cd android && gradle testDebugUnitTest`: JVM tests of the app's rules (the converter,
|
|
programmer mode, saved state), no device needed.
|
|
- `cd android && gradle connectedAndroidTest`: the engine through the real app on a
|
|
device or emulator. It uninstalls the app afterwards.
|
|
- `mise run apk-signer-test`: the CI signing path end to end against SoftHSM.
|
|
|
|
[Pre-commit](https://pre-commit.com) hooks (run with `prek`, which mise installs) check
|
|
formatting, run zlint, build, and require 80% coverage. They also refuse smart
|
|
punctuation: the source is ASCII.
|
|
|
|
## CI/CD
|
|
|
|
[.forgejo/workflows/build.yaml](.forgejo/workflows/build.yaml) runs on every push:
|
|
|
|
| Job | Runner | What |
|
|
|---------------------|--------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
|
| Engine, CLI and TUI | `ubuntu-latest` | `zig fmt --check`, zlint, build, `zig build test` in Debug and each release mode, `zig build jvm-test` in Debug and ReleaseSmall, then release builds for x86_64 and aarch64 Linux and aarch64 macOS |
|
|
| Android | `ubuntu-latest` | `zig build android`, the library audit, the JVM unit tests, debug and unsigned release APKs (with lint's release checks) |
|
|
| Sign | `ubuntu-latest-with-hsm` | master only: signs the APK with the HSM, then every release file with a detached signature logged to sigstore |
|
|
| Publish | `ubuntu-latest` | master only: uploads the signed files to the generic package registry under the short SHA and `latest` |
|
|
| F-Droid repo | `ubuntu-latest-with-hsm` | master only: adds the signed APK to the F-Droid repository on the HSM runner's host and regenerates its index |
|
|
|
|
Every job reports to ntfy. Zlint and Gradle are downloaded at pinned versions and checked
|
|
against pinned SHA-256 digests; Zig's version comes from `build.zig.zon`.
|
|
|
|
Secrets: `HSM_USER_PIN`, `PACKAGE_PUSH`, and `NTFY_HOST`, `NTFY_TOPIC`, `NTFY_USER`,
|
|
`NTFY_PASSWORD`.
|
|
|
|
### Signing
|
|
|
|
The Sign job signs with [action-hsm-sign](https://git.lerch.org/lobo/action-hsm-sign),
|
|
in one step: the APK first (schemes v2 and v3, through Debian's `apksigner` and Java's
|
|
SunPKCS11 provider), then a detached signature of every release file, the signed APK
|
|
included. The action power cycles the HSM on its smart USB hub for the duration, and
|
|
the private key never leaves the HSM.
|
|
|
|
An APK signature carries a certificate, and Java only offers a PKCS#11 key that has one
|
|
on the token; whatever certificate the token holds for the key is used. It is not
|
|
Tally's own: the token holds one certificate per key, so every app signed with the key
|
|
carries it, and its subject is generic. It is the identity of all of them on every
|
|
device: an APK signed with a different one is refused as an update, and users uninstall
|
|
and reinstall. So it is chosen once, kept in this repository as
|
|
[android/signing-cert.pem](android/signing-cert.pem) (it is public), and pinned in the
|
|
workflow by its SHA-256 (`apk_cert_sha256`), so a certificate replaced on the token fails
|
|
the build instead of silently changing the app's identity.
|
|
|
|
It was made on the HSM host, with the HSM powered, from a clone of action-hsm-sign. The
|
|
certificate is assembled in the container and signed on the token with the user PIN, so
|
|
the key never leaves it, and written to the card with the Admin PIN (the card's own
|
|
rule for certificates). `REPLACE_CERT=1` because the key already had a certificate (the
|
|
card's 2023 "AUT certificate", which signed the very first build, 255f239):
|
|
|
|
```
|
|
docker build -t hsm-signer signer
|
|
read -rs PKCS11_PIN && export PKCS11_PIN
|
|
read -rs ADMIN_PIN && export ADMIN_PIN
|
|
docker run --rm -v /run/pcscd/pcscd.comm:/run/pcscd/pcscd.comm:ro \
|
|
-e PKCS11_PIN -e ADMIN_PIN hsm-signer check-pin
|
|
docker run --rm -v /run/pcscd/pcscd.comm:/run/pcscd/pcscd.comm:ro \
|
|
-e PKCS11_PIN -e ADMIN_PIN -e REPLACE_CERT=1 hsm-signer make-cert '/CN=Emil Lerch/O=lerch.org'
|
|
```
|
|
|
|
`check-pin` tries each PIN once first; a wrong PIN costs a try, and the card locks a PIN
|
|
after three. `make-cert` prints the certificate and its `sha256 (apk_cert_sha256)`:
|
|
`4b41d0b1b3a184342fdc348e67563c8ca6cbc3765fa7341dee8a4a8fbb5d4aad` (`CN=Emil Lerch,
|
|
O=lerch.org`, valid to 2056). The key is `03` by default, the one the detached
|
|
signatures use (`KEY_ID` changes it); a new certificate does not change the key, so the
|
|
detached signatures and `serverpublic.pem` are unaffected. Its 30-year validity does not
|
|
matter for APKs, whose certificate dates Android does not check, but is long enough for
|
|
Google Play (validity past 2033-10-22).
|
|
|
|
`mise run apk-signer-test` builds the release APK and runs action-hsm-sign's
|
|
`signer/test.sh` on it, at the version the workflow uses (or a local checkout named by
|
|
`ACTION_HSM_SIGN`). That is the same signer image against SoftHSM: a token with a bare
|
|
key, a detached signature, `make-cert` (and its refusal to replace a certificate), APK
|
|
signing with the certificate pinned, `apksigner verify`, and a check that the APK's
|
|
signing key is the token's. What it cannot cover is the HSM itself and the host's pcscd,
|
|
so the first run on the HSM runner is the real test of those.
|
|
|
|
## License
|
|
|
|
MIT. See [LICENSE](LICENSE).
|