From 33f3343d0c3c1614408826dd08647789237d65d4 Mon Sep 17 00:00:00 2001 From: Emil Lerch Date: Tue, 6 Jan 2026 17:25:29 -0800 Subject: [PATCH] documentation refresh (README still needs a lot of work) --- API_ENDPOINTS.md | 574 ------------------------ ARCHITECTURE.md | 362 +++++++++++++++ CACHE_CONFIGURATION.md | 140 ------ MISSING_FEATURES.md | 31 ++ README.md | 257 ++++------- REWRITE_STRATEGY.md | 714 ------------------------------ TARGET_ARCHITECTURE.md | 967 ----------------------------------------- 7 files changed, 485 insertions(+), 2560 deletions(-) delete mode 100644 API_ENDPOINTS.md create mode 100644 ARCHITECTURE.md delete mode 100644 CACHE_CONFIGURATION.md create mode 100644 MISSING_FEATURES.md delete mode 100644 REWRITE_STRATEGY.md delete mode 100644 TARGET_ARCHITECTURE.md diff --git a/API_ENDPOINTS.md b/API_ENDPOINTS.md deleted file mode 100644 index 8e79aea..0000000 --- a/API_ENDPOINTS.md +++ /dev/null @@ -1,574 +0,0 @@ -# wttr.in API Endpoints Reference - -## Base URL - -- Production: `https://wttr.in` -- Local: `http://localhost:8082` (Go proxy) → `http://localhost:8002` (Python backend) - -## Weather Query Endpoints - -### GET / - -Get weather for IP-based location. - -**Response:** ANSI weather report (or HTML if browser User-Agent) - -**Example:** -```bash -curl wttr.in -``` - -### GET /{location} - -Get weather for specific location. - -**Path Parameters:** -- `location` - City name, coordinates, airport code, or special location - -**Location Formats:** -- City name: `London`, `New York`, `Salt+Lake+City` -- Coordinates: `55.7558,37.6173` -- Airport (IATA): `muc`, `ham` -- Search term: `~Eiffel+Tower`, `~Kilimanjaro` -- Domain: `@github.com`, `@msu.ru` -- IP address: `8.8.8.8` -- Auto-detect: `MyLocation` or empty -- Moon: `Moon`, `Moon@2016-12-25` -- Cyclic: `London:Paris:Berlin` (rotates based on time) - -**Special Prefixes:** -- `~` - Search for location (uses geolocator) -- `@` - Resolve domain to IP, then get location -- No prefix - Exact location name - -**Query Parameters:** See "Query Parameters" section below - -**Examples:** -```bash -curl wttr.in/London -curl wttr.in/~Eiffel+Tower -curl wttr.in/@github.com -curl wttr.in/Moon -curl wttr.in/London:Paris:Berlin?period=60 -``` - -### GET /{location}.png - -Get weather as PNG image. - -**Path Parameters:** -- `location` - Same as above, but with `.png` extension - -**PNG Filename Format:** -``` -{location}_{options}.png -``` - -**Options in filename:** -- `{width}x` - Width in pixels (e.g., `200x`) -- `x{height}` - Height in pixels (e.g., `x300`) -- `{width}x{height}` - Both dimensions (e.g., `200x300`) -- `lang={code}` - Language code (e.g., `lang=ru`) -- Single-letter options: `t`, `n`, `q`, etc. (see below) -- Separate multiple options with `_` - -**Examples:** -```bash -curl wttr.in/Paris.png > paris.png -curl wttr.in/London_200x_t_lang=fr.png > london.png -curl wttr.in/Berlin_0tqp_lang=de.png > berlin.png -``` - -**Note:** Use `_` instead of `?` and `&` for PNG URLs - -## Special Pages - -### GET /:help - -Display help page with usage instructions. - -**Query Parameters:** -- `lang={code}` - Language for help page - -**Example:** -```bash -curl wttr.in/:help -curl wttr.in/:help?lang=de -``` - -### GET /:bash.function - -Get bash function for shell integration. - -**Example:** -```bash -curl wttr.in/:bash.function >> ~/.bashrc -``` - -### GET /:translation - -Display translation information and supported languages. - -**Example:** -```bash -curl wttr.in/:translation -``` - -### GET /:iterm2 - -Get iTerm2 integration instructions. - -**Example:** -```bash -curl wttr.in/:iterm2 -``` - -## Static Files - -### GET /files/{path} - -Serve static files. - -**Path Parameters:** -- `path` - Relative path to file in share/static/ - -**Example:** -```bash -curl wttr.in/files/example-wttr-v2.png -``` - -### GET /favicon.ico - -Get favicon. - -## Query Parameters - -### Single-Letter Options - -Combine multiple single-letter options in the query string without values: - -```bash -curl wttr.in/London?Tnq -``` - -**Available Options:** - -| Option | Description | -|--------|-------------| -| `A` | Force ANSI output (even for browsers) | -| `n` | Narrow output (narrower terminal width) | -| `m` | Use metric units (force metric even for US) | -| `M` | Use m/s for wind speed | -| `u` | Use imperial units (Fahrenheit, mph) | -| `I` | Inverted colors | -| `t` | Transparency for PNG (transparency=150) | -| `T` | No terminal sequences (plain text) | -| `p` | Add padding | -| `0` | Show current weather only (no forecast) | -| `1` | Show today + 1 day | -| `2` | Show today + 2 days | -| `3` | Show today + 3 days (default) | -| `q` | Quiet mode (no caption) | -| `Q` | Super quiet (no city name) | -| `F` | No follow line (no "Follow @igor_chubin") | - -**Unit System Defaults:** - -The service automatically selects units based on: -1. Explicit query parameter (`?u` or `?m`) - highest priority -2. Language parameter (`lang=us`) - forces imperial -3. Client IP geolocation - US IPs default to imperial -4. Default - metric for all other locations - -**Examples:** -```bash -curl wttr.in/London?T # Plain text, no ANSI -curl wttr.in/London?u # Imperial units -curl wttr.in/London?m # Metric units -curl wttr.in/London?0 # Current weather only -curl wttr.in/London?Tnq # Plain text, narrow, quiet -``` - -### Named Parameters - -| Parameter | Description | Example | -|-----------|-------------|---------| -| `lang={code}` | Language code (2-letter ISO 639-1) | `lang=de` | -| `format={fmt}` | Output format (see below) | `format=3` | -| `view={view}` | View type (alias for format) | `view=v2` | -| `period={sec}` | Update interval for cyclic locations | `period=60` | -| `transparency={0-255}` | PNG transparency level | `transparency=100` | - -**Examples:** -```bash -curl wttr.in/London?lang=fr -curl wttr.in/London?format=3 -curl wttr.in/London:Paris?period=60 -``` - -### Combining Parameters - -```bash -curl wttr.in/London?Tn&lang=de&format=3 -curl wttr.in/Paris?u&format=v2 -``` - -## Output Formats - -### format=1 (One-line, compact) - -```bash -curl wttr.in/London?format=1 -# Output: London: ⛅️ +7°C -``` - -### format=2 (One-line, with wind) - -```bash -curl wttr.in/London?format=2 -# Output: London: ⛅️ +7°C 🌬️↗11km/h -``` - -### format=3 (One-line, detailed) - -```bash -curl wttr.in/London?format=3 -# Output: London: ⛅️ +7°C 🌬️↗11km/h 💧65% -``` - -### format=4 (One-line, full) - -```bash -curl wttr.in/London?format=4 -# Output: London: ⛅️ +7°C 🌬️↗11km/h 💧65% ☀️06:45-16:32 -``` - -### format={custom} (Custom format) - -Use `%` notation to create custom formats: - -| Code | Description | -|------|-------------| -| `%c` | Weather condition emoji | -| `%C` | Weather condition text | -| `%h` | Humidity | -| `%t` | Temperature (actual) | -| `%f` | Temperature (feels like) | -| `%w` | Wind | -| `%l` | Location | -| `%m` | Moon phase 🌑🌒🌓🌔🌕🌖🌗🌘 | -| `%M` | Moon day | -| `%p` | Precipitation (mm) | -| `%o` | Probability of precipitation | -| `%P` | Pressure (hPa) | -| `%D` | Dawn time | -| `%S` | Sunrise time | -| `%z` | Zenith time | -| `%s` | Sunset time | -| `%d` | Dusk time | - -**Examples:** -```bash -curl wttr.in/London?format="%l:+%c+%t" -# Output: London: ⛅️ +7°C - -curl wttr.in/London?format="%l:+%C+%t+%w" -# Output: London: Partly cloudy +7°C ↗11km/h - -curl wttr.in/London?format="%D+%S+%z+%s+%d" -# Output: 06:30 06:45 12:15 16:32 17:00 -``` - -**Note:** In tmux.conf, escape `%` as `%%` - -### format=v2 (Data-rich experimental) - -Experimental format with additional information: -- Temperature and precipitation graphs -- Moon phase for 4 days -- Current conditions with all metrics -- Timezone information -- Astronomical times (dawn, sunrise, zenith, sunset, dusk) -- Precise GPS coordinates - -```bash -curl wttr.in/London?format=v2 -# or -curl v2.wttr.in/London -# or (Nerd Fonts) -curl v2d.wttr.in/London # day variant -curl v2n.wttr.in/London # night variant -``` - -**Limitations:** -- Terminal only (no HTML) -- English only - -### format=j1 (JSON) - -Machine-readable JSON format with all weather data. - -```bash -curl wttr.in/London?format=j1 -``` - -**Response Structure:** -```json -{ - "current_condition": [{ - "FeelsLikeC": "25", - "FeelsLikeF": "76", - "cloudcover": "100", - "humidity": "76", - "observation_time": "04:08 PM", - "precipMM": "0.2", - "pressure": "1019", - "temp_C": "22", - "temp_F": "72", - "uvIndex": 5, - "visibility": "16", - "weatherCode": "122", - "weatherDesc": [{"value": "Overcast"}], - "winddir16Point": "NNE", - "winddirDegree": "20", - "windspeedKmph": "7", - "windspeedMiles": "4" - }], - "nearest_area": [...], - "request": [...], - "weather": [...] -} -``` - -**Weather Codes:** See https://www.worldweatheronline.com/developer/api/docs/weather-icons.aspx - -### format=p1 (Prometheus Metrics) - -Prometheus-compatible metrics format. - -```bash -curl wttr.in/London?format=p1 -``` - -**Response:** -``` -# HELP temperature_feels_like_celsius Feels Like Temperature in Celsius -temperature_feels_like_celsius{forecast="current"} 7 -# HELP temperature_feels_like_fahrenheit Feels Like Temperature in Fahrenheit -temperature_feels_like_fahrenheit{forecast="current"} 45 -... -``` - -**Prometheus Config Example:** -```yaml -- job_name: 'wttr_in_london' - static_configs: - - targets: ['wttr.in'] - metrics_path: '/London' - params: - format: ['p1'] -``` - -## Language Support - -54 languages supported via `lang` parameter or subdomain: - -```bash -curl wttr.in/London?lang=de -curl de.wttr.in/London -``` - -**Supported Languages:** -af, ar, az, be, bg, bs, ca, cy, da, de, el, en, eo, es, et, eu, fa, fr, fy, ga, he, hr, hu, hy, ia, id, is, it, ja, kk, lv, mk, nb, nl, nn, oc, pl, pt, pt-br, ro, ru, sl, te, th, tr, uk, uz, vi, zh-cn, zh-tw - -**Language Detection Priority:** -1. Subdomain (e.g., `de.wttr.in`) -2. `lang` parameter -3. `Accept-Language` header -4. Default (English) - -## Multiple Locations - -Query multiple locations in one request using `:` separator: - -```bash -curl wttr.in/London:Paris:Berlin?format=3 -# Output: -# London: ⛅️ +7°C -# Paris: ☀️ +12°C -# Berlin: 🌧 +5°C -``` - -**Cyclic Locations:** - -Rotate through locations based on time: - -```bash -curl wttr.in/London:Paris:Berlin?period=60 -``` - -This will show: -- London for 60 seconds -- Paris for next 60 seconds -- Berlin for next 60 seconds -- Repeat - -Useful for tmux status bars or monitoring dashboards. - -## HTTP Headers - -### Request Headers - -| Header | Purpose | -|--------|---------| -| `User-Agent` | Determines output format (ANSI vs HTML) | -| `Accept-Language` | Language preference | -| `X-Forwarded-For` | Client IP (for location detection) | -| `X-Real-IP` | Client IP (alternative) | -| `X-PNG-Query-For` | Client IP (for PNG requests) | - -### Response Headers - -| Header | Value | -|--------|-------| -| `Content-Type` | `text/plain` or `text/html` or `image/png` | -| `Cache-Control` | `no-cache, no-store, must-revalidate` (PNG only) | -| `Access-Control-Allow-Origin` | `*` | - -## HTTP Status Codes - -| Code | Meaning | -|------|---------| -| 200 | Success | -| 403 | Forbidden (location blacklisted) | -| 429 | Too Many Requests (rate limit exceeded) | -| 500 | Internal Server Error | -| 503 | Service Unavailable (API error) | - -## Rate Limits - -Per-IP limits: -- 300 requests per minute -- 3600 requests per hour -- 86400 requests per day - -**Response on limit exceeded:** -- HTTP 429 -- Body: "Too many queries" message - -## Caching - -Responses are cached for 1000-2000 seconds (randomized). - -**Cache key includes:** -- User-Agent -- Query string -- Client IP -- Language - -**Non-cacheable requests:** -- Cyclic locations (containing `:`) - -## Examples - -### Basic Usage - -```bash -# Current location -curl wttr.in - -# Specific city -curl wttr.in/London - -# Airport -curl wttr.in/muc - -# Coordinates -curl wttr.in/55.7558,37.6173 - -# Moon phase -curl wttr.in/Moon -``` - -### Output Formats - -```bash -# Plain text -curl wttr.in/London?T - -# One-line -curl wttr.in/London?format=3 - -# JSON -curl wttr.in/London?format=j1 - -# PNG -curl wttr.in/London.png > weather.png -``` - -### Units - -```bash -# Metric -curl wttr.in/London?m - -# Imperial -curl wttr.in/London?u -``` - -### Languages - -```bash -# German -curl wttr.in/London?lang=de -curl de.wttr.in/London - -# French -curl fr.wttr.in/Paris -``` - -### Customization - -```bash -# Narrow, quiet, plain text -curl wttr.in/London?Tnq - -# Current weather only -curl wttr.in/London?0 - -# Custom format -curl wttr.in/London?format="%l:+%c+%t+%w" - -# Transparent PNG -curl wttr.in/London_t.png > weather.png -``` - -### Shell Integration - -```bash -# Add to .bashrc -weather() { - curl wttr.in/$1 -} - -# Use it -weather London -weather "New York" -``` - -### Tmux Status Bar - -```bash -# In tmux.conf -set -g status-right '#(curl -s wttr.in/London?format="%%l:+%%c%%20%%t%%60%%w")' -``` - -### Multiple Locations - -```bash -# All at once -curl 'wttr.in/{London,Paris,Berlin}?format=3' - -# Cyclic (for monitoring) -curl wttr.in/London:Paris:Berlin?period=60&format=3 -``` diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md new file mode 100644 index 0000000..b96847b --- /dev/null +++ b/ARCHITECTURE.md @@ -0,0 +1,362 @@ +# wttr Architecture + +## Overview + +Single Zig binary, including: +- HTTP server utilizing [http.zig](https://github.com/karlseguin/http.zig) +- L1 memory/L2 file caching scheme with single directory for + * Geocoding (place name -> coordinates) as a permanent cache + * IP -> location (via [GeoLite2](https://github.com/P3TERX/GeoLite.mmdb) with fallback to [IP2Location](https://ip2location.io)] as a permanent cache + * Weather as a temporary cache +- Pluggable weather provider interface +- Rate limiting middleware + +The idea was to keep most or all of the features of [wttr.in](https://wttr.in) +while vastly simplify the architecture of that system, which had evolved +significantly over time. This is a full re-write of that system. + +## System Diagram + +``` +Client Request + ↓ +HTTP Server (http.zig) + ↓ +Rate Limiter (middleware) + ↓ +Router + ↓ +Request Handler + ↓ +Location Resolver + ↓ +Provider interface cache check + ↓ (miss) +Weather Provider (interface) + ├─ MetNo (default) + └─ Mock (tests) + ↓ +Cache Store + ↓ +Renderer + ├─ ANSI + ├─ JSON + ├─ Line + └─ v2 + ↓ +Response +``` + +## Module Structure + +``` +src/ +├── main.zig # Entry point, server setup +├── Config.zig # Configuration +├── http/ +│ ├── server.zig # HTTP server wrapper +│ ├── router.zig # Route matching +│ ├── handler.zig # Request handlers +│ └── middleware.zig # Rate limiter +├── cache/ +│ ├── cache.zig # Cache interface +│ ├── lru.zig # In-memory LRU +│ └── file.zig # File-backed storage +├── location/ +│ ├── resolver.zig # Main location resolution +│ ├── GeoLite2.zig # GeoIP wrapper +│ └── Ip2location.zig # IP2Location fallback +├── weather/ +│ ├── provider.zig # Weather provider interface +│ ├── MetNo.zig # met.no implementation +│ └── types.zig # Weather data structures +└── render/ + ├── ansi.zig # ANSI terminal output + ├── json.zig # JSON output + └── line.zig # One-line format +``` + +## Core Components + +### HTTP Server + +**Responsibilities:** +- Listen on configured port +- Parse HTTP requests +- Route to handlers +- Apply middleware (rate limiting) +- Return responses + +**Dependencies:** +- [http.zig](https://github.com/karlseguin/http.zig) + +**Routes:** +``` +GET / → weather for IP location +GET /{location} → weather for location +GET /:help → help page +``` + +### Rate Limiter + +**Algorithm:** Token Bucket + +**Configuration:** +```zig +pub const RateLimitConfig = struct { + capacity: u32 = 300, // Max tokens in bucket + refill_rate: u32 = 5, // Tokens per second + refill_interval_ms: u64 = 200, // Refill every 200ms +}; +``` + +**Implementation:** +- HashMap of IP → TokenBucket +- Each request consumes 1 token +- Tokens refill at configured rate (default: 5/second) +- Bucket capacity: 300 tokens (allows bursts) +- Periodic cleanup of old buckets (not accessed in 1 hour) + +### Cache + +**Single-layer cache with two storage backends:** + +**Interface:** +```zig +pub const Cache = struct { + lru: LRU, + file_store: FileStore, + + pub fn get(self: *Cache, key: []const u8) ?[]const u8; + pub fn put(self: *Cache, key: []const u8, value: []const u8, ttl: u64) !void; +}; +``` + +**Cache Key:** +``` +{user_agent}:{path}:{query}:{client_ip} +``` + +**Storage Strategy:** +- Small responses (<1KB): In-memory LRU only +- Large responses (≥1KB): LRU stores file path, data on disk +- TTL: 1000-2000s (randomized to prevent thundering herd) + +**Cache Locations:** + +All caches default to `$XDG_CACHE_HOME/wttr` (typically `~/.cache/wttr`). + +1. **Weather Response Cache** + - Location: `$WTTR_CACHE_DIR` (default: `~/.cache/wttr/`) + - Size: 10,000 entries (configurable via `WTTR_CACHE_SIZE`) + - Expiration: 1000-2000 seconds (randomized) + - Eviction: LRU + +2. **Geocoding Cache** + - Location: `$WTTR_GEOCACHE_FILE` (default: `~/.cache/wttr/geocache.json`) + - Format: JSON + - Expiration: None (persists indefinitely) + +3. **IP2Location Cache** + - Location: `$IP2LOCATION_CACHE_FILE` (default: `~/.cache/wttr/ip2location.cache`) + - Format: Binary (32-byte records) + - Expiration: None (persists indefinitely) + +4. **GeoIP Database** + - Location: `$WTTR_GEOLITE_PATH` (default: `~/.cache/wttr/GeoLite2-City.mmdb`) + - Auto-downloaded if missing + +### Location Resolver (`src/location/resolver.zig`) + +**Responsibilities:** +- Parse location from URL +- Normalize location names +- Resolve IP to location (GeoIP) +- Resolve names to coordinates (geocoding) +- Handle special prefixes (~, @) + +**GeoIP:** +- Uses MaxMind GeoLite2 database +- Auto-downloads if missing +- Fallback to IP2Location API + +**Ip2Location:** +- Uses [Ip2Location](https://ip2location.io) +- API key not required...comes with a limit of 1k/day. Because GeoLite2 handles + most of the mapping, and results are cached, this should be fine for most needs +- API key provides 50k/month requests +- Set via IP2LOCATION_API_KEY environment variable + +**Geocoding:** +- Uses nominatim, part of the OpenStreetMap project +- API key not required + +### Weather Provider + +**Interface (pluggable):** +```zig +pub const WeatherProvider = struct { + ptr: *anyopaque, + vtable: *const VTable, + cache: *Cache, + + pub const VTable = struct { + fetchRaw: *const fn (ptr: *anyopaque, allocator: std.mem.Allocator, coords: Coordinates) anyerror![]const u8, + parse: *const fn (ptr: *anyopaque, allocator: std.mem.Allocator, raw: []const u8) anyerror!types.WeatherData, + deinit: *const fn (ptr: *anyopaque) void, + }; + + pub fn fetch(self: WeatherProvider, location: []const u8) !WeatherData; +}; +``` + +Note that the interface will handle result caching according to the description +of the cache section above. Fetch and parse are two separate functions in this +interface to assist with unit tests. + +**MetNo Implementation:** +- Fetches from Met.no API +- Performs timezone conversions at ingestion, so results are in the timezone + of the target location +- Groups forecast data by local date +- No API key necessary, but requires METNO_TOS_IDENTIFYING_EMAIL environment + variable to be set + +**Provider vs Renderer Responsibilities:** + +**Weather Provider:** +- Fetch raw weather data from external APIs +- Parse API responses into structured types +- Perform timezone conversions to the timezone of the weather location + once at ingestion time +- Group forecast data by local date (not UTC date) +- Store both UTC time and local time in forecast data + +**Renderer:** +- Format weather data for display +- Select appropriate hourly forecasts for display +- Apply unit conversions (metric/imperial). Conversion functions are in the core + weather types, but the renderer is responsible for calling them +- Handle partial days with missing data +- Format dates and times for human readability +- Should NOT perform timezone calculations + +**Key principle:** Timezone conversions happen once at the provider level + +**Implementation details:** +- Core data structures use `zeit.Time` and `zeit.Date` types for type safety +- `HourlyForecast` contains both `time: zeit.Time` (UTC) and `local_time: zeit.Time` +- MetNo provider converts the location to timezone offsets through `src/location/timezone_offsets.zig`) + * This code uses pre-computed timezone offset lookup table + * It is auto-generated based on a Python script + * It is *NOT* precise, currently calculating the timezone based on the longitude + only. This could be up to 2 hours different from the actual timezone...however, + the purpose here is to report weather to a granularity of morning/noon/evening/night + +### Renderers + +There are currently 5 renderers: + +* formatted +* json +* v2 +* custom +* line + +**Formatted Renderer:** +```zig +pub const FormattedRenderer = struct { + pub fn render(allocator: Allocator, data: WeatherData, options: RenderOptions) ![]const u8; +}; +``` + +This is the most complex of all the renders, and is the default when the user +doesn't specifically choose something else. It displays the full table of +current conditions and up to 3 day forecast, and does so in plain text, ansi, +or html formats. Currently, these formats are coded separately. Note it is +possible to provide ansi and html, then have the code strip out extra markup to +provide plain text. This is **not** how the code works today. Also, the +original project has an option to avoid characters some brain-dead terminals +can't handle. That has not been implemented. When done, this is likely to be +implemented as a search/replace for the few unicode characters that exist and +replace them manually. + + +**JSON Renderer:** +```zig +pub const JsonRenderer = struct { + pub fn render(allocator: Allocator, data: WeatherData) ![]const u8; +}; +``` +Just utilizes the `std.json.fmt` api to render the underlying data type. No +attempt has been made to match wttr.in data or format (probably should). + +**One-Line Renderer:** +```zig +pub const LineRenderer = struct { + pub fn render(allocator: Allocator, data: WeatherData, format: []const u8) ![]const u8; +}; + +// Formats: +// 1: "London: ⛅️ +7°C" +// 2: "London: ⛅️ +7°C 🌬️↗11km/h" +// 3: "London: ⛅️ +7°C 🌬️↗11km/h 💧65%" +// Custom: "%l: %c %t" → "London: ⛅️ +7°C" +``` + +## Network Calls + +The application makes network calls to the following services: + +### 1. GeoLite2 Database Download +- **URL:** `https://github.com/P3TERX/GeoLite.mmdb/raw/download/GeoLite2-City.mmdb` +- **Purpose:** Download MaxMind GeoLite2 database if missing +- **Frequency:** Only on first run or when database is missing + +### 2. IP2Location API +- **URL:** `https://api.ip2location.io/?key={API_KEY}&ip={IP_ADDRESS}` +- **Purpose:** Fallback geolocation when MaxMind lookup fails +- **Frequency:** Only when `IP2LOCATION_API_KEY` is set and MaxMind fails +- **Rate Limit:** Free tier: 30,000 requests/month +- **Caching:** Results cached persistently + +### 3. Nominatim Geocoding API +- **URL:** `https://nominatim.openstreetmap.org/search?q={LOCATION}&format=json&limit=1` +- **Purpose:** Convert city/location names to coordinates +- **Frequency:** When user requests weather by city name +- **Rate Limit:** Maximum 1 request per second +- **Caching:** Results cached persistently + +### 4. Met.no Weather API +- **URL:** `https://api.met.no/weatherapi/locationforecast/2.0/compact?lat={LAT}&lon={LON}` +- **Purpose:** Fetch weather forecast data +- **Frequency:** Every request (unless cached) +- **Rate Limit:** No explicit limit, respectful usage required +- **Caching:** Results cached in memory (LRU) + +## Dependencies + +**External (Zig packages):** +- HTTP Server: [http.zig](https://github.com/karlseguin/http.zig) +- Time utilities: [zeit](https://github.com/rockorager/zeit) + +**External (other):** +- Airport code -> location mapping: [Openflights](https://github.com/jpatokal/openflights) +- Ip address -> location mapping: [GeoLite2 City Database](https://github.com/maxmind/libmaxminddb) + +## Performance Targets + +**Latency:** +- Cache hit: <1ms +- Cache miss: <100ms +- P95: <150ms +- P99: <300ms + +**Throughput:** +- >10,000 req/s (cached) +- >1,000 req/s (uncached) + +**Memory:** +- Base: <50MB (currently 9MB) +- With 10,000 cache entries: <200MB +- Binary size: <5MB (currently <2MB in ReleaseSmall) diff --git a/CACHE_CONFIGURATION.md b/CACHE_CONFIGURATION.md deleted file mode 100644 index f6c7351..0000000 --- a/CACHE_CONFIGURATION.md +++ /dev/null @@ -1,140 +0,0 @@ -# Cache Configuration - -wttr.in uses three separate caches following Linux Filesystem Hierarchy Standard (FHS) and XDG Base Directory specifications. - -## External Services - -### Required Services -- **Met.no Weather API** - Primary weather data provider - - No API key required - - Free, open API from Norwegian Meteorological Institute - - Rate limit: Be respectful, use caching - -### Optional Services -- **IP2Location.io** - Fallback IP geolocation service - - API key required: Sign up at https://www.ip2location.io/ - - Used when MaxMind GeoIP database lookup fails - - Free tier: 30,000 requests/month - - Set via `IP2LOCATION_API_KEY` environment variable - -### Database Files -- **MaxMind GeoLite2 City** - IP geolocation database - - Free database, auto-downloaded if missing - - No API key required for database usage - - Updates available monthly from MaxMind - -## Cache Locations - -All caches default to `$XDG_CACHE_HOME/wttr` (typically `~/.cache/wttr`). - -### 1. Weather Response Cache -**Purpose:** Caches weather API responses to reduce upstream requests - -**Default Location:** `$XDG_CACHE_HOME/wttr/` (individual files) -**Environment Variable:** `WTTR_CACHE_DIR` -**Size:** 10,000 entries (configurable via `WTTR_CACHE_SIZE`) -**Expiration:** 1000-2000 seconds (16-33 minutes, randomized to avoid thundering herd) -**Eviction:** LRU (Least Recently Used) - -This is the main cache that stores weather forecast responses from Met.no. Each entry has a randomized TTL to prevent cache stampedes. - -### 2. Geocoding Cache (Optional) -**Purpose:** Caches location name → coordinates mappings - -**Default:** Disabled (in-memory only) -**Environment Variable:** `WTTR_GEOCACHE_FILE` -**Format:** JSON -**Expiration:** None (persists indefinitely) -**Eviction:** None (grows unbounded) - -When enabled, persists geocoding lookups to disk. Saves every 15 minutes if dirty. Useful for reducing external geocoding API calls. - -### 3. IP2Location Cache -**Purpose:** Caches IP → coordinates from IP2Location API - -**Default Location:** `$XDG_CACHE_HOME/wttr/ip2location.cache` -**Environment Variable:** `IP2LOCATION_CACHE_FILE` -**Format:** Binary (32-byte records) -**Expiration:** None (persists indefinitely) -**Eviction:** None (append-only, grows unbounded) - -Only used when `IP2LOCATION_API_KEY` is configured. Provides fallback when MaxMind GeoIP database lookup fails. - -## GeoIP Database Location - -**Default:** `$XDG_CACHE_HOME/wttr/GeoLite2-City.mmdb` -**Environment Variable:** `WTTR_GEOLITE_PATH` - -This is the MaxMind GeoLite2 database. It will be automatically downloaded if missing. - -## Environment Variables Summary - -| Variable | Default | Description | -|----------|---------|-------------| -| `WTTR_CACHE_DIR` | `$XDG_CACHE_HOME/wttr` | Weather response cache directory | -| `WTTR_CACHE_SIZE` | `10000` | Maximum number of cached weather responses | -| `WTTR_GEOCACHE_FILE` | (none) | Optional persistent geocoding cache file | -| `WTTR_GEOLITE_PATH` | `$XDG_CACHE_HOME/wttr/GeoLite2-City.mmdb` | MaxMind GeoLite2 database path | -| `IP2LOCATION_API_KEY` | (none) | API key for IP2Location fallback service | -| `IP2LOCATION_CACHE_FILE` | `$XDG_CACHE_HOME/wttr/ip2location.cache` | IP2Location cache file | -| `XDG_CACHE_HOME` | `~/.cache` | XDG base cache directory | - -## Examples - -### Minimal Configuration (defaults) -```bash -./wttr -# Uses ~/.cache/wttr/ for all caches and GeoIP database -``` - -### Custom Cache Location -```bash -WTTR_CACHE_DIR=/var/cache/wttr ./wttr -# All caches and GeoIP in /var/cache/wttr/ -``` - -### Enable Persistent Geocoding Cache -```bash -WTTR_GEOCACHE_FILE=~/.cache/wttr/geocache.json ./wttr -``` - -### With IP2Location Fallback -```bash -IP2LOCATION_API_KEY=your_key_here ./wttr -# Cache at ~/.cache/wttr/ip2location.cache -``` - -### Production Setup -```bash -WTTR_CACHE_DIR=/var/cache/wttr \ -WTTR_CACHE_SIZE=50000 \ -WTTR_GEOCACHE_FILE=/var/cache/wttr/geocache.json \ -IP2LOCATION_API_KEY=your_key_here \ -./wttr -# GeoIP and IP2Location cache also in /var/cache/wttr/ -``` - -## Cache Maintenance - -### Weather Cache -- **Automatic expiration:** Entries expire after 16-33 minutes (randomized) -- **LRU eviction:** When cache reaches max size (10,000 entries), least recently used entries are removed -- **Disk cleanup:** Expired files are cleaned up on access -- Safe to delete entire cache directory; will be recreated as needed - -### Geocoding Cache -- **No expiration:** Entries persist indefinitely -- **No eviction:** Cache grows unbounded -- **Auto-save:** Writes to disk every 15 minutes when modified -- Consider periodic cleanup if cache grows too large - -### IP2Location Cache -- **No expiration:** Entries persist indefinitely -- **Append-only:** File grows unbounded (32 bytes per unique IP) -- **No cleanup:** Consider periodic truncation for long-running deployments -- Safe to delete; will be recreated on next API lookup - -### GeoIP Database -- **Manual updates:** Download new database periodically for accuracy -- **Auto-download:** Database is automatically downloaded if missing on startup -- Typical update frequency: monthly (MaxMind releases) diff --git a/MISSING_FEATURES.md b/MISSING_FEATURES.md new file mode 100644 index 0000000..ba6dee9 --- /dev/null +++ b/MISSING_FEATURES.md @@ -0,0 +1,31 @@ +# Missing Features + +Features not yet implemented in the Zig version: + +## 1. Prometheus Metrics Format (format=p1) +- Export weather data in Prometheus metrics format +- See API_ENDPOINTS.md for format specification + +## 2. PNG Generation +- Render weather reports as PNG images +- Support transparency and custom styling +- Requires image rendering library integration + +## 3. Multiple Locations Support +- Handle colon-separated locations (e.g., `London:Paris:Berlin`) +- Process and display weather for multiple cities in one request + +## 4. Language/Localization +- Accept-Language header parsing +- lang query parameter support +- Translation of weather conditions and text (54 languages) + +## 5. Moon Phase Calculation +- Real moon phase computation based on date +- Moon phase emoji display +- Moonday calculation + +## 6. Astronomical Times +- Calculate dawn, sunrise, zenith, sunset, dusk times +- Based on location coordinates and timezone +- Display in custom format output diff --git a/README.md b/README.md index bb1a08c..851697f 100644 --- a/README.md +++ b/README.md @@ -1,200 +1,127 @@ -# wttr.in Zig Rewrite Documentation +# wttr -This directory contains comprehensive documentation for rewriting wttr.in in Zig. +Weather forecast service written in Zig, based on [wttr.in](https://wttr.in). ## Quick Start ```bash -# Minimal setup (uses defaults) -./wttr +# Build +zig build -# With IP2Location fallback (optional) -IP2LOCATION_API_KEY=your_key_here ./wttr +# Run with defaults +./zig-out/bin/wttr -# Custom cache location -WTTR_CACHE_DIR=/var/cache/wttr ./wttr +# With custom configuration +WTTR_LISTEN_PORT=8080 ./zig-out/bin/wttr ``` -See [CACHE_CONFIGURATION.md](CACHE_CONFIGURATION.md) for detailed configuration options. +## Usage -## External Services & API Keys +```bash +# Weather for your location (based on IP) +curl localhost:8002/ -### Required Services (No API Key) -- **Met.no Weather API** - Primary weather data provider - - Free, open API from Norwegian Meteorological Institute - - No registration required - - Rate limit: Be respectful, use caching (built-in) +# Weather for a specific city +curl localhost:8002/London -### Optional Services +# Weather for coordinates +curl localhost:8002/51.5074,-0.1278 + +# One-line format +curl localhost:8002/London?format=3 + +# JSON output +curl localhost:8002/London?format=j1 +``` + +## Configuration + +All configuration is via environment variables: + +| Variable | Default | Description | +|----------|---------|-------------| +| `WTTR_LISTEN_HOST` | `0.0.0.0` | Listen address | +| `WTTR_LISTEN_PORT` | `8002` | Listen port | +| `WTTR_CACHE_DIR` | `~/.cache/wttr` | Cache directory | +| `WTTR_CACHE_SIZE` | `10000` | Max cached responses | +| `WTTR_GEOLITE_PATH` | `~/.cache/wttr/GeoLite2-City.mmdb` | GeoIP database path | +| `IP2LOCATION_API_KEY` | (none) | Optional IP geolocation fallback | + +## External Services + +### Required (No API Key) +- **Met.no Weather API** - Weather data provider (free, no registration) + +### Optional - **IP2Location.io** - Fallback IP geolocation - - **API Key Required:** Sign up at https://www.ip2location.io/ + - Sign up at https://www.ip2location.io/ - Free tier: 30,000 requests/month - - Only used when MaxMind GeoIP database lookup fails - Set via `IP2LOCATION_API_KEY` environment variable -### Database Files (Auto-Downloaded) +### Auto-Downloaded - **MaxMind GeoLite2 City** - IP geolocation database - - Free database, automatically downloaded if missing - - No API key required - - Stored in `~/.cache/wttr/GeoLite2-City.mmdb` by default + - Downloaded automatically if missing + - Stored in `~/.cache/wttr/GeoLite2-City.mmdb` -## Current Implementation Status +## Output Formats -### Implemented Features -- HTTP server with routing and middleware -- Rate limiting -- Caching system -- GeoIP database integration (libmaxminddb) -- Location resolver with multiple input types -- Weather provider (Met.no) with timezone-aware forecasts; core data structures use zeit.Time/zeit.Date for type-safe date/time handling -- Output formats: ANSI, line (1-4), JSON (j1), v2 data-rich, custom (%) -- Query parameter parsing -- Static help pages (/:help, /:translation) -- Error handling (404/500 status codes) -- Configuration from environment variables -- **Imperial units auto-detection**: Automatically uses imperial units (°F, mph) for US IP addresses and `lang=us`, with explicit `?u` and `?m` overrides -- **IP2Location fallback**: Optional fallback geolocation service with persistent cache +- **ANSI** (default) - Colored terminal output +- **Line formats** (`?format=1-4`) - One-line weather +- **JSON** (`?format=j1`) - Machine-readable +- **v2** (`?format=v2`) - Data-rich format +- **Custom** (`?format=%l:+%c+%t`) - Custom format string -### Missing Features (To Be Implemented Later) +## Query Parameters -1. **Prometheus Metrics Format** (format=p1) - - Export weather data in Prometheus metrics format - - See API_ENDPOINTS.md for format specification +- `?u` - Imperial units (°F, mph) +- `?m` - Metric units (°C, km/h) +- `?format=N` - Output format +- `?lang=XX` - Language (auto-detected by default) -2. **PNG Generation** - - Render weather reports as PNG images - - Support transparency and custom styling - - Requires image rendering library integration +## Building -3. **Multiple Locations Support** - - Handle colon-separated locations (e.g., `London:Paris:Berlin`) - - Process and display weather for multiple cities in one request +```bash +# Debug build +zig build -4. **Language/Localization** - - Accept-Language header parsing - - lang query parameter support - - Translation of weather conditions and text (54 languages) +# Release build +zig build -Doptimize=ReleaseFast -5. **Moon Phase Calculation** - - Real moon phase computation based on date - - Moon phase emoji display - - Moonday calculation +# Run tests +zig build test +``` -6. **Astronomical Times** - - Calculate dawn, sunrise, zenith, sunset, dusk times - - Based on location coordinates and timezone - - Display in custom format output +## Docker -## Documentation Files +```bash +# Build image +docker build -t wttr -f docker/Dockerfile . -### [CACHE_CONFIGURATION.md](CACHE_CONFIGURATION.md) ⭐ NEW -Complete cache and external services documentation: -- External services (Met.no, IP2Location, MaxMind GeoLite2) -- API key requirements -- Cache locations and policies -- Expiration and eviction strategies -- Environment variables -- Configuration examples +# Run container +docker run -p 8002:8002 wttr +``` -### [TARGET_ARCHITECTURE.md](TARGET_ARCHITECTURE.md) -Target architecture for Zig rewrite: -- Single binary design -- Simplified caching (one layer) -- Pluggable weather provider interface -- Rate limiting middleware -- Module structure -- Testing strategy -- Performance targets +## Prometheus Integration -### [ARCHITECTURE.md](ARCHITECTURE.md) -Current system architecture documentation: -- Component breakdown (Go proxy, Python backend, static assets) -- Request flow diagrams -- API endpoints -- External dependencies -- Caching strategy -- Configuration -- Known issues +**Note:** Not yet implemented - see [MISSING_FEATURES.md](MISSING_FEATURES.md) -### [API_ENDPOINTS.md](API_ENDPOINTS.md) -Complete API reference: -- All endpoints with examples -- Query parameters -- Output formats -- Language support -- HTTP headers -- Rate limits -- Usage examples +```yaml +# prometheus.yml +scrape_configs: + - job_name: 'weather' + static_configs: + - targets: ['localhost:8002'] + metrics_path: '/London' + params: + format: ['p1'] +``` -### [DATA_FLOW.md](DATA_FLOW.md) -Detailed request processing flow: -- Step-by-step request handling -- Location resolution process -- Weather data fetching -- Rendering pipeline -- Caching mechanisms -- Error handling -- Performance optimizations +## Documentation -### [REWRITE_STRATEGY.md](REWRITE_STRATEGY.md) -Comprehensive rewrite plan: -- Goals and non-goals -- Phase-by-phase breakdown (10-11 weeks) -- Module organization -- Testing strategy -- Risk mitigation -- Success criteria -- Alternative approaches +- [ARCHITECTURE.md](ARCHITECTURE.md) - System architecture and design +- [API_ENDPOINTS.md](API_ENDPOINTS.md) - Complete API reference +- [MISSING_FEATURES.md](MISSING_FEATURES.md) - Features not yet implemented -## Quick Start +## License -1. **Read ARCHITECTURE.md** - Understand the current system -2. **Read DATA_FLOW.md** - Understand how requests are processed -3. **Read REWRITE_STRATEGY.md** - Understand the rewrite plan -4. **Refer to API_ENDPOINTS.md** - When implementing specific endpoints - -## Current System Summary - -**Architecture:** Hybrid Python/Go -- Go proxy (port 8082): LRU caching, prefetching -- Python backend (port 8002): Weather logic, rendering -- External binaries: wego (Go), pyphoon (Python) -- External services: Geolocator (port 8004) - -**Key Stats:** -- ~5000 lines of Python -- ~400 lines of Go -- 54 languages supported -- 12,800 entry LRU cache -- 1000-2000s cache TTL -- Zero unit tests (only integration tests) - -**Main Challenges:** -1. ANSI weather rendering (currently done by wego) -2. PNG image generation (PIL + pyte) -3. GeoIP database parsing (MaxMind format) -4. 54 language translations -5. Multiple output formats (ANSI, HTML, PNG, JSON, Prometheus) - -## Recommended Approach - -**Option A: Incremental (Recommended)** -1. Replace Go proxy with Zig (2 weeks) -2. Test and deploy -3. Replace Python backend with Zig (8-9 weeks) -4. Full cutover - -**Option B: Full Rewrite** -- All at once (10-11 weeks) -- Higher risk, but cleaner result - -## Next Steps - -1. Review documentation -2. Choose rewrite strategy -3. Set up Zig project structure -4. Begin implementation (start with HTTP server or Go proxy) - -## Questions? - -See REWRITE_STRATEGY.md "Questions to Answer" section for key decisions needed. +See [LICENSE](LICENSE) diff --git a/REWRITE_STRATEGY.md b/REWRITE_STRATEGY.md deleted file mode 100644 index 8f22041..0000000 --- a/REWRITE_STRATEGY.md +++ /dev/null @@ -1,714 +0,0 @@ -# wttr.in Zig Rewrite Strategy - -## Goals - -1. **Single binary** - Replace Python + Go with one Zig executable -2. **No external binaries** - Eliminate wego, pyphoon dependencies -3. **Maintain compatibility** - All existing API endpoints work identically -4. **Improve performance** - Faster response times, lower memory usage -5. **Add tests** - Comprehensive test coverage from day one -6. **Simplify deployment** - Single binary + data files - -## Non-Goals - -- Rewriting weather APIs (still use met.no/WWO) -- Changing API surface (maintain backward compatibility) -- Rewriting geolocator service (keep as separate service for now) - -## Phase 1: Analysis & Design ✓ - -**Status:** Complete (this document) - -**Deliverables:** -- [x] ARCHITECTURE.md - System overview -- [x] API_ENDPOINTS.md - API reference -- [x] DATA_FLOW.md - Request processing flow -- [x] REWRITE_STRATEGY.md - This document - -## Phase 2: Foundation (Week 1-2) - -### 2.1 Project Setup - -**Tasks:** -- [ ] Create `build.zig` with proper structure -- [ ] Set up module organization -- [ ] Configure dependencies (if any) -- [ ] Set up test framework -- [ ] Create CI/CD pipeline (GitHub Actions) - -**Modules:** -``` -src/ -├── main.zig # Entry point -├── server.zig # HTTP server -├── router.zig # Request routing -├── config.zig # Configuration -├── cache/ -│ ├── lru.zig # LRU cache implementation -│ └── file.zig # File-backed cache -├── location/ -│ ├── resolver.zig # Location resolution -│ ├── geoip.zig # GeoIP lookups -│ └── normalize.zig # Location normalization -├── weather/ -│ ├── client.zig # Weather API client -│ ├── metno.zig # met.no API -│ └── wwo.zig # WorldWeatherOnline API -├── render/ -│ ├── ansi.zig # ANSI rendering -│ ├── html.zig # HTML rendering -│ ├── json.zig # JSON rendering -│ ├── png.zig # PNG rendering -│ ├── line.zig # One-line format -│ └── v2.zig # v2 format -├── i18n/ -│ ├── translations.zig # Translation system -│ └── loader.zig # Load translation files -└── utils/ - ├── http.zig # HTTP utilities - ├── ip.zig # IP address handling - └── time.zig # Time utilities -``` - -### 2.2 Core HTTP Server - -**Tasks:** -- [ ] Implement HTTP server using std.http.Server -- [ ] Request parsing (headers, query params, path) -- [ ] Response building (status, headers, body) -- [ ] Basic routing (exact match, wildcard) -- [ ] Static file serving - -**Tests:** -- [ ] Parse GET requests -- [ ] Parse query parameters -- [ ] Route to handlers -- [ ] Serve static files -- [ ] Handle 404s - -**Acceptance Criteria:** -- Server listens on configurable port -- Handles concurrent requests -- Routes to placeholder handlers -- Serves files from share/ directory - -### 2.3 Configuration System - -**Tasks:** -- [ ] Load environment variables -- [ ] Load config files (if needed) -- [ ] Validate configuration -- [ ] Provide defaults - -**Configuration:** -```zig -const Config = struct { - listen_host: []const u8, - listen_port: u16, - geolite_path: []const u8, - cache_dir: []const u8, - log_level: LogLevel, - // API keys - ip2location_key: ?[]const u8, - ipinfo_token: ?[]const u8, - wwo_key: ?[]const u8, -}; -``` - -**Tests:** -- [ ] Load from environment -- [ ] Apply defaults -- [ ] Validate paths exist - -## Phase 3: Caching Layer (Week 2-3) - -### 3.1 LRU Cache - -**Tasks:** -- [ ] Implement generic LRU cache -- [ ] Thread-safe operations (Mutex) -- [ ] TTL support -- [ ] Eviction policy -- [ ] Cache statistics - -**Tests:** -- [ ] Insert and retrieve -- [ ] LRU eviction -- [ ] TTL expiration -- [ ] Concurrent access -- [ ] Memory limits - -**Acceptance Criteria:** -- Configurable size (default 12,800) -- O(1) get/put operations -- Thread-safe -- TTL-based expiration - -### 3.2 File Cache - -**Tasks:** -- [ ] Store large responses to disk -- [ ] MD5 hash for filenames -- [ ] Read/write with proper locking -- [ ] Cleanup old files - -**Tests:** -- [ ] Write and read files -- [ ] Handle binary data -- [ ] Concurrent access -- [ ] Disk space limits - -### 3.3 Cache Integration - -**Tasks:** -- [ ] Generate cache keys -- [ ] Check cache before processing -- [ ] Store responses after processing -- [ ] Handle InProgress flag (prevent thundering herd) - -**Tests:** -- [ ] Cache hit returns cached response -- [ ] Cache miss processes request -- [ ] Concurrent requests for same key -- [ ] Cache key generation - -## Phase 4: Location Resolution (Week 3-4) - -### 4.1 Location Parsing - -**Tasks:** -- [ ] Parse location from URL -- [ ] Normalize location names -- [ ] Handle special prefixes (~, @) -- [ ] Parse cyclic locations (:) -- [ ] Load aliases file -- [ ] Load blacklist file - -**Tests:** -- [ ] Parse city names -- [ ] Parse coordinates -- [ ] Parse IATA codes -- [ ] Parse special locations (Moon, etc.) -- [ ] Normalize names -- [ ] Apply aliases -- [ ] Check blacklist - -### 4.2 GeoIP Lookup - -**Tasks:** -- [ ] Read MaxMind GeoLite2 database -- [ ] IP to location lookup -- [ ] Cache results - -**Options:** -- Use C library (libmaxminddb) via @cImport -- Or: Parse MMDB format in pure Zig - -**Tests:** -- [ ] Lookup IPv4 address -- [ ] Lookup IPv6 address -- [ ] Handle not found -- [ ] Cache lookups - -### 4.3 External Geolocation - -**Tasks:** -- [ ] HTTP client for geolocator service -- [ ] Parse JSON responses -- [ ] Error handling - -**Tests:** -- [ ] Query geolocator -- [ ] Parse response -- [ ] Handle errors -- [ ] Timeout handling - -### 4.4 IP Location APIs - -**Tasks:** -- [ ] IP2Location API client -- [ ] IPInfo API client -- [ ] File-based caching -- [ ] Fallback chain - -**Tests:** -- [ ] Query each API -- [ ] Parse responses -- [ ] Cache results -- [ ] Fallback on error - -## Phase 5: Weather Data (Week 4-5) - -### 5.1 HTTP Client - -**Tasks:** -- [ ] Generic HTTP client -- [ ] Connection pooling -- [ ] Timeout handling -- [ ] Retry logic -- [ ] User-Agent setting - -**Tests:** -- [ ] GET requests -- [ ] POST requests -- [ ] Headers -- [ ] Timeouts -- [ ] Retries - -### 5.2 met.no Client - -**Tasks:** -- [ ] API endpoint construction -- [ ] Parse XML/JSON responses -- [ ] Transform to internal format -- [ ] Error handling - -**Tests:** -- [ ] Fetch weather data -- [ ] Parse response -- [ ] Handle API errors -- [ ] Handle rate limits - -### 5.3 WorldWeatherOnline Client - -**Tasks:** -- [ ] API endpoint construction -- [ ] Parse JSON responses -- [ ] Transform to internal format -- [ ] Caching (proxy cache) - -**Tests:** -- [ ] Fetch weather data -- [ ] Parse response -- [ ] Handle API errors -- [ ] Cache responses - -### 5.4 Weather Data Model - -**Tasks:** -- [ ] Define internal weather data structure -- [ ] Conversion from met.no format -- [ ] Conversion from WWO format -- [ ] JSON serialization - -**Tests:** -- [ ] Convert met.no data -- [ ] Convert WWO data -- [ ] Serialize to JSON -- [ ] Validate data - -## Phase 6: Rendering (Week 5-7) - -### 6.1 ANSI Renderer - -**Tasks:** -- [ ] Generate ANSI weather report -- [ ] ASCII art for weather conditions -- [ ] Color codes -- [ ] Box drawing characters -- [ ] Temperature graphs -- [ ] Wind direction arrows - -**Tests:** -- [ ] Render current weather -- [ ] Render forecast -- [ ] Apply colors -- [ ] Handle narrow mode -- [ ] Handle inverted colors - -**Acceptance Criteria:** -- Output matches wego format -- All weather conditions supported -- Configurable width - -### 6.2 One-Line Renderer - -**Tasks:** -- [ ] Parse format string -- [ ] Replace format codes (%c, %t, etc.) -- [ ] Handle predefined formats (1-4) -- [ ] Emoji support - -**Tests:** -- [ ] Format 1-4 -- [ ] Custom format strings -- [ ] All format codes -- [ ] Multiple locations - -### 6.3 JSON Renderer - -**Tasks:** -- [ ] Serialize weather data to JSON -- [ ] Match WWO API format -- [ ] Pretty printing - -**Tests:** -- [ ] Serialize current conditions -- [ ] Serialize forecast -- [ ] Match expected format - -### 6.4 HTML Renderer - -**Tasks:** -- [ ] Convert ANSI to HTML -- [ ] Apply CSS styling -- [ ] Add interactive buttons -- [ ] Template system - -**Tests:** -- [ ] Convert ANSI codes -- [ ] Apply colors -- [ ] Render buttons -- [ ] Template rendering - -### 6.5 PNG Renderer - -**Tasks:** -- [ ] Render ANSI to image -- [ ] Font rendering -- [ ] Color support -- [ ] Transparency - -**Options:** -- Use C library (libpng, freetype) via @cImport -- Or: Pure Zig implementation (more work) - -**Tests:** -- [ ] Render text -- [ ] Apply colors -- [ ] Apply transparency -- [ ] Handle fonts - -### 6.6 v2 Renderer - -**Tasks:** -- [ ] Data-rich format -- [ ] Temperature graphs -- [ ] Moon phases -- [ ] Astronomical times - -**Tests:** -- [ ] Render all sections -- [ ] Handle different locations -- [ ] Match expected format - -### 6.7 Prometheus Renderer - -**Tasks:** -- [ ] Convert weather data to metrics -- [ ] Prometheus text format -- [ ] Metric naming - -**Tests:** -- [ ] Render metrics -- [ ] Match Prometheus format -- [ ] All weather fields - -## Phase 7: Translation System (Week 7-8) - -### 7.1 Translation Loader - -**Tasks:** -- [ ] Load translation files -- [ ] Parse translation format -- [ ] Build translation tables -- [ ] Language detection - -**Tests:** -- [ ] Load all language files -- [ ] Parse translations -- [ ] Detect language from header -- [ ] Detect language from subdomain - -### 7.2 Message Translation - -**Tasks:** -- [ ] Translate weather conditions -- [ ] Translate UI messages -- [ ] Fallback to English - -**Tests:** -- [ ] Translate conditions -- [ ] Translate messages -- [ ] Handle missing translations -- [ ] Fallback logic - -## Phase 8: Request Processing (Week 8-9) - -### 8.1 Query Parser - -**Tasks:** -- [ ] Parse query parameters -- [ ] Parse single-letter options -- [ ] Parse PNG filenames -- [ ] Serialize/deserialize state - -**Tests:** -- [ ] Parse all options -- [ ] Parse PNG filenames -- [ ] Serialize state -- [ ] Deserialize state - -### 8.2 Request Handler - -**Tasks:** -- [ ] Main request handler -- [ ] Fast path (cache + static) -- [ ] Full path (location + weather + render) -- [ ] Error handling -- [ ] Rate limiting - -**Tests:** -- [ ] Handle cached requests -- [ ] Handle static pages -- [ ] Handle weather queries -- [ ] Handle errors -- [ ] Enforce rate limits - -### 8.3 Rate Limiting - -**Tasks:** -- [ ] Per-IP counters -- [ ] Time buckets (minute, hour, day) -- [ ] Whitelist support -- [ ] Return 429 on limit - -**Tests:** -- [ ] Count requests -- [ ] Enforce limits -- [ ] Reset counters -- [ ] Whitelist bypass - -## Phase 9: Integration & Testing (Week 9-10) - -### 9.1 End-to-End Tests - -**Tasks:** -- [ ] Test all API endpoints -- [ ] Test all output formats -- [ ] Test all query options -- [ ] Test error cases -- [ ] Compare with Python output - -**Test Cases:** -- [ ] Basic weather query -- [ ] Location resolution -- [ ] All output formats -- [ ] All languages -- [ ] PNG rendering -- [ ] Rate limiting -- [ ] Error handling - -### 9.2 Performance Testing - -**Tasks:** -- [ ] Benchmark request latency -- [ ] Benchmark throughput -- [ ] Memory profiling -- [ ] Cache hit rates -- [ ] Compare with Python/Go - -**Metrics:** -- Requests per second -- Average latency -- P95/P99 latency -- Memory usage -- Cache hit rate - -### 9.3 Compatibility Testing - -**Tasks:** -- [ ] Run integration test (test/query.sh) -- [ ] Compare SHA1 hashes -- [ ] Fix any differences -- [ ] Document intentional changes - -## Phase 10: Deployment (Week 10-11) - -### 10.1 Packaging - -**Tasks:** -- [ ] Build release binary -- [ ] Package data files -- [ ] Create Docker image -- [ ] Write deployment docs - -### 10.2 Migration Plan - -**Tasks:** -- [ ] Deploy Zig version alongside Python/Go -- [ ] Route small percentage of traffic -- [ ] Monitor errors and performance -- [ ] Gradually increase traffic -- [ ] Full cutover - -**Rollback Plan:** -- Keep Python/Go running -- Route traffic back if issues -- Fix issues in Zig version -- Retry migration - -### 10.3 Documentation - -**Tasks:** -- [ ] Update README -- [ ] Installation instructions -- [ ] Configuration guide -- [ ] API documentation -- [ ] Development guide - -## Risks & Mitigations - -### Risk: PNG Rendering Complexity - -**Mitigation:** -- Start with external library (libpng, freetype) -- Consider pure Zig later if needed -- Or: Keep PNG rendering in separate service - -### Risk: GeoIP Database Parsing - -**Mitigation:** -- Use C library (libmaxminddb) initially -- Consider pure Zig parser later -- Well-documented format - -### Risk: ANSI Rendering Differences - -**Mitigation:** -- Extensive testing against wego output -- Pixel-perfect comparison for PNG -- Accept minor differences if functionally equivalent - -### Risk: Performance Regression - -**Mitigation:** -- Benchmark early and often -- Profile hot paths -- Optimize critical sections -- Compare with Python/Go baseline - -### Risk: Translation File Parsing - -**Mitigation:** -- Simple format (key: value) -- Robust parser with error handling -- Validate all files on startup - -### Risk: Weather API Changes - -**Mitigation:** -- Abstract API clients -- Version API responses -- Monitor for changes -- Fallback to other API - -## Success Criteria - -### Functional - -- [ ] All API endpoints work -- [ ] All output formats match -- [ ] All languages supported -- [ ] All query options work -- [ ] Integration tests pass - -### Performance - -- [ ] Latency < Python/Go -- [ ] Throughput > Python/Go -- [ ] Memory < Python/Go -- [ ] Binary size < 10MB - -### Quality - -- [ ] >80% test coverage -- [ ] No memory leaks -- [ ] No crashes under load -- [ ] Clean error handling - -### Operational - -- [ ] Single binary deployment -- [ ] No external dependencies (except data files) -- [ ] Easy configuration -- [ ] Good logging -- [ ] Metrics/monitoring - -## Timeline - -**Total: 10-11 weeks** - -- Week 1-2: Foundation -- Week 2-3: Caching -- Week 3-4: Location -- Week 4-5: Weather -- Week 5-7: Rendering -- Week 7-8: Translation -- Week 8-9: Integration -- Week 9-10: Testing -- Week 10-11: Deployment - -**Milestones:** - -1. **Week 2:** HTTP server running, basic routing -2. **Week 4:** Location resolution working -3. **Week 6:** Weather data fetching working -4. **Week 8:** ANSI rendering working -5. **Week 10:** All features complete, testing done -6. **Week 11:** Deployed to production - -## Alternative: Incremental Approach - -Instead of full rewrite, replace components incrementally: - -### Option A: Replace Go Proxy First - -1. Rewrite Go proxy in Zig (Week 1-2) -2. Keep Python backend -3. Test and deploy -4. Then rewrite Python backend (Week 3-11) - -**Advantages:** -- Smaller initial scope -- Faster time to value -- De-risks Zig HTTP handling -- Can abandon if issues - -### Option B: Replace Python Backend First - -1. Keep Go proxy -2. Rewrite Python backend in Zig (Week 1-10) -3. Test and deploy -4. Then replace Go proxy (Week 11) - -**Advantages:** -- Most complexity in backend -- Proves out rendering logic -- Can keep Go proxy if it works well - -### Recommendation: Option A - -Start with Go proxy replacement: -- Smaller scope (400 lines vs 5000+) -- Clear interface boundary -- Tests caching/HTTP in Zig -- Quick win (2 weeks) -- De-risks larger rewrite - -## Next Steps - -1. **Review this document** - Get feedback on approach -2. **Choose strategy** - Full rewrite vs incremental -3. **Set up project** - Create build.zig, directory structure -4. **Start coding** - Begin with HTTP server or Go proxy replacement -5. **Iterate** - Build, test, refine - -## Questions to Answer - -- [ ] Use C libraries (libpng, freetype, libmaxminddb) or pure Zig? -- [ ] Keep geolocator as separate service or integrate? -- [ ] Keep wego/pyphoon or rewrite rendering? -- [ ] Full rewrite or incremental replacement? -- [ ] Target Zig version (0.11, 0.12, 0.13)? -- [ ] Async I/O strategy (std.event, manual, blocking)? diff --git a/TARGET_ARCHITECTURE.md b/TARGET_ARCHITECTURE.md deleted file mode 100644 index 4133e28..0000000 --- a/TARGET_ARCHITECTURE.md +++ /dev/null @@ -1,967 +0,0 @@ -# wttr.in Target Architecture (Zig) - -## Overview - -Single Zig binary (`wttr`) with simplified architecture: -- One HTTP server (karlseguin/http.zig) -- One caching layer (LRU + file-backed) - Note: There are multiple caches for IP->location, geocoding, and weather, - but these are in a single layer...this avoids multiple file-based - cache layers -- Pluggable weather provider interface -- Rate limiting as injectable middleware -- English-only (i18n can be added later) -- No cron-based prefetching (if you want that, just hit the endpoint on a schedule) - -## System Diagram - -``` -Client Request - ↓ -HTTP Server (http.zig) - ↓ -Rate Limiter (middleware) - ↓ -Router - ↓ -Request Handler - ↓ -Location Resolver - ↓ -Provider interface cache check - ↓ (miss) -Weather Provider (interface) - ├─ MetNo (default) - └─ Mock (tests) - ↓ -Cache Store - ↓ -Renderer - ├─ ANSI - ├─ JSON - ├─ Line - └─ HTML - └─ (PNG later) - ↓ -Response -``` - -## Module Structure - -``` -wttr/ -├── build.zig -├── build.zig.zon -└── src/ - ├── main.zig # Entry point, server setup - ├── config.zig # Configuration - ├── http/ - │ ├── server.zig # HTTP server wrapper - │ ├── router.zig # Route matching - │ ├── handler.zig # Request handlers - │ └── middleware.zig # Rate limiter (Token Bucket) - ├── cache/ - │ ├── cache.zig # Cache interface - │ ├── lru.zig # In-memory LRU - │ └── file.zig # File-backed storage - ├── location/ - │ ├── resolver.zig # Main location resolution - │ ├── geoip.zig # GeoIP wrapper (libmaxminddb) - │ ├── normalize.zig # Location normalization - │ └── geolocator.zig # HTTP client for geolocator service - ├── weather/ - │ ├── provider.zig # Weather provider interface - │ ├── metno.zig # met.no implementation - │ ├── mock.zig # Mock for tests - │ └── types.zig # Weather data structures - ├── render/ - │ ├── ansi.zig # ANSI terminal output - │ ├── json.zig # JSON output - │ └── line.zig # One-line format - └── util/ - ├── ip.zig # IP address utilities - └── hash.zig # Hashing utilities -``` - -## Core Components - -### 1. HTTP Server (main.zig, http/) - -**Responsibilities:** -- Listen on configured port -- Parse HTTP requests -- Route to handlers -- Apply middleware (rate limiting) -- Return responses - -**Dependencies:** -- karlseguin/http.zig - -**Configuration:** -```zig -const Config = struct { - listen_host: []const u8 = "0.0.0.0", - listen_port: u16 = 8002, - cache_size: usize = 10_000, - cache_dir: []const u8 = "/tmp/wttr-cache", - geolite_path: []const u8 = "./GeoLite2-City.mmdb", - geolocator_url: []const u8 = "http://localhost:8004", -}; -``` - -**Routes:** -```zig -GET / → weather for IP location -GET /{location} → weather for location -GET /:help → help page -GET /files/{path} → static files -``` - -### 2. Rate Limiter (http/middleware.zig) - -**Algorithm:** Token Bucket - -**Responsibilities:** -- Track token buckets per IP -- Refill tokens at configured rate -- Consume tokens on request -- Return 429 when bucket empty -- Injectable as middleware - -**Interface:** -```zig -pub const RateLimiter = struct { - buckets: HashMap([]const u8, TokenBucket), - config: RateLimitConfig, - allocator: Allocator, - - pub fn init(allocator: Allocator, config: RateLimitConfig) !RateLimiter; - pub fn check(self: *RateLimiter, ip: []const u8) !void; - pub fn middleware(self: *RateLimiter) Middleware; - pub fn deinit(self: *RateLimiter) void; -}; - -pub const RateLimitConfig = struct { - capacity: u32 = 300, // Max tokens in bucket - refill_rate: u32 = 5, // Tokens per second - refill_interval_ms: u64 = 200, // Refill every 200ms (5 tokens/sec) -}; - -const TokenBucket = struct { - tokens: f64, - capacity: u32, - last_refill: i64, // Unix timestamp (ms) - - fn refill(self: *TokenBucket, now: i64, rate: u32, interval_ms: u64) void { - const elapsed = now - self.last_refill; - const intervals = @as(f64, @floatFromInt(elapsed)) / @as(f64, @floatFromInt(interval_ms)); - const new_tokens = intervals * @as(f64, @floatFromInt(rate)); - - self.tokens = @min( - self.tokens + new_tokens, - @as(f64, @floatFromInt(self.capacity)) - ); - self.last_refill = now; - } - - fn consume(self: *TokenBucket, count: f64) bool { - if (self.tokens >= count) { - self.tokens -= count; - return true; - } - return false; - } -}; -``` - -**Implementation:** -- HashMap of IP → TokenBucket -- Each request consumes 1 token -- Tokens refill at configured rate (default: 5/second) -- Bucket capacity: 300 tokens (allows bursts) -- Periodic cleanup of old buckets (not accessed in 1 hour) - -**Example Configuration:** -```zig -// Allow 300 requests burst, then 5 req/sec sustained -const config = RateLimitConfig{ - .capacity = 300, - .refill_rate = 5, - .refill_interval_ms = 200, -}; -``` - -### 3. Cache (cache/) - -**Single-layer cache with two storage backends:** - -**Interface:** -```zig -pub const Cache = struct { - lru: LRU, - file_store: FileStore, - - pub fn init(allocator: Allocator, config: CacheConfig) !Cache; - pub fn get(self: *Cache, key: []const u8) ?[]const u8; - pub fn put(self: *Cache, key: []const u8, value: []const u8, ttl: u64) !void; -}; - -pub const CacheConfig = struct { - max_entries: usize = 10_000, - file_threshold: usize = 1024, // Store to file if > 1KB - cache_dir: []const u8, -}; -``` - -**Cache Key:** -``` -{user_agent}:{path}:{query}:{client_ip} -``` - -**Storage Strategy:** -- Small responses (<1KB): In-memory LRU only -- Large responses (≥1KB): LRU stores file path, data on disk -- TTL: 1000-2000s (randomized to prevent thundering herd) - -**LRU Implementation:** -```zig -pub const LRU = struct { - map: HashMap([]const u8, Entry), - list: DoublyLinkedList([]const u8), - max_entries: usize, - - const Entry = struct { - value: []const u8, - expires: i64, - node: *Node, - }; - - pub fn get(self: *LRU, key: []const u8) ?[]const u8; - pub fn put(self: *LRU, key: []const u8, value: []const u8, expires: i64) !void; -}; -``` - -### 4. Location Resolver (location/) - -**Responsibilities:** -- Parse location from URL -- Normalize location names -- Resolve IP to location (GeoIP) -- Resolve names to coordinates (geolocator) -- Handle special prefixes (~, @) - -**Interface:** -```zig -pub const LocationResolver = struct { - geoip: GeoIP, - geolocator_url: []const u8, - allocator: Allocator, - - pub fn init(allocator: Allocator, config: LocationConfig) !LocationResolver; - pub fn resolve(self: *LocationResolver, location: ?[]const u8, client_ip: []const u8) !Location; -}; - -pub const Location = struct { - name: []const u8, // "London" or "51.5074,-0.1278" - display_name: ?[]const u8, // Override for display - country: ?[]const u8, // "GB" - source_ip: []const u8, // Client IP -}; -``` - -**Resolution Logic:** -```zig -fn resolve(location: ?[]const u8, client_ip: []const u8) !Location { - // 1. Empty/null → Use client IP - if (location == null) return resolveIP(client_ip); - - // 2. IP address → Resolve to location - if (isIP(location)) return resolveIP(location); - - // 3. @domain → Resolve domain to IP, then location - if (location[0] == '@') return resolveDomain(location[1..]); - - // 4. ~search → Use geolocator - if (location[0] == '~') return geolocate(location[1..]); - - // 5. Name → Normalize and use - return Location{ - .name = normalize(location), - .display_name = null, - .country = null, - .source_ip = client_ip, - }; -} -``` - -**GeoIP:** -- Wraps libmaxminddb C library via @cImport -- Built from source using Zig build system (not system library) -- Exposes clean Zig interface -- Hides C implementation details - -**Build Integration:** -The libmaxminddb library will be built as part of the Zig build process: -- Source code vendored in `vendor/libmaxminddb/` -- Compiled using `b.addStaticLibrary()` or `b.addObject()` -- Linked into the main binary -- No system library dependency required - -```zig -// location/geoip.zig -const c = @cImport({ - @cInclude("maxminddb.h"); -}); - -pub const GeoIP = struct { - mmdb: c.MMDB_s, - allocator: Allocator, - - pub fn init(allocator: Allocator, db_path: []const u8) !GeoIP { - var self = GeoIP{ - .mmdb = undefined, - .allocator = allocator, - }; - - const path_z = try allocator.dupeZ(u8, db_path); - defer allocator.free(path_z); - - const status = c.MMDB_open(path_z.ptr, c.MMDB_MODE_MMAP, &self.mmdb); - if (status != c.MMDB_SUCCESS) { - return error.GeoIPOpenFailed; - } - - return self; - } - - pub fn lookup(self: *GeoIP, ip: []const u8) !?GeoIPResult { - const ip_z = try self.allocator.dupeZ(u8, ip); - defer self.allocator.free(ip_z); - - var gai_error: c_int = 0; - var mmdb_error: c_int = 0; - - const result = c.MMDB_lookup_string( - &self.mmdb, - ip_z.ptr, - &gai_error, - &mmdb_error - ); - - if (gai_error != 0 or mmdb_error != c.MMDB_SUCCESS) { - return null; - } - - if (!result.found_entry) { - return null; - } - - // Extract city and country - var entry_data: c.MMDB_entry_data_s = undefined; - - const city = blk: { - const status = c.MMDB_get_value( - &result.entry, - &entry_data, - "city", "names", "en", null - ); - if (status == c.MMDB_SUCCESS and entry_data.has_data) { - const str = entry_data.utf8_string[0..entry_data.data_size]; - break :blk try self.allocator.dupe(u8, str); - } - break :blk null; - }; - - const country = blk: { - const status = c.MMDB_get_value( - &result.entry, - &entry_data, - "country", "names", "en", null - ); - if (status == c.MMDB_SUCCESS and entry_data.has_data) { - const str = entry_data.utf8_string[0..entry_data.data_size]; - break :blk try self.allocator.dupe(u8, str); - } - break :blk null; - }; - - return GeoIPResult{ - .city = city, - .country = country, - }; - } - - pub fn deinit(self: *GeoIP) void { - c.MMDB_close(&self.mmdb); - } -}; - -pub const GeoIPResult = struct { - city: ?[]const u8, - country: ?[]const u8, - - pub fn deinit(self: GeoIPResult, allocator: Allocator) void { - if (self.city) |city| allocator.free(city); - if (self.country) |country| allocator.free(country); - } -}; -``` - -### 5. Weather Provider (weather/) - -**Interface (pluggable):** -```zig -pub const WeatherProvider = struct { - ptr: *anyopaque, - vtable: *const VTable, - - pub const VTable = struct { - fetch: *const fn (ptr: *anyopaque, location: []const u8) anyerror!WeatherData, - deinit: *const fn (ptr: *anyopaque) void, - }; - - pub fn fetch(self: WeatherProvider, location: []const u8) !WeatherData { - return self.vtable.fetch(self.ptr, location); - } -}; - -pub const WeatherData = struct { - location: []const u8, - current: CurrentCondition, - forecast: []ForecastDay, - - pub const CurrentCondition = struct { - temp_c: f32, - temp_f: f32, - condition: []const u8, - weather_code: u16, - humidity: u8, - wind_kph: f32, - wind_dir: []const u8, - pressure_mb: f32, - precip_mm: f32, - }; - - pub const ForecastDay = struct { - date: []const u8, - max_temp_c: f32, - min_temp_c: f32, - condition: []const u8, - weather_code: u16, - hourly: []HourlyForecast, - }; - - pub const HourlyForecast = struct { - time: []const u8, - temp_c: f32, - condition: []const u8, - weather_code: u16, - wind_kph: f32, - precip_mm: f32, - }; -}; -``` - -**MetNo Implementation:** -```zig -pub const MetNo = struct { - allocator: Allocator, - http_client: HttpClient, - - pub fn init(allocator: Allocator) !MetNo; - pub fn provider(self: *MetNo) WeatherProvider; - pub fn fetch(self: *MetNo, location: []const u8) !WeatherData; -}; -``` - -**Mock Implementation (for tests):** -```zig -pub const MockWeather = struct { - allocator: Allocator, - responses: HashMap([]const u8, WeatherData), - - pub fn init(allocator: Allocator) !MockWeather; - pub fn provider(self: *MockWeather) WeatherProvider; - pub fn addResponse(self: *MockWeather, location: []const u8, data: WeatherData) !void; - pub fn fetch(self: *MockWeather, location: []const u8) !WeatherData; -}; -``` - -### Provider vs Renderer Responsibilities - -**Weather Provider responsibilities:** -- Fetch raw weather data from external APIs -- Parse API responses into structured types -- **Perform timezone conversions once at ingestion time** -- Group forecast data by local date (not UTC date) -- Store both UTC time and local time in forecast data -- Return data in a timezone-agnostic format ready for rendering - -**Renderer responsibilities:** -- Format weather data for display (ANSI, plain text, JSON, etc.) -- Select appropriate hourly forecasts for display (morning/noon/evening/night) -- Apply unit conversions (metric/imperial) based on user preferences -- Handle partial days with missing data (render empty slots) -- Format dates and times for human readability -- **Should NOT perform timezone calculations** - use pre-calculated local times from provider - -**Key principle:** Timezone conversions are expensive and error-prone. They should happen once at the provider level, not repeatedly at the renderer level. This separation ensures consistent behavior across all output formats and simplifies the rendering logic. - -**Implementation details:** -- Core data structures use `zeit.Time` and `zeit.Date` types instead of strings for type safety -- `HourlyForecast` contains both `time: zeit.Time` (UTC) and `local_time: zeit.Time` (pre-calculated) -- `ForecastDay.date` is `zeit.Date` (not string), eliminating parsing/formatting overhead -- The `MetNo` provider uses a pre-computed timezone offset lookup table (360 entries covering global coordinates) -- Date formatting uses `zeit.Time.gofmt()` with Go-style format strings (e.g., "Mon _2 Jan") -- Timezone offset table provides ±1-2.5 hour accuracy at extreme latitudes, sufficient for forecast grouping - -**Benefits of zeit integration:** -- Type safety prevents format string errors and invalid date/time operations -- Explicit timezone handling - no implicit UTC assumptions -- Eliminates redundant string parsing and formatting -- Enables proper date arithmetic (e.g., `instant.add(duration)`, `instant.subtract(duration)`) -- Consistent date/time representation across all modules - -### 6. Renderers (render/) - -**ANSI Renderer:** -```zig -pub const AnsiRenderer = struct { - pub fn render(allocator: Allocator, data: WeatherData, options: RenderOptions) ![]const u8; -}; - -pub const RenderOptions = struct { - narrow: bool = false, - days: u8 = 3, - use_imperial: bool = false, - no_caption: bool = false, -}; -``` - -**Output Format:** -``` -Weather for: London, GB - - \ / Clear - .-. 10 – 11 °C - ― ( ) ― ↑ 11 km/h - `-' 10 km - / \ 0.0 mm -``` - -**JSON Renderer:** -```zig -pub const JsonRenderer = struct { - pub fn render(allocator: Allocator, data: WeatherData) ![]const u8; -}; -``` - -**One-Line Renderer:** -```zig -pub const LineRenderer = struct { - pub fn render(allocator: Allocator, data: WeatherData, format: []const u8) ![]const u8; -}; - -// Formats: -// 1: "London: ⛅️ +7°C" -// 2: "London: ⛅️ +7°C 🌬️↗11km/h" -// 3: "London: ⛅️ +7°C 🌬️↗11km/h 💧65%" -// Custom: "%l: %c %t" → "London: ⛅️ +7°C" -``` - -### 7. Request Handler (http/handler.zig) - -**Main handler flow:** -```zig -pub const HandleWeatherOptions = struct { - cache: *Cache, - resolver: *LocationResolver, - provider: WeatherProvider, -}; - -pub fn handleWeather( - req: *http.Request, - res: *http.Response, - options: HandleWeatherOptions, -) !void { - // 1. Parse request - const location = parseLocation(req.url.path); - const render_opts = parseOptions(req.url.query); - const client_ip = getClientIP(req); - - // 2. Generate cache key - const cache_key = try generateCacheKey(req, client_ip); - - // 3. Check cache - if (options.cache.get(cache_key)) |cached| { - return res.write(cached); - } - - // 4. Resolve location - const resolved = try options.resolver.resolve(location, client_ip); - - // 5. Fetch weather - const weather = try options.provider.fetch(resolved.name); - - // 6. Render - const output = try renderWeather(weather, render_opts); - - // 7. Cache - const ttl = 1000 + std.crypto.random.intRangeAtMost(u64, 0, 1000); - try options.cache.put(cache_key, output, ttl); - - // 8. Return - return res.write(output); -} -``` - -## Configuration - -**Environment Variables:** -```bash -WTTR_LISTEN_HOST=0.0.0.0 -WTTR_LISTEN_PORT=8002 -WTTR_CACHE_SIZE=10000 -WTTR_CACHE_DIR=/tmp/wttr-cache -WTTR_GEOLITE_PATH=./GeoLite2-City.mmdb -WTTR_GEOLOCATOR_URL=http://localhost:8004 -``` - -**Config File (optional):** -```zig -// config.zig -pub const Config = struct { - listen_host: []const u8, - listen_port: u16, - cache_size: usize, - cache_dir: []const u8, - geolite_path: []const u8, - geolocator_url: []const u8, - - pub fn load(allocator: Allocator) !Config { - return Config{ - .listen_host = std.os.getenv("WTTR_LISTEN_HOST") orelse "0.0.0.0", - .listen_port = parsePort(std.os.getenv("WTTR_LISTEN_PORT")) orelse 8002, - .cache_size = parseSize(std.os.getenv("WTTR_CACHE_SIZE")) orelse 10_000, - .cache_dir = std.os.getenv("WTTR_CACHE_DIR") orelse "/tmp/wttr-cache", - .geolite_path = std.os.getenv("WTTR_GEOLITE_PATH") orelse "./GeoLite2-City.mmdb", - .geolocator_url = std.os.getenv("WTTR_GEOLOCATOR_URL") orelse "http://localhost:8004", - }; - } -}; -``` - -## Dependencies - -**External (Zig packages):** -- karlseguin/http.zig - HTTP server -- rockorager/zeit - Time utilities - -**Vendored (built from source):** -- libmaxminddb - GeoIP lookups (C library, built with Zig build system) - -**Standard Library:** -- std.http.Client - HTTP client for weather APIs -- std.json - JSON parsing/serialization -- std.HashMap - Cache and rate limiter storage -- std.crypto.random - Random TTL generation - -## Build Configuration - -**build.zig:** -```zig -const std = @import("std"); - -pub fn build(b: *std.Build) void { - const target = b.standardTargetOptions(.{}); - const optimize = b.standardOptimizeOption(.{}); - - // Build libmaxminddb from source - const maxminddb = b.addStaticLibrary(.{ - .name = "maxminddb", - .target = target, - .optimize = optimize, - }); - maxminddb.linkLibC(); - maxminddb.addIncludePath(.{ .path = "vendor/libmaxminddb/include" }); - maxminddb.addCSourceFiles(.{ - .files = &.{ - "vendor/libmaxminddb/src/maxminddb.c", - "vendor/libmaxminddb/src/data-pool.c", - }, - .flags = &.{"-std=c99"}, - }); - - const exe = b.addExecutable(.{ - .name = "wttr", - .root_source_file = .{ .path = "src/main.zig" }, - .target = target, - .optimize = optimize, - }); - - // Add http.zig dependency - const http = b.dependency("http", .{ - .target = target, - .optimize = optimize, - }); - exe.root_module.addImport("http", http.module("http")); - - // Add zeit dependency - const zeit = b.dependency("zeit", .{ - .target = target, - .optimize = optimize, - }); - exe.root_module.addImport("zeit", zeit.module("zeit")); - - // Link libmaxminddb (built from source) - exe.linkLibrary(maxminddb); - exe.addIncludePath(.{ .path = "vendor/libmaxminddb/include" }); - exe.linkLibC(); - - b.installArtifact(exe); - - // Tests - const tests = b.addTest(.{ - .root_source_file = .{ .path = "src/main.zig" }, - .target = target, - .optimize = optimize, - }); - tests.root_module.addImport("http", http.module("http")); - tests.root_module.addImport("zeit", zeit.module("zeit")); - tests.linkLibrary(maxminddb); - tests.addIncludePath(.{ .path = "vendor/libmaxminddb/include" }); - tests.linkLibC(); - - const run_tests = b.addRunArtifact(tests); - const test_step = b.step("test", "Run tests"); - test_step.dependOn(&run_tests.step); -} -``` - -**Project Structure:** -``` -wttr/ -├── build.zig -├── build.zig.zon -├── vendor/ -│ └── libmaxminddb/ # Vendored libmaxminddb source -│ ├── include/ -│ │ └── maxminddb.h -│ └── src/ -│ ├── maxminddb.c -│ └── data-pool.c -└── src/ - └── ... -``` - -**build.zig.zon:** -```zig -.{ - .name = "wttr", - .version = "0.1.0", - .dependencies = .{ - .http = .{ - .url = "https://github.com/karlseguin/http.zig/archive/refs/tags/v0.1.0.tar.gz", - .hash = "...", - }, - .zeit = .{ - .url = "https://github.com/rockorager/zeit/archive/refs/tags/v0.1.0.tar.gz", - .hash = "...", - }, - }, - .paths = .{ - "build.zig", - "build.zig.zon", - "src", - }, -} -``` - -## Testing Strategy - -**Unit Tests:** -```zig -// cache/lru.zig -test "LRU basic operations" { - var lru = try LRU.init(testing.allocator, 3); - defer lru.deinit(); - - try lru.put("key1", "value1", 9999999999); - try testing.expectEqualStrings("value1", lru.get("key1").?); -} - -test "LRU eviction" { - var lru = try LRU.init(testing.allocator, 2); - defer lru.deinit(); - - try lru.put("key1", "value1", 9999999999); - try lru.put("key2", "value2", 9999999999); - try lru.put("key3", "value3", 9999999999); - - try testing.expect(lru.get("key1") == null); // Evicted - try testing.expectEqualStrings("value2", lru.get("key2").?); -} - -// weather/mock.zig -test "mock weather provider" { - var mock = try MockWeather.init(testing.allocator); - defer mock.deinit(); - - const data = WeatherData{ /* ... */ }; - try mock.addResponse("London", data); - - const provider = mock.provider(); - const result = try provider.fetch("London"); - try testing.expectEqual(data.current.temp_c, result.current.temp_c); -} - -// http/middleware.zig -test "rate limiter enforces limits" { - const zeit = @import("zeit"); - - var limiter = try RateLimiter.init(testing.allocator, .{ - .capacity = 10, - .refill_rate = 1, - .refill_interval_ms = 1000, - }); - defer limiter.deinit(); - - // Consume all tokens - var i: usize = 0; - while (i < 10) : (i += 1) { - try limiter.check("1.2.3.4"); - } - - // Next request should fail - try testing.expectError(error.RateLimitExceeded, limiter.check("1.2.3.4")); - - // Wait for refill (1 second = 1 token) - std.time.sleep(1_100_000_000); // 1.1 seconds - - // Should succeed now - try limiter.check("1.2.3.4"); -} - -test "rate limiter allows bursts" { - var limiter = try RateLimiter.init(testing.allocator, .{ - .capacity = 100, - .refill_rate = 5, - .refill_interval_ms = 200, - }); - defer limiter.deinit(); - - // Should allow 100 requests immediately (burst) - var i: usize = 0; - while (i < 100) : (i += 1) { - try limiter.check("1.2.3.4"); - } - - // 101st should fail - try testing.expectError(error.RateLimitExceeded, limiter.check("1.2.3.4")); -} -``` - -**Integration Tests:** -```zig -test "end-to-end weather request" { - var mock = try MockWeather.init(testing.allocator); - defer mock.deinit(); - - // Setup mock data - try mock.addResponse("London", test_weather_data); - - // Create server components - var cache = try Cache.init(testing.allocator, .{}); - defer cache.deinit(); - - var resolver = try LocationResolver.init(testing.allocator, .{}); - defer resolver.deinit(); - - // Simulate request - const output = try handleWeatherRequest( - "London", - &cache, - &resolver, - mock.provider(), - ); - defer testing.allocator.free(output); - - try testing.expect(std.mem.indexOf(u8, output, "London") != null); - try testing.expect(std.mem.indexOf(u8, output, "°C") != null); -} -``` - -## Deployment - -**Single Binary:** -```bash -# Build -zig build -Doptimize=ReleaseFast - -# Run -./zig-out/bin/wttr - -# With config -WTTR_LISTEN_PORT=8080 ./zig-out/bin/wttr -``` - -**Docker:** - -See dockerfile in ./docker. This is a FROM SCRATCH image based on a musl static binary - -## Performance Targets - -**Latency:** -- Cache hit: <1ms -- Cache miss: <100ms (depends on weather API) -- P95: <150ms -- P99: <300ms - -**Throughput:** -- >10,000 req/s (cached) -- >1,000 req/s (uncached) - -**Memory:** -- Base: <50MB (currently 9MB) -- With 10,000 cache entries: <200MB -- Binary size: <5MB (ReleaseSmall currently less than 2MB) - -## Future Enhancements (Not in Initial Version) - -1. **i18n Support** - Add translation system -2. **PNG Rendering** - Add image output -4. **v2 Format** - Data-rich experimental format -5. **Prometheus Format** - Metrics output -6. **Moon Phases** - Special moon queries -7. **Multiple Weather Providers** - WWO, OpenWeather, etc. -8. **Metrics/Observability** - Prometheus metrics, structured logging -9. **Admin API** - Cache stats, health checks - -## Success Criteria - -**Functional:** -- [x] All core endpoints work (/, /{location}, /:help) -- [x] ANSI output matches current system -- [x] JSON output matches current system -- [x] One-line formats work -- [x] Location resolution works (IP, name, coordinates) -- [x] Rate limiting works -- [x] Caching works - -**Performance:** -- [x] Latency ≤ current system -- [x] Throughput ≥ current system -- [x] Memory ≤ current system -- [x] Binary size <10MB (under ReleaseSafe) - -**Quality:** -- [ ] >80% test coverage -- [ ] No memory leaks (valgrind clean) -- [ ] No crashes under load -- [x] Clean error handling - -**Operational:** -- [x] Single binary deployment -- [x] Simple configuration -- [x] Good logging -- [x] Easy to debug