Compare commits

..

No commits in common. "35c9831ba86f0240e80df38f53f655062183ed7b" and "1e6a5e28ca3e8ca11426264c80adc748225819fc" have entirely different histories.

21 changed files with 3899 additions and 897 deletions

574
API_ENDPOINTS.md Normal file
View file

@ -0,0 +1,574 @@
# wttr.in API Endpoints Reference
## Base URL
- Production: `https://wttr.in`
- Local: `http://localhost:8082` (Go proxy) → `http://localhost:8002` (Python backend)
## Weather Query Endpoints
### GET /
Get weather for IP-based location.
**Response:** ANSI weather report (or HTML if browser User-Agent)
**Example:**
```bash
curl wttr.in
```
### GET /{location}
Get weather for specific location.
**Path Parameters:**
- `location` - City name, coordinates, airport code, or special location
**Location Formats:**
- City name: `London`, `New York`, `Salt+Lake+City`
- Coordinates: `55.7558,37.6173`
- Airport (IATA): `muc`, `ham`
- Search term: `~Eiffel+Tower`, `~Kilimanjaro`
- Domain: `@github.com`, `@msu.ru`
- IP address: `8.8.8.8`
- Auto-detect: `MyLocation` or empty
- Moon: `Moon`, `Moon@2016-12-25`
- Cyclic: `London:Paris:Berlin` (rotates based on time)
**Special Prefixes:**
- `~` - Search for location (uses geolocator)
- `@` - Resolve domain to IP, then get location
- No prefix - Exact location name
**Query Parameters:** See "Query Parameters" section below
**Examples:**
```bash
curl wttr.in/London
curl wttr.in/~Eiffel+Tower
curl wttr.in/@github.com
curl wttr.in/Moon
curl wttr.in/London:Paris:Berlin?period=60
```
### GET /{location}.png
Get weather as PNG image.
**Path Parameters:**
- `location` - Same as above, but with `.png` extension
**PNG Filename Format:**
```
{location}_{options}.png
```
**Options in filename:**
- `{width}x` - Width in pixels (e.g., `200x`)
- `x{height}` - Height in pixels (e.g., `x300`)
- `{width}x{height}` - Both dimensions (e.g., `200x300`)
- `lang={code}` - Language code (e.g., `lang=ru`)
- Single-letter options: `t`, `n`, `q`, etc. (see below)
- Separate multiple options with `_`
**Examples:**
```bash
curl wttr.in/Paris.png > paris.png
curl wttr.in/London_200x_t_lang=fr.png > london.png
curl wttr.in/Berlin_0tqp_lang=de.png > berlin.png
```
**Note:** Use `_` instead of `?` and `&` for PNG URLs
## Special Pages
### GET /:help
Display help page with usage instructions.
**Query Parameters:**
- `lang={code}` - Language for help page
**Example:**
```bash
curl wttr.in/:help
curl wttr.in/:help?lang=de
```
### GET /:bash.function
Get bash function for shell integration.
**Example:**
```bash
curl wttr.in/:bash.function >> ~/.bashrc
```
### GET /:translation
Display translation information and supported languages.
**Example:**
```bash
curl wttr.in/:translation
```
### GET /:iterm2
Get iTerm2 integration instructions.
**Example:**
```bash
curl wttr.in/:iterm2
```
## Static Files
### GET /files/{path}
Serve static files.
**Path Parameters:**
- `path` - Relative path to file in share/static/
**Example:**
```bash
curl wttr.in/files/example-wttr-v2.png
```
### GET /favicon.ico
Get favicon.
## Query Parameters
### Single-Letter Options
Combine multiple single-letter options in the query string without values:
```bash
curl wttr.in/London?Tnq
```
**Available Options:**
| Option | Description |
|--------|-------------|
| `A` | Force ANSI output (even for browsers) |
| `n` | Narrow output (narrower terminal width) |
| `m` | Use metric units (force metric even for US) |
| `M` | Use m/s for wind speed |
| `u` | Use imperial units (Fahrenheit, mph) |
| `I` | Inverted colors |
| `t` | Transparency for PNG (transparency=150) |
| `T` | No terminal sequences (plain text) |
| `p` | Add padding |
| `0` | Show current weather only (no forecast) |
| `1` | Show today + 1 day |
| `2` | Show today + 2 days |
| `3` | Show today + 3 days (default) |
| `q` | Quiet mode (no caption) |
| `Q` | Super quiet (no city name) |
| `F` | No follow line (no "Follow @igor_chubin") |
**Unit System Defaults:**
The service automatically selects units based on:
1. Explicit query parameter (`?u` or `?m`) - highest priority
2. Language parameter (`lang=us`) - forces imperial
3. Client IP geolocation - US IPs default to imperial
4. Default - metric for all other locations
**Examples:**
```bash
curl wttr.in/London?T # Plain text, no ANSI
curl wttr.in/London?u # Imperial units
curl wttr.in/London?m # Metric units
curl wttr.in/London?0 # Current weather only
curl wttr.in/London?Tnq # Plain text, narrow, quiet
```
### Named Parameters
| Parameter | Description | Example |
|-----------|-------------|---------|
| `lang={code}` | Language code (2-letter ISO 639-1) | `lang=de` |
| `format={fmt}` | Output format (see below) | `format=3` |
| `view={view}` | View type (alias for format) | `view=v2` |
| `period={sec}` | Update interval for cyclic locations | `period=60` |
| `transparency={0-255}` | PNG transparency level | `transparency=100` |
**Examples:**
```bash
curl wttr.in/London?lang=fr
curl wttr.in/London?format=3
curl wttr.in/London:Paris?period=60
```
### Combining Parameters
```bash
curl wttr.in/London?Tn&lang=de&format=3
curl wttr.in/Paris?u&format=v2
```
## Output Formats
### format=1 (One-line, compact)
```bash
curl wttr.in/London?format=1
# Output: London: ⛅️ +7°C
```
### format=2 (One-line, with wind)
```bash
curl wttr.in/London?format=2
# Output: London: ⛅️ +7°C 🌬↗11km/h
```
### format=3 (One-line, detailed)
```bash
curl wttr.in/London?format=3
# Output: London: ⛅️ +7°C 🌬↗11km/h 💧65%
```
### format=4 (One-line, full)
```bash
curl wttr.in/London?format=4
# Output: London: ⛅️ +7°C 🌬↗11km/h 💧65% ☀06:45-16:32
```
### format={custom} (Custom format)
Use `%` notation to create custom formats:
| Code | Description |
|------|-------------|
| `%c` | Weather condition emoji |
| `%C` | Weather condition text |
| `%h` | Humidity |
| `%t` | Temperature (actual) |
| `%f` | Temperature (feels like) |
| `%w` | Wind |
| `%l` | Location |
| `%m` | Moon phase 🌑🌒🌓🌔🌕🌖🌗🌘 |
| `%M` | Moon day |
| `%p` | Precipitation (mm) |
| `%o` | Probability of precipitation |
| `%P` | Pressure (hPa) |
| `%D` | Dawn time |
| `%S` | Sunrise time |
| `%z` | Zenith time |
| `%s` | Sunset time |
| `%d` | Dusk time |
**Examples:**
```bash
curl wttr.in/London?format="%l:+%c+%t"
# Output: London: ⛅️ +7°C
curl wttr.in/London?format="%l:+%C+%t+%w"
# Output: London: Partly cloudy +7°C ↗11km/h
curl wttr.in/London?format="%D+%S+%z+%s+%d"
# Output: 06:30 06:45 12:15 16:32 17:00
```
**Note:** In tmux.conf, escape `%` as `%%`
### format=v2 (Data-rich experimental)
Experimental format with additional information:
- Temperature and precipitation graphs
- Moon phase for 4 days
- Current conditions with all metrics
- Timezone information
- Astronomical times (dawn, sunrise, zenith, sunset, dusk)
- Precise GPS coordinates
```bash
curl wttr.in/London?format=v2
# or
curl v2.wttr.in/London
# or (Nerd Fonts)
curl v2d.wttr.in/London # day variant
curl v2n.wttr.in/London # night variant
```
**Limitations:**
- Terminal only (no HTML)
- English only
### format=j1 (JSON)
Machine-readable JSON format with all weather data.
```bash
curl wttr.in/London?format=j1
```
**Response Structure:**
```json
{
"current_condition": [{
"FeelsLikeC": "25",
"FeelsLikeF": "76",
"cloudcover": "100",
"humidity": "76",
"observation_time": "04:08 PM",
"precipMM": "0.2",
"pressure": "1019",
"temp_C": "22",
"temp_F": "72",
"uvIndex": 5,
"visibility": "16",
"weatherCode": "122",
"weatherDesc": [{"value": "Overcast"}],
"winddir16Point": "NNE",
"winddirDegree": "20",
"windspeedKmph": "7",
"windspeedMiles": "4"
}],
"nearest_area": [...],
"request": [...],
"weather": [...]
}
```
**Weather Codes:** See https://www.worldweatheronline.com/developer/api/docs/weather-icons.aspx
### format=p1 (Prometheus Metrics)
Prometheus-compatible metrics format.
```bash
curl wttr.in/London?format=p1
```
**Response:**
```
# HELP temperature_feels_like_celsius Feels Like Temperature in Celsius
temperature_feels_like_celsius{forecast="current"} 7
# HELP temperature_feels_like_fahrenheit Feels Like Temperature in Fahrenheit
temperature_feels_like_fahrenheit{forecast="current"} 45
...
```
**Prometheus Config Example:**
```yaml
- job_name: 'wttr_in_london'
static_configs:
- targets: ['wttr.in']
metrics_path: '/London'
params:
format: ['p1']
```
## Language Support
54 languages supported via `lang` parameter or subdomain:
```bash
curl wttr.in/London?lang=de
curl de.wttr.in/London
```
**Supported Languages:**
af, ar, az, be, bg, bs, ca, cy, da, de, el, en, eo, es, et, eu, fa, fr, fy, ga, he, hr, hu, hy, ia, id, is, it, ja, kk, lv, mk, nb, nl, nn, oc, pl, pt, pt-br, ro, ru, sl, te, th, tr, uk, uz, vi, zh-cn, zh-tw
**Language Detection Priority:**
1. Subdomain (e.g., `de.wttr.in`)
2. `lang` parameter
3. `Accept-Language` header
4. Default (English)
## Multiple Locations
Query multiple locations in one request using `:` separator:
```bash
curl wttr.in/London:Paris:Berlin?format=3
# Output:
# London: ⛅️ +7°C
# Paris: ☀️ +12°C
# Berlin: 🌧 +5°C
```
**Cyclic Locations:**
Rotate through locations based on time:
```bash
curl wttr.in/London:Paris:Berlin?period=60
```
This will show:
- London for 60 seconds
- Paris for next 60 seconds
- Berlin for next 60 seconds
- Repeat
Useful for tmux status bars or monitoring dashboards.
## HTTP Headers
### Request Headers
| Header | Purpose |
|--------|---------|
| `User-Agent` | Determines output format (ANSI vs HTML) |
| `Accept-Language` | Language preference |
| `X-Forwarded-For` | Client IP (for location detection) |
| `X-Real-IP` | Client IP (alternative) |
| `X-PNG-Query-For` | Client IP (for PNG requests) |
### Response Headers
| Header | Value |
|--------|-------|
| `Content-Type` | `text/plain` or `text/html` or `image/png` |
| `Cache-Control` | `no-cache, no-store, must-revalidate` (PNG only) |
| `Access-Control-Allow-Origin` | `*` |
## HTTP Status Codes
| Code | Meaning |
|------|---------|
| 200 | Success |
| 403 | Forbidden (location blacklisted) |
| 429 | Too Many Requests (rate limit exceeded) |
| 500 | Internal Server Error |
| 503 | Service Unavailable (API error) |
## Rate Limits
Per-IP limits:
- 300 requests per minute
- 3600 requests per hour
- 86400 requests per day
**Response on limit exceeded:**
- HTTP 429
- Body: "Too many queries" message
## Caching
Responses are cached for 1000-2000 seconds (randomized).
**Cache key includes:**
- User-Agent
- Query string
- Client IP
- Language
**Non-cacheable requests:**
- Cyclic locations (containing `:`)
## Examples
### Basic Usage
```bash
# Current location
curl wttr.in
# Specific city
curl wttr.in/London
# Airport
curl wttr.in/muc
# Coordinates
curl wttr.in/55.7558,37.6173
# Moon phase
curl wttr.in/Moon
```
### Output Formats
```bash
# Plain text
curl wttr.in/London?T
# One-line
curl wttr.in/London?format=3
# JSON
curl wttr.in/London?format=j1
# PNG
curl wttr.in/London.png > weather.png
```
### Units
```bash
# Metric
curl wttr.in/London?m
# Imperial
curl wttr.in/London?u
```
### Languages
```bash
# German
curl wttr.in/London?lang=de
curl de.wttr.in/London
# French
curl fr.wttr.in/Paris
```
### Customization
```bash
# Narrow, quiet, plain text
curl wttr.in/London?Tnq
# Current weather only
curl wttr.in/London?0
# Custom format
curl wttr.in/London?format="%l:+%c+%t+%w"
# Transparent PNG
curl wttr.in/London_t.png > weather.png
```
### Shell Integration
```bash
# Add to .bashrc
weather() {
curl wttr.in/$1
}
# Use it
weather London
weather "New York"
```
### Tmux Status Bar
```bash
# In tmux.conf
set -g status-right '#(curl -s wttr.in/London?format="%%l:+%%c%%20%%t%%60%%w")'
```
### Multiple Locations
```bash
# All at once
curl 'wttr.in/{London,Paris,Berlin}?format=3'
# Cyclic (for monitoring)
curl wttr.in/London:Paris:Berlin?period=60&format=3
```

View file

@ -1,362 +1,432 @@
# wttr Architecture # wttr.in Architecture Documentation
## Overview ## System Overview
Single Zig binary, including: wttr.in is a console-oriented weather forecast service with a hybrid Python/Go architecture:
- HTTP server utilizing [http.zig](https://github.com/karlseguin/http.zig) - **Go proxy layer** (cmd/): LRU caching proxy with prefetching
- L1 memory/L2 file caching scheme with single directory for - **Python backend** (lib/, bin/): Weather data fetching, formatting, rendering
* Geocoding (place name -> coordinates) as a permanent cache - **Static assets** (share/): Translations, templates, emoji, help files
* IP -> location (via [GeoLite2](https://github.com/P3TERX/GeoLite.mmdb) with fallback to [IP2Location](https://ip2location.io)] as a permanent cache
* Weather as a temporary cache
- Pluggable weather provider interface
- Rate limiting middleware
The idea was to keep most or all of the features of [wttr.in](https://wttr.in) ## Request Flow
while vastly simplify the architecture of that system, which had evolved
significantly over time. This is a full re-write of that system.
## System Diagram
``` ```
Client Request Client Request
HTTP Server (http.zig) Go Proxy (port 8082) - LRU cache + prefetch
↓ (cache miss)
Python Backend (port 8002) - Flask/gevent
Rate Limiter (middleware) Location Resolution (GeoIP/IP2Location/IPInfo)
Router Weather API (met.no or WorldWeatherOnline)
Request Handler Format & Render (ANSI/HTML/PNG/JSON/Prometheus)
Location Resolver Response (cached with TTL 1000-2000s)
Provider interface cache check
↓ (miss)
Weather Provider (interface)
├─ MetNo (default)
└─ Mock (tests)
Cache Store
Renderer
├─ ANSI
├─ JSON
├─ Line
└─ v2
Response
``` ```
## Module Structure ## Component Breakdown
``` ### 1. Go Proxy Layer (cmd/)
src/
├── main.zig # Entry point, server setup
├── Config.zig # Configuration
├── http/
│ ├── server.zig # HTTP server wrapper
│ ├── router.zig # Route matching
│ ├── handler.zig # Request handlers
│ └── middleware.zig # Rate limiter
├── cache/
│ ├── cache.zig # Cache interface
│ ├── lru.zig # In-memory LRU
│ └── file.zig # File-backed storage
├── location/
│ ├── resolver.zig # Main location resolution
│ ├── GeoLite2.zig # GeoIP wrapper
│ └── Ip2location.zig # IP2Location fallback
├── weather/
│ ├── provider.zig # Weather provider interface
│ ├── MetNo.zig # met.no implementation
│ └── types.zig # Weather data structures
└── render/
├── ansi.zig # ANSI terminal output
├── json.zig # JSON output
└── line.zig # One-line format
```
## Core Components **Files:**
- `cmd/srv.go` - Main HTTP server (port 8082)
### HTTP Server - `cmd/processRequest.go` - Request processing & caching logic
- `cmd/peakHandling.go` - Peak time prefetching (cron-based)
**Responsibilities:** **Responsibilities:**
- Listen on configured port - LRU cache (12,800 entries, 1000-1500s TTL)
- Parse HTTP requests - Cache key: `UserAgent:Host+URI:ClientIP:AcceptLanguage`
- Route to handlers - Prefetch popular requests at :24 and :54 past the hour
- Apply middleware (rate limiting) - Forward cache misses to Python backend (127.0.0.1:9002)
- Return responses - Handle concurrent requests (InProgress flag prevents thundering herd)
**Dependencies:** **Key Logic:**
- [http.zig](https://github.com/karlseguin/http.zig) - `dontCache()`: Skip caching for cyclic requests (location contains `:`)
- `getCacheDigest()`: Generate cache key from request metadata
- `processRequest()`: Main request handler with cache-aside pattern
- `savePeakRequest()`: Record requests at :30 and :00 for prefetching
**Routes:** ### 2. Python Backend (bin/, lib/)
```
GET / → weather for IP location
GET /{location} → weather for location
GET /:help → help page
```
### Rate Limiter #### Entry Points (bin/)
**Algorithm:** Token Bucket **bin/srv.py** - Main Flask application
- Listens on port 8002 (configurable via WTTRIN_SRV_PORT)
- Routes: `/`, `/<location>`, `/files/<path>`, `/favicon.ico`
- Uses gevent WSGI server for async I/O
- Delegates to `wttr_srv.wttr()` for all weather requests
**Configuration:** **bin/proxy.py** - Weather API proxy (separate service)
```zig - Caches weather API responses
pub const RateLimitConfig = struct { - Transforms met.no/WWO data to standard JSON
capacity: u32 = 300, // Max tokens in bucket - Test mode support (WTTRIN_TEST env var)
refill_rate: u32 = 5, // Tokens per second - Handles translations for weather conditions
refill_interval_ms: u64 = 200, // Refill every 200ms
};
```
**Implementation:** **bin/geo-proxy.py** - Geolocation service proxy
- HashMap of IP → TokenBucket - Not examined in detail (separate microservice)
- Each request consumes 1 token
- Tokens refill at configured rate (default: 5/second)
- Bucket capacity: 300 tokens (allows bursts)
- Periodic cleanup of old buckets (not accessed in 1 hour)
### Cache #### Core Logic (lib/)
**Single-layer cache with two storage backends:** **lib/wttr_srv.py** - Main request handler
- `wttr(location, request)` - Entry point for all weather queries
- `parse_request()` - Parse location, language, format from request
- `_response()` - Generate response (checks cache, calls renderers)
- Rate limiting (300/min, 3600/hour, 24*3600/day per IP)
- ThreadPool (25 workers) for PNG rendering
- Two-phase processing: fast path (cache/static) then full path
**Interface:** **lib/parse_query.py** - Query string parsing
```zig - Parse single-letter options: `n`=narrow, `m`=metric, `u`=imperial, `T`=no-terminal, etc.
pub const Cache = struct { - Parse PNG filenames: `City_200x_lang=ru.png` → structured dict
lru: LRU, - Serialize/deserialize query state (base64+zlib for short URLs)
file_store: FileStore, - Metric vs imperial logic (US IPs default to imperial)
pub fn get(self: *Cache, key: []const u8) ?[]const u8; **lib/location.py** - Location resolution
pub fn put(self: *Cache, key: []const u8, value: []const u8, ttl: u64) !void; - `location_processing()` - Main entry point
}; - IP → Location: GeoIP2 (MaxMind), IP2Location API, IPInfo API
``` - Location normalization (lowercase, strip special chars)
- Geolocator service (localhost:8004) for GPS coordinates
- IATA airport code support
- Alias resolution (share/aliases file)
- Blacklist checking (share/blacklist file)
- Hemisphere detection for moon phases
- Special prefixes:
- `~` = search term (use geolocator)
- `@` = domain name (resolve to IP first)
- No prefix = exact location name
**Cache Key:** **lib/globals.py** - Configuration
``` - Environment variables: WTTR_MYDIR, WTTR_GEOLITE, WTTR_WEGO, etc.
{user_agent}:{path}:{query}:{client_ip} - File paths: cache dirs, static files, translations
``` - API keys: IP2Location, IPInfo, WorldWeatherOnline
- Constants: NOT_FOUND_LOCATION, PLAIN_TEXT_AGENTS, QUERY_LIMITS
- IP location order: geoip → ip2location → ipinfo
**Storage Strategy:** **lib/cache.py** - LRU cache (Python side)
- Small responses (<1KB): In-memory LRU only - In-memory LRU (10,000 entries, pylru)
- Large responses (≥1KB): LRU stores file path, data on disk - File cache for large responses (>80 bytes)
- TTL: 1000-2000s (randomized to prevent thundering herd) - TTL: 1000-2000s (randomized)
- Cache key: `UserAgent:QueryString:ClientIP:Lang`
- Dynamic timestamp replacement: `%{{NOW(timezone)}}`
**Cache Locations:** **lib/limits.py** - Rate limiting
- Per-IP query limits (minute/hour/day buckets)
- Whitelist support
- Returns 429 on limit exceeded
All caches default to `$XDG_CACHE_HOME/wttr` (typically `~/.cache/wttr`). #### View Renderers (lib/view/)
1. **Weather Response Cache** **lib/view/wttr.py** - Main weather view
- Location: `$WTTR_CACHE_DIR` (default: `~/.cache/wttr/`) - Calls `wego` (Go binary) for weather rendering
- Size: 10,000 entries (configurable via `WTTR_CACHE_SIZE`) - Passes flags: -inverse, -wind_in_ms, -narrow, -lang, -imperial
- Expiration: 1000-2000 seconds (randomized) - Post-processes output (location name, formatting)
- Eviction: LRU - Converts to HTML if needed
2. **Geocoding Cache** **lib/view/line.py** - One-line format
- Location: `$WTTR_GEOCACHE_FILE` (default: `~/.cache/wttr/geocache.json`) - Formats: 1, 2, 3, 4, or custom with % notation
- Format: JSON - Custom format codes: %c=condition, %t=temp, %h=humidity, %w=wind, etc.
- Expiration: None (persists indefinitely) - Supports multiple locations (`:` separated)
3. **IP2Location Cache** **lib/view/v2.py** - Data-rich v2 format
- Location: `$IP2LOCATION_CACHE_FILE` (default: `~/.cache/wttr/ip2location.cache`) - Experimental format with more detail
- Format: Binary (32-byte records) - Moon phase, astronomical times, temperature graphs
- Expiration: None (persists indefinitely) - Terminal-only, English-only
4. **GeoIP Database** **lib/view/moon.py** - Moon phase view
- Location: `$WTTR_GEOLITE_PATH` (default: `~/.cache/wttr/GeoLite2-City.mmdb`) - Uses `pyphoon-lolcat` for rendering
- Auto-downloaded if missing - Supports date selection: `Moon@2016-12-25`
### Location Resolver (`src/location/resolver.zig`) **lib/view/prometheus.py** - Prometheus metrics
- Exports weather data as Prometheus metrics
- Format: `p1`
**Responsibilities:** #### Formatters (lib/fmt/)
- Parse location from URL
- Normalize location names
- Resolve IP to location (GeoIP)
- Resolve names to coordinates (geocoding)
- Handle special prefixes (~, @)
**GeoIP:** **lib/fmt/png.py** - PNG rendering
- Uses MaxMind GeoLite2 database - Converts ANSI terminal output to PNG images
- Auto-downloads if missing - Uses pyte (terminal emulator) + PIL
- Fallback to IP2Location API - Transparency support
- Font rendering
**Ip2Location:** **lib/fmt/unicodedata2.py** - Unicode handling
- Uses [Ip2Location](https://ip2location.io) - Character width calculations for terminal rendering
- API key not required...comes with a limit of 1k/day. Because GeoLite2 handles
most of the mapping, and results are cached, this should be fine for most needs
- API key provides 50k/month requests
- Set via IP2LOCATION_API_KEY environment variable
**Geocoding:** #### Other Modules (lib/)
- Uses nominatim, part of the OpenStreetMap project
- API key not required
### Weather Provider **lib/translations.py** - i18n support
- 54 languages supported
- Weather condition translations
- Help file translations (share/translations/)
- Language detection from Accept-Language header
**Interface (pluggable):** **lib/constants.py** - Weather constants
```zig - Weather codes (WWO API)
pub const WeatherProvider = struct { - Condition mappings
ptr: *anyopaque, - Emoji mappings
vtable: *const VTable,
cache: *Cache,
pub const VTable = struct { **lib/buttons.py** - HTML UI elements
fetchRaw: *const fn (ptr: *anyopaque, allocator: std.mem.Allocator, coords: Coordinates) anyerror![]const u8, - Add interactive buttons to HTML output
parse: *const fn (ptr: *anyopaque, allocator: std.mem.Allocator, raw: []const u8) anyerror!types.WeatherData,
deinit: *const fn (ptr: *anyopaque) void,
};
pub fn fetch(self: WeatherProvider, location: []const u8) !WeatherData; **lib/fields.py** - Data field extraction
}; - Parse weather API responses
```
Note that the interface will handle result caching according to the description **lib/weather_data.py** - Weather data structures
of the cache section above. Fetch and parse are two separate functions in this
interface to assist with unit tests.
**MetNo Implementation:** **lib/airports.py** - IATA code handling
- Fetches from Met.no API
- Performs timezone conversions at ingestion, so results are in the timezone
of the target location
- Groups forecast data by local date
- No API key necessary, but requires METNO_TOS_IDENTIFYING_EMAIL environment
variable to be set
**Provider vs Renderer Responsibilities:** **lib/metno.py** - met.no API client
- Norwegian Meteorological Institute API
- Transforms to standard JSON format
**Weather Provider:** ### 3. Static Assets (share/)
- Fetch raw weather data from external APIs
- Parse API responses into structured types
- Perform timezone conversions to the timezone of the weather location
once at ingestion time
- Group forecast data by local date (not UTC date)
- Store both UTC time and local time in forecast data
**Renderer:** **share/translations/** - 54 language files
- Format weather data for display - Format: `{lang}.txt` (weather conditions)
- Select appropriate hourly forecasts for display - Format: `{lang}-help.txt` (help pages)
- Apply unit conversions (metric/imperial). Conversion functions are in the core
weather types, but the renderer is responsible for calling them
- Handle partial days with missing data
- Format dates and times for human readability
- Should NOT perform timezone calculations
**Key principle:** Timezone conversions happen once at the provider level **share/emoji/** - Weather emoji PNGs
- Used for PNG rendering
**Implementation details:** **share/static/** - Web assets
- Core data structures use `zeit.Time` and `zeit.Date` types for type safety - favicon.ico
- `HourlyForecast` contains both `time: zeit.Time` (UTC) and `local_time: zeit.Time` - style.css
- MetNo provider converts the location to timezone offsets through `src/location/timezone_offsets.zig`) - example images
* This code uses pre-computed timezone offset lookup table
* It is auto-generated based on a Python script
* It is *NOT* precise, currently calculating the timezone based on the longitude
only. This could be up to 2 hours different from the actual timezone...however,
the purpose here is to report weather to a granularity of morning/noon/evening/night
### Renderers **share/templates/** - Jinja2 templates
- index.html (HTML output wrapper)
There are currently 5 renderers: **share/** - Data files
- `aliases` - Location aliases (from:to format)
- `blacklist` - Blocked locations
- `list-of-iata-codes.txt` - Airport codes
- `help.txt` - English help
- `bash-function.txt` - Shell integration
- `translation.txt` - Translation info page
* formatted ## API Endpoints
* json
* v2
* custom
* line
**Formatted Renderer:** ### Weather Queries
```zig
pub const FormattedRenderer = struct {
pub fn render(allocator: Allocator, data: WeatherData, options: RenderOptions) ![]const u8;
};
```
This is the most complex of all the renders, and is the default when the user - `GET /` - Weather for IP-based location
doesn't specifically choose something else. It displays the full table of - `GET /{location}` - Weather for specific location
current conditions and up to 3 day forecast, and does so in plain text, ansi, - `GET /{location}.png` - PNG image output
or html formats. Currently, these formats are coded separately. Note it is - `GET /{location}?{options}` - Weather with options
possible to provide ansi and html, then have the code strip out extra markup to
provide plain text. This is **not** how the code works today. Also, the
original project has an option to avoid characters some brain-dead terminals
can't handle. That has not been implemented. When done, this is likely to be
implemented as a search/replace for the few unicode characters that exist and
replace them manually.
### Special Pages
**JSON Renderer:** - `GET /:help` - Help page
```zig - `GET /:bash.function` - Shell function
pub const JsonRenderer = struct { - `GET /:translation` - Translation info
pub fn render(allocator: Allocator, data: WeatherData) ![]const u8; - `GET /:iterm2` - iTerm2 integration
};
```
Just utilizes the `std.json.fmt` api to render the underlying data type. No
attempt has been made to match wttr.in data or format (probably should).
**One-Line Renderer:** ### Static Files
```zig
pub const LineRenderer = struct {
pub fn render(allocator: Allocator, data: WeatherData, format: []const u8) ![]const u8;
};
// Formats: - `GET /files/{path}` - Static assets
// 1: "London: ⛅️ +7°C" - `GET /favicon.ico` - Favicon
// 2: "London: ⛅️ +7°C 🌬↗11km/h"
// 3: "London: ⛅️ +7°C 🌬↗11km/h 💧65%"
// Custom: "%l: %c %t" → "London: ⛅️ +7°C"
```
## Network Calls ## Query Parameters
The application makes network calls to the following services: ### Single-letter Options (combined in query string)
### 1. GeoLite2 Database Download - `A` - Force ANSI output
- **URL:** `https://github.com/P3TERX/GeoLite.mmdb/raw/download/GeoLite2-City.mmdb` - `n` - Narrow output
- **Purpose:** Download MaxMind GeoLite2 database if missing - `m` - Metric units
- **Frequency:** Only on first run or when database is missing - `M` - m/s for wind speed
- `u` - Imperial units
- `I` - Inverted colors
- `t` - Transparency (PNG)
- `T` - No terminal sequences
- `p` - Padding
- `0-3` - Number of days
- `q` - No caption
- `Q` - No city name
- `F` - No follow line
### 2. IP2Location API ### Named Parameters
- **URL:** `https://api.ip2location.io/?key={API_KEY}&ip={IP_ADDRESS}`
- **Purpose:** Fallback geolocation when MaxMind lookup fails
- **Frequency:** Only when `IP2LOCATION_API_KEY` is set and MaxMind fails
- **Rate Limit:** Free tier: 30,000 requests/month
- **Caching:** Results cached persistently
### 3. Nominatim Geocoding API - `lang={code}` - Language override
- **URL:** `https://nominatim.openstreetmap.org/search?q={LOCATION}&format=json&limit=1` - `format={fmt}` - Output format (1-4, v2, j1, p1, custom)
- **Purpose:** Convert city/location names to coordinates - `view={view}` - View type (alias for format)
- **Frequency:** When user requests weather by city name - `period={sec}` - Update interval for cyclic locations
- **Rate Limit:** Maximum 1 request per second
- **Caching:** Results cached persistently
### 4. Met.no Weather API ### PNG Filename Format
- **URL:** `https://api.met.no/weatherapi/locationforecast/2.0/compact?lat={LAT}&lon={LON}`
- **Purpose:** Fetch weather forecast data
- **Frequency:** Every request (unless cached)
- **Rate Limit:** No explicit limit, respectful usage required
- **Caching:** Results cached in memory (LRU)
## Dependencies `{location}_{width}x{height}_{options}_lang={lang}.png`
**External (Zig packages):** Example: `London_200x_t_lang=ru.png`
- HTTP Server: [http.zig](https://github.com/karlseguin/http.zig)
- Time utilities: [zeit](https://github.com/rockorager/zeit)
**External (other):** ## Output Formats
- Airport code -> location mapping: [Openflights](https://github.com/jpatokal/openflights)
- Ip address -> location mapping: [GeoLite2 City Database](https://github.com/maxmind/libmaxminddb)
## Performance Targets 1. **ANSI** - Terminal with colors/formatting
2. **Plain text** - No ANSI codes (T option)
3. **HTML** - Web browser output
4. **PNG** - Image file
5. **JSON** (j1) - Machine-readable data
6. **Prometheus** (p1) - Metrics format
7. **One-line** (1-4) - Compact formats
8. **v2** - Data-rich experimental format
**Latency:** ## External Dependencies
- Cache hit: <1ms
- Cache miss: <100ms
- P95: <150ms
- P99: <300ms
**Throughput:** ### Weather APIs
- >10,000 req/s (cached)
- >1,000 req/s (uncached)
**Memory:** - **met.no** (Norwegian Meteorological Institute) - Primary, free
- Base: <50MB (currently 9MB) - **WorldWeatherOnline** - Fallback, requires API key
- With 10,000 cache entries: <200MB
- Binary size: <5MB (currently <2MB in ReleaseSmall) ### Geolocation
- **GeoLite2** (MaxMind) - Free GeoIP database (required)
- **IP2Location** - Commercial API (optional, needs key)
- **IPInfo** - Commercial API (optional, needs key)
- **Geolocator service** - localhost:8004 (GPS coordinates)
### External Binaries
- **wego** (we-lang) - Go weather rendering binary
- **pyphoon-lolcat** - Moon phase rendering
### Python Libraries
- Flask - Web framework
- gevent - Async I/O
- geoip2 - GeoIP lookups
- geopy - Geocoding
- requests - HTTP client
- PIL - Image processing
- pyte - Terminal emulator
- pytz - Timezone handling
- pylru - LRU cache
### Go Libraries
- github.com/hashicorp/golang-lru - LRU cache
- github.com/robfig/cron - Cron scheduler
## Configuration
### Environment Variables
- `WTTR_MYDIR` - Installation directory
- `WTTR_GEOLITE` - Path to GeoLite2-City.mmdb
- `WTTR_WEGO` - Path to wego binary
- `WTTR_LISTEN_HOST` - Bind address (default: "")
- `WTTR_LISTEN_PORT` - Port (default: 8002)
- `WTTR_USER_AGENT` - Custom user agent
- `WTTR_IPLOCATION_ORDER` - IP location method order
- `WTTRIN_SRV_PORT` - Override listen port
- `WTTRIN_TEST` - Enable test mode
### API Key Files
- `~/.wwo.key` - WorldWeatherOnline API key
- `~/.ip2location.key` - IP2Location API key
- `~/.ipinfo.key` - IPInfo token
- `~/.wegorc` - Wego configuration (JSON)
### Data Directories
- `/wttr.in/cache/ip2l/` - IP location cache
- `/wttr.in/cache/png/` - PNG cache
- `/wttr.in/cache/lru/` - LRU file cache
- `/wttr.in/cache/proxy-wwo/` - Weather API cache
- `/wttr.in/log/` - Log files
## Caching Strategy
### Three-tier Cache
1. **Go LRU** (12,800 entries, 1000-1500s TTL)
- In-memory, fastest
- Full HTTP responses
- Shared across all requests
2. **Python LRU** (10,000 entries, 1000-2000s TTL)
- In-memory for small responses (<80 bytes)
- File-backed for large responses
- Per-process cache
3. **File Cache**
- IP location cache (persistent)
- Weather API cache (persistent)
- PNG cache (persistent)
### Cache Keys
- Go: `UserAgent:Host+URI:ClientIP:AcceptLanguage`
- Python: `UserAgent:QueryString:ClientIP:Lang`
### Cache Invalidation
- TTL-based expiration (no manual invalidation)
- Randomized TTL prevents thundering herd
- Non-cacheable: cyclic requests (location contains `:`)
## Prefetching
- Cron jobs at :24 and :54 past the hour
- Records popular requests at :30 and :00
- Spreads prefetch over 5 minutes (300s)
- Prevents cache expiry during peak times
## Rate Limiting
- Per-IP limits: 300/min, 3600/hour, 24*3600/day
- Whitelist support (MY_EXTERNAL_IP)
- Returns HTTP 429 on limit exceeded
- Implemented in Python layer only
## Error Handling
- Location not found → "not found" location (fallback weather)
- API errors → 503 Service Unavailable
- Malformed requests → 500 Internal Server Error (HTML) or error message (text)
- Blocked locations → 403 Forbidden
- Rate limit → 429 Too Many Requests
## Logging
- Main log: `/wttr.in/log/main.log`
- Debug log: `/tmp/wttr.in-debug.log`
- Go proxy logs to stdout
## Testing
- No unit tests
- Integration test: `test/query.sh`
- Makes HTTP requests to running server
- Compares SHA1 hashes of responses
- Test data in `test/test-data/signatures`
- CI: flake8 linting only (no actual tests run)
## Known Issues & Limitations
1. No unit test coverage
2. v2 format is experimental (terminal-only, English-only)
3. Moon phase Unicode ambiguity (hemisphere-dependent)
4. Hardcoded IP whitelist (MY_EXTERNAL_IP)
5. Multiple cache layers with different keys
6. Mixed Python/Go codebase
7. External binary dependencies (wego, pyphoon)
8. Requires external geolocator service (port 8004)
9. File cache grows unbounded
10. No cache warming on startup
## Performance Characteristics
- Go proxy handles ~12,800 cached requests in memory
- Python backend spawns 25 threads for PNG rendering
- Gevent provides async I/O for Python
- Prefetching reduces latency during peak times
- File cache avoids memory pressure for large responses
- Rate limiting prevents abuse
## Security Considerations
- IP-based rate limiting
- Location blacklist
- No authentication required
- User-provided location names passed to external APIs
- File cache uses MD5 hashes (not cryptographic)
- No input sanitization for location names
- Trusts X-Forwarded-For header

140
CACHE_CONFIGURATION.md Normal file
View file

@ -0,0 +1,140 @@
# Cache Configuration
wttr.in uses three separate caches following Linux Filesystem Hierarchy Standard (FHS) and XDG Base Directory specifications.
## External Services
### Required Services
- **Met.no Weather API** - Primary weather data provider
- No API key required
- Free, open API from Norwegian Meteorological Institute
- Rate limit: Be respectful, use caching
### Optional Services
- **IP2Location.io** - Fallback IP geolocation service
- API key required: Sign up at https://www.ip2location.io/
- Used when MaxMind GeoIP database lookup fails
- Free tier: 30,000 requests/month
- Set via `IP2LOCATION_API_KEY` environment variable
### Database Files
- **MaxMind GeoLite2 City** - IP geolocation database
- Free database, auto-downloaded if missing
- No API key required for database usage
- Updates available monthly from MaxMind
## Cache Locations
All caches default to `$XDG_CACHE_HOME/wttr` (typically `~/.cache/wttr`).
### 1. Weather Response Cache
**Purpose:** Caches weather API responses to reduce upstream requests
**Default Location:** `$XDG_CACHE_HOME/wttr/` (individual files)
**Environment Variable:** `WTTR_CACHE_DIR`
**Size:** 10,000 entries (configurable via `WTTR_CACHE_SIZE`)
**Expiration:** 1000-2000 seconds (16-33 minutes, randomized to avoid thundering herd)
**Eviction:** LRU (Least Recently Used)
This is the main cache that stores weather forecast responses from Met.no. Each entry has a randomized TTL to prevent cache stampedes.
### 2. Geocoding Cache (Optional)
**Purpose:** Caches location name → coordinates mappings
**Default:** Disabled (in-memory only)
**Environment Variable:** `WTTR_GEOCACHE_FILE`
**Format:** JSON
**Expiration:** None (persists indefinitely)
**Eviction:** None (grows unbounded)
When enabled, persists geocoding lookups to disk. Saves every 15 minutes if dirty. Useful for reducing external geocoding API calls.
### 3. IP2Location Cache
**Purpose:** Caches IP → coordinates from IP2Location API
**Default Location:** `$XDG_CACHE_HOME/wttr/ip2location.cache`
**Environment Variable:** `IP2LOCATION_CACHE_FILE`
**Format:** Binary (32-byte records)
**Expiration:** None (persists indefinitely)
**Eviction:** None (append-only, grows unbounded)
Only used when `IP2LOCATION_API_KEY` is configured. Provides fallback when MaxMind GeoIP database lookup fails.
## GeoIP Database Location
**Default:** `$XDG_CACHE_HOME/wttr/GeoLite2-City.mmdb`
**Environment Variable:** `WTTR_GEOLITE_PATH`
This is the MaxMind GeoLite2 database. It will be automatically downloaded if missing.
## Environment Variables Summary
| Variable | Default | Description |
|----------|---------|-------------|
| `WTTR_CACHE_DIR` | `$XDG_CACHE_HOME/wttr` | Weather response cache directory |
| `WTTR_CACHE_SIZE` | `10000` | Maximum number of cached weather responses |
| `WTTR_GEOCACHE_FILE` | (none) | Optional persistent geocoding cache file |
| `WTTR_GEOLITE_PATH` | `$XDG_CACHE_HOME/wttr/GeoLite2-City.mmdb` | MaxMind GeoLite2 database path |
| `IP2LOCATION_API_KEY` | (none) | API key for IP2Location fallback service |
| `IP2LOCATION_CACHE_FILE` | `$XDG_CACHE_HOME/wttr/ip2location.cache` | IP2Location cache file |
| `XDG_CACHE_HOME` | `~/.cache` | XDG base cache directory |
## Examples
### Minimal Configuration (defaults)
```bash
./wttr
# Uses ~/.cache/wttr/ for all caches and GeoIP database
```
### Custom Cache Location
```bash
WTTR_CACHE_DIR=/var/cache/wttr ./wttr
# All caches and GeoIP in /var/cache/wttr/
```
### Enable Persistent Geocoding Cache
```bash
WTTR_GEOCACHE_FILE=~/.cache/wttr/geocache.json ./wttr
```
### With IP2Location Fallback
```bash
IP2LOCATION_API_KEY=your_key_here ./wttr
# Cache at ~/.cache/wttr/ip2location.cache
```
### Production Setup
```bash
WTTR_CACHE_DIR=/var/cache/wttr \
WTTR_CACHE_SIZE=50000 \
WTTR_GEOCACHE_FILE=/var/cache/wttr/geocache.json \
IP2LOCATION_API_KEY=your_key_here \
./wttr
# GeoIP and IP2Location cache also in /var/cache/wttr/
```
## Cache Maintenance
### Weather Cache
- **Automatic expiration:** Entries expire after 16-33 minutes (randomized)
- **LRU eviction:** When cache reaches max size (10,000 entries), least recently used entries are removed
- **Disk cleanup:** Expired files are cleaned up on access
- Safe to delete entire cache directory; will be recreated as needed
### Geocoding Cache
- **No expiration:** Entries persist indefinitely
- **No eviction:** Cache grows unbounded
- **Auto-save:** Writes to disk every 15 minutes when modified
- Consider periodic cleanup if cache grows too large
### IP2Location Cache
- **No expiration:** Entries persist indefinitely
- **Append-only:** File grows unbounded (32 bytes per unique IP)
- **No cleanup:** Consider periodic truncation for long-running deployments
- Safe to delete; will be recreated on next API lookup
### GeoIP Database
- **Manual updates:** Download new database periodically for accuracy
- **Auto-download:** Database is automatically downloaded if missing on startup
- Typical update frequency: monthly (MaxMind releases)

641
DATA_FLOW.md Normal file
View file

@ -0,0 +1,641 @@
# wttr.in Data Flow Documentation
## Request Processing Flow
### 1. Initial Request
```
Client → Go Proxy (port 8082)
```
**Input:**
- HTTP request with location in URL
- Headers: User-Agent, Accept-Language, X-Forwarded-For
- Query parameters
**Go Proxy Actions:**
1. Extract cache key: `UserAgent:Host+URI:ClientIP:AcceptLanguage`
2. Check if request is cacheable (no `:` in location)
3. Look up in LRU cache (12,800 entries)
**Cache Hit Path:**
```
Go Proxy → Check expiry → Return cached response
```
**Cache Miss Path:**
```
Go Proxy → Set InProgress flag → Forward to Python Backend
```
### 2. Python Backend Processing
```
Go Proxy → Python Backend (port 8002) → Flask Router
```
**Flask Routes:**
- `/``wttr_srv.wttr(None, request)`
- `/{location}``wttr_srv.wttr(location, request)`
- `/:help`, `/:bash.function`, etc. → Static file handlers
### 3. Request Parsing (wttr_srv.py)
**Phase 1: Fast Path (Cache + Static)**
```python
parse_request(location, request, query, fast_mode=True)
_response(parsed_query, query, fast_mode=True)
Check Python LRU cache
Check if static page (:help, :bash.function, etc.)
Return if found, else continue to slow path
```
**Phase 2: Full Processing**
```python
parse_request(location, request, query, fast_mode=False)
Location Processing
_response(parsed_query, query, fast_mode=False)
Render weather
Cache and return
```
### 4. Location Processing (location.py)
**Input:** Location string, Client IP
**Processing Steps:**
```
1. Detect location type
├─ Empty/MyLocation → Use client IP
├─ IP address → Resolve to location
├─ @domain → Resolve domain to IP, then location
├─ ~search → Use geolocator service
├─ Moon → Special moon handler
└─ Name → Use as-is
2. Normalize location
├─ Lowercase
├─ Replace _ and + with space
└─ Remove special chars (!@#$*;:\)
3. Check aliases (share/aliases)
└─ from:to mapping
4. Check blacklist (share/blacklist)
└─ Return 403 if blocked
5. Resolve location
├─ IP → Location (GeoIP/IP2Location/IPInfo)
├─ Name → GPS coords (Geolocator service)
└─ IATA code → Airport location
6. Get hemisphere (for moon queries)
└─ GPS latitude > 0 = North
```
**Output:**
- `location` - Normalized location or GPS coords
- `override_location_name` - Display name
- `full_address` - Full address from geolocator
- `country` - Country name
- `query_source_location` - Client's location (city, country)
- `hemisphere` - True=North, False=South
### 5. IP to Location Resolution
**Method Priority (configurable via WTTR_IPLOCATION_ORDER):**
```
1. GeoIP (MaxMind GeoLite2)
├─ Read from GeoLite2-City.mmdb
├─ Extract city and country
└─ Fast, local, free
2. IP2Location API (optional)
├─ HTTP GET to api.ip2location.com
├─ Requires API key (~/.ip2location.key)
├─ Cache result in /wttr.in/cache/ip2l/{ip}
└─ Format: city;country
3. IPInfo API (optional)
├─ HTTP GET to ipinfo.io
├─ Requires token (~/.ipinfo.key)
├─ Cache result in /wttr.in/cache/ip2l/{ip}
└─ JSON response
Fallback: NOT_FOUND_LOCATION ("not found")
```
**Caching:**
- File cache: `/wttr.in/cache/ip2l/{ip_address}`
- Format: `city;country` or `location;country;extra;city`
- Persistent across restarts
### 6. Geolocator Service
**For search terms (~location) and non-ASCII names:**
```
Python Backend → HTTP GET localhost:8004/{location}
Geolocator Service (separate microservice)
Returns JSON:
{
"latitude": 48.8582602,
"longitude": 2.29449905432,
"address": "Tour Eiffel, 5, Avenue Anatole France..."
}
```
**Used for:**
- `~Eiffel Tower` → GPS coordinates
- `~Kilimanjaro` → GPS coordinates
- Non-ASCII location names
- IATA airport codes
### 7. Weather Data Fetching
**Two data sources (configured via WWO_KEY):**
#### Option A: met.no (Norwegian Meteorological Institute)
```
Python Backend → metno.py
HTTP GET to api.met.no
Parse XML/JSON response
Transform to standard JSON format
Return weather data
```
**Advantages:**
- Free, no API key required
- High quality data
- No rate limits
#### Option B: WorldWeatherOnline (WWO)
```
Python Backend → bin/proxy.py (separate service)
Check proxy cache (/wttr.in/cache/proxy-wwo/)
If miss: HTTP GET to api.worldweatheronline.com
Cache response
Return weather data
```
**Advantages:**
- More locations supported
- Historical data available
**Disadvantages:**
- Requires API key (~/.wwo.key)
- Rate limited (500 queries/day free tier)
**Weather Data Structure:**
```json
{
"current_condition": [{
"temp_C": "22",
"temp_F": "72",
"weatherCode": "122",
"weatherDesc": [{"value": "Overcast"}],
"windspeedKmph": "7",
"humidity": "76",
...
}],
"weather": [
{
"date": "2025-12-17",
"maxtempC": "25",
"mintempC": "18",
"hourly": [...]
}
]
}
```
### 8. Weather Rendering
**Route to appropriate renderer based on query:**
```
parsed_query → Determine view type
├─ format=1,2,3,4 → view/line.py (one-line format)
├─ format=j1 → Return raw JSON
├─ format=p1 → view/prometheus.py
├─ format=v2 → view/v2.py (data-rich)
├─ location=Moon → view/moon.py
└─ default → view/wttr.py (main view)
```
#### Main View (view/wttr.py)
```
get_wetter(parsed_query)
Call wego binary (Go program)
├─ Pass flags: -city, -lang, -imperial, -narrow, etc.
├─ wego fetches weather data
├─ wego renders ANSI output
└─ Return ANSI text
Post-process output
├─ Add location name override
├─ Add "not found" message if needed
└─ Format for display
If HTML output:
└─ Convert ANSI to HTML (ansi2html.sh)
```
**wego Command Example:**
```bash
/path/to/we-lang \
--city=London,GB \
-lang=en \
-imperial \
-narrow \
-location_name="London"
```
#### One-Line View (view/line.py)
```
wttr_line(query, parsed_query)
Get weather data (JSON)
Parse format string
├─ Predefined: 1, 2, 3, 4
└─ Custom: %c, %t, %h, %w, etc.
Replace format codes with data
├─ %c → Weather emoji
├─ %t → Temperature
├─ %h → Humidity
└─ etc.
Return formatted string
```
**Format Examples:**
- `format=3``London: ⛅️ +7°C`
- `format=%l:+%c+%t``London: ⛅️ +7°C`
#### Moon View (view/moon.py)
```
get_moon(parsed_query)
Parse date from location (Moon@2016-12-25)
Call pyphoon-lolcat binary
├─ Pass date parameter
└─ Return ASCII moon phase art
Return moon phase output
```
#### v2 View (view/v2.py)
```
Experimental data-rich format
Get weather data
Render:
├─ Temperature graph (ASCII)
├─ Precipitation graph (ASCII)
├─ Moon phases (4 days)
├─ Current conditions (detailed)
├─ Astronomical times (dawn, sunrise, etc.)
└─ GPS coordinates
Return formatted output
```
#### Prometheus View (view/prometheus.py)
```
Get weather data (JSON)
Convert to Prometheus metrics format
├─ temperature_feels_like_celsius{forecast="current"} 7
├─ humidity_percent{forecast="current"} 65
└─ etc.
Return metrics text
```
### 9. PNG Rendering (fmt/png.py)
**For .png requests:**
```
ANSI text output
Spawn thread (ThreadPool, 25 workers)
render_ansi(output, options)
Create virtual terminal (pyte)
├─ Feed ANSI sequences
└─ Capture terminal state
Render to image (PIL)
├─ Draw characters with font
├─ Apply colors from ANSI codes
└─ Apply transparency if requested
Return PNG bytes
Cache in /wttr.in/cache/png/
```
**Options:**
- `t` - Transparency (150)
- `transparency={0-255}` - Custom transparency
- `{width}x{height}` - Image dimensions
### 10. Translation (translations.py)
**Language Detection:**
```
1. Check subdomain (de.wttr.in → lang=de)
2. Check lang parameter (?lang=de)
3. Check Accept-Language header
4. Default to English
```
**Translation Files:**
- `share/translations/{lang}.txt` - Weather conditions
- `share/translations/{lang}-help.txt` - Help pages
**Translation Process:**
```
Weather condition text (English)
Look up in translations.py:TRANSLATIONS dict
Find translation for target language
Return translated text
```
**Example:**
```python
TRANSLATIONS = {
"en": {"Partly cloudy": "Partly cloudy"},
"de": {"Partly cloudy": "Teilweise bewölkt"},
"fr": {"Partly cloudy": "Partiellement nuageux"}
}
```
### 11. Caching (cache.py)
**Python LRU Cache:**
```
Request → Generate cache signature
signature = f"{user_agent}:{query_string}:{client_ip}:{lang}"
Check in-memory LRU (10,000 entries)
If found and not expired:
├─ If value starts with "file:" or "bfile:"
│ └─ Read from /wttr.in/cache/lru/{md5_hash}
└─ Return value
If not found:
├─ Generate response
├─ If response > 80 bytes:
│ ├─ Write to /wttr.in/cache/lru/{md5_hash}
│ └─ Store "file:{md5_hash}" in LRU
└─ Else: Store value directly in LRU
Set expiry: current_time + random(1000, 2000) seconds
Return response
```
**Dynamic Timestamps:**
```
Cached response with %{{NOW(timezone)}}
On retrieval: Replace with current time in timezone
Example: %{{NOW(Europe/London)}} → 14:32:15+0000
```
### 12. Response Wrapping
**Final response preparation:**
```
Response text/bytes
Determine content type
├─ PNG → image/png
├─ HTML → text/html
└─ ANSI/text → text/plain
Add buttons (if HTML and not format query)
├─ Add interactive UI elements
└─ Wrap in HTML template
Set HTTP headers
├─ Content-Type
├─ Cache-Control (PNG only)
└─ Access-Control-Allow-Origin: *
Return Flask response
```
### 13. Go Proxy Caching
**After Python backend returns:**
```
Python Backend → Response
Go Proxy receives response
If status code 200 or 304:
├─ Store in LRU cache
├─ Set expiry: current_time + random(1000, 1500) seconds
└─ Remove InProgress flag
Else (error):
└─ Remove from cache
Return response to client
```
### 14. Peak Request Prefetching
**Cron-based prefetching:**
```
Every hour at :30 and :00
Record incoming requests in sync.Map
At :24 and :54 (5 minutes before peak)
Iterate through recorded requests
For each request:
├─ Spawn goroutine
├─ Call processRequest() (refreshes cache)
├─ Sleep (spread over 300 seconds)
└─ Delete from sync.Map
Cache is warm for peak time
```
**Peak Times:**
- :30 past the hour (recorded at :30, prefetched at :24)
- :00 on the hour (recorded at :00, prefetched at :54)
## Data Structures
### Parsed Query
```python
{
"location": "London,GB", # Normalized location
"orig_location": "London", # Original input
"override_location_name": None, # Display name override
"full_address": "London, UK", # Full address
"country": "GB", # Country code
"query_source_location": ("Paris", "France"), # Client location
"hemisphere": True, # North=True, South=False
"lang": "en", # Language code
"view": None, # View type (v2, etc.)
"html_output": False, # HTML vs ANSI
"png_filename": None, # PNG filename if .png request
"ip_addr": "1.2.3.4", # Client IP
"user_agent": "curl/7.68.0", # User agent
"request_url": "http://...", # Full request URL
# Query options
"use_metric": True, # Metric units
"use_imperial": False, # Imperial units
"use_ms_for_wind": False, # m/s for wind
"narrow": False, # Narrow output
"inverted_colors": False, # Inverted colors
"no-terminal": False, # Plain text
"no-caption": False, # No caption
"no-city": False, # No city name
"no-follow-line": False, # No follow line
"days": "3", # Number of days
"transparency": None, # PNG transparency
"padding": False, # Add padding
"force-ansi": False, # Force ANSI
}
```
### Cache Entry (Go)
```go
type responseWithHeader struct {
InProgress bool // Request being processed
Expires time.Time // Expiration time
Body []byte // Response body
Header http.Header // HTTP headers
StatusCode int // HTTP status code
}
```
### Cache Entry (Python)
```python
{
"val": "response text" or "file:md5hash",
"expiry": 1702834567.123 # Unix timestamp
}
```
## Error Handling Flow
### Location Not Found
```
Location resolution fails
Set location = NOT_FOUND_LOCATION ("not found")
Fetch weather for default location (Oymyakon)
Append "not found" message in user's language
Return response
```
### API Error
```
Weather API returns error
Log error
If HTML output:
└─ Return malformed-response.html (500)
Else:
└─ Return "capacity limit reached" message (503)
```
### Rate Limit Exceeded
```
Check IP against limits (300/min, 3600/hour, 86400/day)
If exceeded:
└─ Return 429 with error message
```
### Blocked Location
```
Check location against blacklist
If blocked:
└─ Return 403 Forbidden
```
## Performance Optimizations
1. **Two-tier caching** (Go + Python)
2. **Fast path** (cache + static files checked first)
3. **File cache** for large responses (>80 bytes)
4. **Prefetching** at peak times
5. **ThreadPool** for PNG rendering (25 workers)
6. **Gevent** for async I/O in Python
7. **LRU eviction** prevents memory bloat
8. **Randomized TTL** prevents thundering herd
9. **InProgress flag** prevents duplicate work
10. **IP location caching** (persistent file cache)

98
IMPERIAL_UNITS.md Normal file
View file

@ -0,0 +1,98 @@
# Imperial Units Implementation
## Overview
The application automatically selects between metric and imperial (USCS) units based on the request context, matching the behavior of the original Python implementation.
## Unit Selection Priority
The system determines which units to use in the following priority order:
1. **Explicit query parameter** (highest priority)
- `?u` - Force imperial units (°F, mph, inches, inHg)
- `?m` - Force metric units (°C, km/h, mm, hPa)
2. **Language parameter**
- `?lang=us` - Use imperial units (for us.wttr.in subdomain)
3. **Client IP geolocation**
- Requests from US IP addresses automatically use imperial units
- Uses GeoIP database to detect country code
4. **Default**
- Metric units for all other cases
## Implementation Details
### Code Changes
1. **GeoIP Module** (`src/location/geoip.zig`)
- Added `isUSIP()` method to detect US IP addresses
- Queries MaxMind database for country ISO code
- Returns `true` if country code is "US"
2. **Handler** (`src/http/handler.zig`)
- Added `geoip` to `HandleWeatherOptions`
- Implements priority logic for unit selection
- Extracts client IP from headers (X-Forwarded-For, X-Real-IP)
- Passes `use_imperial` flag to all renderers
3. **Renderers** (all updated to support imperial units)
- `ansi.zig` - Shows °F instead of °C
- `line.zig` - Shows °F and mph for formats 1-4
- `custom.zig` - Converts all units (temp, wind, precip, pressure)
- `v2.zig` - Shows imperial units in detailed format
- `json.zig` - Already outputs both units (no changes needed)
### Conversions
- Temperature: `°F = °C × 9/5 + 32`
- Wind speed: `mph = km/h × 0.621371`
- Precipitation: `inches = mm × 0.0393701`
- Pressure: `inHg = hPa × 0.02953`
## Testing
### Unit Tests
All renderers have unit tests verifying imperial units:
- `test "render with imperial units"` in ansi.zig
- `test "format 2 with imperial units"` in line.zig
- `test "render custom format with imperial units"` in custom.zig
- `test "render v2 format with imperial units"` in v2.zig
- `test "imperial units selection logic"` in handler.zig
- `test "isUSIP detects US IPs"` in geoip.zig
### Integration Testing
```bash
# Test with lang=us
curl "http://localhost:8002/London?lang=us&format=2"
# Output: 51.5074,-0.1278: ☁️ 54°F 🌬SW19mph
# Test with explicit ?u
curl "http://localhost:8002/London?format=2&u"
# Output: 51.5074,-0.1278: ☁️ 54°F 🌬SW19mph
# Test metric override
curl "http://localhost:8002/London?lang=us&format=2&m"
# Output: 51.5074,-0.1278: ☁️ 12°C 🌬SW30km/h
# Test from US IP (automatic detection)
curl -H "X-Forwarded-For: 1.1.1.1" "http://localhost:8002/London?format=2"
# Output: Uses imperial as 1.1.1.1 is detected as US IP
```
## Documentation Updates
- **API_ENDPOINTS.md** - Added "Unit System Defaults" section explaining priority
- **README.md** - Updated "Implemented Features" to mention auto-detection
- **IMPERIAL_UNITS.md** - This document
## Compatibility
This implementation matches the original Python behavior in `lib/parse_query.py`:
- Same priority order
- Same detection logic
- Same unit conversions
- Compatible with existing clients and scripts

View file

@ -1,34 +0,0 @@
# Missing Features
Features not yet implemented in the Zig version:
## 1. Prometheus Metrics Format (format=p1)
- Export weather data in Prometheus metrics format
- See API_ENDPOINTS.md for format specification
## 2. PNG Generation
- Render weather reports as PNG images
- Support transparency and custom styling
- Requires image rendering library integration
## 3. Multiple Locations Support
- Handle colon-separated locations (e.g., `London:Paris:Berlin`)
- Process and display weather for multiple cities in one request
## 4. Language/Localization
- Accept-Language header parsing
- lang query parameter support
- Translation of weather conditions and text (54 languages)
## 5. Moon Phase Calculation
- Real moon phase computation based on date
- Moon phase emoji display
- Moonday calculation
## 6. Astronomical Times
- Calculate dawn, sunrise, zenith, sunset, dusk times
- Based on location coordinates and timezone
- Display in custom format output
## 7. Json output
- Does not match wttr.in format

420
README.md
View file

@ -1,268 +1,200 @@
# wttr — console-oriented weather forecast service # wttr.in Zig Rewrite Documentation
*wttr* is a console-oriented weather forecast service written in Zig, based on [wttr.in](https://wttr.in). This directory contains comprehensive documentation for rewriting wttr.in in Zig.
wttr supports various information representation methods like terminal-oriented ## Quick Start
ANSI-sequences for console HTTP clients (curl, httpie, or wget), HTML for web
browsers, or PNG for graphical viewers.
## Usage
You can access the service from a shell or from a Web browser:
```bash ```bash
$ curl localhost:8002 # Minimal setup (uses defaults)
Weather for City: Paris, France ./wttr
\ / Clear # With IP2Location fallback (optional)
.-. 10 11 °C IP2LOCATION_API_KEY=your_key_here ./wttr
― ( ) ― ↑ 11 km/h
`-' 10 km # Custom cache location
/ \ 0.0 mm WTTR_CACHE_DIR=/var/cache/wttr ./wttr
``` ```
Or in PowerShell: See [CACHE_CONFIGURATION.md](CACHE_CONFIGURATION.md) for detailed configuration options.
```powershell ## External Services & API Keys
Invoke-RestMethod http://localhost:8002
```
Get weather for a specific location by adding it to the URL: ### Required Services (No API Key)
- **Met.no Weather API** - Primary weather data provider
```bash - Free, open API from Norwegian Meteorological Institute
$ curl localhost:8002/London - No registration required
$ curl localhost:8002/Moscow - Rate limit: Be respectful, use caching (built-in)
$ curl localhost:8002/Salt+Lake+City
```
If you omit the location name, you'll get the report for your current location based on your IP address.
Use 3-letter airport codes to get weather at a specific airport:
```bash
$ curl localhost:8002/muc # Munich International Airport
$ curl localhost:8002/ham # Hamburg Airport
```
Look up special locations (attractions, mountains, etc.) by prefixing with `~`:
```bash
$ curl localhost:8002/~Vostok+Station
$ curl localhost:8002/~Eiffel+Tower
$ curl localhost:8002/~Kilimanjaro
```
You can also use IP addresses or domain names (prefixed with `@`):
```bash
$ curl localhost:8002/@github.com
$ curl localhost:8002/@msu.ru
```
To get detailed information, access the help page:
```bash
$ curl localhost:8002/:help
```
## Weather Units
By default the USCS units are used for the queries from the USA and the metric
system for the rest of the world. You can override this behavior by adding `?u`
or `?m` to a URL:
```bash
$ curl localhost:8002/Amsterdam?u # USCS (used by default in US)
$ curl localhost:8002/Amsterdam?m # metric (SI) (used by default everywhere except US)
```
## One-line output
For one-line output format, specify the `format` parameter:
```bash
$ curl localhost:8002/Nuremberg?format=3
Nuremberg: 🌦 +11⁰C
```
Available preconfigured formats: 1, 2, 3, 4 and custom format using percent notation.
* `format=1`: `🌦 +11⁰C`
* `format=2`: `🌦 🌡️+11°C 🌬↓4km/h`
* `format=3`: `Nuremberg: 🌦 +11⁰C`
* `format=4`: `Nuremberg: 🌦 🌡️+11°C 🌬↓4km/h`
You can specify multiple locations separated with `:`:
**Note:** Not yet fully implemented - see [MISSING_FEATURES.md](MISSING_FEATURES.md)
```bash
$ curl localhost:8002/Nuremberg:Hamburg:Berlin?format=3
Nuremberg: 🌦 +11⁰C
Hamburg: 🌦 +8⁰C
Berlin: 🌦 +8⁰C
```
Or process them all at once:
```bash
$ curl -s 'localhost:8002/{Nuremberg,Hamburg,Berlin}?format=3'
```
To specify your own custom output format, use the special `%`-notation:
```
c Weather condition emoji
C Weather condition textual name
h Humidity
t Temperature (Actual)
f Temperature (Feels Like)
w Wind
l Location
m Moon phase 🌑🌒🌓🌔🌕🌖🌗🌘
M Moon day
p Precipitation (mm/3 hours)
P Pressure (hPa)
D Dawn
S Sunrise
z Zenith
s Sunset
d Dusk
```
Example:
```bash
$ curl localhost:8002/London?format="%l:+%c+%t"
London: ⛅️ +7⁰C
```
## Data-rich output format (v2)
**Note:** Not yet fully implemented - see [MISSING_FEATURES.md](MISSING_FEATURES.md)
In the data-rich output format (view code `v2`), additional weather and astronomical information is available:
```bash
$ curl localhost:8002/München?format=v2
```
## JSON output
To fetch information in JSON format:
**Note:** Not yet fully implemented - see [MISSING_FEATURES.md](MISSING_FEATURES.md)
```bash
$ curl localhost:8002/Detroit?format=j1
```
The result will look like:
```json
{
"current_condition": [{
"FeelsLikeC": "25",
"FeelsLikeF": "76",
"cloudcover": "100",
"humidity": "76",
"temp_C": "22",
"temp_F": "72",
"weatherCode": "122",
"weatherDesc": [{"value": "Overcast"}],
"windspeedKmph": "7"
}],
...
}
```
## Prometheus Metrics Output
**Note:** Not yet implemented - see [MISSING_FEATURES.md](MISSING_FEATURES.md)
To fetch information in Prometheus format:
```bash
$ curl localhost:8002/Detroit?format=p1
```
A possible Prometheus configuration:
```yaml
- job_name: 'wttr_in_detroit'
static_configs:
- targets: ['localhost:8002']
metrics_path: '/Detroit'
params:
format: ['p1']
```
## Building
```bash
# Debug build
zig build
# Release build
zig build -Doptimize=ReleaseSafe
# Run tests
zig build test
# Run
./zig-out/bin/wttr
```
## Configuration
All configuration is via environment variables:
| Variable | Default | Description |
|----------|---------|-------------|
| `WTTR_LISTEN_HOST` | `0.0.0.0` | Listen address |
| `WTTR_LISTEN_PORT` | `8002` | Listen port |
| `WTTR_CACHE_DIR` | `~/.cache/wttr` | Cache directory |
| `WTTR_CACHE_SIZE` | `10000` | Max cached responses |
| `WTTR_GEOLITE_PATH` | `~/.cache/wttr/GeoLite2-City.mmdb` | GeoIP database path |
| `IP2LOCATION_API_KEY` | (none) | Optional IP geolocation fallback |
## External Services
### Required (No API Key)
- **Met.no Weather API** - Weather data provider (free, no registration)
- Set METNO_TOS_IDENTIFYING_EMAIL to identify yourself to Met.No
### Optional Services
- **IP2Location.io** - Fallback IP geolocation - **IP2Location.io** - Fallback IP geolocation
- Sign up at https://www.ip2location.io/ - **API Key Required:** Sign up at https://www.ip2location.io/
- Free tier: 30,000 requests/month - Free tier: 30,000 requests/month
- Only used when MaxMind GeoIP database lookup fails
- Set via `IP2LOCATION_API_KEY` environment variable - Set via `IP2LOCATION_API_KEY` environment variable
### Auto-Downloaded ### Database Files (Auto-Downloaded)
- **MaxMind GeoLite2 City** - IP geolocation database - **MaxMind GeoLite2 City** - IP geolocation database
- Downloaded automatically if missing - Free database, automatically downloaded if missing
- Stored in `~/.cache/wttr/GeoLite2-City.mmdb` - No API key required
- Stored in `~/.cache/wttr/GeoLite2-City.mmdb` by default
## Docker ## Current Implementation Status
Prebuilt images are available from https://git.lerch.org/lobo/-/packages/container/wttr/latest ### Implemented Features
and can be pulled with `docker pull git.lerch.org/lobo/wttr:latest` or - HTTP server with routing and middleware
`docker pull git.lerch.org/lobo/wttr:<shortsha>` for a specific revision - Rate limiting
- Caching system
- GeoIP database integration (libmaxminddb)
- Location resolver with multiple input types
- Weather provider (Met.no) with timezone-aware forecasts; core data structures use zeit.Time/zeit.Date for type-safe date/time handling
- Output formats: ANSI, line (1-4), JSON (j1), v2 data-rich, custom (%)
- Query parameter parsing
- Static help pages (/:help, /:translation)
- Error handling (404/500 status codes)
- Configuration from environment variables
- **Imperial units auto-detection**: Automatically uses imperial units (°F, mph) for US IP addresses and `lang=us`, with explicit `?u` and `?m` overrides
- **IP2Location fallback**: Optional fallback geolocation service with persistent cache
### Missing Features (To Be Implemented Later)
```bash 1. **Prometheus Metrics Format** (format=p1)
# Build image. Note the binary must be in the directory - Export weather data in Prometheus metrics format
docker build -t wttr -f docker/Dockerfile . - See API_ENDPOINTS.md for format specification
# Run container 2. **PNG Generation**
docker run -p 8002:8002 wttr - Render weather reports as PNG images
``` - Support transparency and custom styling
- Requires image rendering library integration
## Documentation 3. **Multiple Locations Support**
- Handle colon-separated locations (e.g., `London:Paris:Berlin`)
- Process and display weather for multiple cities in one request
- [ARCHITECTURE.md](ARCHITECTURE.md) - System architecture and design 4. **Language/Localization**
- [MISSING_FEATURES.md](MISSING_FEATURES.md) - Features not yet implemented - Accept-Language header parsing
- lang query parameter support
- Translation of weather conditions and text (54 languages)
## License 5. **Moon Phase Calculation**
- Real moon phase computation based on date
- Moon phase emoji display
- Moonday calculation
Apache v2, matching wttr.in project from which this is derived. See [LICENSE](LICENSE) 6. **Astronomical Times**
- Calculate dawn, sunrise, zenith, sunset, dusk times
- Based on location coordinates and timezone
- Display in custom format output
## Documentation Files
### [CACHE_CONFIGURATION.md](CACHE_CONFIGURATION.md) ⭐ NEW
Complete cache and external services documentation:
- External services (Met.no, IP2Location, MaxMind GeoLite2)
- API key requirements
- Cache locations and policies
- Expiration and eviction strategies
- Environment variables
- Configuration examples
### [TARGET_ARCHITECTURE.md](TARGET_ARCHITECTURE.md)
Target architecture for Zig rewrite:
- Single binary design
- Simplified caching (one layer)
- Pluggable weather provider interface
- Rate limiting middleware
- Module structure
- Testing strategy
- Performance targets
### [ARCHITECTURE.md](ARCHITECTURE.md)
Current system architecture documentation:
- Component breakdown (Go proxy, Python backend, static assets)
- Request flow diagrams
- API endpoints
- External dependencies
- Caching strategy
- Configuration
- Known issues
### [API_ENDPOINTS.md](API_ENDPOINTS.md)
Complete API reference:
- All endpoints with examples
- Query parameters
- Output formats
- Language support
- HTTP headers
- Rate limits
- Usage examples
### [DATA_FLOW.md](DATA_FLOW.md)
Detailed request processing flow:
- Step-by-step request handling
- Location resolution process
- Weather data fetching
- Rendering pipeline
- Caching mechanisms
- Error handling
- Performance optimizations
### [REWRITE_STRATEGY.md](REWRITE_STRATEGY.md)
Comprehensive rewrite plan:
- Goals and non-goals
- Phase-by-phase breakdown (10-11 weeks)
- Module organization
- Testing strategy
- Risk mitigation
- Success criteria
- Alternative approaches
## Quick Start
1. **Read ARCHITECTURE.md** - Understand the current system
2. **Read DATA_FLOW.md** - Understand how requests are processed
3. **Read REWRITE_STRATEGY.md** - Understand the rewrite plan
4. **Refer to API_ENDPOINTS.md** - When implementing specific endpoints
## Current System Summary
**Architecture:** Hybrid Python/Go
- Go proxy (port 8082): LRU caching, prefetching
- Python backend (port 8002): Weather logic, rendering
- External binaries: wego (Go), pyphoon (Python)
- External services: Geolocator (port 8004)
**Key Stats:**
- ~5000 lines of Python
- ~400 lines of Go
- 54 languages supported
- 12,800 entry LRU cache
- 1000-2000s cache TTL
- Zero unit tests (only integration tests)
**Main Challenges:**
1. ANSI weather rendering (currently done by wego)
2. PNG image generation (PIL + pyte)
3. GeoIP database parsing (MaxMind format)
4. 54 language translations
5. Multiple output formats (ANSI, HTML, PNG, JSON, Prometheus)
## Recommended Approach
**Option A: Incremental (Recommended)**
1. Replace Go proxy with Zig (2 weeks)
2. Test and deploy
3. Replace Python backend with Zig (8-9 weeks)
4. Full cutover
**Option B: Full Rewrite**
- All at once (10-11 weeks)
- Higher risk, but cleaner result
## Next Steps
1. Review documentation
2. Choose rewrite strategy
3. Set up Zig project structure
4. Begin implementation (start with HTTP server or Go proxy)
## Questions?
See REWRITE_STRATEGY.md "Questions to Answer" section for key decisions needed.

714
REWRITE_STRATEGY.md Normal file
View file

@ -0,0 +1,714 @@
# wttr.in Zig Rewrite Strategy
## Goals
1. **Single binary** - Replace Python + Go with one Zig executable
2. **No external binaries** - Eliminate wego, pyphoon dependencies
3. **Maintain compatibility** - All existing API endpoints work identically
4. **Improve performance** - Faster response times, lower memory usage
5. **Add tests** - Comprehensive test coverage from day one
6. **Simplify deployment** - Single binary + data files
## Non-Goals
- Rewriting weather APIs (still use met.no/WWO)
- Changing API surface (maintain backward compatibility)
- Rewriting geolocator service (keep as separate service for now)
## Phase 1: Analysis & Design ✓
**Status:** Complete (this document)
**Deliverables:**
- [x] ARCHITECTURE.md - System overview
- [x] API_ENDPOINTS.md - API reference
- [x] DATA_FLOW.md - Request processing flow
- [x] REWRITE_STRATEGY.md - This document
## Phase 2: Foundation (Week 1-2)
### 2.1 Project Setup
**Tasks:**
- [ ] Create `build.zig` with proper structure
- [ ] Set up module organization
- [ ] Configure dependencies (if any)
- [ ] Set up test framework
- [ ] Create CI/CD pipeline (GitHub Actions)
**Modules:**
```
src/
├── main.zig # Entry point
├── server.zig # HTTP server
├── router.zig # Request routing
├── config.zig # Configuration
├── cache/
│ ├── lru.zig # LRU cache implementation
│ └── file.zig # File-backed cache
├── location/
│ ├── resolver.zig # Location resolution
│ ├── geoip.zig # GeoIP lookups
│ └── normalize.zig # Location normalization
├── weather/
│ ├── client.zig # Weather API client
│ ├── metno.zig # met.no API
│ └── wwo.zig # WorldWeatherOnline API
├── render/
│ ├── ansi.zig # ANSI rendering
│ ├── html.zig # HTML rendering
│ ├── json.zig # JSON rendering
│ ├── png.zig # PNG rendering
│ ├── line.zig # One-line format
│ └── v2.zig # v2 format
├── i18n/
│ ├── translations.zig # Translation system
│ └── loader.zig # Load translation files
└── utils/
├── http.zig # HTTP utilities
├── ip.zig # IP address handling
└── time.zig # Time utilities
```
### 2.2 Core HTTP Server
**Tasks:**
- [ ] Implement HTTP server using std.http.Server
- [ ] Request parsing (headers, query params, path)
- [ ] Response building (status, headers, body)
- [ ] Basic routing (exact match, wildcard)
- [ ] Static file serving
**Tests:**
- [ ] Parse GET requests
- [ ] Parse query parameters
- [ ] Route to handlers
- [ ] Serve static files
- [ ] Handle 404s
**Acceptance Criteria:**
- Server listens on configurable port
- Handles concurrent requests
- Routes to placeholder handlers
- Serves files from share/ directory
### 2.3 Configuration System
**Tasks:**
- [ ] Load environment variables
- [ ] Load config files (if needed)
- [ ] Validate configuration
- [ ] Provide defaults
**Configuration:**
```zig
const Config = struct {
listen_host: []const u8,
listen_port: u16,
geolite_path: []const u8,
cache_dir: []const u8,
log_level: LogLevel,
// API keys
ip2location_key: ?[]const u8,
ipinfo_token: ?[]const u8,
wwo_key: ?[]const u8,
};
```
**Tests:**
- [ ] Load from environment
- [ ] Apply defaults
- [ ] Validate paths exist
## Phase 3: Caching Layer (Week 2-3)
### 3.1 LRU Cache
**Tasks:**
- [ ] Implement generic LRU cache
- [ ] Thread-safe operations (Mutex)
- [ ] TTL support
- [ ] Eviction policy
- [ ] Cache statistics
**Tests:**
- [ ] Insert and retrieve
- [ ] LRU eviction
- [ ] TTL expiration
- [ ] Concurrent access
- [ ] Memory limits
**Acceptance Criteria:**
- Configurable size (default 12,800)
- O(1) get/put operations
- Thread-safe
- TTL-based expiration
### 3.2 File Cache
**Tasks:**
- [ ] Store large responses to disk
- [ ] MD5 hash for filenames
- [ ] Read/write with proper locking
- [ ] Cleanup old files
**Tests:**
- [ ] Write and read files
- [ ] Handle binary data
- [ ] Concurrent access
- [ ] Disk space limits
### 3.3 Cache Integration
**Tasks:**
- [ ] Generate cache keys
- [ ] Check cache before processing
- [ ] Store responses after processing
- [ ] Handle InProgress flag (prevent thundering herd)
**Tests:**
- [ ] Cache hit returns cached response
- [ ] Cache miss processes request
- [ ] Concurrent requests for same key
- [ ] Cache key generation
## Phase 4: Location Resolution (Week 3-4)
### 4.1 Location Parsing
**Tasks:**
- [ ] Parse location from URL
- [ ] Normalize location names
- [ ] Handle special prefixes (~, @)
- [ ] Parse cyclic locations (:)
- [ ] Load aliases file
- [ ] Load blacklist file
**Tests:**
- [ ] Parse city names
- [ ] Parse coordinates
- [ ] Parse IATA codes
- [ ] Parse special locations (Moon, etc.)
- [ ] Normalize names
- [ ] Apply aliases
- [ ] Check blacklist
### 4.2 GeoIP Lookup
**Tasks:**
- [ ] Read MaxMind GeoLite2 database
- [ ] IP to location lookup
- [ ] Cache results
**Options:**
- Use C library (libmaxminddb) via @cImport
- Or: Parse MMDB format in pure Zig
**Tests:**
- [ ] Lookup IPv4 address
- [ ] Lookup IPv6 address
- [ ] Handle not found
- [ ] Cache lookups
### 4.3 External Geolocation
**Tasks:**
- [ ] HTTP client for geolocator service
- [ ] Parse JSON responses
- [ ] Error handling
**Tests:**
- [ ] Query geolocator
- [ ] Parse response
- [ ] Handle errors
- [ ] Timeout handling
### 4.4 IP Location APIs
**Tasks:**
- [ ] IP2Location API client
- [ ] IPInfo API client
- [ ] File-based caching
- [ ] Fallback chain
**Tests:**
- [ ] Query each API
- [ ] Parse responses
- [ ] Cache results
- [ ] Fallback on error
## Phase 5: Weather Data (Week 4-5)
### 5.1 HTTP Client
**Tasks:**
- [ ] Generic HTTP client
- [ ] Connection pooling
- [ ] Timeout handling
- [ ] Retry logic
- [ ] User-Agent setting
**Tests:**
- [ ] GET requests
- [ ] POST requests
- [ ] Headers
- [ ] Timeouts
- [ ] Retries
### 5.2 met.no Client
**Tasks:**
- [ ] API endpoint construction
- [ ] Parse XML/JSON responses
- [ ] Transform to internal format
- [ ] Error handling
**Tests:**
- [ ] Fetch weather data
- [ ] Parse response
- [ ] Handle API errors
- [ ] Handle rate limits
### 5.3 WorldWeatherOnline Client
**Tasks:**
- [ ] API endpoint construction
- [ ] Parse JSON responses
- [ ] Transform to internal format
- [ ] Caching (proxy cache)
**Tests:**
- [ ] Fetch weather data
- [ ] Parse response
- [ ] Handle API errors
- [ ] Cache responses
### 5.4 Weather Data Model
**Tasks:**
- [ ] Define internal weather data structure
- [ ] Conversion from met.no format
- [ ] Conversion from WWO format
- [ ] JSON serialization
**Tests:**
- [ ] Convert met.no data
- [ ] Convert WWO data
- [ ] Serialize to JSON
- [ ] Validate data
## Phase 6: Rendering (Week 5-7)
### 6.1 ANSI Renderer
**Tasks:**
- [ ] Generate ANSI weather report
- [ ] ASCII art for weather conditions
- [ ] Color codes
- [ ] Box drawing characters
- [ ] Temperature graphs
- [ ] Wind direction arrows
**Tests:**
- [ ] Render current weather
- [ ] Render forecast
- [ ] Apply colors
- [ ] Handle narrow mode
- [ ] Handle inverted colors
**Acceptance Criteria:**
- Output matches wego format
- All weather conditions supported
- Configurable width
### 6.2 One-Line Renderer
**Tasks:**
- [ ] Parse format string
- [ ] Replace format codes (%c, %t, etc.)
- [ ] Handle predefined formats (1-4)
- [ ] Emoji support
**Tests:**
- [ ] Format 1-4
- [ ] Custom format strings
- [ ] All format codes
- [ ] Multiple locations
### 6.3 JSON Renderer
**Tasks:**
- [ ] Serialize weather data to JSON
- [ ] Match WWO API format
- [ ] Pretty printing
**Tests:**
- [ ] Serialize current conditions
- [ ] Serialize forecast
- [ ] Match expected format
### 6.4 HTML Renderer
**Tasks:**
- [ ] Convert ANSI to HTML
- [ ] Apply CSS styling
- [ ] Add interactive buttons
- [ ] Template system
**Tests:**
- [ ] Convert ANSI codes
- [ ] Apply colors
- [ ] Render buttons
- [ ] Template rendering
### 6.5 PNG Renderer
**Tasks:**
- [ ] Render ANSI to image
- [ ] Font rendering
- [ ] Color support
- [ ] Transparency
**Options:**
- Use C library (libpng, freetype) via @cImport
- Or: Pure Zig implementation (more work)
**Tests:**
- [ ] Render text
- [ ] Apply colors
- [ ] Apply transparency
- [ ] Handle fonts
### 6.6 v2 Renderer
**Tasks:**
- [ ] Data-rich format
- [ ] Temperature graphs
- [ ] Moon phases
- [ ] Astronomical times
**Tests:**
- [ ] Render all sections
- [ ] Handle different locations
- [ ] Match expected format
### 6.7 Prometheus Renderer
**Tasks:**
- [ ] Convert weather data to metrics
- [ ] Prometheus text format
- [ ] Metric naming
**Tests:**
- [ ] Render metrics
- [ ] Match Prometheus format
- [ ] All weather fields
## Phase 7: Translation System (Week 7-8)
### 7.1 Translation Loader
**Tasks:**
- [ ] Load translation files
- [ ] Parse translation format
- [ ] Build translation tables
- [ ] Language detection
**Tests:**
- [ ] Load all language files
- [ ] Parse translations
- [ ] Detect language from header
- [ ] Detect language from subdomain
### 7.2 Message Translation
**Tasks:**
- [ ] Translate weather conditions
- [ ] Translate UI messages
- [ ] Fallback to English
**Tests:**
- [ ] Translate conditions
- [ ] Translate messages
- [ ] Handle missing translations
- [ ] Fallback logic
## Phase 8: Request Processing (Week 8-9)
### 8.1 Query Parser
**Tasks:**
- [ ] Parse query parameters
- [ ] Parse single-letter options
- [ ] Parse PNG filenames
- [ ] Serialize/deserialize state
**Tests:**
- [ ] Parse all options
- [ ] Parse PNG filenames
- [ ] Serialize state
- [ ] Deserialize state
### 8.2 Request Handler
**Tasks:**
- [ ] Main request handler
- [ ] Fast path (cache + static)
- [ ] Full path (location + weather + render)
- [ ] Error handling
- [ ] Rate limiting
**Tests:**
- [ ] Handle cached requests
- [ ] Handle static pages
- [ ] Handle weather queries
- [ ] Handle errors
- [ ] Enforce rate limits
### 8.3 Rate Limiting
**Tasks:**
- [ ] Per-IP counters
- [ ] Time buckets (minute, hour, day)
- [ ] Whitelist support
- [ ] Return 429 on limit
**Tests:**
- [ ] Count requests
- [ ] Enforce limits
- [ ] Reset counters
- [ ] Whitelist bypass
## Phase 9: Integration & Testing (Week 9-10)
### 9.1 End-to-End Tests
**Tasks:**
- [ ] Test all API endpoints
- [ ] Test all output formats
- [ ] Test all query options
- [ ] Test error cases
- [ ] Compare with Python output
**Test Cases:**
- [ ] Basic weather query
- [ ] Location resolution
- [ ] All output formats
- [ ] All languages
- [ ] PNG rendering
- [ ] Rate limiting
- [ ] Error handling
### 9.2 Performance Testing
**Tasks:**
- [ ] Benchmark request latency
- [ ] Benchmark throughput
- [ ] Memory profiling
- [ ] Cache hit rates
- [ ] Compare with Python/Go
**Metrics:**
- Requests per second
- Average latency
- P95/P99 latency
- Memory usage
- Cache hit rate
### 9.3 Compatibility Testing
**Tasks:**
- [ ] Run integration test (test/query.sh)
- [ ] Compare SHA1 hashes
- [ ] Fix any differences
- [ ] Document intentional changes
## Phase 10: Deployment (Week 10-11)
### 10.1 Packaging
**Tasks:**
- [ ] Build release binary
- [ ] Package data files
- [ ] Create Docker image
- [ ] Write deployment docs
### 10.2 Migration Plan
**Tasks:**
- [ ] Deploy Zig version alongside Python/Go
- [ ] Route small percentage of traffic
- [ ] Monitor errors and performance
- [ ] Gradually increase traffic
- [ ] Full cutover
**Rollback Plan:**
- Keep Python/Go running
- Route traffic back if issues
- Fix issues in Zig version
- Retry migration
### 10.3 Documentation
**Tasks:**
- [ ] Update README
- [ ] Installation instructions
- [ ] Configuration guide
- [ ] API documentation
- [ ] Development guide
## Risks & Mitigations
### Risk: PNG Rendering Complexity
**Mitigation:**
- Start with external library (libpng, freetype)
- Consider pure Zig later if needed
- Or: Keep PNG rendering in separate service
### Risk: GeoIP Database Parsing
**Mitigation:**
- Use C library (libmaxminddb) initially
- Consider pure Zig parser later
- Well-documented format
### Risk: ANSI Rendering Differences
**Mitigation:**
- Extensive testing against wego output
- Pixel-perfect comparison for PNG
- Accept minor differences if functionally equivalent
### Risk: Performance Regression
**Mitigation:**
- Benchmark early and often
- Profile hot paths
- Optimize critical sections
- Compare with Python/Go baseline
### Risk: Translation File Parsing
**Mitigation:**
- Simple format (key: value)
- Robust parser with error handling
- Validate all files on startup
### Risk: Weather API Changes
**Mitigation:**
- Abstract API clients
- Version API responses
- Monitor for changes
- Fallback to other API
## Success Criteria
### Functional
- [ ] All API endpoints work
- [ ] All output formats match
- [ ] All languages supported
- [ ] All query options work
- [ ] Integration tests pass
### Performance
- [ ] Latency < Python/Go
- [ ] Throughput > Python/Go
- [ ] Memory < Python/Go
- [ ] Binary size < 10MB
### Quality
- [ ] >80% test coverage
- [ ] No memory leaks
- [ ] No crashes under load
- [ ] Clean error handling
### Operational
- [ ] Single binary deployment
- [ ] No external dependencies (except data files)
- [ ] Easy configuration
- [ ] Good logging
- [ ] Metrics/monitoring
## Timeline
**Total: 10-11 weeks**
- Week 1-2: Foundation
- Week 2-3: Caching
- Week 3-4: Location
- Week 4-5: Weather
- Week 5-7: Rendering
- Week 7-8: Translation
- Week 8-9: Integration
- Week 9-10: Testing
- Week 10-11: Deployment
**Milestones:**
1. **Week 2:** HTTP server running, basic routing
2. **Week 4:** Location resolution working
3. **Week 6:** Weather data fetching working
4. **Week 8:** ANSI rendering working
5. **Week 10:** All features complete, testing done
6. **Week 11:** Deployed to production
## Alternative: Incremental Approach
Instead of full rewrite, replace components incrementally:
### Option A: Replace Go Proxy First
1. Rewrite Go proxy in Zig (Week 1-2)
2. Keep Python backend
3. Test and deploy
4. Then rewrite Python backend (Week 3-11)
**Advantages:**
- Smaller initial scope
- Faster time to value
- De-risks Zig HTTP handling
- Can abandon if issues
### Option B: Replace Python Backend First
1. Keep Go proxy
2. Rewrite Python backend in Zig (Week 1-10)
3. Test and deploy
4. Then replace Go proxy (Week 11)
**Advantages:**
- Most complexity in backend
- Proves out rendering logic
- Can keep Go proxy if it works well
### Recommendation: Option A
Start with Go proxy replacement:
- Smaller scope (400 lines vs 5000+)
- Clear interface boundary
- Tests caching/HTTP in Zig
- Quick win (2 weeks)
- De-risks larger rewrite
## Next Steps
1. **Review this document** - Get feedback on approach
2. **Choose strategy** - Full rewrite vs incremental
3. **Set up project** - Create build.zig, directory structure
4. **Start coding** - Begin with HTTP server or Go proxy replacement
5. **Iterate** - Build, test, refine
## Questions to Answer
- [ ] Use C libraries (libpng, freetype, libmaxminddb) or pure Zig?
- [ ] Keep geolocator as separate service or integrate?
- [ ] Keep wego/pyphoon or rewrite rendering?
- [ ] Full rewrite or incremental replacement?
- [ ] Target Zig version (0.11, 0.12, 0.13)?
- [ ] Async I/O strategy (std.event, manual, blocking)?

988
TARGET_ARCHITECTURE.md Normal file
View file

@ -0,0 +1,988 @@
# wttr.in Target Architecture (Zig)
## Overview
Single Zig binary (`wttr`) with simplified architecture:
- One HTTP server (karlseguin/http.zig)
- One caching layer (LRU + file-backed)
- Pluggable weather provider interface
- Rate limiting as injectable middleware
- English-only (i18n can be added later)
- No cron-based prefetching (operational concern)
## System Diagram
```
Client Request
HTTP Server (http.zig)
Rate Limiter (middleware)
Router
Request Handler
Location Resolver
Provider interface cache check
↓ (miss)
Weather Provider (interface)
├─ MetNo (default)
└─ Mock (tests)
Renderer
├─ ANSI
├─ JSON
├─ Line
└─ (PNG/HTML later)
Cache Store
Response
```
## Module Structure
```
wttr/
├── build.zig
├── build.zig.zon
└── src/
├── main.zig # Entry point, server setup
├── config.zig # Configuration
├── http/
│ ├── server.zig # HTTP server wrapper
│ ├── router.zig # Route matching
│ ├── handler.zig # Request handlers
│ └── middleware.zig # Rate limiter (Token Bucket)
├── cache/
│ ├── cache.zig # Cache interface
│ ├── lru.zig # In-memory LRU
│ └── file.zig # File-backed storage
├── location/
│ ├── resolver.zig # Main location resolution
│ ├── geoip.zig # GeoIP wrapper (libmaxminddb)
│ ├── normalize.zig # Location normalization
│ └── geolocator.zig # HTTP client for geolocator service
├── weather/
│ ├── provider.zig # Weather provider interface
│ ├── metno.zig # met.no implementation
│ ├── mock.zig # Mock for tests
│ └── types.zig # Weather data structures
├── render/
│ ├── ansi.zig # ANSI terminal output
│ ├── json.zig # JSON output
│ └── line.zig # One-line format
└── util/
├── ip.zig # IP address utilities
└── hash.zig # Hashing utilities
```
## Core Components
### 1. HTTP Server (main.zig, http/)
**Responsibilities:**
- Listen on configured port
- Parse HTTP requests
- Route to handlers
- Apply middleware (rate limiting)
- Return responses
**Dependencies:**
- karlseguin/http.zig
**Configuration:**
```zig
const Config = struct {
listen_host: []const u8 = "0.0.0.0",
listen_port: u16 = 8002,
cache_size: usize = 10_000,
cache_dir: []const u8 = "/tmp/wttr-cache",
geolite_path: []const u8 = "./GeoLite2-City.mmdb",
geolocator_url: []const u8 = "http://localhost:8004",
};
```
**Routes:**
```zig
GET / → weather for IP location
GET /{location} → weather for location
GET /:help → help page
GET /files/{path} → static files
```
### 2. Rate Limiter (http/middleware.zig)
**Algorithm:** Token Bucket
**Responsibilities:**
- Track token buckets per IP
- Refill tokens at configured rate
- Consume tokens on request
- Return 429 when bucket empty
- Injectable as middleware
**Interface:**
```zig
pub const RateLimiter = struct {
buckets: HashMap([]const u8, TokenBucket),
config: RateLimitConfig,
allocator: Allocator,
pub fn init(allocator: Allocator, config: RateLimitConfig) !RateLimiter;
pub fn check(self: *RateLimiter, ip: []const u8) !void;
pub fn middleware(self: *RateLimiter) Middleware;
pub fn deinit(self: *RateLimiter) void;
};
pub const RateLimitConfig = struct {
capacity: u32 = 300, // Max tokens in bucket
refill_rate: u32 = 5, // Tokens per second
refill_interval_ms: u64 = 200, // Refill every 200ms (5 tokens/sec)
};
const TokenBucket = struct {
tokens: f64,
capacity: u32,
last_refill: i64, // Unix timestamp (ms)
fn refill(self: *TokenBucket, now: i64, rate: u32, interval_ms: u64) void {
const elapsed = now - self.last_refill;
const intervals = @as(f64, @floatFromInt(elapsed)) / @as(f64, @floatFromInt(interval_ms));
const new_tokens = intervals * @as(f64, @floatFromInt(rate));
self.tokens = @min(
self.tokens + new_tokens,
@as(f64, @floatFromInt(self.capacity))
);
self.last_refill = now;
}
fn consume(self: *TokenBucket, count: f64) bool {
if (self.tokens >= count) {
self.tokens -= count;
return true;
}
return false;
}
};
```
**Implementation:**
- HashMap of IP → TokenBucket
- Each request consumes 1 token
- Tokens refill at configured rate (default: 5/second)
- Bucket capacity: 300 tokens (allows bursts)
- Periodic cleanup of old buckets (not accessed in 1 hour)
**Example Configuration:**
```zig
// Allow 300 requests burst, then 5 req/sec sustained
const config = RateLimitConfig{
.capacity = 300,
.refill_rate = 5,
.refill_interval_ms = 200,
};
```
### 3. Cache (cache/)
**Single-layer cache with two storage backends:**
**Interface:**
```zig
pub const Cache = struct {
lru: LRU,
file_store: FileStore,
pub fn init(allocator: Allocator, config: CacheConfig) !Cache;
pub fn get(self: *Cache, key: []const u8) ?[]const u8;
pub fn put(self: *Cache, key: []const u8, value: []const u8, ttl: u64) !void;
};
pub const CacheConfig = struct {
max_entries: usize = 10_000,
file_threshold: usize = 1024, // Store to file if > 1KB
cache_dir: []const u8,
};
```
**Cache Key:**
```
{user_agent}:{path}:{query}:{client_ip}
```
**Storage Strategy:**
- Small responses (<1KB): In-memory LRU only
- Large responses (≥1KB): LRU stores file path, data on disk
- TTL: 1000-2000s (randomized to prevent thundering herd)
**LRU Implementation:**
```zig
pub const LRU = struct {
map: HashMap([]const u8, Entry),
list: DoublyLinkedList([]const u8),
max_entries: usize,
const Entry = struct {
value: []const u8,
expires: i64,
node: *Node,
};
pub fn get(self: *LRU, key: []const u8) ?[]const u8;
pub fn put(self: *LRU, key: []const u8, value: []const u8, expires: i64) !void;
};
```
### 4. Location Resolver (location/)
**Responsibilities:**
- Parse location from URL
- Normalize location names
- Resolve IP to location (GeoIP)
- Resolve names to coordinates (geolocator)
- Handle special prefixes (~, @)
**Interface:**
```zig
pub const LocationResolver = struct {
geoip: GeoIP,
geolocator_url: []const u8,
allocator: Allocator,
pub fn init(allocator: Allocator, config: LocationConfig) !LocationResolver;
pub fn resolve(self: *LocationResolver, location: ?[]const u8, client_ip: []const u8) !Location;
};
pub const Location = struct {
name: []const u8, // "London" or "51.5074,-0.1278"
display_name: ?[]const u8, // Override for display
country: ?[]const u8, // "GB"
source_ip: []const u8, // Client IP
};
```
**Resolution Logic:**
```zig
fn resolve(location: ?[]const u8, client_ip: []const u8) !Location {
// 1. Empty/null → Use client IP
if (location == null) return resolveIP(client_ip);
// 2. IP address → Resolve to location
if (isIP(location)) return resolveIP(location);
// 3. @domain → Resolve domain to IP, then location
if (location[0] == '@') return resolveDomain(location[1..]);
// 4. ~search → Use geolocator
if (location[0] == '~') return geolocate(location[1..]);
// 5. Name → Normalize and use
return Location{
.name = normalize(location),
.display_name = null,
.country = null,
.source_ip = client_ip,
};
}
```
**GeoIP:**
- Wraps libmaxminddb C library via @cImport
- Built from source using Zig build system (not system library)
- Exposes clean Zig interface
- Hides C implementation details
**Build Integration:**
The libmaxminddb library will be built as part of the Zig build process:
- Source code vendored in `vendor/libmaxminddb/`
- Compiled using `b.addStaticLibrary()` or `b.addObject()`
- Linked into the main binary
- No system library dependency required
```zig
// location/geoip.zig
const c = @cImport({
@cInclude("maxminddb.h");
});
pub const GeoIP = struct {
mmdb: c.MMDB_s,
allocator: Allocator,
pub fn init(allocator: Allocator, db_path: []const u8) !GeoIP {
var self = GeoIP{
.mmdb = undefined,
.allocator = allocator,
};
const path_z = try allocator.dupeZ(u8, db_path);
defer allocator.free(path_z);
const status = c.MMDB_open(path_z.ptr, c.MMDB_MODE_MMAP, &self.mmdb);
if (status != c.MMDB_SUCCESS) {
return error.GeoIPOpenFailed;
}
return self;
}
pub fn lookup(self: *GeoIP, ip: []const u8) !?GeoIPResult {
const ip_z = try self.allocator.dupeZ(u8, ip);
defer self.allocator.free(ip_z);
var gai_error: c_int = 0;
var mmdb_error: c_int = 0;
const result = c.MMDB_lookup_string(
&self.mmdb,
ip_z.ptr,
&gai_error,
&mmdb_error
);
if (gai_error != 0 or mmdb_error != c.MMDB_SUCCESS) {
return null;
}
if (!result.found_entry) {
return null;
}
// Extract city and country
var entry_data: c.MMDB_entry_data_s = undefined;
const city = blk: {
const status = c.MMDB_get_value(
&result.entry,
&entry_data,
"city", "names", "en", null
);
if (status == c.MMDB_SUCCESS and entry_data.has_data) {
const str = entry_data.utf8_string[0..entry_data.data_size];
break :blk try self.allocator.dupe(u8, str);
}
break :blk null;
};
const country = blk: {
const status = c.MMDB_get_value(
&result.entry,
&entry_data,
"country", "names", "en", null
);
if (status == c.MMDB_SUCCESS and entry_data.has_data) {
const str = entry_data.utf8_string[0..entry_data.data_size];
break :blk try self.allocator.dupe(u8, str);
}
break :blk null;
};
return GeoIPResult{
.city = city,
.country = country,
};
}
pub fn deinit(self: *GeoIP) void {
c.MMDB_close(&self.mmdb);
}
};
pub const GeoIPResult = struct {
city: ?[]const u8,
country: ?[]const u8,
pub fn deinit(self: GeoIPResult, allocator: Allocator) void {
if (self.city) |city| allocator.free(city);
if (self.country) |country| allocator.free(country);
}
};
```
### 5. Weather Provider (weather/)
**Interface (pluggable):**
```zig
pub const WeatherProvider = struct {
ptr: *anyopaque,
vtable: *const VTable,
pub const VTable = struct {
fetch: *const fn (ptr: *anyopaque, location: []const u8) anyerror!WeatherData,
deinit: *const fn (ptr: *anyopaque) void,
};
pub fn fetch(self: WeatherProvider, location: []const u8) !WeatherData {
return self.vtable.fetch(self.ptr, location);
}
};
pub const WeatherData = struct {
location: []const u8,
current: CurrentCondition,
forecast: []ForecastDay,
pub const CurrentCondition = struct {
temp_c: f32,
temp_f: f32,
condition: []const u8,
weather_code: u16,
humidity: u8,
wind_kph: f32,
wind_dir: []const u8,
pressure_mb: f32,
precip_mm: f32,
};
pub const ForecastDay = struct {
date: []const u8,
max_temp_c: f32,
min_temp_c: f32,
condition: []const u8,
weather_code: u16,
hourly: []HourlyForecast,
};
pub const HourlyForecast = struct {
time: []const u8,
temp_c: f32,
condition: []const u8,
weather_code: u16,
wind_kph: f32,
precip_mm: f32,
};
};
```
**MetNo Implementation:**
```zig
pub const MetNo = struct {
allocator: Allocator,
http_client: HttpClient,
pub fn init(allocator: Allocator) !MetNo;
pub fn provider(self: *MetNo) WeatherProvider;
pub fn fetch(self: *MetNo, location: []const u8) !WeatherData;
};
```
**Mock Implementation (for tests):**
```zig
pub const MockWeather = struct {
allocator: Allocator,
responses: HashMap([]const u8, WeatherData),
pub fn init(allocator: Allocator) !MockWeather;
pub fn provider(self: *MockWeather) WeatherProvider;
pub fn addResponse(self: *MockWeather, location: []const u8, data: WeatherData) !void;
pub fn fetch(self: *MockWeather, location: []const u8) !WeatherData;
};
```
### Provider vs Renderer Responsibilities
**Weather Provider responsibilities:**
- Fetch raw weather data from external APIs
- Parse API responses into structured types
- **Perform timezone conversions once at ingestion time**
- Group forecast data by local date (not UTC date)
- Store both UTC time and local time in forecast data
- Return data in a timezone-agnostic format ready for rendering
**Renderer responsibilities:**
- Format weather data for display (ANSI, plain text, JSON, etc.)
- Select appropriate hourly forecasts for display (morning/noon/evening/night)
- Apply unit conversions (metric/imperial) based on user preferences
- Handle partial days with missing data (render empty slots)
- Format dates and times for human readability
- **Should NOT perform timezone calculations** - use pre-calculated local times from provider
**Key principle:** Timezone conversions are expensive and error-prone. They should happen once at the provider level, not repeatedly at the renderer level. This separation ensures consistent behavior across all output formats and simplifies the rendering logic.
**Implementation details:**
- Core data structures use `zeit.Time` and `zeit.Date` types instead of strings for type safety
- `HourlyForecast` contains both `time: zeit.Time` (UTC) and `local_time: zeit.Time` (pre-calculated)
- `ForecastDay.date` is `zeit.Date` (not string), eliminating parsing/formatting overhead
- The `MetNo` provider uses a pre-computed timezone offset lookup table (360 entries covering global coordinates)
- Date formatting uses `zeit.Time.gofmt()` with Go-style format strings (e.g., "Mon _2 Jan")
- Timezone offset table provides ±1-2.5 hour accuracy at extreme latitudes, sufficient for forecast grouping
**Benefits of zeit integration:**
- Type safety prevents format string errors and invalid date/time operations
- Explicit timezone handling - no implicit UTC assumptions
- Eliminates redundant string parsing and formatting
- Enables proper date arithmetic (e.g., `instant.add(duration)`, `instant.subtract(duration)`)
- Consistent date/time representation across all modules
### 6. Renderers (render/)
**ANSI Renderer:**
```zig
pub const AnsiRenderer = struct {
pub fn render(allocator: Allocator, data: WeatherData, options: RenderOptions) ![]const u8;
};
pub const RenderOptions = struct {
narrow: bool = false,
days: u8 = 3,
use_imperial: bool = false,
no_caption: bool = false,
};
```
**Output Format:**
```
Weather for: London, GB
\ / Clear
.-. 10 11 °C
― ( ) ― ↑ 11 km/h
`-' 10 km
/ \ 0.0 mm
```
**JSON Renderer:**
```zig
pub const JsonRenderer = struct {
pub fn render(allocator: Allocator, data: WeatherData) ![]const u8;
};
```
**One-Line Renderer:**
```zig
pub const LineRenderer = struct {
pub fn render(allocator: Allocator, data: WeatherData, format: []const u8) ![]const u8;
};
// Formats:
// 1: "London: ⛅️ +7°C"
// 2: "London: ⛅️ +7°C 🌬↗11km/h"
// 3: "London: ⛅️ +7°C 🌬↗11km/h 💧65%"
// Custom: "%l: %c %t" → "London: ⛅️ +7°C"
```
### 7. Request Handler (http/handler.zig)
**Main handler flow:**
```zig
pub const HandleWeatherOptions = struct {
cache: *Cache,
resolver: *LocationResolver,
provider: WeatherProvider,
};
pub fn handleWeather(
req: *http.Request,
res: *http.Response,
options: HandleWeatherOptions,
) !void {
// 1. Parse request
const location = parseLocation(req.url.path);
const render_opts = parseOptions(req.url.query);
const client_ip = getClientIP(req);
// 2. Generate cache key
const cache_key = try generateCacheKey(req, client_ip);
// 3. Check cache
if (options.cache.get(cache_key)) |cached| {
return res.write(cached);
}
// 4. Resolve location
const resolved = try options.resolver.resolve(location, client_ip);
// 5. Fetch weather
const weather = try options.provider.fetch(resolved.name);
// 6. Render
const output = try renderWeather(weather, render_opts);
// 7. Cache
const ttl = 1000 + std.crypto.random.intRangeAtMost(u64, 0, 1000);
try options.cache.put(cache_key, output, ttl);
// 8. Return
return res.write(output);
}
```
## Configuration
**Environment Variables:**
```bash
WTTR_LISTEN_HOST=0.0.0.0
WTTR_LISTEN_PORT=8002
WTTR_CACHE_SIZE=10000
WTTR_CACHE_DIR=/tmp/wttr-cache
WTTR_GEOLITE_PATH=./GeoLite2-City.mmdb
WTTR_GEOLOCATOR_URL=http://localhost:8004
```
**Config File (optional):**
```zig
// config.zig
pub const Config = struct {
listen_host: []const u8,
listen_port: u16,
cache_size: usize,
cache_dir: []const u8,
geolite_path: []const u8,
geolocator_url: []const u8,
pub fn load(allocator: Allocator) !Config {
return Config{
.listen_host = std.os.getenv("WTTR_LISTEN_HOST") orelse "0.0.0.0",
.listen_port = parsePort(std.os.getenv("WTTR_LISTEN_PORT")) orelse 8002,
.cache_size = parseSize(std.os.getenv("WTTR_CACHE_SIZE")) orelse 10_000,
.cache_dir = std.os.getenv("WTTR_CACHE_DIR") orelse "/tmp/wttr-cache",
.geolite_path = std.os.getenv("WTTR_GEOLITE_PATH") orelse "./GeoLite2-City.mmdb",
.geolocator_url = std.os.getenv("WTTR_GEOLOCATOR_URL") orelse "http://localhost:8004",
};
}
};
```
## Dependencies
**External (Zig packages):**
- karlseguin/http.zig - HTTP server
- rockorager/zeit - Time utilities
**Vendored (built from source):**
- libmaxminddb - GeoIP lookups (C library, built with Zig build system)
**Standard Library:**
- std.http.Client - HTTP client for weather APIs
- std.json - JSON parsing/serialization
- std.HashMap - Cache and rate limiter storage
- std.crypto.random - Random TTL generation
## Build Configuration
**build.zig:**
```zig
const std = @import("std");
pub fn build(b: *std.Build) void {
const target = b.standardTargetOptions(.{});
const optimize = b.standardOptimizeOption(.{});
// Build libmaxminddb from source
const maxminddb = b.addStaticLibrary(.{
.name = "maxminddb",
.target = target,
.optimize = optimize,
});
maxminddb.linkLibC();
maxminddb.addIncludePath(.{ .path = "vendor/libmaxminddb/include" });
maxminddb.addCSourceFiles(.{
.files = &.{
"vendor/libmaxminddb/src/maxminddb.c",
"vendor/libmaxminddb/src/data-pool.c",
},
.flags = &.{"-std=c99"},
});
const exe = b.addExecutable(.{
.name = "wttr",
.root_source_file = .{ .path = "src/main.zig" },
.target = target,
.optimize = optimize,
});
// Add http.zig dependency
const http = b.dependency("http", .{
.target = target,
.optimize = optimize,
});
exe.root_module.addImport("http", http.module("http"));
// Add zeit dependency
const zeit = b.dependency("zeit", .{
.target = target,
.optimize = optimize,
});
exe.root_module.addImport("zeit", zeit.module("zeit"));
// Link libmaxminddb (built from source)
exe.linkLibrary(maxminddb);
exe.addIncludePath(.{ .path = "vendor/libmaxminddb/include" });
exe.linkLibC();
b.installArtifact(exe);
// Tests
const tests = b.addTest(.{
.root_source_file = .{ .path = "src/main.zig" },
.target = target,
.optimize = optimize,
});
tests.root_module.addImport("http", http.module("http"));
tests.root_module.addImport("zeit", zeit.module("zeit"));
tests.linkLibrary(maxminddb);
tests.addIncludePath(.{ .path = "vendor/libmaxminddb/include" });
tests.linkLibC();
const run_tests = b.addRunArtifact(tests);
const test_step = b.step("test", "Run tests");
test_step.dependOn(&run_tests.step);
}
```
**Project Structure:**
```
wttr/
├── build.zig
├── build.zig.zon
├── vendor/
│ └── libmaxminddb/ # Vendored libmaxminddb source
│ ├── include/
│ │ └── maxminddb.h
│ └── src/
│ ├── maxminddb.c
│ └── data-pool.c
└── src/
└── ...
```
**build.zig.zon:**
```zig
.{
.name = "wttr",
.version = "0.1.0",
.dependencies = .{
.http = .{
.url = "https://github.com/karlseguin/http.zig/archive/refs/tags/v0.1.0.tar.gz",
.hash = "...",
},
.zeit = .{
.url = "https://github.com/rockorager/zeit/archive/refs/tags/v0.1.0.tar.gz",
.hash = "...",
},
},
.paths = .{
"build.zig",
"build.zig.zon",
"src",
},
}
```
## Testing Strategy
**Unit Tests:**
```zig
// cache/lru.zig
test "LRU basic operations" {
var lru = try LRU.init(testing.allocator, 3);
defer lru.deinit();
try lru.put("key1", "value1", 9999999999);
try testing.expectEqualStrings("value1", lru.get("key1").?);
}
test "LRU eviction" {
var lru = try LRU.init(testing.allocator, 2);
defer lru.deinit();
try lru.put("key1", "value1", 9999999999);
try lru.put("key2", "value2", 9999999999);
try lru.put("key3", "value3", 9999999999);
try testing.expect(lru.get("key1") == null); // Evicted
try testing.expectEqualStrings("value2", lru.get("key2").?);
}
// weather/mock.zig
test "mock weather provider" {
var mock = try MockWeather.init(testing.allocator);
defer mock.deinit();
const data = WeatherData{ /* ... */ };
try mock.addResponse("London", data);
const provider = mock.provider();
const result = try provider.fetch("London");
try testing.expectEqual(data.current.temp_c, result.current.temp_c);
}
// http/middleware.zig
test "rate limiter enforces limits" {
const zeit = @import("zeit");
var limiter = try RateLimiter.init(testing.allocator, .{
.capacity = 10,
.refill_rate = 1,
.refill_interval_ms = 1000,
});
defer limiter.deinit();
// Consume all tokens
var i: usize = 0;
while (i < 10) : (i += 1) {
try limiter.check("1.2.3.4");
}
// Next request should fail
try testing.expectError(error.RateLimitExceeded, limiter.check("1.2.3.4"));
// Wait for refill (1 second = 1 token)
std.time.sleep(1_100_000_000); // 1.1 seconds
// Should succeed now
try limiter.check("1.2.3.4");
}
test "rate limiter allows bursts" {
var limiter = try RateLimiter.init(testing.allocator, .{
.capacity = 100,
.refill_rate = 5,
.refill_interval_ms = 200,
});
defer limiter.deinit();
// Should allow 100 requests immediately (burst)
var i: usize = 0;
while (i < 100) : (i += 1) {
try limiter.check("1.2.3.4");
}
// 101st should fail
try testing.expectError(error.RateLimitExceeded, limiter.check("1.2.3.4"));
}
```
**Integration Tests:**
```zig
test "end-to-end weather request" {
var mock = try MockWeather.init(testing.allocator);
defer mock.deinit();
// Setup mock data
try mock.addResponse("London", test_weather_data);
// Create server components
var cache = try Cache.init(testing.allocator, .{});
defer cache.deinit();
var resolver = try LocationResolver.init(testing.allocator, .{});
defer resolver.deinit();
// Simulate request
const output = try handleWeatherRequest(
"London",
&cache,
&resolver,
mock.provider(),
);
defer testing.allocator.free(output);
try testing.expect(std.mem.indexOf(u8, output, "London") != null);
try testing.expect(std.mem.indexOf(u8, output, "°C") != null);
}
```
## Deployment
**Single Binary:**
```bash
# Build
zig build -Doptimize=ReleaseFast
# Run
./zig-out/bin/wttr
# With config
WTTR_LISTEN_PORT=8080 ./zig-out/bin/wttr
```
**Docker:**
```dockerfile
FROM alpine:latest
COPY zig-out/bin/wttr /usr/local/bin/wttr
COPY GeoLite2-City.mmdb /data/GeoLite2-City.mmdb
ENV WTTR_GEOLITE_PATH=/data/GeoLite2-City.mmdb
EXPOSE 8002
CMD ["/usr/local/bin/wttr"]
```
## Performance Targets
**Latency:**
- Cache hit: <1ms
- Cache miss: <100ms (depends on weather API)
- P95: <150ms
- P99: <300ms
**Throughput:**
- >10,000 req/s (cached)
- >1,000 req/s (uncached)
**Memory:**
- Base: <50MB
- With 10,000 cache entries: <200MB
- Binary size: <5MB
## Future Enhancements (Not in Initial Version)
1. **i18n Support** - Add translation system
2. **PNG Rendering** - Add image output
3. **HTML Output** - Add web browser support
4. **v2 Format** - Data-rich experimental format
5. **Prometheus Format** - Metrics output
6. **Moon Phases** - Special moon queries
7. **Multiple Weather Providers** - WWO, OpenWeather, etc.
8. **Metrics/Observability** - Prometheus metrics, structured logging
9. **Admin API** - Cache stats, health checks
## Migration from Current System
**Phase 1: Parallel Deployment**
- Deploy Zig version on different port (8003)
- Route 1% of traffic to Zig version
- Monitor errors and performance
- Compare outputs
**Phase 2: Gradual Rollout**
- Increase traffic to 10%, 25%, 50%, 100%
- Monitor at each step
- Rollback if issues
**Phase 3: Cutover**
- Switch all traffic to Zig version
- Keep Python/Go as backup for 1 week
- Decommission old system
## Success Criteria
**Functional:**
- [ ] All core endpoints work (/, /{location}, /:help)
- [ ] ANSI output matches current system
- [ ] JSON output matches current system
- [ ] One-line formats work
- [ ] Location resolution works (IP, name, coordinates)
- [ ] Rate limiting works
- [ ] Caching works
**Performance:**
- [ ] Latency ≤ current system
- [ ] Throughput ≥ current system
- [ ] Memory ≤ current system
- [ ] Binary size <10MB
**Quality:**
- [ ] >80% test coverage
- [ ] No memory leaks (valgrind clean)
- [ ] No crashes under load
- [ ] Clean error handling
**Operational:**
- [ ] Single binary deployment
- [ ] Simple configuration
- [ ] Good logging
- [ ] Easy to debug

View file

@ -3,7 +3,7 @@
.version = "0.1.0", .version = "0.1.0",
.dependencies = .{ .dependencies = .{
.httpz = .{ .httpz = .{
.url = "git+https://github.com/karlseguin/http.zig/#1c0ec3751c53e276d5e0c42b6d5481e72d9d1971", .url = "https://github.com/karlseguin/http.zig/archive/refs/heads/master.tar.gz",
.hash = "httpz-0.0.0-PNVzrEktBwCzPoiua-S8LAYo2tILqczm3tSpneEzLQ9L", .hash = "httpz-0.0.0-PNVzrEktBwCzPoiua-S8LAYo2tILqczm3tSpneEzLQ9L",
}, },
.maxminddb = .{ .maxminddb = .{

View file

@ -1,151 +0,0 @@
//!Units:
//!
//! m # metric (SI) (used by default everywhere except US)
//! u # USCS (used by default in US)
//! M # show wind speed in m/s
//!
//!View options:
//!
//! 0 # only current weather
//! 1 # current weather + today's forecast
//! 2 # current weather + today's + tomorrow's forecast
//! A # ignore User-Agent and force ANSI output format (terminal)
//! d # restrict output to standard console font glyphs
//! F # do not show the "Follow" line
//! n # narrow version (only day and night)
//! q # quiet version (no "Weather report" text)
//! Q # superquiet version (no "Weather report", no city name)
//! T # switch terminal sequences off (no colors)
const std = @import("std");
const RenderOptions = @import("../render/Formatted.zig").RenderOptions;
const QueryParams = @This();
format: ?[]const u8 = null,
lang: ?[]const u8 = null,
location: ?[]const u8 = null,
transparency: ?u8 = null,
/// A: Ignore user agent and force ansi mode
ansi: bool = false,
/// T: Avoid terminal sequences and just output plain text
text_only: bool = false,
/// This is necessary because it it imporant to know if the user explicitly
/// requested imperial/metric
use_imperial: ?bool = null,
render_options: RenderOptions,
pub fn parse(allocator: std.mem.Allocator, query_string: []const u8) !QueryParams {
// SAFETY: function adds render_options at end of function before return
var params = QueryParams{ .render_options = undefined };
var iter = std.mem.splitScalar(u8, query_string, '&');
var render_options = RenderOptions{};
while (iter.next()) |pair| {
if (pair.len == 0) continue;
var kv = std.mem.splitScalar(u8, pair, '=');
const key = kv.next() orelse continue;
const value = kv.next();
if (key.len == 1) {
switch (key[0]) {
'0' => render_options.days = 0,
'1' => render_options.days = 1,
'2' => render_options.days = 2,
'u' => params.use_imperial = true,
'm' => params.use_imperial = false,
'n' => render_options.narrow = true,
'q' => render_options.quiet = true,
'Q' => render_options.super_quiet = true,
'A' => params.ansi = true,
'T' => params.text_only = true,
't' => params.transparency = 150,
else => continue,
}
}
if (std.mem.eql(u8, key, "format")) {
params.format = if (value) |v| try allocator.dupe(u8, v) else null;
} else if (std.mem.eql(u8, key, "lang")) {
params.lang = if (value) |v| try allocator.dupe(u8, v) else null;
} else if (std.mem.eql(u8, key, "location")) {
params.location = if (value) |v| try allocator.dupe(u8, v) else null;
} else if (std.mem.eql(u8, key, "use_imperial")) {
params.use_imperial = true;
} else if (std.mem.eql(u8, key, "use_metric")) {
params.use_imperial = false;
} else if (std.mem.eql(u8, key, "transparency")) {
if (value) |v| {
params.transparency = try std.fmt.parseInt(u8, v, 10);
}
}
}
if (params.use_imperial) |u| render_options.use_imperial = u;
params.render_options = render_options;
return params;
}
test "parse empty query" {
const allocator = std.testing.allocator;
const params = try QueryParams.parse(allocator, "");
try std.testing.expect(params.format == null);
try std.testing.expect(params.lang == null);
try std.testing.expect(params.use_imperial == null);
}
test "parse format parameter" {
const allocator = std.testing.allocator;
const params = try QueryParams.parse(allocator, "format=j1");
defer if (params.format) |f| allocator.free(f);
try std.testing.expect(params.format != null);
try std.testing.expectEqualStrings("j1", params.format.?);
}
test "parse units with question mark" {
const allocator = std.testing.allocator;
// Test with just "u" (no question mark in query string)
const params1 = try QueryParams.parse(allocator, "u");
try std.testing.expect(params1.use_imperial.?);
// Test with "u=" (empty value)
const params2 = try QueryParams.parse(allocator, "u=");
try std.testing.expect(params2.use_imperial.?);
// Test combined with other params
const params3 = try QueryParams.parse(allocator, "format=3&u");
defer if (params3.format) |f| allocator.free(f);
try std.testing.expect(params3.use_imperial.?);
}
test "parse units parameters" {
const allocator = std.testing.allocator;
const params_m = try QueryParams.parse(allocator, "m");
try std.testing.expect(!params_m.use_imperial.?);
const params_u = try QueryParams.parse(allocator, "u");
try std.testing.expect(params_u.use_imperial.?);
const params_u_query = try QueryParams.parse(allocator, "u=");
try std.testing.expect(params_u_query.use_imperial.?);
}
test "parse multiple parameters" {
const allocator = std.testing.allocator;
const params = try QueryParams.parse(allocator, "format=3&lang=de&m");
defer if (params.format) |f| allocator.free(f);
defer if (params.lang) |l| allocator.free(l);
try std.testing.expectEqualStrings("3", params.format.?);
try std.testing.expectEqualStrings("de", params.lang.?);
try std.testing.expect(!params.use_imperial.?);
}
test "parse transparency" {
const allocator = std.testing.allocator;
const params_t = try QueryParams.parse(allocator, "t");
try std.testing.expect(params_t.transparency != null);
try std.testing.expectEqual(@as(u8, 150), params_t.transparency.?);
const params_custom = try QueryParams.parse(allocator, "transparency=200");
try std.testing.expectEqual(@as(u8, 200), params_custom.transparency.?);
}

View file

@ -2,12 +2,12 @@ const std = @import("std");
const httpz = @import("httpz"); const httpz = @import("httpz");
const WeatherProvider = @import("../weather/Provider.zig"); const WeatherProvider = @import("../weather/Provider.zig");
const Resolver = @import("../location/resolver.zig").Resolver; const Resolver = @import("../location/resolver.zig").Resolver;
const QueryParams = @import("QueryParams.zig"); const QueryParams = @import("query.zig").QueryParams;
const Formatted = @import("../render/Formatted.zig"); const formatted = @import("../render/formatted.zig");
const Line = @import("../render/Line.zig"); const line = @import("../render/line.zig");
const Json = @import("../render/Json.zig"); const json = @import("../render/json.zig");
const V2 = @import("../render/V2.zig"); const v2 = @import("../render/v2.zig");
const Custom = @import("../render/Custom.zig"); const custom = @import("../render/custom.zig");
const help = @import("help.zig"); const help = @import("help.zig");
const log = std.log.scoped(.handler); const log = std.log.scoped(.handler);
@ -153,21 +153,20 @@ fn handleWeatherInternal(
if (params.lang) |l| req_alloc.free(l); if (params.lang) |l| req_alloc.free(l);
} }
var render_options = params.render_options;
// Determine if imperial units should be used // Determine if imperial units should be used
// Priority: explicit ?u or ?m > lang=us > US IP > default metric // Priority: explicit ?u or ?m > lang=us > US IP > default metric
if (params.use_imperial == null) { const use_imperial = blk: {
// User did not ask for anything explicitly if (params.units) |u|
break :blk u == .uscs;
// Check if lang=us if (params.lang) |lang|
if (params.lang) |lang| {
if (std.mem.eql(u8, lang, "us")) if (std.mem.eql(u8, lang, "us"))
render_options.use_imperial = true; break :blk true;
}
if (!render_options.use_imperial and client_ip.len > 0 and opts.geoip.isUSIp(client_ip)) if (client_ip.len > 0 and opts.geoip.isUSIp(client_ip))
render_options.use_imperial = true; // this is a US IP break :blk true;
} break :blk false;
};
// Add coordinates header using response allocator // Add coordinates header using response allocator
const coords_header = try std.fmt.allocPrint(res.arena, "{d:.4},{d:.4}", .{ location.coords.latitude, location.coords.longitude }); const coords_header = try std.fmt.allocPrint(res.arena, "{d:.4},{d:.4}", .{ location.coords.latitude, location.coords.longitude });
@ -179,27 +178,27 @@ fn handleWeatherInternal(
res.content_type = .TEXT; res.content_type = .TEXT;
if (std.mem.eql(u8, fmt, "j1")) { if (std.mem.eql(u8, fmt, "j1")) {
res.content_type = .JSON; // reset to json res.content_type = .JSON; // reset to json
break :blk try Json.render(req_alloc, weather); break :blk try json.render(req_alloc, weather);
} }
if (std.mem.eql(u8, fmt, "v2")) if (std.mem.eql(u8, fmt, "v2"))
break :blk try V2.render(req_alloc, weather, render_options.use_imperial); break :blk try v2.render(req_alloc, weather, use_imperial);
if (std.mem.startsWith(u8, fmt, "%")) if (std.mem.startsWith(u8, fmt, "%"))
break :blk try Custom.render(req_alloc, weather, fmt, render_options.use_imperial); break :blk try custom.render(req_alloc, weather, fmt, use_imperial);
// fall back to line if we don't understand the format parameter // fall back to line if we don't understand the format parameter
break :blk try Line.render(req_alloc, weather, fmt, render_options.use_imperial); break :blk try line.render(req_alloc, weather, fmt, use_imperial);
} else { } else {
render_options.format = determineFormat(params, req.headers.get("user-agent")); const format: formatted.Format = determineFormat(params, req.headers.get("user-agent"));
log.debug( log.debug(
"Format: {}. params.ansi {}, params.text {}, user agent: {?s}", "Format: {}. params.ansi {}, params.text {}, user agent: {?s}",
.{ render_options.format, params.ansi, params.text_only, req.headers.get("user-agent") }, .{ format, params.ansi, params.text_only, req.headers.get("user-agent") },
); );
if (render_options.format != .html) res.content_type = .TEXT else res.content_type = .HTML; if (format != .html) res.content_type = .TEXT else res.content_type = .HTML;
break :blk try Formatted.render(req_alloc, weather, render_options); break :blk try formatted.render(req_alloc, weather, .{ .use_imperial = use_imperial, .format = format });
} }
}; };
} }
fn determineFormat(params: QueryParams, user_agent: ?[]const u8) Formatted.Format { fn determineFormat(params: QueryParams, user_agent: ?[]const u8) formatted.Format {
if (params.ansi or params.text_only) { if (params.ansi or params.text_only) {
// user explicitly requested something. If both are set, text will win // user explicitly requested something. If both are set, text will win
if (params.text_only) return .plain_text; if (params.text_only) return .plain_text;
@ -256,10 +255,10 @@ test "imperial units selection logic" {
const allocator = std.testing.allocator; const allocator = std.testing.allocator;
const params_u = try QueryParams.parse(allocator, "u"); const params_u = try QueryParams.parse(allocator, "u");
try std.testing.expect(params_u.use_imperial.?); try std.testing.expectEqual(QueryParams.Units.uscs, params_u.units.?);
const params_m = try QueryParams.parse(allocator, "m"); const params_m = try QueryParams.parse(allocator, "m");
try std.testing.expect(!params_m.use_imperial.?); try std.testing.expectEqual(QueryParams.Units.metric, params_m.units.?);
const params_lang = try QueryParams.parse(allocator, "lang=us"); const params_lang = try QueryParams.parse(allocator, "lang=us");
defer allocator.free(params_lang.lang.?); defer allocator.free(params_lang.lang.?);

View file

@ -1,117 +1,39 @@
const std = @import("std"); const std = @import("std");
pub const help_page = pub const help_page =
\\wttr - Weather Forecast Service \\wttr.in - Weather Forecast Service
\\
\\Based on wttr.in. items below with an * are not implemented in this version
\\ \\
\\Usage: \\Usage:
\\ curl wttr.in # Weather for your location
\\ curl wttr.in/London # Weather for London
\\ curl wttr.in/~Eiffel+Tower # Weather for special location
\\ curl wttr.in/@github.com # Weather for domain location
\\ curl wttr.in/muc # Weather for airport (IATA code)
\\ \\
\\ $ curl <base url> # current location \\Query Parameters:
\\ $ curl <base url>/muc # weather in the Munich airport \\ ?format=FORMAT Output format (1,2,3,4,j1,p1,v2)
\\ ?lang=LANG Language code (en,de,fr,etc)
\\ ?u Use USCS units
\\ ?m Use metric units
\\ ?t Transparency for PNG
\\ \\
\\Supported location types: \\Special Endpoints:
\\ /:help This help page
\\ /:translation Translation information
\\ \\
\\ /paris # city name \\Examples:
\\ /~Eiffel+tower # any location (+ for spaces) \\ curl wttr.in/Paris?format=3
\\ /Москва # Unicode name of any location in any language \\ curl wttr.in/Berlin?lang=de
\\ /muc # airport code (3 letters) \\ curl wttr.in/Tokyo?m
\\ /@stackoverflow.com # domain name
\\ /94107 # area codes
\\ /-78.46,106.79 # GPS coordinates
\\ \\
\\Moon phase information (not yet implemented): \\For more information visit: https://github.com/chubin/wttr.in
\\ \\
\\ /moon # Moon phase (add ,+US or ,+France for these cities)
\\ /moon@2016-10-25 # Moon phase for the date (@2016-10-25)
\\
\\Query string units:
\\
\\ m # metric (SI) (used by default everywhere except US)
\\ u # USCS (used by default in US)
\\ M # * show wind speed in m/s
\\
\\Query string view options:
\\
\\ 0 # only current weather
\\ 1 # current weather + today's forecast
\\ 2 # current weather + today's + tomorrow's forecast
\\ A # ignore User-Agent and force ANSI output format (terminal)
\\ d # * restrict output to standard console font glyphs
\\ F # do not show the "Follow" line (not necessary - this version does not have a follow line)
\\ n # narrow version (only day and night)
\\ q # quiet version (no "Weather report" text)
\\ Q # superquiet version (no "Weather report", no city name)
\\ T # switch terminal sequences off (no colors)
\\
\\Output formats:
\\
\\ format=1 # one-line: "London: ⛅️ +7°C"
\\ format=2 # one-line with wind: "London: ⛅️ +7°C 🌬↗11km/h"
\\ format=3 # one-line detailed: "London: ⛅️ +7°C 🌬↗11km/h 💧65%"
\\ format=4 # one-line full: "London: ⛅️ +7°C 🌬↗11km/h 💧65% ☀06:45-16:32"
\\ format=j1 # JSON output
\\ format=v2 # * data-rich experimental format
\\ format=p1 # * Prometheus metrics format
\\ format=%l:+%c+%t # custom format (see format codes below)
\\
\\Custom format codes:
\\
\\ %c # weather condition emoji
\\ %C # weather condition text
\\ %h # humidity
\\ %t # temperature (actual)
\\ %f # temperature (feels like)
\\ %w # wind
\\ %l # location
\\ %m # * moon phase emoji
\\ %M # * moon day
\\ %p # precipitation (mm)
\\ %o # probability of precipitation
\\ %P # pressure (hPa)
\\ %D # * dawn time
\\ %S # * sunrise time
\\ %z # * zenith time
\\ %s # * sunset time
\\ %d # * dusk time
\\
\\PNG options:
\\
\\ /paris.png # generate a PNG file
\\ p # add frame around the output
\\ t # transparency 150
\\ transparency=... # transparency from 0 to 255 (255 = not transparent)
\\ background=... # background color in form RRGGBB, e.g. 00aaaa
\\
\\Options can be combined:
\\
\\ /Paris?0pq
\\ /Paris?0pq&lang=fr
\\ /Paris_0pq.png # in PNG the file mode are specified after _
\\ /Rome_0pq_lang=it.png # long options are separated with underscore
\\
\\* Localization:
\\
\\ $ curl fr.wttr.in/Paris
\\ $ curl wttr.in/paris?lang=fr
\\ $ curl -H "Accept-Language: fr" wttr.in/paris
\\
\\* Supported languages:
\\
\\ am ar af be bn ca da de el et fr fa gl hi hu ia id it lt mg nb nl oc pl pt-br ro ru ta tr th uk vi zh-cn zh-tw (supported)
\\ az bg bs cy cs eo es eu fi ga hi hr hy is ja jv ka kk ko ky lv mk ml mr nl fy nn pt pt-br sk sl sr sr-lat sv sw te uz zh zu he (in progress)
\\
\\Special URLs:
\\
\\ /:help # show this page
\\ /:bash.function # show recommended bash function wttr()
\\ /:translation # show the information about the translators
; ;
pub const translation_page = pub const translation_page =
\\wttr Translation \\wttr.in Translation
\\ \\
\\wttr is currently translated into 54 languages. \\wttr.in is currently translated into 54 languages.
\\ \\
\\NOTE: Translations are not implemented in this version! \\NOTE: Translations are not implemented in this version!
\\ \\

142
src/http/query.zig Normal file
View file

@ -0,0 +1,142 @@
const std = @import("std");
///Units:
///
/// m # metric (SI) (used by default everywhere except US)
/// u # USCS (used by default in US)
/// M # show wind speed in m/s
///
///View options:
///
/// 0 # only current weather
/// 1 # current weather + today's forecast
/// 2 # current weather + today's + tomorrow's forecast
/// A # ignore User-Agent and force ANSI output format (terminal)
/// d # restrict output to standard console font glyphs
/// F # do not show the "Follow" line
/// n # narrow version (only day and night)
/// q # quiet version (no "Weather report" text)
/// Q # superquiet version (no "Weather report", no city name)
/// T # switch terminal sequences off (no colors)
pub const QueryParams = struct {
format: ?[]const u8 = null,
lang: ?[]const u8 = null,
location: ?[]const u8 = null,
units: ?Units = null,
transparency: ?u8 = null,
/// A: Ignore user agent and force ansi mode
ansi: bool = false,
/// T: Avoid terminal sequences and just output plain text
text_only: bool = false,
pub const Units = enum {
metric,
uscs,
};
pub fn parse(allocator: std.mem.Allocator, query_string: []const u8) !QueryParams {
var params = QueryParams{};
var iter = std.mem.splitScalar(u8, query_string, '&');
while (iter.next()) |pair| {
if (pair.len == 0) continue;
var kv = std.mem.splitScalar(u8, pair, '=');
const key = kv.next() orelse continue;
const value = kv.next();
if (key.len == 1) {
switch (key[0]) {
'u' => params.units = .uscs,
'm' => params.units = .metric,
'A' => params.ansi = true,
'T' => params.text_only = true,
't' => params.transparency = 150,
else => continue,
}
}
if (std.mem.eql(u8, key, "format")) {
params.format = if (value) |v| try allocator.dupe(u8, v) else null;
} else if (std.mem.eql(u8, key, "lang")) {
params.lang = if (value) |v| try allocator.dupe(u8, v) else null;
} else if (std.mem.eql(u8, key, "location")) {
params.location = if (value) |v| try allocator.dupe(u8, v) else null;
} else if (std.mem.eql(u8, key, "use_imperial")) {
params.units = .uscs;
} else if (std.mem.eql(u8, key, "use_metric")) {
params.units = .metric;
} else if (std.mem.eql(u8, key, "transparency")) {
if (value) |v| {
params.transparency = try std.fmt.parseInt(u8, v, 10);
}
}
}
return params;
}
};
test "parse empty query" {
const allocator = std.testing.allocator;
const params = try QueryParams.parse(allocator, "");
try std.testing.expect(params.format == null);
try std.testing.expect(params.lang == null);
try std.testing.expect(params.units == null);
}
test "parse format parameter" {
const allocator = std.testing.allocator;
const params = try QueryParams.parse(allocator, "format=j1");
defer if (params.format) |f| allocator.free(f);
try std.testing.expect(params.format != null);
try std.testing.expectEqualStrings("j1", params.format.?);
}
test "parse units with question mark" {
const allocator = std.testing.allocator;
// Test with just "u" (no question mark in query string)
const params1 = try QueryParams.parse(allocator, "u");
try std.testing.expectEqual(QueryParams.Units.uscs, params1.units.?);
// Test with "u=" (empty value)
const params2 = try QueryParams.parse(allocator, "u=");
try std.testing.expectEqual(QueryParams.Units.uscs, params2.units.?);
// Test combined with other params
const params3 = try QueryParams.parse(allocator, "format=3&u");
defer if (params3.format) |f| allocator.free(f);
try std.testing.expectEqual(QueryParams.Units.uscs, params3.units.?);
}
test "parse units parameters" {
const allocator = std.testing.allocator;
const params_m = try QueryParams.parse(allocator, "m");
try std.testing.expectEqual(QueryParams.Units.metric, params_m.units.?);
const params_u = try QueryParams.parse(allocator, "u");
try std.testing.expectEqual(QueryParams.Units.uscs, params_u.units.?);
const params_u_query = try QueryParams.parse(allocator, "u=");
try std.testing.expectEqual(QueryParams.Units.uscs, params_u_query.units.?);
}
test "parse multiple parameters" {
const allocator = std.testing.allocator;
const params = try QueryParams.parse(allocator, "format=3&lang=de&m");
defer if (params.format) |f| allocator.free(f);
defer if (params.lang) |l| allocator.free(l);
try std.testing.expectEqualStrings("3", params.format.?);
try std.testing.expectEqualStrings("de", params.lang.?);
try std.testing.expectEqual(QueryParams.Units.metric, params.units.?);
}
test "parse transparency" {
const allocator = std.testing.allocator;
const params_t = try QueryParams.parse(allocator, "t");
try std.testing.expect(params_t.transparency != null);
try std.testing.expectEqual(@as(u8, 150), params_t.transparency.?);
const params_custom = try QueryParams.parse(allocator, "transparency=200");
try std.testing.expectEqual(@as(u8, 200), params_custom.transparency.?);
}

View file

@ -24,19 +24,6 @@ pub const LocationType = enum {
domain_name, domain_name,
}; };
/// Primary way to resolve a string to some sort of location. The string
/// can represent:
///
/// * An IP address:
/// Uses GeoIp, which checks the Geolite2 database, then falls back to
/// ip2location.info if not found. ip2location has a permanent cache
/// * Domain name:
/// Resolves the domain name to an IP address, then follows the IP flow
/// * Airport code:
/// Uses Airports, which uses openflights data to determine location
/// * Place name (also "special location, when a user uses '~' as the query):
/// Uses Nominatum (open street map) online service to resolve. This also
/// has a permanent cache
pub const Resolver = struct { pub const Resolver = struct {
allocator: std.mem.Allocator, allocator: std.mem.Allocator,
geoip: ?*GeoIp, geoip: ?*GeoIp,

View file

@ -90,13 +90,12 @@ test {
_ = @import("cache/Lru.zig"); _ = @import("cache/Lru.zig");
_ = @import("weather/Mock.zig"); _ = @import("weather/Mock.zig");
_ = @import("http/RateLimiter.zig"); _ = @import("http/RateLimiter.zig");
_ = @import("http/QueryParams.zig"); _ = @import("http/query.zig");
_ = @import("http/help.zig"); _ = @import("http/help.zig");
_ = @import("render/Formatted.zig"); _ = @import("render/line.zig");
_ = @import("render/Line.zig"); _ = @import("render/json.zig");
_ = @import("render/Json.zig"); _ = @import("render/v2.zig");
_ = @import("render/V2.zig"); _ = @import("render/custom.zig");
_ = @import("render/Custom.zig");
_ = @import("location/GeoIp.zig"); _ = @import("location/GeoIp.zig");
_ = @import("location/GeoCache.zig"); _ = @import("location/GeoCache.zig");
_ = @import("location/Airports.zig"); _ = @import("location/Airports.zig");

View file

@ -99,10 +99,9 @@ fn countInvisible(bytes: []const u8, format: Format) usize {
pub const RenderOptions = struct { pub const RenderOptions = struct {
narrow: bool = false, narrow: bool = false,
quiet: bool = false,
super_quiet: bool = false,
days: u8 = 3, days: u8 = 3,
use_imperial: bool = false, use_imperial: bool = false,
no_caption: bool = false,
format: Format = .ansi, format: Format = .ansi,
}; };
@ -112,11 +111,8 @@ pub fn render(allocator: std.mem.Allocator, data: types.WeatherData, options: Re
const w = &output.writer; const w = &output.writer;
if (options.format == .html) try w.writeAll("<pre>"); if (options.format == .html) try w.writeAll("<pre>");
if (!options.super_quiet) if (!options.no_caption)
try w.print( try w.print("Weather report: {s}\n\n", .{data.locationDisplayName()});
"{s}{s}\n\n",
.{ if (!options.quiet) "Weather report: " else "", data.locationDisplayName() },
);
try renderCurrent(w, data.current, options); try renderCurrent(w, data.current, options);
@ -230,33 +226,21 @@ fn renderForecastDay(w: *std.Io.Writer, day: types.ForecastDay, options: RenderO
try date_time.gofmt(date_stream.writer(), "Mon _2 Jan"); try date_time.gofmt(date_stream.writer(), "Mon _2 Jan");
const date_len = date_stream.pos; const date_len = date_stream.pos;
if (!options.narrow) {
try w.writeAll(" ┌─────────────┐\n"); try w.writeAll(" ┌─────────────┐\n");
try w.print("┌──────────────────────────────┬───────────────────────┤ {s} ├───────────────────────┬──────────────────────────────┐\n", .{ try w.print("┌──────────────────────────────┬───────────────────────┤ {s} ├───────────────────────┬──────────────────────────────┐\n", .{
date_str[0..date_len], date_str[0..date_len],
}); });
try w.writeAll("│ Morning │ Noon └──────┬──────┘ Evening │ Night │\n"); try w.writeAll("│ Morning │ Noon └──────┬──────┘ Evening │ Night │\n");
try w.writeAll("├──────────────────────────────┼──────────────────────────────┼──────────────────────────────┼──────────────────────────────┤\n"); try w.writeAll("├──────────────────────────────┼──────────────────────────────┼──────────────────────────────┼──────────────────────────────┤\n");
} else {
// narrow mode
try w.writeAll(" ┌─────────────┐\n");
try w.print("┌───────────────────────┤ {s} ├──────────────────────┐\n", .{
date_str[0..date_len],
});
try w.writeAll("│ Noon └──────┬──────┘ Night │\n");
try w.writeAll("├──────────────────────────────┼─────────────────────────────┤\n");
}
const last_cell: u3 = if (options.narrow) 2 else 4;
for (0..5) |line| { for (0..5) |line| {
try w.writeAll(""); try w.writeAll("");
for (selected_hours[0..4], 0..) |maybe_hour, i| { for (selected_hours[0..4], 0..) |maybe_hour, i| {
if (options.narrow and i % 2 == 0) continue;
if (maybe_hour) |hour| if (maybe_hour) |hour|
try renderHourlyCell(w, hour, line, options) try renderHourlyCell(w, hour, line, options)
else else
try w.splatByteAll(' ', total_cell_width); try w.splatByteAll(' ', total_cell_width);
if (i < last_cell - 1) { if (i < 3) {
try w.writeAll(""); try w.writeAll("");
} else { } else {
try w.writeAll(""); try w.writeAll("");
@ -265,10 +249,7 @@ fn renderForecastDay(w: *std.Io.Writer, day: types.ForecastDay, options: RenderO
try w.writeAll("\n"); try w.writeAll("\n");
} }
if (!options.narrow) try w.writeAll("└──────────────────────────────┴──────────────────────────────┴──────────────────────────────┴──────────────────────────────┘\n");
try w.writeAll("└──────────────────────────────┴──────────────────────────────┴──────────────────────────────┴──────────────────────────────┘\n")
else
try w.writeAll("└──────────────────────────────┴─────────────────────────────┘\n");
} }
const total_cell_width = 28; const total_cell_width = 28;
@ -923,7 +904,7 @@ test "unknown weather code art" {
} }
test "temperature matches between ansi and custom format" { test "temperature matches between ansi and custom format" {
const custom = @import("Custom.zig"); const custom = @import("custom.zig");
const data = types.WeatherData{ const data = types.WeatherData{
.location = "PDX", .location = "PDX",

View file

@ -16,7 +16,7 @@ pub fn render(allocator: std.mem.Allocator, weather: types.WeatherData) ![]const
.weather = weather.forecast, .weather = weather.forecast,
}; };
return try std.fmt.allocPrint(allocator, "{f}", .{std.json.fmt(data, .{})}); return try std.fmt.allocPrint(allocator, "{any}", .{std.json.fmt(data, .{})});
} }
test "render json format" { test "render json format" {