Andreas/mtg_python_deckbuilder
1
0
Fork 0
mirror of https://github.com/mwisnowski/mtg_python_deckbuilder.git synced 2025-12-16 15:40:12 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions

docs: refresh docker and readme guides

Browse source
This commit is contained in:
matt 2025-10-02 16:28:19 -07:00
parent 0448419d9f
commit 84749da214
7 changed files with 1737 additions and 208 deletions
Download patch file Download diff file Expand all files Collapse all files

228
docs/archive/DOCKER_2025-10-02.md Normal file
View file

@ -0,0 +1,228 @@
# Docker Guide
Run the MTG Deckbuilder (CLI and Web UI) in Docker with persistent volumes and optional headless mode.
## Quick start
### PowerShell (recommended)
```powershell
docker compose build
docker compose run --rm mtg-deckbuilder
```
### From Docker Hub (PowerShell)
```powershell
docker run -it --rm `
-v "${PWD}/deck_files:/app/deck_files" `
-v "${PWD}/logs:/app/logs" `
-v "${PWD}/csv_files:/app/csv_files" `
-v "${PWD}/owned_cards:/app/owned_cards" `
-v "${PWD}/config:/app/config" `
mwisnowski/mtg-python-deckbuilder:latest
```
## Web UI (new)
The web UI runs the same deckbuilding logic behind a browser-based interface.
### PowerShell (recommended)
```powershell
docker compose up --build --no-deps -d web
```
Then open http://localhost:8080
Volumes are the same as the CLI service, so deck exports/logs/configs persist in your working folder.
The app serves a favicon at `/favicon.ico` and exposes a health endpoint at `/healthz`.
Compare view offers a Copy summary button to copy a plain-text diff of two runs. The sidebar has a subtle depth shadow for clearer separation.
Web UI feature highlights:
- Locks: Click a card or the lock control in Step 5; locks persist across reruns.
- Replace: Enable Replace in Step 5, click a card to open Alternatives (filters include Owned-only), then choose a swap.
- Permalinks: Copy a permalink from Step 5 or a Finished deck; paste via “Open Permalink…” to restore.
- Compare: Use the Compare page from Finished Decks; quick actions include Latest two and Swap A/B.
Virtualized lists and lazy images (opt‑in)
- Set `WEB_VIRTUALIZE=1` to enable virtualization in Step 5 grids/lists and the Owned library for smoother scrolling on large sets.
- Example (Compose):
```yaml
services:
web:
environment:
- WEB_VIRTUALIZE=1
```
- Example (Docker Hub):
```powershell
docker run --rm -p 8080:8080 `
-e WEB_VIRTUALIZE=1 `
-v "${PWD}/deck_files:/app/deck_files" `
-v "${PWD}/logs:/app/logs" `
-v "${PWD}/csv_files:/app/csv_files" `
-v "${PWD}/owned_cards:/app/owned_cards" `
-v "${PWD}/config:/app/config" `
-e SHOW_DIAGNOSTICS=1 ` # optional: enables diagnostics tools and overlay
mwisnowski/mtg-python-deckbuilder:latest `
bash -lc "cd /app && uvicorn code.web.app:app --host 0.0.0.0 --port 8080"
```
### Diagnostics and logs (optional)
Enable internal diagnostics and a read-only logs viewer with environment flags.
- `SHOW_DIAGNOSTICS=1` — adds a Diagnostics nav link and `/diagnostics` tools
- `SHOW_LOGS=1` — enables `/logs` and `/status/logs?tail=200`
When enabled:
- `/logs` supports an auto-refresh toggle with interval, a level filter (All/Error/Warning/Info/Debug), and a Copy button to copy the visible tail.
- `/status/sys` returns a simple system summary (version, uptime, UTC server time, and feature flags) and is shown on the Diagnostics page when `SHOW_DIAGNOSTICS=1`.
- Virtualization overlay: press `v` on pages with virtualized grids to toggle per-grid overlays and a global summary bubble.
Compose example (web service):
```yaml
environment:
- SHOW_LOGS=1
- SHOW_DIAGNOSTICS=1
```
Docker Hub (PowerShell) example:
```powershell
docker run --rm `
-p 8080:8080 `
-e SHOW_LOGS=1 -e SHOW_DIAGNOSTICS=1 -e ENABLE_THEMES=1 -e THEME=system `
-e SPLASH_ADAPTIVE=1 -e SPLASH_ADAPTIVE_SCALE="1:1.0,2:1.0,3:1.0,4:0.6,5:0.35" ` # optional experiment
-e RANDOM_MODES=1 -e RANDOM_UI=1 -e RANDOM_MAX_ATTEMPTS=5 -e RANDOM_TIMEOUT_MS=5000 `
-v "${PWD}/deck_files:/app/deck_files" `
-v "${PWD}/logs:/app/logs" `
-v "${PWD}/csv_files:/app/csv_files" `
-v "${PWD}/owned_cards:/app/owned_cards" `
-v "${PWD}/config:/app/config" `
mwisnowski/mtg-python-deckbuilder:latest `
bash -lc "cd /app && uvicorn code.web.app:app --host 0.0.0.0 --port 8080"
```
### Setup speed: parallel tagging (Web)
First-time setup or stale data triggers card tagging. The web service uses parallel workers by default.
Configure via environment variables on the `web` service:
- `WEB_TAG_PARALLEL=1|0` — enable/disable parallel tagging (default: 1)
- `WEB_TAG_WORKERS=<N>` — number of worker processes (default: 4 in compose)
If parallel initialization fails, the service falls back to sequential tagging and continues.
### From Docker Hub (PowerShell)
If you prefer not to build locally, pull `mwisnowski/mtg-python-deckbuilder:latest` and run uvicorn:
```powershell
docker run --rm `
-p 8080:8080 `
-v "${PWD}/deck_files:/app/deck_files" `
-v "${PWD}/logs:/app/logs" `
-v "${PWD}/csv_files:/app/csv_files" `
-v "${PWD}/owned_cards:/app/owned_cards" `
-v "${PWD}/config:/app/config" `
mwisnowski/mtg-python-deckbuilder:latest `
bash -lc "cd /app && uvicorn code.web.app:app --host 0.0.0.0 --port 8080"
```
Health check:
```text
GET http://localhost:8080/healthz -> { "status": "ok", "version": "dev", "uptime_seconds": 123 }
```
Theme preference reset (client-side): use the header’s Reset Theme control to clear the saved browser preference; the server default (THEME) applies on next paint.
### Random Modes (alpha) and test dataset override
Enable experimental Random Modes and UI controls in Web runs by setting:
```yaml
services:
web:
environment:
- RANDOM_MODES=1
- RANDOM_UI=1
- RANDOM_MAX_ATTEMPTS=5
- RANDOM_TIMEOUT_MS=5000
```
For deterministic tests or development, you can point the app to a frozen dataset snapshot:
```yaml
services:
web:
environment:
- CSV_FILES_DIR=/app/csv_files/testdata
```
### Taxonomy snapshot (maintainers)
Capture the current bracket taxonomy into an auditable JSON file inside the container:
```powershell
docker compose run --rm web bash -lc "python -m code.scripts.snapshot_taxonomy"
```
Artifacts appear under `./logs/taxonomy_snapshots/` on your host via the mounted volume.
To force a new snapshot even when the content hash matches the latest, pass `--force` to the module.
## Volumes
- `/app/deck_files` ↔ `./deck_files`
- `/app/logs` ↔ `./logs`
- `/app/csv_files` ↔ `./csv_files`
- `/app/owned_cards` ↔ `./owned_cards` (owned cards lists: .txt/.csv)
- Optional: `/app/config` ↔ `./config` (JSON configs for headless)
## Interactive vs headless
- Interactive: attach a TTY (compose run or `docker run -it`)
- Headless auto-run:
```powershell
docker compose run --rm -e DECK_MODE=headless mtg-deckbuilder
```
- Headless with JSON config:
```powershell
docker compose run --rm `
-e DECK_MODE=headless `
-e DECK_CONFIG=/app/config/deck.json `
mtg-deckbuilder
```
### Common env vars
- DECK_MODE=headless
- DECK_CONFIG=/app/config/deck.json
- DECK_COMMANDER, DECK_PRIMARY_CHOICE
- DECK_ADD_LANDS, DECK_FETCH_COUNT
- DECK_TAG_MODE=AND|OR (combine mode used by the builder)
### Web UI tuning env vars
- WEB_TAG_PARALLEL=1|0 (parallel tagging on/off)
- WEB_TAG_WORKERS=<N> (process count; set based on CPU/memory)
- WEB_VIRTUALIZE=1 (enable virtualization)
- SHOW_DIAGNOSTICS=1 (enables diagnostics pages and overlay hotkey `v`)
- RANDOM_MODES=1 (enable random build endpoints)
- RANDOM_UI=1 (show Surprise/Theme/Reroll/Share controls)
- RANDOM_MAX_ATTEMPTS=5 (cap retry attempts)
- (Upcoming) Multi-theme inputs: once UI ships, Random Mode will accept `primary_theme`, `secondary_theme`, `tertiary_theme` fields; current backend already supports the cascade + diagnostics.
- RANDOM_TIMEOUT_MS=5000 (per-build timeout in ms)
Testing/determinism helper (dev):
- CSV_FILES_DIR=csv_files/testdata — override CSV base dir to a frozen set for tests
## Manual build/run
```powershell
docker build -t mtg-deckbuilder .
docker run -it --rm `
-v "${PWD}/deck_files:/app/deck_files" `
-v "${PWD}/logs:/app/logs" `
-v "${PWD}/csv_files:/app/csv_files" `
-v "${PWD}/owned_cards:/app/owned_cards" `
-v "${PWD}/config:/app/config" `
mtg-deckbuilder
```
## Troubleshooting
- No prompts? Use `docker compose run --rm` (not `up`) or add `-it` to `docker run`
- Files not saving? Verify volume mounts and that folders exist
- Headless not picking config? Ensure `./config` is mounted to `/app/config` and `DECK_CONFIG` points to a JSON file
- Owned-cards prompt not seeing files? Ensure `./owned_cards` is mounted to `/app/owned_cards`
## Tips
- Use `docker compose run`, not `up`, for interactive mode
- Exported decks appear in `deck_files/`
- JSON run-config is exported only in interactive runs; headless skips it

951
docs/archive/README_2025-10-02.md Normal file
View file

@ -0,0 +1,951 @@
# 🃏 MTG Python Deckbuilder
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Python 3.11+](https://img.shields.io/badge/python-3.11+-blue.svg)](https://www.python.org/downloads/)
[![Docker](https://img.shields.io/badge/docker-supported-blue.svg)](https://www.docker.com/)
A fast, menu-driven deckbuilder for MTG Commander/EDH with both CLI and Web UI. Supports interactive and headless runs, CSV/TXT exports, owned-card libraries, and Docker.
## 🔹 Features
- Web UI and CLI, powered by the same builder
- Commander search with smart theme tagging (primary/secondary/tertiary)
- Power bracket selection and ideal count prompts
- **Include/Exclude Card Lists**: Must-include and must-exclude functionality with fuzzy matching, visual validation, and strict enforcement (set `ALLOW_MUST_HAVES=true`)
- CSV and TXT exports with stable filenames
- Headless mode (non-interactive) and a headless submenu in the main menu
- Config precedence: CLI > env > JSON > defaults
- Visual summaries in Web UI: Mana curve, Color pips and Sources with hover-to-highlight and copyable tooltips
- Web UI: Locks, Replace flow, Compare builds, and shareable permalinks
- Unified “New Deck” modal (steps 1–3): commander search, themes, bracket, options, and optional Name (used in export filenames)
- Multi-copy archetypes (opt-in modal): choose quantity for supported archetypes and optionally add Thrumming Stone; added early with target adjustments and a hard 100-card clamp
- Combos & Synergies: detect curated pairs, show chip-style lists with badges, and dual-card hover previews; optional auto-complete adds missing partners based on your plan.
## ✨ Highlights
- Smart tagging and suggestions for commander + themes, with AND/OR combine modes
- Theme governance: generated theme catalog with whitelist normalization, enforced synergies, and capped synergy lists (top 5) for concise UI; roadmap includes YAML authoring export.
- Exports CSV and TXT decklists to `deck_files/`
- Random Full Build (alpha) now outputs a single authoritative artifact set (CSV, TXT, compliance JSON, summary JSON) without duplicate suffixed files; API returns paths for integration. Summary sidecars carry multi-theme metadata (`primary_theme`, `secondary_theme`, `tertiary_theme`, `resolved_themes`, fallback flags) plus legacy aliases for incremental consumer upgrades.
- Random Mode (alpha) multi-theme groundwork: curated pools now support manual exclusions via `config/random_theme_exclusions.yml` (documented in `docs/random_theme_exclusions.md` and exportable with `python code/scripts/report_random_theme_pool.py --write-exclusions`); UI/diagnostics surface fallback telemetry while the upcoming picker adds Primary/Secondary/Tertiary inputs with deterministic fallback (P+S+T → P+S → P+T → P → synergy overlap → full pool) and an inline “Clear themes” control quickly resets stored inputs.
- Random Mode instrumentation: `python code/scripts/profile_multi_theme_filter.py` captures mean/p95 filter timings, while `python code/scripts/check_random_theme_perf.py` guards against regressions relative to `config/random_theme_perf_baseline.json` (`--update-baseline` refreshes).
- Owned-cards mode: point to one or more `.txt`/`.csv` files in `owned_cards/` and choose "Use only owned cards?"
- If you opt out, final CSV marks owned cards with an `Owned` column
- Build options include Prefer-owned: bias picks toward owned cards while allowing unowned fallback
- Interactive menu with a headless submenu
- Works locally or in Docker (Windows PowerShell friendly)
- Card images and data via Scryfall (attribution shown in the Web UI)
- Web visual summaries: cross-highlight cards from charts; sources include non-lands and colorless 'C' with an on/off toggle
- New Deck modal keyboard UX: Enter selects the first suggestion; arrow navigation removed; browser autofill disabled
- Export naming: optional Name in the modal becomes the filename stem for CSV/TXT/JSON; decks include a sidecar `.summary.json` with `meta.name`
- Finished Decks and deck banners prefer your custom Name when present
- Compare page includes a Copy summary button to quickly share diffs
- New Deck modal shows your selected themes and their order (1 → 3) while picking; Bracket options are labeled (e.g., "Bracket 3: Upgraded"). Default bracket is 3.
- Bracket policy enforcement: disallowed categories (e.g., Game Changers in Brackets 1–2) are pruned from the pool up-front; UI blocks progression until violations are resolved, with inline Replace to pick compliant alternatives.
- Bracket compliance UX: the compliance panel auto-opens when non-compliant and shows a colored overall chip (green/WARN amber/red). WARN thresholds (e.g., tutors/extra turns) are advisory—tiles show with a subtle style and no gating; FAIL continues to gate and enables enforcement.
### Commander eligibility rules (clarified)
The builder now applies stricter Commander legality filtering when generating `commander_cards.csv`:
1. Automatically eligible:
- Legendary Creatures (includes Artifact Creature / Enchantment Creature lines)
- Legendary Artifact Vehicles or Spacecraft that have printed power and toughness
- Any card whose rules text explicitly contains "can be your commander"
2. Not auto‑eligible (unless they have the explicit text above):
- Plain Legendary Planeswalkers
- Plain Legendary Enchantments without the Creature type
- Generic Legendary Artifacts (non‑Vehicle/Spacecraft or without P/T)
This prevents over-broad inclusion of non-creature legendary permanents.
## 🚀 Quick start
### Docker Hub (PowerShell)
```powershell
docker run -it --rm `
-v "${PWD}/deck_files:/app/deck_files" `
-v "${PWD}/logs:/app/logs" `
-v "${PWD}/csv_files:/app/csv_files" `
-v "${PWD}/owned_cards:/app/owned_cards" `
-v "${PWD}/config:/app/config" `
mwisnowski/mtg-python-deckbuilder:latest
```
### From source
```powershell
pip install -r requirements.txt
python code/main.py
```
### From this repo with Docker Compose (PowerShell)
```powershell
docker compose build
docker compose run --rm mtg-deckbuilder
```
## 🌐 Web UI
Run the browser-based UI backed by the same deckbuilding engine.
### From source
```powershell
pip install -r requirements.txt
uvicorn code.web.app:app --host 127.0.0.1 --port 8080
```
Open http://127.0.0.1:8080
### Docker Compose (PowerShell)
```powershell
docker compose build web
docker compose up --no-deps web
```
Then open http://localhost:8080
Notes:
- Exports/logs/configs use the same folders and will appear under your mounted volumes (see compose file).
- The footer includes Scryfall attribution per their guidelines.
- Favicon is bundled; browsers load `/favicon.ico` (auto-falls back to PNG).
- Bracket enforcement is applied in both web and headless runs; see "Bracket policy & enforcement" below.
### Run with Docker Compose (Docker Hub image)
Use the prebuilt image with the provided `dockerhub-docker-compose.yml`. The image defaults to the Web UI.
Web (default):
```powershell
docker compose -f dockerhub-docker-compose.yml up -d
```
Pin to a version (edit the compose file image tag to `:X.Y.Z`) to avoid `latest` drift.
CLI mode (override default):
```powershell
docker compose -f dockerhub-docker-compose.yml run --rm `
-e APP_MODE=cli `
web
```
Notes:
- Volumes under `${PWD}` persist your exports/logs/configs locally next to the compose file.
- Health checks are built into the image for the Web UI. In CLI, `APP_MODE=cli` bypasses health.
#### Theme defaults and flags
- Enable the Theme selector with `ENABLE_THEMES=1` (on by default in `web` in compose).
- Set the initial default with `THEME=system|light|dark` (applies only when the browser has no prior preference saved):
- `THEME=system` (recommended): follows the OS theme; Light uses the consolidated Light (Blend) palette.
- `THEME=light`: maps to the consolidated Light (Blend) theme in the UI.
- `THEME=dark`: uses the dark theme.
- You can also force a one-off override via URL: `?theme=system|light|dark|high-contrast|cb-friendly`.
- Example: `http://localhost:8080/?theme=dark`
- The choice is saved in localStorage; the URL parameter is removed after applying.
- Reset: Use the “Reset theme” control in the header to clear your preference and re‑apply the server default (or system).
- YAML export strategy: By default the app now always regenerates per-theme YAML files even on fast refreshes to keep editorial artifacts synchronized with `theme_list.json`. To opt-out on incremental refreshes (rare), set `THEME_YAML_FAST_SKIP=1` (compose variable). This only skips export during fast-path (no retag) theme refreshes and never during full builds.
#### Theme catalog editorial pipeline
Build script: `python code/scripts/build_theme_catalog.py`
Key flags / env vars:
- `--limit N` (preview subset; guarded from overwriting canonical JSON unless `--allow-limit-write`)
- `--output path` (write catalog to alternate location; suppresses YAML backfill so tests don't mutate source files)
- `--backfill-yaml` or `EDITORIAL_BACKFILL_YAML=1` (write auto `description` + derived/pinned `popularity_bucket` into each YAML if missing)
- `--force-backfill-yaml` (overwrite existing description/popularity_bucket)
- `EDITORIAL_SEED=<int>` (deterministic ordering for inference and any randomized fallbacks)
- `EDITORIAL_AGGRESSIVE_FILL=1` (pad ultra-sparse themes with extra inferred synergies)
- `EDITORIAL_POP_BOUNDARIES="a,b,c,d"` (override popularity bucket thresholds)
- `EDITORIAL_POP_EXPORT=1` (emit `theme_popularity_metrics.json` summary)
Derived fields:
- `popularity_bucket` (Very Common / Common / Uncommon / Niche / Rare) based on summed color frequency; authors may pin a value in YAML.
- `description` auto-generated if absent using heuristics keyed to theme class (tribal, counters, graveyard, tokens, etc.).
Editorial quality & metadata info:
- Optional `editorial_quality: draft|reviewed|final` can be set in per-theme YAML (or force-backfilled) to track curation progress.
- Backfills now stamp a `metadata_info` block (formerly `provenance`) with keys such as `last_backfill`, `script`, and `version`. The legacy key `provenance` is still parsed for a short deprecation window; if both appear a warning is emitted (suppress via `SUPPRESS_PROVENANCE_DEPRECATION=1`). Planned removal of the legacy alias: version 2.4.0.
- A running specialization trend is appended to `config/themes/description_fallback_history.jsonl` whenever `EDITORIAL_INCLUDE_FALLBACK_SUMMARY=1` is set.
- When `EDITORIAL_INCLUDE_FALLBACK_SUMMARY=1`, the catalog additionally embeds a `description_fallback_summary` object exposing coverage KPIs (see below) to power regression tests.
Synergy pairs & clusters:
- `config/themes/synergy_pairs.yml` provides curated fallback synergies (applied only if a theme YAML lacks `curated_synergies`).
- `config/themes/theme_clusters.yml` groups themes (Tokens, Counters, Graveyard, etc.) for upcoming UI filters and analytics.
- Validator checks structure; UI integration planned in Phase E.
Description mapping externalization:
- `config/themes/description_mapping.yml` fully replaces internal mapping when present.
- Items: `triggers` (list of substrings, lowercase-insensitive) and `description` string.
- `{SYNERGIES}` placeholder expands to a short clause with top synergy names or is removed gracefully if none.
- Mapping validator (`scripts/validate_description_mapping.py`) reports duplicate triggers, suggests `{SYNERGIES}` usage, and validates synergy pairs & clusters.
KPI targets (initial schedule):
- Wave 1 (current): Maintain regression test thresholds (generic_pct < 52%).
- Wave 2: Ratchet to <45% after next mapping expansion.
- Wave 3: <38%; Wave 4: <30% (issue templates will adjust thresholds once prior target is achieved and history confirms downward trend).
Lint (`code/scripts/lint_theme_editorial.py`) enhancements:
- `--require-description` / env `EDITORIAL_REQUIRE_DESCRIPTION=1` enforces descriptions present.
- `--require-popularity` / env `EDITORIAL_REQUIRE_POPULARITY=1` enforces popularity bucket present.
- Strict mode also escalates missing cornerstone examples and missing description/popularity to errors.
Testing:
Determinism, boundary overrides, and backfill isolation validated in `code/tests/test_theme_catalog_generation.py` using the new `--output` flag.
### Commander catalog upkeep
- The Commander Browser and auto-select flows read from `csv_files/commander_cards.csv`; keep it version-controlled so web and CLI stay in sync.
- Regenerate the file after MTGJSON updates or commander policy tweaks:
```powershell
python -c "from file_setup.setup import regenerate_csvs_all; regenerate_csvs_all()"
```
- For quick fixes, you can refresh just the commander list:
```powershell
python -c "from file_setup.setup import determine_commanders; determine_commanders()"
```
- Full onboarding guidance (required columns, caching notes, validation steps) lives in `docs/commander_catalog.md`.
- Staging toggle: set `SHOW_COMMANDERS=0` to hide the sidebar link while testing privately; defaults to on.
### Theme Catalog API
Two new endpoints expose the merged catalog for the upcoming theme picker UI.
List themes:
`GET /themes/api/themes`
Parameters:
- `q` substring across theme name & synergies
- `archetype` filter by `deck_archetype`
- `bucket` popularity bucket (Very Common|Common|Uncommon|Niche|Rare)
- `colors` comma-separated color initials (e.g. `G,W`)
- `limit` (default 50, max 200) & `offset`
- `diagnostics=1` (requires env `WEB_THEME_PICKER_DIAGNOSTICS=1`) to expose `has_fallback_description` + `editorial_quality`
Sample response snippet:
```
{
"ok": true,
"count": 314,
"items": [
{
"id": "plus1-plus1-counters",
"theme": "+1/+1 Counters",
"primary_color": "Green",
"secondary_color": "White",
"popularity_bucket": "Very Common",
"deck_archetype": "Counters",
"description": "...",
"synergies": ["Adapt","Evolve","Proliferate","Counters Matter","Hydra Kindred"],
"synergy_count": 5
// diagnostics-only: has_fallback_description, editorial_quality
}
],
"next_offset": 50,
"generated_at": "2025-09-19T12:34:56",
"diagnostics": false
}
```
Theme detail:
`GET /themes/api/theme/{id}`
Parameters:
- `uncapped=1` (diagnostics only) returns `uncapped_synergies` (full curated+enforced+inferred ordered set)
- `diagnostics=1` enables extra fields (fallback + editorial quality)
Detail response excerpt:
```
{
"ok": true,
"theme": {
"id": "plus1-plus1-counters",
"theme": "+1/+1 Counters",
"description": "...",
"synergies": ["Adapt","Evolve","Proliferate","Counters Matter","Hydra Kindred"],
"curated_synergies": [...],
"enforced_synergies": [...],
"inferred_synergies": [...],
"example_commanders": [...],
"example_cards": [...],
"synergy_commanders": [...]
// diagnostics-only: uncapped_synergies, editorial_quality, has_fallback_description
}
}
```
Caching & freshness: ETag header reflects catalog size/mtime + newest YAML mtime. Use `/themes/status` for stale detection; POST `/themes/refresh` to rebuild in background.
Environment flag: `WEB_THEME_PICKER_DIAGNOSTICS=1` enables diagnostics extras and uncapped synergy toggle.
Performance & optimization:
- Fast list filtering uses precomputed lowercase search haystacks + memoized filtered slug cache (keyed by catalog ETag and query params) for sub‑50ms typical responses once warm.
- Skeleton loaders (picker shell, table list fragment, preview modal) improve perceived load time; real content swaps in via HTMX fragment requests.
- Metrics endpoint `/themes/metrics` (gated by `WEB_THEME_PICKER_DIAGNOSTICS=1`) exposes:
- Catalog filter requests, cache hits, cache entry count
- Preview requests, cache hits, average build time (ms), cache size, TTL
- Response headers: `X-ThemeCatalog-Filter-Duration-ms` and `ETag` let you observe server-side filter cost and enable conditional requests.
### Governance & Editorial
The theme catalog follows lightweight governance so curation quality scales:
1. Minimum Examples Threshold: Target 2 example cards and 1 example commander for every established theme. Enforcement flips from warning to required once coverage >90% (tracked via metrics). Until then, themes below threshold are flagged but not blocked.
2. Curation Ordering: Preview assembly order is deterministic — examples, curated synergy examples, sampled role cards (payoff/enabler/support/wildcard), then synthetic placeholders only if needed to reach target count.
3. Splash Relax Policy: Four- and five-color commanders may include a single off-color enabler (scored with a -0.3 penalty) to avoid over-pruning multi-color creative lines. This is documented to prevent accidental removal during future sampling refactors.
4. Popularity Buckets Are Non-Scoring: Popularity is for filtering and subtle UI hints only and must never directly influence sampling scores (avoids positive feedback loops that crowd out niche archetypes).
5. Taxonomy Expansion Criteria: Proposed new high-level themes (Combo, Storm, Extra Turns, Group Hug / Politics, Pillowfort, Toolbox / Tutors, Treasure Matters, Monarch / Initiative) must demonstrate: (a) a distinct strategic pattern, (b) at least 8 broadly played representative cards, (c) not a strict subset of another existing theme.
6. Editorial Quality Tracking: Optional `editorial_quality: draft|reviewed|final` in theme YAML guides review pass prioritization; metrics aggregate coverage to spot stagnation.
7. Deterministic Sampling: Seed = `theme|commander` hash; changes to sampling heuristics must preserve deterministic output for identical inputs (regression tests rely on this). Score contributors must append reasons for transparency (`reasons[]`).
For deeper rationale & checklist see `docs/theme_taxonomy_rationale.md`.
- Preview sampling caches results (TTL 600s) keyed by (slug, limit, colors, commander, ETag) ensuring catalog changes invalidate prior samples.
Governance tooling & experiments:
- Taxonomy snapshot: `python -m code.scripts.snapshot_taxonomy` writes `logs/taxonomy_snapshots/taxonomy_<timestamp>.json` with a SHA-256 `hash` for auditability. Identical content is skipped unless forced. Use this before tuning taxonomy-aware sampling heuristics.
- Adaptive splash penalty (optional): set `SPLASH_ADAPTIVE=1` to scale off-color enabler penalties based on commander color count; tune with `SPLASH_ADAPTIVE_SCALE` (e.g., `1:1.0,2:1.0,3:1.0,4:0.6,5:0.35`). Analytics recognize both static and adaptive reasons; experiments can compare counters when enabled vs disabled.
Preview endpoint:
`GET /themes/api/theme/{id}/preview?limit=12&colors=G,W&commander=Name`
Returns a deterministic sample (seeded by theme + optional commander) combining:
- Pinned curated `example_cards` (role `example`)
- Pinned curated `synergy_example_cards` (role `curated_synergy`)
- Heuristically sampled cards bucketed into payoff / enabler / support / wildcard with diversity quotas (~40/40/20 after pinned examples)
- Synthetic `[Synergy]` placeholders only if real sampling underfills the requested limit
Commander bias heuristics (color identity restriction + overlap/theme bonuses) influence scoring & ordering.
Preview sampling now includes curated examples, curated synergy layer, role-based heuristic sampling (payoff / enabler / support / wildcard) with diversity quotas, commander bias heuristics (color identity restriction + diminishing overlap bonuses), rarity diminishing weights, splash leniency (single off-color allowance with small penalty for 4–5 color commanders), role saturation penalty, refined overlap scaling curve, and tooltip-expanded heuristic reasons. Synthetic placeholders are only added if the sample underfills.
Edge considerations (Theme picker & preview):
- Empty dataset: graceful skeleton + 'No themes match your filters' message.
- High latency first load: skeleton shimmer + optional prewarm (when `WEB_THEME_FILTER_PREWARM=1`).
- Catalog reload or tagging completion: filter + preview caches bust; stale indicator triggers background refresh.
- Commander not found: commander bias reasons omitted; sampling still deterministic via seed.
- Color filter reduces pool below quota: synthetic `[Synergy]` placeholders pad to requested limit.
- Duplicate card names across tag shards: final defensive dedupe pass before payload.
- Oversized preview cache: FIFO eviction maintains <= THEME_PREVIEW_CACHE_MAX (default 400).
- Disabled diagnostics: uncapped synergies, editorial quality, fallback description markers, raw YAML & metrics all suppressed.
- Tooltip safety: JS errors are swallowed; preview remains functional without enhancements.
#### Editorial coverage metrics / specialization progress
- Set `EDITORIAL_INCLUDE_FALLBACK_SUMMARY=1` when running `build_theme_catalog.py` to embed a `description_fallback_summary` block in the resulting JSON. This includes:
- `generic_total`, `generic_with_synergies`, `generic_plain`, and `generic_pct`
- `top_generic_by_frequency` (capped list of the highest-frequency themes still using generic fallback phrasing)
- Used by `test_theme_description_fallback_regression.py` to enforce a shrinking ceiling on generic descriptions after each optimization wave.
- History persisted to `description_fallback_history.jsonl` for KPI trend visualization and automated threshold ratcheting.
#### External description mapping
- Extend or override auto-description heuristics without code edits via `config/themes/description_mapping.yml`.
- Each list item: `triggers: ["substring1", ...]`, `description: "Text with optional {SYNERGIES} placeholder"`
- First match wins; triggers are lowercase substring checks.
- `{SYNERGIES}` is replaced with a short clause (e.g., `Synergies like Tokens and Sacrifice reinforce the plan.`) when synergy context exists; omitted cleanly otherwise.
- If the file is absent, the internal mapping rules are used as a fallback.
### Quick guide (locks • replace • compare • permalinks)
- New Deck modal (Steps 1–3 unified): search for your commander, pick themes, choose bracket (defaults to 3), and optionally set Name. Submitting starts the build immediately. Name becomes the export filename stem and display name.
- Lock a card: click the card image or the lock control in Step 5. Locked cards are pinned for reruns. A small “Last action” chip confirms lock/unlock.
- Replace a card: toggle Replace in Step 5, then click a card to open Alternatives. Filter (including Owned-only) and pick a swap; the deck updates in place.
- When bracket violations exist, Step 5 shows them first and disables Continue/Rerun. Pick replacements inline; alternatives exclude commanders, locked, in-deck, and just-added cards and prefer role-consistent options.
- Permalink: use “Copy permalink” from Step 5 or a Finished deck to share/restore a run (commander, themes, bracket, ideals, flags). “Open Permalink…” lets you paste one to restore to review.
- Compare: from Finished Decks, open Compare and pick A and B (quick actions: Latest two, Swap A/B). Use Changed-only to focus diffs. Copy summary or Download .txt.
- Keyboard: In the New Deck modal, Enter selects the first commander result. Browser autofill is disabled for cleaner inputs. In Step 5, use L to lock/unlock the focused card, R to open Alternatives, and C to copy a permalink.
### Setup speed: parallel tagging (Web)
- On first run or when data is stale, the Web UI prepares and tags the card database.
- This tagging runs in parallel by default for faster setup.
- Tuning via env vars (set on the `web` service in `docker-compose.yml`):
- `WEB_TAG_PARALLEL=1|0` — enable/disable parallel workers (default: 1/enabled)
- `WEB_TAG_WORKERS=<N>` — number of worker processes (default: 4 in compose; omit to auto-pick)
- The UI shows progress and falls back to sequential tagging automatically if parallel init fails.
Virtualized lists and image polish (opt-in)
- Opt-in with `WEB_VIRTUALIZE=1` to enable virtualization in Step 5 lists/grids and the Owned library for smoother scrolling on large sets.
- Virtualization diagnostics overlay (for debugging): enable `SHOW_DIAGNOSTICS=1`, then press `v` to toggle an overlay per grid and a global summary bubble; shows visible range, total, rows, row height, render time, and counters.
- Thumbnails use lazy-loading with srcset/sizes; LQIP blur/fade-in is applied to Step 5 and Owned thumbnails, and the commander preview image.
- Respects reduced motion: when the OS prefers-reduced-motion, blur/fade transitions and smooth scrolling are disabled for accessibility.
### Build options (Web)
- Use only owned cards: builds strictly from your `owned_cards/` lists (commander exempt).
- Prefer owned cards: gently prioritizes owned cards across creatures and spells but allows unowned when they’re better fits.
- Implemented via a stable owned-first reorder and small weight boosts; preserves existing sort intent.
Notes:
- Steps 1–3 are consolidated into a single New Deck modal. Submitting starts the build immediately, skipping the old review page.
- The modal includes an optional Name; when set, it is used as the export filename stem and display name in the UI.
- The modal displays selected themes in order (1, 2, 3). The Bracket selector shows numbers (e.g., "Bracket 3: Upgraded") and defaults to 3 for new decks.
### Staged build visibility
- Step 5 can optionally “Show skipped stages” so you can see phases that added no cards with a clear annotation.
### Multi-copy archetypes (Web)
- When your commander + themes suggest a multi-copy strategy (e.g., petitioners, approach, apostles), the Web UI shows a one-time modal to add a package.
- Choose how many copies (bounded by the printed cap) and optionally include 1× Thrumming Stone when it synergizes.
- The package is applied as the first stage so later phases account for the volume.
- Targets adjust automatically (reduce creatures for creature packages, or spread reductions across spells). The UI shows an “Adjusted targets” note on that stage.
- A final safety clamp trims overflow from this stage to keep the deck at 100. A “Clamped N” chip appears when this happens.
- Suggestions won’t re-prompt repeatedly for the same commander+theme context unless you change selections; you can also dismiss the modal.
### Visual summaries (Web)
- Mana Curve bars with hover-to-highlight of matching cards in both list and thumbnail views
- Color Pips (requirements) and Sources (generation) bars with per-color tooltips listing cards
- Cross-highlighting from charts to the card views; list-mode highlights only the name span
- Sources detection includes non-land producers (artifacts/creatures/etc.) and colorless 'C'
- Fetch lands are not counted as mana sources; basic lands and Wastes are handled reliably
- Optional: Show colorless (C) toggle for Sources; persisted per browser
- Tooltips include a Copy button to copy the card list
### Combos & Synergies (Web)
- Detection: curated two-card combos/synergies are detected from the final deck (commander included) and shown with list version badges.
- UI: chip-style rows with badges (cheap/early, setup). Hover a row to preview both cards side-by-side; hover an individual name for single-card preview.
- Auto-Complete Combos: when enabled in the New Deck modal, the builder completes up to N pairs before theme fill/monolithic spells so partners stick.
- Settings: Prefer combos (on/off), How many combos (target), Balance (early/late/mix) to bias partner selection.
- Existing completed pairs are counted first; only missing partners are added.
- Color identity enforced via the filtered card pool; off-color/unavailable partners are skipped.
### Owned page (Web)
- View and manage your owned lists with:
- Export TXT/CSV, sort controls, and a live “N shown” counter
- Color identity dots and exact color-identity combo filters (including 4-color)
- Viewport-filling list with styled scrollbar
- Optional virtualization when `WEB_VIRTUALIZE=1` (improves performance on very large libraries)
- Uploading `.txt` or `.csv` lists enriches and deduplicates entries at upload-time and persists them, so page loads are fast.
### Finished Decks (Web)
- Theme filters are a dropdown with shareable state for easier browsing of saved builds.
- Each finished deck has a permalink you can copy and revisit; Compare mode lets you diff against another run. The Compare page has a Copy summary button to copy a plain-text diff.
- Locks and Replace flow: lock-in picks you like or replace a card with an alternative during iteration.
### Diagnostics (Web)
- Health endpoint: `GET /healthz` returns `{ status, version, uptime_seconds }`
- Responses include `X-Request-ID` for easier error correlation; unhandled errors return JSON with `request_id`
Logs and system tools:
- Logs page (`/logs`, enable with `SHOW_LOGS=1`):
- Auto-refresh toggle with adjustable interval
- Level filter (All/Error/Warning/Info/Debug) and keyword filter
- Copy button to copy the visible log tail
- System summary (`GET /status/sys`, shown on `/diagnostics` when `SHOW_DIAGNOSTICS=1`):
- Returns `{ version, uptime_seconds, server_time_utc, flags }`
- Shows resolved theme and stored preference; includes a Reset preference button.
Compose: enable diagnostics/logs (optional)
```yaml
services:
web:
environment:
- SHOW_LOGS=1
- SHOW_DIAGNOSTICS=1
# Random Modes (optional; alpha)
- RANDOM_MODES=1
- RANDOM_UI=1
- RANDOM_MAX_ATTEMPTS=5
- RANDOM_TIMEOUT_MS=5000
# RANDOM_BUILD_SUPPRESS_INITIAL_EXPORT: # (Default behavior auto-enables suppression; set to 0 to debug legacy double-export)
```
### Budget Mode and price/legal status
- Price and legality snippet integration is deferred to a dedicated Budget Mode initiative. This will centralize price sourcing, caching, legality checks, and UI surfaces. Track progress in `logs/roadmaps/roadmap_9_budget_mode.md`.
## 🤖 Headless mode (no prompts)
- Auto-headless: set `DECK_MODE=headless`
- Example (PowerShell):
```powershell
docker compose run --rm -e DECK_MODE=headless mtg-deckbuilder
```
- Use a JSON config: mount `./config` to `/app/config` and set `DECK_CONFIG=/app/config/deck.json`
- Example (PowerShell):
```powershell
docker compose run --rm `
-e DECK_MODE=headless `
-e DECK_CONFIG=/app/config/deck.json `
mtg-deckbuilder
```
- Override via env vars (subset): `DECK_COMMANDER`, `DECK_PRIMARY_CHOICE`, `DECK_SECONDARY_CHOICE`, `DECK_TERTIARY_CHOICE`, `DECK_ADD_LANDS`, `DECK_FETCH_COUNT`, `DECK_BRACKET_LEVEL`
- Precedence: CLI > env > JSON > defaults
### Enhanced CLI with Type Safety & Theme Names
The headless runner now features comprehensive CLI arguments with type indicators and intelligent theme selection:
```powershell
# Show all available options with type information
python code/headless_runner.py --help
# Build with theme names instead of index numbers
python code/headless_runner.py `
--commander "Aang, Airbending Master" `
--primary-tag "Airbending" `
--secondary-tag "Exile Matters" `
--bracket-level 4
# Override ideal deck composition counts
```
## 🧪 Theme Catalog Validation & Schema
The theme system now uses a merged catalog plus per-theme YAML files.
Validator usage:
```powershell
python code/scripts/validate_theme_catalog.py # Standard validation
python code/scripts/validate_theme_catalog.py --rebuild-pass # Ensures idempotent rebuild
python code/scripts/validate_theme_catalog.py --schema # Catalog JSON Schema
python code/scripts/validate_theme_catalog.py --yaml-schema # Per-file YAML JSON Schema
python code/scripts/validate_theme_catalog.py --strict-alias # Fail if alias display_name still present
```
CI runs the non-strict validator every push. A second step runs `--strict-alias` as an allowed failure until legacy alias YAMLs (e.g., `reanimator.yml`) are removed, after which the step will be enforced.
Per-file YAML structure (excerpt):
```yaml
id: plus1-plus1-counters
display_name: +1/+1 Counters
curated_synergies: [Proliferate, Counters Matter]
enforced_synergies: []
inferred_synergies: []
synergies: [Proliferate, Counters Matter, Adapt, Evolve]
primary_color: Green
secondary_color: White
notes: ''
```
Add or edit a theme by creating/updating a file in `config/themes/catalog/` and running the build script:
```powershell
python code/scripts/build_theme_catalog.py
```
Metadata info (formerly provenance) is written to `theme_list.json` and consumed by tests and future UI diagnostics. During the deprecation period the builder accepts either key; prefer `metadata_info` in all new tooling / docs.
### Duplicate suppression & example_cards quality controls
Example card generation (and rebuild / padding via `synergy_promote_fill.py`) includes safeguards to reduce noisy staples and over-represented picks:
- `--common-card-threshold <fraction>`: Excludes candidate example cards that already appear in more than this fraction of themes (default 0.18 = 18%). This curbs ubiquitous staples (e.g., Sol Ring) from crowding lists.
- `--print-dup-metrics`: After a run (dry-run or apply) prints global frequency distribution so thresholds can be tuned.
Typical loop (dry run with metrics):
```powershell
python code/scripts/synergy_promote_fill.py --fill-example-cards --common-card-threshold 0.18 --print-dup-metrics
```
Adjustment guidance:
- If many mid-frequency utility cards still leak through: lower the threshold (e.g., 0.15).
- If high-value, legitimately thematic cards are being suppressed prematurely: raise slightly (e.g., 0.22) and re-run with metrics to confirm net effect.
These controls complement existing staple suppression heuristics (`staples_block` list) and color-fallback restraint flags.
## 📝 Theme Editorial Enrichment
Phase D introduces automated editorial helpers to surface representative cards and synergy-informed commander suggestions per theme while preserving manual curation:
### Key scripts
- `code/scripts/generate_theme_editorial_suggestions.py`: Scans card & commander CSVs to propose `example_cards`, `example_commanders`, and a derived `synergy_commanders` list.
- Synergy commanders: up to 3 / 2 / 1 picks from the theme's top three synergies (3/2/1 pattern), falling back to legendary card hits if a synergy lacks direct commander-tagged entries.
- Promotion: ensures a minimum (default 5) example commanders by promoting synergy picks with annotations.
- Annotation format: `Name - Synergy (Synergy Theme)` for both promoted examples and residual synergy-only entries.
- Duplicate filtering: commanders already present (base name) in `example_commanders` are omitted from `synergy_commanders` to avoid redundancy.
- Augmentation (`--augment-synergies`): heuristically pads sparse `synergies` arrays (e.g., injects `Counters Matter`, `Proliferate` for counter variants) before deriving synergy picks.
- `code/scripts/lint_theme_editorial.py`: Validates editorial consistency (annotation correctness, min counts, deduplication, size bounds) and emits warnings (non-fatal by default).
### Usage examples
```powershell
# Dry run (first 25 themes)
python code/scripts/generate_theme_editorial_suggestions.py
# Apply to all themes with augmentation and minimum example commanders of 5
python code/scripts/generate_theme_editorial_suggestions.py --apply --augment-synergies --min-examples 5
# Lint results
python code/scripts/lint_theme_editorial.py
```
### Non-determinism & source control guidance
The editorial output depends on current CSV card data, ranks, and tagging heuristics. Regeneration on a different machine or after data refresh may yield a different ordering or composition. For that reason:
- Commit the tooling and schema changes.
- Prefer selective or manually reviewed YAML edits (especially cornerstone themes) instead of bulk committing the entire regenerated catalog.
- Treat full-catalog regeneration as an operational task (e.g., during a release preparation) rather than every feature branch.
### Schema fields
Per-theme YAML now may include:
```yaml
example_cards: [ ... ] # Representative non-commander cards
example_commanders: [ ... ] # Curated + promoted synergy commanders (annotated)
synergy_commanders: [ ... ] # Remaining annotated synergy picks not already promoted
```
### Lint expectations (default)
- example_commanders: <=12, aim for ≥5 (warnings below minimum)
- example_cards: ≤20
- synergy_commanders: ≤6 (3/2/1 pattern) after filtering duplicates
- Annotations must reference a synergy present in `synergies`.
Cornerstone themes (Landfall, Reanimate, Superfriends, Tokens Matter, +1/+1 Counters) should always have curated coverage; automated promotion fills gaps but manual review is encouraged.
python code/headless_runner.py `
--commander "Krenko, Mob Boss" `
--primary-tag "Goblin Kindred" `
--creature-count 35 `
--land-count 33 `
--ramp-count 12
# Include/exclude specific cards (semicolon for comma-containing names)
python code/headless_runner.py `
--commander "Jace, Vryn's Prodigy" `
--include-cards "Sol Ring;Jace, the Mind Sculptor" `
--exclude-cards "Chaos Orb;Shahrazad" `
--enforcement-mode strict
```
**New CLI Features:**
- **Type-safe help text**: All arguments show expected types (PATH, NAME, INT, BOOL)
- **Ideal count arguments**: `--ramp-count`, `--land-count`, `--creature-count`, etc.
- **Theme tag names**: `--primary-tag "Theme Name"` instead of `--primary-choice 1`
- **Include/exclude CLI**: `--include-cards`, `--exclude-cards` with semicolon support
- **Console diagnostics**: Detailed summary output for validation and results
Headless submenu notes:
- If one JSON exists in `config/`, it auto-runs it
- If multiple exist, they’re listed as "Commander - Theme1, Theme2, Theme3"; `deck.json` shows as "Default"
- CSV/TXT are exported as usual; JSON run-config is exported only in interactive runs
### Run locally (no Docker)
```powershell
# Show resolved settings (no run)
python code/headless_runner.py --config config/deck.json --dry-run
# Run with a specific config file
python code/headless_runner.py --config config/deck.json
# Point to a folder; if exactly one config exists, it's auto-used
python code/headless_runner.py --config config
# Override via CLI
python code/headless_runner.py --commander "Pantlaza, Sun-Favored" --primary-choice 2 --secondary-choice 0 --add-lands true --fetch-count 3
```
### CLI flag reference
Each flag below mirrors an environment variable (env vars override JSON, CLI overrides both). Types match `--help` output.
#### Core selection & flow
| Flag | Type | Description | JSON key / Env var |
| --- | --- | --- | --- |
| `--config PATH` | string | Path to JSON config file or directory (auto-detects a single file) | `DECK_CONFIG` |
| `--commander NAME` | string | Commander search term | `commander` / `DECK_COMMANDER` |
| `--primary-choice INT` | int | Primary theme menu index (1-based) | `primary_choice` / `DECK_PRIMARY_CHOICE` |
| `--secondary-choice INT` | optional int | Secondary theme index | `secondary_choice` / `DECK_SECONDARY_CHOICE` |
| `--tertiary-choice INT` | optional int | Tertiary theme index | `tertiary_choice` / `DECK_TERTIARY_CHOICE` |
| `--primary-tag NAME` | string | Primary theme name (auto-maps to index) | `primary_tag` / `DECK_PRIMARY_TAG` |
| `--secondary-tag NAME` | string | Secondary theme name | `secondary_tag` / `DECK_SECONDARY_TAG` |
| `--tertiary-tag NAME` | string | Tertiary theme name | `tertiary_tag` / `DECK_TERTIARY_TAG` |
| `--bracket-level 1-5` | int | Power bracket selection | `bracket_level` / `DECK_BRACKET_LEVEL` |
| `--dry-run` | flag | Print the resolved configuration and exit | — |
#### Deck composition & counts
| Flag | Type | Description | JSON key / Env var |
| --- | --- | --- | --- |
| `--ramp-count INT` | int | Target ramp spells | `ideal_counts.ramp` |
| `--land-count INT` | int | Total land target | `ideal_counts.lands` |
| `--basic-land-count INT` | int | Minimum basic lands | `ideal_counts.basic_lands` |
| `--creature-count INT` | int | Creature count target | `ideal_counts.creatures` |
| `--removal-count INT` | int | Spot removal target | `ideal_counts.removal` |
| `--wipe-count INT` | int | Board wipe target | `ideal_counts.wipes` |
| `--card-advantage-count INT` | int | Card advantage target | `ideal_counts.card_advantage` |
| `--protection-count INT` | int | Protection spell target | `ideal_counts.protection` |
| `--fetch-count INT` | optional int | Requested fetch land count | `fetch_count` / `DECK_FETCH_COUNT` |
| `--dual-count INT` | optional int | Dual land request | `dual_count` / `DECK_DUAL_COUNT` |
| `--triple-count INT` | optional int | Three-color land request | `triple_count` / `DECK_TRIPLE_COUNT` |
| `--utility-count INT` | optional int | Utility land request | `utility_count` / `DECK_UTILITY_COUNT` |
#### Card type toggles
| Flag | Type | Description | JSON key / Env var |
| --- | --- | --- | --- |
| `--add-lands BOOL` | bool | Run the land builder sequence | `add_lands` / `DECK_ADD_LANDS` |
| `--add-creatures BOOL` | bool | Add creatures | `add_creatures` / `DECK_ADD_CREATURES` |
| `--add-non-creature-spells BOOL` | bool | Bulk add non-creature spells (if available) | `add_non_creature_spells` / `DECK_ADD_NON_CREATURE_SPELLS` |
| `--add-ramp BOOL` | bool | Add ramp spells | `add_ramp` / `DECK_ADD_RAMP` |
| `--add-removal BOOL` | bool | Add removal spells | `add_removal` / `DECK_ADD_REMOVAL` |
| `--add-wipes BOOL` | bool | Add board wipes | `add_wipes` / `DECK_ADD_WIPES` |
| `--add-card-advantage BOOL` | bool | Add card draw | `add_card_advantage` / `DECK_ADD_CARD_ADVANTAGE` |
| `--add-protection BOOL` | bool | Add the protection suite | `add_protection` / `DECK_ADD_PROTECTION` |
#### Include / exclude controls
| Flag | Type | Description | JSON key / Env var |
| --- | --- | --- | --- |
| `--include-cards LIST` | string | Force-include cards (comma or semicolon separated) | `include_cards` |
| `--exclude-cards LIST` | string | Exclude cards | `exclude_cards` |
| `--enforcement-mode warn|strict` | string | Handling when includes cannot be satisfied | `enforcement_mode` |
| `--allow-illegal BOOL` | bool | Permit non-legal cards in include/exclude lists | `allow_illegal` |
| `--fuzzy-matching BOOL` | bool | Enable fuzzy card name matching | `fuzzy_matching` |
#### Random mode (web parity)
| Flag | Type | Description | Env var / JSON key |
| --- | --- | --- | --- |
| `--random-mode` | flag | Force the headless random builder path | `HEADLESS_RANDOM_MODE` / `random_mode` |
| `--random-theme NAME` | string | Legacy single-theme alias (maps to primary) | `RANDOM_THEME` / `random.theme` |
| `--random-primary-theme NAME` | string | Primary theme slug | `RANDOM_PRIMARY_THEME` / `random.primary_theme` |
| `--random-secondary-theme NAME` | string | Secondary theme slug | `RANDOM_SECONDARY_THEME` / `random.secondary_theme` |
| `--random-tertiary-theme NAME` | string | Tertiary theme slug | `RANDOM_TERTIARY_THEME` / `random.tertiary_theme` |
| `--random-auto-fill BOOL` | bool | Auto-fill missing theme slots | `RANDOM_AUTO_FILL` / `random.auto_fill` |
| `--random-auto-fill-secondary BOOL` | bool | Override secondary auto-fill | `RANDOM_AUTO_FILL_SECONDARY` / `random.auto_fill_secondary` |
| `--random-auto-fill-tertiary BOOL` | bool | Override tertiary auto-fill | `RANDOM_AUTO_FILL_TERTIARY` / `random.auto_fill_tertiary` |
| `--random-strict-theme-match BOOL` | bool | Require exact theme matches when selecting commanders | `RANDOM_STRICT_THEME_MATCH` / `random.strict_theme_match` |
| `--random-attempts INT` | int | Retry attempts before giving up | `RANDOM_MAX_ATTEMPTS` / `random.attempts` |
| `--random-timeout-ms INT` | int | Timeout per attempt (milliseconds) | `RANDOM_TIMEOUT_MS` / `random.timeout_ms` |
| `--random-seed VALUE` | int or string | Deterministic seed for reproducible runs | `RANDOM_SEED` / `random.seed` |
| `--random-constraints JSON_OR_PATH` | string | Inline JSON or path to constraint file | `RANDOM_CONSTRAINTS` / `RANDOM_CONSTRAINTS_PATH` / `random.constraints` |
| `--random-output-json PATH` | string | Write random build payload to file or directory | `RANDOM_OUTPUT_JSON` / `random.output_json` |
Booleans accept: 1/0, true/false, yes/no, on/off.
### Random mode parity (web → headless)
The headless runner now shares the web UI's random builder pipeline. Set `--random-mode` (or `HEADLESS_RANDOM_MODE=1`) to route through the Surprise/Reroll flow with multi-theme support, auto-fill assistance, constraints, and deterministic seeds. Combine flags as needed:
```powershell
python code/headless_runner.py `
--random-mode `
--random-primary-theme Artifacts `
--random-secondary-theme Tokens `
--random-auto-fill true `
--random-output-json deck_files/random/latest.json
```
The build prints a full summary (commander, seed, theme stack, fallback reasons, attempts vs timeout, auto-fill status) and writes the optional JSON payload when `--random-output-json`/`RANDOM_OUTPUT_JSON` is provided. Use `--random-constraints` (or `RANDOM_CONSTRAINTS[_PATH]`) to supply pool limits, and `--random-seed` for reproducible reruns. Pair any random flag with `--dry-run` to inspect the resolved configuration without building.
### JSON export in headless
- By default, headless runs do not export a JSON run-config to avoid duplicates.
- Opt-in with:
```powershell
$env:HEADLESS_EXPORT_JSON = "1"
python code/headless_runner.py --config config/deck.json
```
- Tip: when opting in, prefer using `--config` instead of a `DECK_CONFIG` file path to avoid creating both a stem-based JSON and a second explicit-path JSON.
Example JSON (`config/deck.json`):
```json
{
"commander": "Pantlaza",
"bracket_level": 3,
"primary_choice": 2,
"secondary_choice": 2,
"tertiary_choice": 2,
"tag_mode": "OR", // OR or AND; Web UI default is OR
"add_lands": true,
"fetch_count": 3,
"ideal_counts": { "ramp": 10, "lands": 36, "basic_lands": 16, "creatures": 28, "removal": 8, "wipes": 3, "card_advantage": 8, "protection": 3 }
}
```
Notes: headless honors `ideal_counts` (leave prompts blank to accept). Only `fetch_count` is tracked/exported for lands.
#### JSON fields for Combos (Web Configs)
When running from a JSON config in the Web UI, the following fields are supported and exported from interactive runs:
```json
{
"prefer_combos": true,
"combo_target_count": 3,
"combo_balance": "mix"
}
```
Auto-Complete Combos runs before theme fill/monolithic spells when `prefer_combos` is true.
## 🔒 Bracket policy & enforcement
- Policy source: `config/brackets.yml` defines per-bracket limits for categories like `game_changers`, `extra_turns`, `mass_land_denial`, `tutors_nonland`, and `two_card_combos`.
- Authoritative lists: JSON under `config/card_lists/` provides names for enforcement and reporting (`game_changers.json`, `extra_turns.json`, `mass_land_denial.json`, `tutors_nonland.json`). A `list_version` may be included for badges.
- Global safety prune: when a category has a limit of 0, the builder removes matching cards from the card pool up-front so they cannot be selected (Game Changers by name; others by tags when present). This runs in both Web and headless builds.
- Preemptive filters: spells and creatures phases also apply bracket-aware pre-filters.
- Inline enforcement (Web): if violations still occur, Step 5 shows them before the summary. You must replace or remove flagged cards before you can Continue or Rerun. Alternatives are role-consistent, exclude the replaced/in-deck/locked/commander cards, and bias toward owned when enabled.
- Game Changer fallback: if `config/card_lists/game_changers.json` is empty, enforcement and reporting fall back to the in-code `builder_constants.GAME_CHANGERS` list so low-bracket decks still exclude those cards.
- Auto-enforce (optional): set `WEB_AUTO_ENFORCE=1` to automatically apply the enforcement plan after build and re-export CSV/TXT when violations are detected.
Status levels
- PASS: within limits; panel remains collapsed by default.
- WARN: advisory thresholds met (from `<category>_warn` keys in `config/brackets.yml` or conservative defaults for low brackets on tutors/extra turns). Panel opens automatically and shows WARN tiles with a subtle amber border and badge; no gating or automatic enforcement.
- FAIL: over hard limits or disallowed categories for the selected bracket. Panel opens automatically, tiles show with red accents, and Continue/Rerun are disabled until resolved; enforcement actions are available.
Compliance report
- Every run writes `[stem]_compliance.json` next to your deck exports, including per-category counts/limits, status (PASS/FAIL), detected cheap/early two-card combos, and list versions.
- Headless JSON builds and Web builds use the same engine; summary and exports are consistent.
## 🧩 Theme combine mode (AND/OR)
- OR (default): Recommend cards that match any selected themes, prioritizing overlap.
- AND: Prioritize multi-theme intersections. For creatures, an AND pre-pass first picks "all selected themes" creatures up to a cap, then fills by weighted overlap.
UI tips:
- Step 2 includes AND/OR radios with a tooltip explaining trade-offs.
- In staged build view, "Creatures: All-Theme" shows which selected themes each card hits.
## 🕹️ Usage (interactive)
1) Start the app (Docker or from source)
2) Pick Build a New Deck
3) Search/confirm commander
4) Pick primary/secondary/tertiary themes (or stop at primary); choose AND/OR combine mode
5) Choose power bracket and review ideal counts
6) Deck builds; CSV/TXT export to `deck_files/`
## ⚙️ Environment variables (common)
- DECK_MODE=headless
- DECK_CONFIG=/app/config/deck.json
- DECK_COMMANDER, DECK_PRIMARY_CHOICE, DECK_SECONDARY_CHOICE, DECK_TERTIARY_CHOICE
- DECK_ADD_LANDS, DECK_FETCH_COUNT
- DECK_ADD_CREATURES, DECK_ADD_NON_CREATURE_SPELLS, DECK_ADD_RAMP, DECK_ADD_REMOVAL, DECK_ADD_WIPES, DECK_ADD_CARD_ADVANTAGE, DECK_ADD_PROTECTION
- DECK_BRACKET_LEVEL
- **ALLOW_MUST_HAVES=true** (enables include/exclude card lists feature with enhanced UI validation)
- Random Modes (alpha): core toggles `RANDOM_MODES=1`, `RANDOM_UI=1`, `RANDOM_MAX_ATTEMPTS=5`, `RANDOM_TIMEOUT_MS=5000` (see detailed list below for theme/auto-fill/seed overrides)
- `RANDOM_BUILD_SUPPRESS_INITIAL_EXPORT=0` — (optional) disable the default suppression of builder auto-export (will produce legacy duplicate artifacts; for debugging only)
- Testing/data overrides:
- `CSV_FILES_DIR=csv_files/testdata` — point the app/tests at a frozen dataset snapshot for determinism
Random Modes (alpha flags):
- `RANDOM_MODES=1` — enable random build endpoints and backend features
- `RANDOM_UI=1` — show Surprise/Reroll/Share controls in UI
- `RANDOM_MAX_ATTEMPTS=5` — cap retry attempts for random generation
- `RANDOM_TIMEOUT_MS=5000` — per-build timeout in milliseconds
- `RANDOM_THEME=`*slug* — legacy single-theme alias (maps to primary when others unset)
- `RANDOM_PRIMARY_THEME=`*slug* — primary theme override
- `RANDOM_SECONDARY_THEME=`*slug* — secondary theme override
- `RANDOM_TERTIARY_THEME=`*slug* — tertiary theme override
- `RANDOM_AUTO_FILL=0|1` — auto-fill missing secondary/tertiary slots when enabled
- `RANDOM_AUTO_FILL_SECONDARY=0|1` / `RANDOM_AUTO_FILL_TERTIARY=0|1` — explicit per-slot overrides
- `RANDOM_STRICT_THEME_MATCH=0|1` — require exact theme matches when rolling commanders
- `RANDOM_CONSTRAINTS=`*inline JSON* and/or `RANDOM_CONSTRAINTS_PATH=/app/config/random_constraints.json`
- `RANDOM_SEED=`*value* — deterministic run seed (int or string)
- `RANDOM_OUTPUT_JSON=/app/deck_files/random_build.json` — write random build payload to file/dir
- `HEADLESS_RANDOM_MODE=1` — force the headless runner to take the random pipeline
- `RANDOM_BUILD_SUPPRESS_INITIAL_EXPORT=0` — (optional) disable the default suppression of builder auto-export (useful for debugging legacy duplicate exports)
Testing/determinism helpers:
- CSV_FILES_DIR=csv_files/testdata — override CSV base dir for tests or pinned snapshots
Optional name-based tag overrides (mapped to indices for the chosen commander):
- DECK_PRIMARY_TAG, DECK_SECONDARY_TAG, DECK_TERTIARY_TAG
Combine mode in headless:
- JSON: set `"tag_mode": "AND" | "OR"`
- Env var: `DECK_TAG_MODE=AND|OR` (if configured in your environment)
Web UI performance tuning:
- WEB_TAG_PARALLEL=1|0
- WEB_TAG_WORKERS=<N>
- WEB_VIRTUALIZE=1 (enable list virtualization)
- SHOW_DIAGNOSTICS=1 (optional: diagnostics tools and virtualization overlay toggle using `v`)
- WEB_AUTO_ENFORCE=1 (optional; after building, auto-apply enforcement and re-export when the compliance report fails)
- SPLASH_ADAPTIVE=1 (optional: enable adaptive off-color penalty by commander color count)
- SPLASH_ADAPTIVE_SCALE="1:1.0,2:1.0,3:1.0,4:0.6,5:0.35" (optional: tuning for adaptive scaling)
Paths and data overrides:
- `DECK_EXPORTS=/app/deck_files` — where finished deck exports are read by the Web UI
- `OWNED_CARDS_DIR=/app/owned_cards` — preferred directory for owned inventory uploads
- `CARD_LIBRARY_DIR=/app/owned_cards` — back‑compat alias for OWNED_CARDS_DIR
- `DECK_CONFIG=/app/config/deck.json` — default headless config path
- `CSV_FILES_DIR=/app/csv_files` — override CSV base dir (tests, snapshots, or alternate datasets)
- `CARD_INDEX_EXTRA_CSV=path/to/synthetic_cards.csv` — inject an extra CSV into the card index for experiments/tests
Theme catalog scan & prewarm:
- `THEME_CATALOG_YAML_SCAN_INTERVAL_SEC=2.0` — poll for YAML changes (dev); omit in production
- `WEB_THEME_FILTER_PREWARM=1` — prewarm common filters for faster first renders
Theme preview cache & Redis (optional):
- `THEME_PREVIEW_CACHE_MAX=400` — max previews cached in memory
- `WEB_THEME_PREVIEW_LOG=0` — 1=verbose cache logs
- `THEME_PREVIEW_ADAPTIVE=0` — 1=adaptive cache policy
- `THEME_PREVIEW_EVICT_COST_THRESHOLDS=5,15,40` — cost tiers for eviction policy
- `THEME_PREVIEW_BG_REFRESH=0` — 1=background refresh worker; `THEME_PREVIEW_BG_REFRESH_INTERVAL=120` seconds
- TTL tuning: `THEME_PREVIEW_TTL_BASE=300`, `THEME_PREVIEW_TTL_MIN=60`, `THEME_PREVIEW_TTL_MAX=900`, `THEME_PREVIEW_TTL_BANDS=0.2,0.5,0.8`, `THEME_PREVIEW_TTL_STEPS=2,4,2,3,1`
- Redis: `THEME_PREVIEW_REDIS_URL=redis://localhost:6379/0`, `THEME_PREVIEW_REDIS_DISABLE=0` (1=disable redis even if URL set)
Rarity weighting and diversity (advanced):
- `RARITY_W_MYTHIC=1.2`, `RARITY_W_RARE=0.9`, `RARITY_W_UNCOMMON=0.65`, `RARITY_W_COMMON=0.4` — diminish or boost rarity selection
- `RARITY_DIVERSITY_TARGETS="mythic:0-1,rare:0-2,uncommon:0-4,common:0-6"` — soft targets per rarity band
- `RARITY_DIVERSITY_OVER_PENALTY=-0.5` — penalty when exceeding targets
Misc utility land tuning (Step 7):
- MISC_LAND_DEBUG=1 Write debug CSVs for misc land step (candidates/post-filter). Otherwise suppressed unless SHOW_DIAGNOSTICS=1.
- MISC_LAND_EDHREC_KEEP_PERCENT_MIN=0.75 Lower bound of dynamic EDHREC keep range (0–1). When MIN & MAX are both set, a random % in [MIN,MAX] is rolled per build.
- MISC_LAND_EDHREC_KEEP_PERCENT_MAX=1.0 Upper bound of dynamic EDHREC keep range (0–1).
- MISC_LAND_EDHREC_KEEP_PERCENT=0.80 Legacy fixed keep % used only if MIN/MAX not both present.
- MISC_LAND_THEME_MATCH_BASE=1.4 Base multiplier when at least one selected theme tag matches a candidate land.
- MISC_LAND_THEME_MATCH_PER_EXTRA=0.15 Incremental multiplier per additional matching theme tag beyond the first.
- MISC_LAND_THEME_MATCH_CAP=2.0 Cap for total theme multiplier after stacking base + extras.
Notes:
- Fetch lands are fully excluded from the misc step (handled earlier).
- Mono-color decks auto-filter broad rainbow/any-color lands except an explicit always-keep list.
- Land Alternatives endpoint: for land seeds, returns land-only suggestions; 12 random picks each request from a randomly sized window within the top 60–100 ranked candidates (per-card, uncached) for variety.
## 📁 Folders
- `deck_files/` — CSV/TXT exports
- `csv_files/` — card data
- `logs/` — logs
- `config/` — JSON configs (optional)
- `owned_cards/` — your owned cards lists (`.txt`/`.csv`); used for owned-only builds and Owned flagging
## 🧰 Troubleshooting
- Use `docker compose run --rm` (not `up`) for interactive sessions
- Ensure volumes are mounted so files persist (`deck_files`, `logs`, `csv_files`)
- For headless with config, mount `./config:/app/config` and set `DECK_CONFIG`
- Card data refresh: if `csv_files/cards.csv` is missing or older than 7 days, the app refreshes data and re-tags automatically. A `.tagging_complete.json` file in `csv_files/` indicates tagging completion.
- Web health: the header dot polls `/healthz` — green is OK, red is degraded. If red, check logs.
- Error details: HTMX errors show a toast with a “Copy details” button including X-Request-ID; include that when filing issues.
- Logs page: set `SHOW_LOGS=1` to enable `/logs` and `/status/logs?tail=200` (read‑only) for quick diagnostics.
- Diagnostics page: set `SHOW_DIAGNOSTICS=1` to enable the nav link and `/diagnostics` test tools.
Data integrity notes:
- Banned cards: per-color/guild CSVs now consistently respect the Commander banned list using exact, case‑insensitive name matching across `name` and `faceName`.
## 🧪 Development and tests
Set up a virtual environment, install dependencies, and run the test suite.
```powershell
# From repo root (PowerShell)
python -m venv .venv
& ".venv/Scripts/Activate.ps1"
pip install -r requirements.txt
pip install -r requirements-dev.txt
# Run tests (pytest is configured to look under code/tests)
python -m pytest -q
```
Notes:
- Test discovery is set to `code/tests` in `pytest.ini`.
- `pytest.ini` disables the built-in debugging plugin to avoid a stdlib module name clash with the project folder `code/`.
- Feature flags for local diagnostics: set `SHOW_DIAGNOSTICS=1` and/or `SHOW_LOGS=1` to expose `/diagnostics` and `/logs`.
- Index testing helper: set `CARD_INDEX_EXTRA_CSV=path\to\synthetic_cards.csv` to inject a temporary CSV during tests (e.g., color identity edge cases) without altering production shards.
### What’s new (quick summary)
- Faster browsing with optional virtualized grids/lists in Step 5 and Owned (`WEB_VIRTUALIZE=1`).
- Image polish: lazy-loading, responsive `srcset/sizes`, and LQIP blur/fade for Step 5, Owned, and commander preview.
- Diagnostics overlay (opt-in with `SHOW_DIAGNOSTICS=1`): press `v` to see visible range, totals, render time, and counters.
- Accessibility: respects reduced-motion (disables blur/fade and smooth scrolling).
- Small caching wins: short‑TTL fragment caching for summary partials and suggestions.
## 📦 Releases
- Release notes are maintained in `RELEASE_NOTES_TEMPLATE.md`. Automated workflows read from this file to populate Docker Hub and GitHub releases.
### Collecting diagnostics for an issue
- Note the Request-ID from the toast or error page.
- Copy logs from `/logs` (enable with `SHOW_LOGS=1`) or `/status/logs?tail=200`.
- Include your `/healthz` JSON and environment flags (SHOW_LOGS/SHOW_DIAGNOSTICS/SHOW_SETUP) when reporting.
## Performance Baselines
A warm preview performance baseline is committed at `logs/perf/theme_preview_warm_baseline.json`.
Tooling:
- Benchmark: `python -m code.scripts.preview_perf_benchmark --all --passes 2 --extract-warm-baseline logs/perf/theme_preview_warm_baseline.json`
- Compare (manual): `python -m code.scripts.preview_perf_compare --baseline logs/perf/theme_preview_warm_baseline.json --candidate logs/perf/new_run.json --warm-only --p95-threshold 5`
- Regression helper: `python -m code.scripts.preview_perf_ci_check --baseline logs/perf/theme_preview_warm_baseline.json --p95-threshold 5` (run locally or in ad-hoc CI when desired)
Policy:
- Treat warm-only p95 regressions beyond 5% as blockers even though enforcement is now manual.
- Update the baseline only after intentional performance improvements or systemic environment shifts.
Workflow to refresh baseline:
1. Ensure no active regressions (run wrapper locally, expect pass).
2. Run a multi-pass benchmark with at least two passes over all themes.
3. Replace `theme_preview_warm_baseline.json` with new warm pass output; commit in same PR with rationale in CHANGELOG.
Optional future enhancements:
- Reintroduce automated gating once preview perf stabilizes on shared runners.
- Separate p50 threshold (e.g., >7% regression) for early warning without failing build.

91
docs/headless_cli_guide.md Normal file
View file

@ -0,0 +1,91 @@
# Headless & CLI Guide
Leverage the shared deckbuilding engine from the command line, in headless mode, or within containers.
## Table of contents
- [Entry points](#entry-points)
- [Switching modes in Docker](#switching-modes-in-docker)
- [Headless JSON configs](#headless-json-configs)
- [Environment overrides](#environment-overrides)
- [CLI argument reference](#cli-argument-reference)
- [Include/exclude lists from the CLI](#includeexclude-lists-from-the-cli)
- [Practical examples](#practical-examples)
---
## Entry points
- Interactive menu: `python code/main.py`
- Headless runner: `python code/headless_runner.py --config config/deck.json`
- Both executables share the same builder core used by the Web UI.
## Switching modes in Docker
Override the container entrypoint to run the CLI or headless flows inside Docker Compose or plain `docker run`.
```powershell
# Compose example
docker compose run --rm -e APP_MODE=cli web
# Compose with headless automation
docker compose run --rm `
-e APP_MODE=cli `
-e DECK_MODE=headless `
-e DECK_CONFIG=/app/config/deck.json `
web
```
Set `APP_MODE=cli` to switch from the Web UI to the textual interface. Add `DECK_MODE=headless` to skip prompts and immediately run the configured deck.
## Headless JSON configs
- Drop JSON files into `config/` (e.g., `config/deck.json`).
- Headless mode auto-runs the lone JSON file; if multiple exist, the CLI lists them with summaries (commander + themes).
- Config fields cover commander, bracket, include/exclude lists, theme preferences, owned-mode toggles, and output naming.
## Environment overrides
When running in containers or automation, environment variables can override JSON settings. Typical variables include:
- `DECK_COMMANDER`
- `DECK_PRIMARY_CHOICE`, `DECK_SECONDARY_CHOICE`, `DECK_TERTIARY_CHOICE`
- `DECK_BRACKET_LEVEL`
- `DECK_ADD_LANDS`, `DECK_LAND_COUNT`, `DECK_CREATURE_COUNT`, `DECK_RAMP_COUNT`
Precedence order: **CLI flags > environment variables > JSON config > defaults**.
## CLI argument reference
Run `python code/headless_runner.py --help` to see the current argument surface. Highlights:
- Type indicators make expectations explicit (e.g., `PATH`, `NAME`, `INT`).
- Theme selection accepts human-readable names: `--primary-tag "Airbending"` instead of numeric indexes.
- Bracket selection via `--bracket-level`.
- Ideal counts such as `--land-count`, `--ramp-count`, `--creature-count`, and more.
## Include/exclude lists from the CLI
You can specify comma- or semicolon-separated lists directly through the CLI:
```powershell
python code/headless_runner.py `
--commander "Jace, Vryn's Prodigy" `
--include-cards "Sol Ring;Jace, the Mind Sculptor" `
--exclude-cards "Chaos Orb;Shahrazad" `
--enforcement-mode strict
```
Semicolons allow card names containing commas. Enforcement modes mirror the Web UI (`off`, `warn`, `strict`).
## Practical examples
```powershell
# Build a Goblins list with tuned counts
python code/headless_runner.py `
--commander "Krenko, Mob Boss" `
--primary-tag "Goblin Kindred" `
--creature-count 35 `
--land-count 33 `
--ramp-count 12
# Fire a headless run via Docker using an alternate config folder
docker compose run --rm `
-e APP_MODE=cli `
-e DECK_MODE=headless `
-e DECK_CONFIG=/app/config/custom_decks `
web
```
The CLI prints a detailed summary at the end of each run, including enforcement results, resolved themes, and export paths. All artifacts land in the same `deck_files/` folder used by the Web UI.

148
docs/theme_catalog_advanced.md Normal file
View file

@ -0,0 +1,148 @@
# Theme Catalog Advanced Guide
Additional details for developers and power users working with the theme catalog, editorial tooling, and diagnostics.
## Table of contents
- [HTMX API endpoints](#htmx-api-endpoints)
- [Caching, diagnostics, and metrics](#caching-diagnostics-and-metrics)
- [Governance principles](#governance-principles)
- [Operational tooling](#operational-tooling)
- [Refreshing catalogs](#refreshing-catalogs)
- [Snapshotting taxonomy](#snapshotting-taxonomy)
- [Adaptive splash penalty experiments](#adaptive-splash-penalty-experiments)
- [Editorial pipeline](#editorial-pipeline)
- [Script summary](#script-summary)
- [Example configuration](#example-configuration)
- [Duplicate suppression controls](#duplicate-suppression-controls)
- [Coverage metrics and KPIs](#coverage-metrics-and-kpis)
- [Description mapping overrides](#description-mapping-overrides)
- [Validation and schema tooling](#validation-and-schema-tooling)
---
## HTMX API endpoints
The upcoming theme picker UI is powered by two FastAPI endpoints.
### `GET /themes/api/themes`
Parameters:
- `q`: substring search across theme names and synergies.
- `archetype`: filter by `deck_archetype`.
- `bucket`: popularity bucket (Very Common, Common, Uncommon, Niche, Rare).
- `colors`: comma-separated color initials (e.g. `G,W`).
- `limit` / `offset`: pagination (limit defaults to 50, max 200).
- `diagnostics=1`: surfaces `has_fallback_description` and `editorial_quality` (requires `WEB_THEME_PICKER_DIAGNOSTICS=1`).
The response includes `count`, the filtered `items`, and `next_offset` for subsequent requests. Diagnostic mode adds extra telemetry fields.
### `GET /themes/api/theme/{id}`
Parameters:
- `uncapped=1`: (diagnostics) returns `uncapped_synergies`, combining curated, enforced, and inferred sets.
- `diagnostics=1`: exposes editorial metadata such as `editorial_quality` and `has_fallback_description`.
The payload merges curated data with editorial artifacts (`example_cards`, `example_commanders`, etc.) and respects the same diagnostic feature flag.
## Caching, diagnostics, and metrics
- Responses include an `ETag` header derived from catalog metadata so consumers can perform conditional GETs.
- `/themes/status` reports freshness and stale indicators; `/themes/refresh` (POST) triggers a background rebuild.
- When `WEB_THEME_PICKER_DIAGNOSTICS=1` is set, the app records:
- Filter cache hits/misses and duration (`X-ThemeCatalog-Filter-Duration-ms`).
- Preview cache metrics (`/themes/metrics` exposes counts, hit rates, TTL, and average build time).
- Skeleton loaders ship with the HTMX fragments to keep perceived latency low.
## Governance principles
To keep the catalog healthy, the project follows a lightweight governance checklist:
1. **Minimum examples** – target at least two example cards and one commander per established theme.
2. **Deterministic preview assembly** – curated examples first, then role-based samples (payoff/enabler/support/wildcard), then placeholders if needed.
3. **Splash relax policy** – four- and five-color commanders may include a single off-color enabler with a small penalty, preventing over-pruning.
4. **Popularity buckets are advisory** – they guide filters and UI hints but never directly influence scoring.
5. **Taxonomy expansion bar** – new high-level archetypes require a distinct pattern, at least eight representative cards, and no overlap with existing themes.
6. **Editorial quality tiers** – optional `editorial_quality: draft|reviewed|final` helps prioritize review passes.
7. **Deterministic sampling** – seeds derive from `theme|commander` hashes; scoring code should emit `reasons[]` to explain decisions and remain regression-test friendly.
See `docs/theme_taxonomy_rationale.md` for the underlying rationale and roadmap.
## Operational tooling
### Refreshing catalogs
- Primary builder: `python code/scripts/build_theme_catalog.py`
- Options:
- `--limit N`: preview a subset without overwriting canonical outputs (unless `--allow-limit-write`).
- `--output path`: write to an alternate path; suppresses YAML backfill to avoid mutating tracked files.
- `--backfill-yaml` or `EDITORIAL_BACKFILL_YAML=1`: fill missing descriptions and popularity buckets in YAML files.
- `--force-backfill-yaml`: overwrite existing description/popularity fields.
- `EDITORIAL_SEED=<int>`: force a deterministic ordering when heuristics use randomness.
- `EDITORIAL_AGGRESSIVE_FILL=1`: pad sparse themes with inferred synergies.
- `EDITORIAL_POP_BOUNDARIES="a,b,c,d"`: tune popularity thresholds.
- `EDITORIAL_POP_EXPORT=1`: emit `theme_popularity_metrics.json` summaries.
### Snapshotting taxonomy
`python -m code.scripts.snapshot_taxonomy` writes `logs/taxonomy_snapshots/taxonomy_<timestamp>.json` with a SHA-256 hash. Identical content is skipped unless you supply `--force`. Use snapshots before experimenting with taxonomy-aware sampling.
### Adaptive splash penalty experiments
Set `SPLASH_ADAPTIVE=1` to scale off-color enabler penalties based on commander color count. Tune with `SPLASH_ADAPTIVE_SCALE` (e.g. `1:1.0,2:1.0,3:1.0,4:0.6,5:0.35`). Analytics aggregate both static and adaptive reasons for comparison.
## Editorial pipeline
### Script summary
- `code/scripts/generate_theme_editorial_suggestions.py`
- Proposes `example_cards`, `example_commanders`, and `synergy_commanders` using card CSVs and tagging heuristics.
- `--augment-synergies` can pad sparse `synergies` arrays prior to suggestion.
- `--apply` writes results; dry runs print suggestions for review.
- `code/scripts/lint_theme_editorial.py`
- Validates annotation formats, min/max counts, and deduplication. Combine with environment toggles (`EDITORIAL_REQUIRE_DESCRIPTION`, `EDITORIAL_REQUIRE_POPULARITY`) for stricter gating.
### Example configuration
```powershell
# Dry run on the first 25 themes
python code/scripts/generate_theme_editorial_suggestions.py
# Apply across the catalog with augmentation and min example commanders set to 5
python code/scripts/generate_theme_editorial_suggestions.py --apply --augment-synergies --min-examples 5
# Lint results
python code/scripts/lint_theme_editorial.py
```
Editorial output depends on current CSV data. Expect ordering or composition changes after upstream dataset refreshes—treat full-catalog regeneration as an operational task and review diffs carefully.
### Duplicate suppression controls
`code/scripts/synergy_promote_fill.py` can rebalance example cards:
```powershell
python code/scripts/synergy_promote_fill.py --fill-example-cards --common-card-threshold 0.18 --print-dup-metrics
```
- `--common-card-threshold`: filters cards appearing in more than the specified fraction of themes (default `0.18`).
- Use metrics output to tune thresholds so staple utility cards stay in check without removing legitimate thematic cards.
### Coverage metrics and KPIs
- `EDITORIAL_INCLUDE_FALLBACK_SUMMARY=1` embeds a `description_fallback_summary` block in the generated catalog (`generic_total`, `generic_plain`, `generic_pct`, etc.).
- Regression tests use these metrics to ratchet down generic descriptions over time.
- Historical trends are appended to `config/themes/description_fallback_history.jsonl` for analysis.
### Description mapping overrides
Customize automatic descriptions without editing code:
- Add `config/themes/description_mapping.yml` with entries:
```yaml
- triggers: ["sacrifice", "aristocrat"]
description: "Leans on sacrifice loops and {SYNERGIES}."
```
- The first matching trigger wins (case-insensitive substring search).
- `{SYNERGIES}` expands to a short clause listing the top synergies when available, and disappears gracefully if not.
- Internal defaults remain as fallbacks when the mapping file is absent.
## Validation and schema tooling
Run validators to maintain catalog quality:
```powershell
python code/scripts/validate_theme_catalog.py
python code/scripts/validate_theme_catalog.py --rebuild-pass
python code/scripts/validate_theme_catalog.py --schema
python code/scripts/validate_theme_catalog.py --yaml-schema
python code/scripts/validate_theme_catalog.py --strict-alias
```
Per-theme YAML files (under `config/themes/catalog/`) are tracked in source control. Keys such as `metadata_info` replace the legacy `provenance`; the validator treats missing migrations as warnings until the deprecation completes.

103
docs/web_ui_deep_dive.md Normal file
View file

@ -0,0 +1,103 @@
# Web UI Deep Dive
A closer look at the rich interactions available in the MTG Python Deckbuilder Web UI. Use this guide after you are comfortable with the basic homepage flows described in the README.
## Table of contents
- [Unified New Deck modal](#unified-new-deck-modal)
- [Stage 5 tools: lock, replace, compare, permalinks](#stage-5-tools-lock-replace-compare-permalinks)
- [Multi-copy archetype packages](#multi-copy-archetype-packages)
- [Bracket compliance and skipped stages](#bracket-compliance-and-skipped-stages)
- [Build options: owned-only and prefer-owned](#build-options-owned-only-and-prefer-owned)
- [Visual summaries](#visual-summaries)
- [Combos & synergies](#combos--synergies)
- [Owned library page](#owned-library-page)
- [Finished decks workspace](#finished-decks-workspace)
- [Keyboard shortcuts](#keyboard-shortcuts)
- [Virtualization, tagging, and performance](#virtualization-tagging-and-performance)
- [Diagnostics and logs](#diagnostics-and-logs)
---
## Unified New Deck modal
The first three steps of deckbuilding live inside a single modal:
1. **Search for a commander** – autocomplete prioritizes color identity matches; press Enter to grab the top result.
2. **Pick primary/secondary/tertiary themes** – the modal displays your selections in order so you can revisit them quickly.
3. **Choose a bracket** – labels such as “Bracket 3: Upgraded” clarify power bands. Bracket 3 is the default tier for new builds.
Optional inputs:
- **Deck name** becomes the export filename stem and is reused in Finished Decks banners.
- **Combo auto-complete** and other preferences persist between runs.
Once you submit, the modal closes and the build starts immediately—no extra confirmation screen.
## Stage 5 tools: lock, replace, compare, permalinks
Stage 5 is the iterative workspace for tuning the deck:
- **Lock** a card by clicking the padlock or the card artwork. Locked cards persist across rerolls and show a “Last action” chip for quick confirmation.
- **Replace** opens the Alternatives drawer. Filters include Owned-only, role alignment, and bracket compliance. The system skips commanders, locked cards, just-added cards, and anything already in the list.
- **Permalink** buttons appear in Stage 5 and Finished Decks. Share a build (commander, themes, bracket, ideals, flags) or restore one by pasting a permalink back into the app.
- **Compare** mode lives in Finished Decks. Pick two builds (quick actions select the latest pair) and triage changes via Changed-only, Copy summary, or download the diff as TXT.
## Multi-copy archetype packages
When a commander + theme combination suggests a multi-copy strategy (e.g., Persistent Petitioners, Shadowborn Apostles), the UI offers an optional package:
- Choose the desired quantity (bounded by printed limits) and optionally add **Thrumming Stone** when it synergizes.
- Packages are inserted before other stages so target counts adjust appropriately.
- A safety clamp trims overflow to keep the deck at 100 cards; the stage displays a “Clamped N” indicator if it triggers.
- You can dismiss the modal, and we won’t re-prompt unless your selections change.
## Bracket compliance and skipped stages
- Bracket policy enforcement prunes disallowed categories before stage execution. Violations block reruns until you resolve them.
- Enforcement options: keep the panel collapsed when compliant, auto-open with a colored status chip (green/amber/red) when action is needed.
- Enable auto-enforcement by setting `WEB_AUTO_ENFORCE=1`.
- Toggle **Show skipped stages** to surface steps that added zero cards, making it easier to review the full pipeline.
## Build options: owned-only and prefer-owned
The modal includes toggles for **Use only owned cards** and **Prefer owned cards**:
- Owned-only builds pull strictly from the inventory in `owned_cards/` (commander exempt).
- Prefer-owned bumps owned cards slightly in the scoring pipeline but still allows unowned all-stars when necessary.
- Both modes respect the Owned Library filters and show Owned badges in the exported CSV (including the `Owned` column when you disable the mode).
## Visual summaries
Stage 5 displays multiple data visualizations that cross-link to the card list:
- **Mana curve** – hover a bar to highlight matching cards in list and thumbnail views.
- **Color requirements vs. sources** – pips show requirements; sources include non-land producers and an optional `C` (colorless) toggle.
- **Tooltips** – each tooltip lists contributing cards and offers a copy-to-clipboard action.
- Visual polish includes lazy-loaded thumbnails, blur-up transitions, and accessibility tweaks that respect `prefers-reduced-motion`.
## Combos & synergies
The builder detects curated two-card combos and synergy pairs in the final deck:
- Chips display badges such as “cheap” or “setup” with hover previews for each card and a split preview when hovering the entire row.
- Enable **Auto-complete combos** to add missing partners before theme filling. Configure target count, balance (early/late/mix), and preference weighting.
- Color identity restrictions keep the algorithm from suggesting off-color partners.
## Owned library page
Open the Owned tile to manage uploaded inventories:
- Upload `.txt` or `.csv` files with one card per line. The app enriches and deduplicates entries on ingestion.
- The page includes sortable columns, exact color-identity filters (including four-color combos), and an export button.
- Large collections benefit from virtualization when `WEB_VIRTUALIZE=1`.
## Finished decks workspace
- Browse historical builds with filterable theme chips.
- Each deck offers Download TXT, Copy summary, Open permalink, and Compare actions.
- Locks, replace history, and compliance metadata are stored per deck and surface alongside the exports.
## Keyboard shortcuts
- **Enter** selects the first commander suggestion while searching.
- Inside Stage 5 lists: **L** locks/unlocks the focused card, **R** opens the Replace drawer, and **C** copies the permalink.
- Browser autofill is disabled in the modal to keep searches clean.
## Virtualization, tagging, and performance
- `WEB_TAG_PARALLEL=1` with `WEB_TAG_WORKERS=4` (compose default) speeds up initial data preparation. The UI falls back to sequential tagging if workers fail to start.
- `WEB_VIRTUALIZE=1` enables virtualized grids in Stage 5 and the Owned library, smoothing large decks or libraries.
- Diagnostics overlays: enable `SHOW_DIAGNOSTICS=1`, then press **v** inside a virtualized grid to inspect render ranges, row counts, and paint timings.
## Diagnostics and logs
- `SHOW_DIAGNOSTICS=1` unlocks the `/diagnostics` page with system summaries (`/status/sys`), feature flags, and per-request `X-Request-ID` headers.
- `SHOW_LOGS=1` turns on the `/logs` viewer with level & keyword filters, auto-refresh, and copy-to-clipboard.
- Health probes live at `/healthz` and return `{status, version, uptime_seconds}` for integration with uptime monitors.