Andreas/mtg_python_deckbuilder
1
0
Fork 0
mirror of https://github.com/mwisnowski/mtg_python_deckbuilder.git synced 2025-12-16 07:30:13 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions

docs: refresh docker and readme guides

Browse source
This commit is contained in:
matt 2025-10-02 16:28:19 -07:00
parent 0448419d9f
commit 84749da214
7 changed files with 1737 additions and 208 deletions
Download patch file Download diff file Expand all files Collapse all files

424
DOCKER.md
View file

@ -1,228 +1,236 @@
# Docker Guide
Run the MTG Deckbuilder (CLI and Web UI) in Docker with persistent volumes and optional headless mode.
Spin up the MTG Python Deckbuilder inside containers. The image defaults to the Web UI; switch to the CLI/headless runner by flipping environment variables. All commands assume Windows PowerShell.
## Quick start
## Build a Deck (Web UI)
### PowerShell (recommended)
```powershell
docker compose build
docker compose run --rm mtg-deckbuilder
```
- Build the image (first run only) and start the `web` service in detached mode:
### From Docker Hub (PowerShell)
```powershell
docker run -it --rm `
-v "${PWD}/deck_files:/app/deck_files" `
-v "${PWD}/logs:/app/logs" `
-v "${PWD}/csv_files:/app/csv_files" `
-v "${PWD}/owned_cards:/app/owned_cards" `
-v "${PWD}/config:/app/config" `
mwisnowski/mtg-python-deckbuilder:latest
```
## Web UI (new)
The web UI runs the same deckbuilding logic behind a browser-based interface.
### PowerShell (recommended)
```powershell
docker compose up --build --no-deps -d web
```
Then open http://localhost:8080
- Open http://localhost:8080 to use the browser experience. First launch seeds data, downloads the latest card catalog, and tags cards automatically (`WEB_AUTO_SETUP=1`, `WEB_TAG_PARALLEL=1`, `WEB_TAG_WORKERS=4` in `docker-compose.yml`).
Volumes are the same as the CLI service, so deck exports/logs/configs persist in your working folder.
The app serves a favicon at `/favicon.ico` and exposes a health endpoint at `/healthz`.
Compare view offers a Copy summary button to copy a plain-text diff of two runs. The sidebar has a subtle depth shadow for clearer separation.
Web UI feature highlights:
- Locks: Click a card or the lock control in Step 5; locks persist across reruns.
- Replace: Enable Replace in Step 5, click a card to open Alternatives (filters include Owned-only), then choose a swap.
- Permalinks: Copy a permalink from Step 5 or a Finished deck; paste via “Open Permalink…” to restore.
- Compare: Use the Compare page from Finished Decks; quick actions include Latest two and Swap A/B.
Virtualized lists and lazy images (opt‑in)
- Set `WEB_VIRTUALIZE=1` to enable virtualization in Step 5 grids/lists and the Owned library for smoother scrolling on large sets.
- Example (Compose):
```yaml
services:
web:
environment:
- WEB_VIRTUALIZE=1
```
- Example (Docker Hub):
```powershell
docker run --rm -p 8080:8080 `
-e WEB_VIRTUALIZE=1 `
-v "${PWD}/deck_files:/app/deck_files" `
-v "${PWD}/logs:/app/logs" `
-v "${PWD}/csv_files:/app/csv_files" `
-v "${PWD}/owned_cards:/app/owned_cards" `
-v "${PWD}/config:/app/config" `
-e SHOW_DIAGNOSTICS=1 ` # optional: enables diagnostics tools and overlay
mwisnowski/mtg-python-deckbuilder:latest `
bash -lc "cd /app && uvicorn code.web.app:app --host 0.0.0.0 --port 8080"
```
### Diagnostics and logs (optional)
Enable internal diagnostics and a read-only logs viewer with environment flags.
- `SHOW_DIAGNOSTICS=1` — adds a Diagnostics nav link and `/diagnostics` tools
- `SHOW_LOGS=1` — enables `/logs` and `/status/logs?tail=200`
When enabled:
- `/logs` supports an auto-refresh toggle with interval, a level filter (All/Error/Warning/Info/Debug), and a Copy button to copy the visible tail.
- `/status/sys` returns a simple system summary (version, uptime, UTC server time, and feature flags) and is shown on the Diagnostics page when `SHOW_DIAGNOSTICS=1`.
- Virtualization overlay: press `v` on pages with virtualized grids to toggle per-grid overlays and a global summary bubble.
Compose example (web service):
```yaml
environment:
- SHOW_LOGS=1
- SHOW_DIAGNOSTICS=1
```
Docker Hub (PowerShell) example:
```powershell
docker run --rm `
-p 8080:8080 `
-e SHOW_LOGS=1 -e SHOW_DIAGNOSTICS=1 -e ENABLE_THEMES=1 -e THEME=system `
-e SPLASH_ADAPTIVE=1 -e SPLASH_ADAPTIVE_SCALE="1:1.0,2:1.0,3:1.0,4:0.6,5:0.35" ` # optional experiment
-e RANDOM_MODES=1 -e RANDOM_UI=1 -e RANDOM_MAX_ATTEMPTS=5 -e RANDOM_TIMEOUT_MS=5000 `
-v "${PWD}/deck_files:/app/deck_files" `
-v "${PWD}/logs:/app/logs" `
-v "${PWD}/csv_files:/app/csv_files" `
-v "${PWD}/owned_cards:/app/owned_cards" `
-v "${PWD}/config:/app/config" `
mwisnowski/mtg-python-deckbuilder:latest `
bash -lc "cd /app && uvicorn code.web.app:app --host 0.0.0.0 --port 8080"
```
### Setup speed: parallel tagging (Web)
First-time setup or stale data triggers card tagging. The web service uses parallel workers by default.
Configure via environment variables on the `web` service:
- `WEB_TAG_PARALLEL=1|0` — enable/disable parallel tagging (default: 1)
- `WEB_TAG_WORKERS=<N>` — number of worker processes (default: 4 in compose)
If parallel initialization fails, the service falls back to sequential tagging and continues.
### From Docker Hub (PowerShell)
If you prefer not to build locally, pull `mwisnowski/mtg-python-deckbuilder:latest` and run uvicorn:
```powershell
docker run --rm `
-p 8080:8080 `
-v "${PWD}/deck_files:/app/deck_files" `
-v "${PWD}/logs:/app/logs" `
-v "${PWD}/csv_files:/app/csv_files" `
-v "${PWD}/owned_cards:/app/owned_cards" `
-v "${PWD}/config:/app/config" `
mwisnowski/mtg-python-deckbuilder:latest `
bash -lc "cd /app && uvicorn code.web.app:app --host 0.0.0.0 --port 8080"
```
Health check:
```text
GET http://localhost:8080/healthz -> { "status": "ok", "version": "dev", "uptime_seconds": 123 }
```
Theme preference reset (client-side): use the header’s Reset Theme control to clear the saved browser preference; the server default (THEME) applies on next paint.
### Random Modes (alpha) and test dataset override
Enable experimental Random Modes and UI controls in Web runs by setting:
```yaml
services:
web:
environment:
- RANDOM_MODES=1
- RANDOM_UI=1
- RANDOM_MAX_ATTEMPTS=5
- RANDOM_TIMEOUT_MS=5000
```
For deterministic tests or development, you can point the app to a frozen dataset snapshot:
```yaml
services:
web:
environment:
- CSV_FILES_DIR=/app/csv_files/testdata
```
### Taxonomy snapshot (maintainers)
Capture the current bracket taxonomy into an auditable JSON file inside the container:
- Stop or restart the service when you're done:
```powershell
docker compose run --rm web bash -lc "python -m code.scripts.snapshot_taxonomy"
docker compose stop web
docker compose start web
```
Artifacts appear under `./logs/taxonomy_snapshots/` on your host via the mounted volume.
To force a new snapshot even when the content hash matches the latest, pass `--force` to the module.
- Prefer the public image? Pull and run it without building locally:
## Volumes
- `/app/deck_files` ↔ `./deck_files`
- `/app/logs` ↔ `./logs`
- `/app/csv_files` ↔ `./csv_files`
- `/app/owned_cards` ↔ `./owned_cards` (owned cards lists: .txt/.csv)
- Optional: `/app/config` ↔ `./config` (JSON configs for headless)
## Interactive vs headless
- Interactive: attach a TTY (compose run or `docker run -it`)
- Headless auto-run:
```powershell
docker compose run --rm -e DECK_MODE=headless mtg-deckbuilder
```
- Headless with JSON config:
```powershell
docker compose run --rm `
-e DECK_MODE=headless `
-e DECK_CONFIG=/app/config/deck.json `
mtg-deckbuilder
```
### Common env vars
- DECK_MODE=headless
- DECK_CONFIG=/app/config/deck.json
- DECK_COMMANDER, DECK_PRIMARY_CHOICE
- DECK_ADD_LANDS, DECK_FETCH_COUNT
- DECK_TAG_MODE=AND|OR (combine mode used by the builder)
### Web UI tuning env vars
- WEB_TAG_PARALLEL=1|0 (parallel tagging on/off)
- WEB_TAG_WORKERS=<N> (process count; set based on CPU/memory)
- WEB_VIRTUALIZE=1 (enable virtualization)
- SHOW_DIAGNOSTICS=1 (enables diagnostics pages and overlay hotkey `v`)
- RANDOM_MODES=1 (enable random build endpoints)
- RANDOM_UI=1 (show Surprise/Theme/Reroll/Share controls)
- RANDOM_MAX_ATTEMPTS=5 (cap retry attempts)
- (Upcoming) Multi-theme inputs: once UI ships, Random Mode will accept `primary_theme`, `secondary_theme`, `tertiary_theme` fields; current backend already supports the cascade + diagnostics.
- RANDOM_TIMEOUT_MS=5000 (per-build timeout in ms)
Testing/determinism helper (dev):
- CSV_FILES_DIR=csv_files/testdata — override CSV base dir to a frozen set for tests
## Manual build/run
```powershell
docker build -t mtg-deckbuilder .
docker run -it --rm `
-v "${PWD}/deck_files:/app/deck_files" `
-v "${PWD}/logs:/app/logs" `
-v "${PWD}/csv_files:/app/csv_files" `
-v "${PWD}/owned_cards:/app/owned_cards" `
-v "${PWD}/config:/app/config" `
mtg-deckbuilder
docker run --rm -p 8080:8080 `
-v "${PWD}/deck_files:/app/deck_files" `
-v "${PWD}/logs:/app/logs" `
-v "${PWD}/csv_files:/app/csv_files" `
-v "${PWD}/config:/app/config" `
-v "${PWD}/owned_cards:/app/owned_cards" `
mwisnowski/mtg-python-deckbuilder:latest
```
Shared volumes persist builds, logs, configs, and owned cards on the host.
## Run a JSON Config
Use the homepage “Run a JSON Config” button or run the same flow in-container:
```powershell
docker compose run --rm `
-e APP_MODE=cli `
-e DECK_MODE=headless `
-e DECK_CONFIG=/app/config/deck.json `
web
```
- `APP_MODE=cli` routes the entrypoint to the CLI menu.
- `DECK_MODE=headless` skips prompts and calls `headless_runner`.
- Mount JSON configs under `config/` so both the UI and CLI can pick them up.
Override counts, theme tags, or include/exclude lists by setting the matching environment variables before running the container (see “Environment variables” below).
## Initial Setup
The homepage “Initial Setup” tile appears when `SHOW_SETUP=1` (enabled in compose). It re-runs:
1. Card downloads and color-filtered CSV generation.
2. Commander catalog rebuild (including multi-face merges).
3. Tagging and caching.
To force a rebuild from the host:
```powershell
docker compose run --rm --entrypoint bash web -lc "python -m code.file_setup.setup"
```
Add `--entrypoint bash ... "python -m code.scripts.refresh_commander_catalog"` when you only need the commander catalog (with MDFC merge and optional compatibility snapshot).
## Owned Library
Store `.txt` or `.csv` lists in `owned_cards/` (mounted to `/app/owned_cards`). The Web UI uses them for:
- Owned-only or prefer-owned builds.
- The Owned Library management page (virtualized when `WEB_VIRTUALIZE=1`).
- Alternative suggestions that respect ownership.
Use `/owned` to upload files and export enriched lists. These files persist through the `owned_cards` volume.
## Browse Commanders
`SHOW_COMMANDERS=1` exposes the commander browser tile.
- Data lives in `csv_files/commander_cards.csv`.
- Refresh the catalog (including MDFC merges) from within the container:
```powershell
docker compose run --rm --entrypoint bash web -lc "python -m code.scripts.refresh_commander_catalog"
```
Pass `--compat-snapshot` if you also need an unmerged compatibility CSV under `csv_files/compat_faces/`.
## Finished Decks
The Finished Decks page reads the `deck_files/` volume for completed builds:
- Each run produces CSV, TXT, compliance JSON, and summary JSON sidecars.
- Locks and replace history persist per deck.
- Compare view can diff and export summaries.
Ensure the deck exports volume remains mounted so these artifacts survive container restarts.
## Browse Themes
The Themes browser exposes the merged theme catalog with search, filters, and diagnostics.
- `ENABLE_THEMES=1` keeps the selector visible.
- `WEB_THEME_PICKER_DIAGNOSTICS=1` unlocks uncapped synergies, extra metadata, and `/themes/metrics`.
- Regenerate the catalog manually:
```powershell
docker compose run --rm --entrypoint bash web -lc "python -m code.scripts.build_theme_catalog"
```
Advanced options (e.g., `EDITORIAL_*` variables) live in `.env.example`.
## Random Build
Enable the Surprise/Reroll flow by setting:
- `RANDOM_MODES=1` to expose backend random endpoints.
- `RANDOM_UI=1` to show the Random Build tile.
- Optional tunables: `RANDOM_MAX_ATTEMPTS`, `RANDOM_TIMEOUT_MS`, `RANDOM_PRIMARY_THEME`, `RANDOM_SEED`, and auto-fill flags.
Headless parity is available by pairing `APP_MODE=cli` with `DECK_MODE=headless` and the same random variables.
## Diagnostics
`SHOW_DIAGNOSTICS=1` unlocks `/diagnostics` for system summaries, feature flags, and performance probes. Highlights:
- `/healthz` returns `{status, version, uptime_seconds}` for external monitoring.
- Press `v` on pages with virtualized grids (when `WEB_VIRTUALIZE=1`) to toggle the range overlay.
- `WEB_AUTO_ENFORCE=1` (optional) applies bracket enforcement automatically after each build.
## View Logs
`SHOW_LOGS=1` enables the logs tile and `/logs` interface:
- Tail the container log with filtering and copy-to-clipboard.
- `/status/logs?tail=200` offers a lightweight JSON endpoint.
- Raw files live under `logs/` on the host; rotate or archive them as needed.
## Environment variables (Docker quick reference)
See `.env.example` for the full catalog. Common knobs:
### Core mode and networking
| Variable | Default | Purpose |
| --- | --- | --- |
| `APP_MODE` | `web` | Switch between Web UI (`web`) and CLI (`cli`). |
| `DECK_MODE` | _(unset)_ | `headless` auto-runs the headless builder when the CLI starts. |
| `DECK_CONFIG` | `/app/config/deck.json` | JSON config file or directory (auto-discovery). |
| `HOST` / `PORT` / `WORKERS` | `0.0.0.0` / `8080` / `1` | Uvicorn binding when `APP_MODE=web`. |
### Homepage visibility & UX
| Variable | Default | Purpose |
| --- | --- | --- |
| `SHOW_SETUP` | `1` | Show the Initial Setup card. |
| `SHOW_LOGS` | `1` | Enable the View Logs tile and endpoints. |
| `SHOW_DIAGNOSTICS` | `1` | Enable Diagnostics tools and overlays. |
| `SHOW_COMMANDERS` | `1` | Expose the commander browser. |
| `ENABLE_THEMES` | `1` | Keep the theme selector and themes explorer visible. |
| `WEB_VIRTUALIZE` | `1` | Opt-in to virtualized lists/grids for large result sets. |
| `ALLOW_MUST_HAVES` | `1` | Enable include/exclude enforcement in Step 5. |
| `THEME` | `dark` | Initial UI theme (`system`, `light`, or `dark`). |
### Random build controls
| Variable | Default | Purpose |
| --- | --- | --- |
| `RANDOM_MODES` | _(unset)_ | Enable random build endpoints. |
| `RANDOM_UI` | _(unset)_ | Show the Random Build homepage tile. |
| `RANDOM_MAX_ATTEMPTS` | `5` | Retry budget for constrained random rolls. |
| `RANDOM_TIMEOUT_MS` | `5000` | Per-attempt timeout in milliseconds. |
| `RANDOM_PRIMARY_THEME` / `RANDOM_SECONDARY_THEME` / `RANDOM_TERTIARY_THEME` | _(blank)_ | Override theme slots for random runs. |
| `RANDOM_SEED` | _(blank)_ | Deterministic seed. |
| `RANDOM_AUTO_FILL` | `1` | Allow automatic backfill of missing theme slots. |
### Automation & performance
| Variable | Default | Purpose |
| --- | --- | --- |
| `WEB_AUTO_SETUP` | `1` | Auto-run data setup when artifacts are missing or stale. |
| `WEB_AUTO_REFRESH_DAYS` | `7` | Refresh `cards.csv` if older than N days. |
| `WEB_TAG_PARALLEL` | `1` | Use parallel workers during tagging. |
| `WEB_TAG_WORKERS` | `4` | Worker count for parallel tagging. |
| `WEB_AUTO_ENFORCE` | `0` | Re-export decks after auto-applying compliance fixes. |
| `WEB_THEME_PICKER_DIAGNOSTICS` | `1` | Enable theme diagnostics endpoints. |
### Paths and data overrides
| Variable | Default | Purpose |
| --- | --- | --- |
| `CSV_FILES_DIR` | `/app/csv_files` | Point the app at an alternate dataset (e.g., test snapshots). |
| `DECK_EXPORTS` | `/app/deck_files` | Override where the web UI looks for exports. |
| `OWNED_CARDS_DIR` / `CARD_LIBRARY_DIR` | `/app/owned_cards` | Override owned library directory. |
| `CARD_INDEX_EXTRA_CSV` | _(blank)_ | Inject a synthetic CSV into the card index for testing. |
Advanced editorial and theme-catalog knobs (`EDITORIAL_*`, `SPLASH_ADAPTIVE`, etc.) are documented inline in `docker-compose.yml` and `.env.example`.
## Shared volumes
| Host path | Container path | Contents |
| --- | --- | --- |
| `deck_files/` | `/app/deck_files` | CSV/TXT exports, summary JSON, compliance reports. |
| `logs/` | `/app/logs` | Application logs and taxonomy snapshots. |
| `csv_files/` | `/app/csv_files` | Card datasets, commander catalog, tagging flags. |
| `config/` | `/app/config` | JSON configs, bracket policy, card list overrides. |
| `owned_cards/` | `/app/owned_cards` | Uploaded inventory files for owned-only flows. |
## Maintenance commands
Run ad-hoc tasks by overriding the entrypoint:
```powershell
# Theme catalog rebuild
docker compose run --rm --entrypoint bash web -lc "python -m code.scripts.build_theme_catalog"
# Snapshot taxonomy (writes logs/taxonomy_snapshots/)
docker compose run --rm --entrypoint bash web -lc "python -m code.scripts.snapshot_taxonomy"
# Preview the MDFC commander diff
docker compose run --rm --entrypoint bash web -lc "python -m code.scripts.preview_dfc_catalog_diff"
```
Use the `--compat-snapshot` or other script arguments as needed.
## Troubleshooting
- No prompts? Use `docker compose run --rm` (not `up`) or add `-it` to `docker run`
- Files not saving? Verify volume mounts and that folders exist
- Headless not picking config? Ensure `./config` is mounted to `/app/config` and `DECK_CONFIG` points to a JSON file
- Owned-cards prompt not seeing files? Ensure `./owned_cards` is mounted to `/app/owned_cards`
## Tips
- Use `docker compose run`, not `up`, for interactive mode
- Exported decks appear in `deck_files/`
- JSON run-config is exported only in interactive runs; headless skips it
- **Container starts but UI stays blank:** check `/healthz` and `/logs` (enable with `SHOW_LOGS=1`), then inspect the `logs/` volume.
- **Files missing on the host:** ensure the host directories exist before starting Compose; Windows will create empty folders if the path is invalid.
- **Long first boot:** dataset downloads and tagging can take several minutes the first time. Watch progress at `/setup`.
- **Random build hangs:** lower `RANDOM_MAX_ATTEMPTS` or raise `RANDOM_TIMEOUT_MS`, and confirm your theme overrides are valid slugs via `/themes/`.
- **Commander catalog outdated:** rerun the refresh command above or delete `csv_files/.tagging_complete.json` to force a full rebuild on next start.

BIN
README.md
View file

Binary file not shown.

228
docs/archive/DOCKER_2025-10-02.md Normal file
View file

@ -0,0 +1,228 @@
# Docker Guide
Run the MTG Deckbuilder (CLI and Web UI) in Docker with persistent volumes and optional headless mode.
## Quick start
### PowerShell (recommended)
```powershell
docker compose build
docker compose run --rm mtg-deckbuilder
```
### From Docker Hub (PowerShell)
```powershell
docker run -it --rm `
-v "${PWD}/deck_files:/app/deck_files" `
-v "${PWD}/logs:/app/logs" `
-v "${PWD}/csv_files:/app/csv_files" `
-v "${PWD}/owned_cards:/app/owned_cards" `
-v "${PWD}/config:/app/config" `
mwisnowski/mtg-python-deckbuilder:latest
```
## Web UI (new)
The web UI runs the same deckbuilding logic behind a browser-based interface.
### PowerShell (recommended)
```powershell
docker compose up --build --no-deps -d web
```
Then open http://localhost:8080
Volumes are the same as the CLI service, so deck exports/logs/configs persist in your working folder.
The app serves a favicon at `/favicon.ico` and exposes a health endpoint at `/healthz`.
Compare view offers a Copy summary button to copy a plain-text diff of two runs. The sidebar has a subtle depth shadow for clearer separation.
Web UI feature highlights:
- Locks: Click a card or the lock control in Step 5; locks persist across reruns.
- Replace: Enable Replace in Step 5, click a card to open Alternatives (filters include Owned-only), then choose a swap.
- Permalinks: Copy a permalink from Step 5 or a Finished deck; paste via “Open Permalink…” to restore.
- Compare: Use the Compare page from Finished Decks; quick actions include Latest two and Swap A/B.
Virtualized lists and lazy images (opt‑in)
- Set `WEB_VIRTUALIZE=1` to enable virtualization in Step 5 grids/lists and the Owned library for smoother scrolling on large sets.
- Example (Compose):
```yaml
services:
web:
environment:
- WEB_VIRTUALIZE=1
```
- Example (Docker Hub):
```powershell
docker run --rm -p 8080:8080 `
-e WEB_VIRTUALIZE=1 `
-v "${PWD}/deck_files:/app/deck_files" `
-v "${PWD}/logs:/app/logs" `
-v "${PWD}/csv_files:/app/csv_files" `
-v "${PWD}/owned_cards:/app/owned_cards" `
-v "${PWD}/config:/app/config" `
-e SHOW_DIAGNOSTICS=1 ` # optional: enables diagnostics tools and overlay
mwisnowski/mtg-python-deckbuilder:latest `
bash -lc "cd /app && uvicorn code.web.app:app --host 0.0.0.0 --port 8080"
```
### Diagnostics and logs (optional)
Enable internal diagnostics and a read-only logs viewer with environment flags.
- `SHOW_DIAGNOSTICS=1` — adds a Diagnostics nav link and `/diagnostics` tools
- `SHOW_LOGS=1` — enables `/logs` and `/status/logs?tail=200`
When enabled:
- `/logs` supports an auto-refresh toggle with interval, a level filter (All/Error/Warning/Info/Debug), and a Copy button to copy the visible tail.
- `/status/sys` returns a simple system summary (version, uptime, UTC server time, and feature flags) and is shown on the Diagnostics page when `SHOW_DIAGNOSTICS=1`.
- Virtualization overlay: press `v` on pages with virtualized grids to toggle per-grid overlays and a global summary bubble.
Compose example (web service):
```yaml
environment:
- SHOW_LOGS=1
- SHOW_DIAGNOSTICS=1
```
Docker Hub (PowerShell) example:
```powershell
docker run --rm `
-p 8080:8080 `
-e SHOW_LOGS=1 -e SHOW_DIAGNOSTICS=1 -e ENABLE_THEMES=1 -e THEME=system `
-e SPLASH_ADAPTIVE=1 -e SPLASH_ADAPTIVE_SCALE="1:1.0,2:1.0,3:1.0,4:0.6,5:0.35" ` # optional experiment
-e RANDOM_MODES=1 -e RANDOM_UI=1 -e RANDOM_MAX_ATTEMPTS=5 -e RANDOM_TIMEOUT_MS=5000 `
-v "${PWD}/deck_files:/app/deck_files" `
-v "${PWD}/logs:/app/logs" `
-v "${PWD}/csv_files:/app/csv_files" `
-v "${PWD}/owned_cards:/app/owned_cards" `
-v "${PWD}/config:/app/config" `
mwisnowski/mtg-python-deckbuilder:latest `
bash -lc "cd /app && uvicorn code.web.app:app --host 0.0.0.0 --port 8080"
```
### Setup speed: parallel tagging (Web)
First-time setup or stale data triggers card tagging. The web service uses parallel workers by default.
Configure via environment variables on the `web` service:
- `WEB_TAG_PARALLEL=1|0` — enable/disable parallel tagging (default: 1)
- `WEB_TAG_WORKERS=<N>` — number of worker processes (default: 4 in compose)
If parallel initialization fails, the service falls back to sequential tagging and continues.
### From Docker Hub (PowerShell)
If you prefer not to build locally, pull `mwisnowski/mtg-python-deckbuilder:latest` and run uvicorn:
```powershell
docker run --rm `
-p 8080:8080 `
-v "${PWD}/deck_files:/app/deck_files" `
-v "${PWD}/logs:/app/logs" `
-v "${PWD}/csv_files:/app/csv_files" `
-v "${PWD}/owned_cards:/app/owned_cards" `
-v "${PWD}/config:/app/config" `
mwisnowski/mtg-python-deckbuilder:latest `
bash -lc "cd /app && uvicorn code.web.app:app --host 0.0.0.0 --port 8080"
```
Health check:
```text
GET http://localhost:8080/healthz -> { "status": "ok", "version": "dev", "uptime_seconds": 123 }
```
Theme preference reset (client-side): use the header’s Reset Theme control to clear the saved browser preference; the server default (THEME) applies on next paint.
### Random Modes (alpha) and test dataset override
Enable experimental Random Modes and UI controls in Web runs by setting:
```yaml
services:
web:
environment:
- RANDOM_MODES=1
- RANDOM_UI=1
- RANDOM_MAX_ATTEMPTS=5
- RANDOM_TIMEOUT_MS=5000
```
For deterministic tests or development, you can point the app to a frozen dataset snapshot:
```yaml
services:
web:
environment:
- CSV_FILES_DIR=/app/csv_files/testdata
```
### Taxonomy snapshot (maintainers)
Capture the current bracket taxonomy into an auditable JSON file inside the container:
```powershell
docker compose run --rm web bash -lc "python -m code.scripts.snapshot_taxonomy"
```
Artifacts appear under `./logs/taxonomy_snapshots/` on your host via the mounted volume.
To force a new snapshot even when the content hash matches the latest, pass `--force` to the module.
## Volumes
- `/app/deck_files` ↔ `./deck_files`
- `/app/logs` ↔ `./logs`
- `/app/csv_files` ↔ `./csv_files`
- `/app/owned_cards` ↔ `./owned_cards` (owned cards lists: .txt/.csv)
- Optional: `/app/config` ↔ `./config` (JSON configs for headless)
## Interactive vs headless
- Interactive: attach a TTY (compose run or `docker run -it`)
- Headless auto-run:
```powershell
docker compose run --rm -e DECK_MODE=headless mtg-deckbuilder
```
- Headless with JSON config:
```powershell
docker compose run --rm `
-e DECK_MODE=headless `
-e DECK_CONFIG=/app/config/deck.json `
mtg-deckbuilder
```
### Common env vars
- DECK_MODE=headless
- DECK_CONFIG=/app/config/deck.json
- DECK_COMMANDER, DECK_PRIMARY_CHOICE
- DECK_ADD_LANDS, DECK_FETCH_COUNT
- DECK_TAG_MODE=AND|OR (combine mode used by the builder)
### Web UI tuning env vars
- WEB_TAG_PARALLEL=1|0 (parallel tagging on/off)
- WEB_TAG_WORKERS=<N> (process count; set based on CPU/memory)
- WEB_VIRTUALIZE=1 (enable virtualization)
- SHOW_DIAGNOSTICS=1 (enables diagnostics pages and overlay hotkey `v`)
- RANDOM_MODES=1 (enable random build endpoints)
- RANDOM_UI=1 (show Surprise/Theme/Reroll/Share controls)
- RANDOM_MAX_ATTEMPTS=5 (cap retry attempts)
- (Upcoming) Multi-theme inputs: once UI ships, Random Mode will accept `primary_theme`, `secondary_theme`, `tertiary_theme` fields; current backend already supports the cascade + diagnostics.
- RANDOM_TIMEOUT_MS=5000 (per-build timeout in ms)
Testing/determinism helper (dev):
- CSV_FILES_DIR=csv_files/testdata — override CSV base dir to a frozen set for tests
## Manual build/run
```powershell
docker build -t mtg-deckbuilder .
docker run -it --rm `
-v "${PWD}/deck_files:/app/deck_files" `
-v "${PWD}/logs:/app/logs" `
-v "${PWD}/csv_files:/app/csv_files" `
-v "${PWD}/owned_cards:/app/owned_cards" `
-v "${PWD}/config:/app/config" `
mtg-deckbuilder
```
## Troubleshooting
- No prompts? Use `docker compose run --rm` (not `up`) or add `-it` to `docker run`
- Files not saving? Verify volume mounts and that folders exist
- Headless not picking config? Ensure `./config` is mounted to `/app/config` and `DECK_CONFIG` points to a JSON file
- Owned-cards prompt not seeing files? Ensure `./owned_cards` is mounted to `/app/owned_cards`
## Tips
- Use `docker compose run`, not `up`, for interactive mode
- Exported decks appear in `deck_files/`
- JSON run-config is exported only in interactive runs; headless skips it

951
docs/archive/README_2025-10-02.md Normal file
View file

@ -0,0 +1,951 @@
# 🃏 MTG Python Deckbuilder
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Python 3.11+](https://img.shields.io/badge/python-3.11+-blue.svg)](https://www.python.org/downloads/)
[![Docker](https://img.shields.io/badge/docker-supported-blue.svg)](https://www.docker.com/)
A fast, menu-driven deckbuilder for MTG Commander/EDH with both CLI and Web UI. Supports interactive and headless runs, CSV/TXT exports, owned-card libraries, and Docker.
## 🔹 Features
- Web UI and CLI, powered by the same builder
- Commander search with smart theme tagging (primary/secondary/tertiary)
- Power bracket selection and ideal count prompts
- **Include/Exclude Card Lists**: Must-include and must-exclude functionality with fuzzy matching, visual validation, and strict enforcement (set `ALLOW_MUST_HAVES=true`)
- CSV and TXT exports with stable filenames
- Headless mode (non-interactive) and a headless submenu in the main menu
- Config precedence: CLI > env > JSON > defaults
- Visual summaries in Web UI: Mana curve, Color pips and Sources with hover-to-highlight and copyable tooltips
- Web UI: Locks, Replace flow, Compare builds, and shareable permalinks
- Unified “New Deck” modal (steps 1–3): commander search, themes, bracket, options, and optional Name (used in export filenames)
- Multi-copy archetypes (opt-in modal): choose quantity for supported archetypes and optionally add Thrumming Stone; added early with target adjustments and a hard 100-card clamp
- Combos & Synergies: detect curated pairs, show chip-style lists with badges, and dual-card hover previews; optional auto-complete adds missing partners based on your plan.
## ✨ Highlights
- Smart tagging and suggestions for commander + themes, with AND/OR combine modes
- Theme governance: generated theme catalog with whitelist normalization, enforced synergies, and capped synergy lists (top 5) for concise UI; roadmap includes YAML authoring export.
- Exports CSV and TXT decklists to `deck_files/`
- Random Full Build (alpha) now outputs a single authoritative artifact set (CSV, TXT, compliance JSON, summary JSON) without duplicate suffixed files; API returns paths for integration. Summary sidecars carry multi-theme metadata (`primary_theme`, `secondary_theme`, `tertiary_theme`, `resolved_themes`, fallback flags) plus legacy aliases for incremental consumer upgrades.
- Random Mode (alpha) multi-theme groundwork: curated pools now support manual exclusions via `config/random_theme_exclusions.yml` (documented in `docs/random_theme_exclusions.md` and exportable with `python code/scripts/report_random_theme_pool.py --write-exclusions`); UI/diagnostics surface fallback telemetry while the upcoming picker adds Primary/Secondary/Tertiary inputs with deterministic fallback (P+S+T → P+S → P+T → P → synergy overlap → full pool) and an inline “Clear themes” control quickly resets stored inputs.
- Random Mode instrumentation: `python code/scripts/profile_multi_theme_filter.py` captures mean/p95 filter timings, while `python code/scripts/check_random_theme_perf.py` guards against regressions relative to `config/random_theme_perf_baseline.json` (`--update-baseline` refreshes).
- Owned-cards mode: point to one or more `.txt`/`.csv` files in `owned_cards/` and choose "Use only owned cards?"
- If you opt out, final CSV marks owned cards with an `Owned` column
- Build options include Prefer-owned: bias picks toward owned cards while allowing unowned fallback
- Interactive menu with a headless submenu
- Works locally or in Docker (Windows PowerShell friendly)
- Card images and data via Scryfall (attribution shown in the Web UI)
- Web visual summaries: cross-highlight cards from charts; sources include non-lands and colorless 'C' with an on/off toggle
- New Deck modal keyboard UX: Enter selects the first suggestion; arrow navigation removed; browser autofill disabled
- Export naming: optional Name in the modal becomes the filename stem for CSV/TXT/JSON; decks include a sidecar `.summary.json` with `meta.name`
- Finished Decks and deck banners prefer your custom Name when present
- Compare page includes a Copy summary button to quickly share diffs
- New Deck modal shows your selected themes and their order (1 → 3) while picking; Bracket options are labeled (e.g., "Bracket 3: Upgraded"). Default bracket is 3.
- Bracket policy enforcement: disallowed categories (e.g., Game Changers in Brackets 1–2) are pruned from the pool up-front; UI blocks progression until violations are resolved, with inline Replace to pick compliant alternatives.
- Bracket compliance UX: the compliance panel auto-opens when non-compliant and shows a colored overall chip (green/WARN amber/red). WARN thresholds (e.g., tutors/extra turns) are advisory—tiles show with a subtle style and no gating; FAIL continues to gate and enables enforcement.
### Commander eligibility rules (clarified)
The builder now applies stricter Commander legality filtering when generating `commander_cards.csv`:
1. Automatically eligible:
- Legendary Creatures (includes Artifact Creature / Enchantment Creature lines)
- Legendary Artifact Vehicles or Spacecraft that have printed power and toughness
- Any card whose rules text explicitly contains "can be your commander"
2. Not auto‑eligible (unless they have the explicit text above):
- Plain Legendary Planeswalkers
- Plain Legendary Enchantments without the Creature type
- Generic Legendary Artifacts (non‑Vehicle/Spacecraft or without P/T)
This prevents over-broad inclusion of non-creature legendary permanents.
## 🚀 Quick start
### Docker Hub (PowerShell)
```powershell
docker run -it --rm `
-v "${PWD}/deck_files:/app/deck_files" `
-v "${PWD}/logs:/app/logs" `
-v "${PWD}/csv_files:/app/csv_files" `
-v "${PWD}/owned_cards:/app/owned_cards" `
-v "${PWD}/config:/app/config" `
mwisnowski/mtg-python-deckbuilder:latest
```
### From source
```powershell
pip install -r requirements.txt
python code/main.py
```
### From this repo with Docker Compose (PowerShell)
```powershell
docker compose build
docker compose run --rm mtg-deckbuilder
```
## 🌐 Web UI
Run the browser-based UI backed by the same deckbuilding engine.
### From source
```powershell
pip install -r requirements.txt
uvicorn code.web.app:app --host 127.0.0.1 --port 8080
```
Open http://127.0.0.1:8080
### Docker Compose (PowerShell)
```powershell
docker compose build web
docker compose up --no-deps web
```
Then open http://localhost:8080
Notes:
- Exports/logs/configs use the same folders and will appear under your mounted volumes (see compose file).
- The footer includes Scryfall attribution per their guidelines.
- Favicon is bundled; browsers load `/favicon.ico` (auto-falls back to PNG).
- Bracket enforcement is applied in both web and headless runs; see "Bracket policy & enforcement" below.
### Run with Docker Compose (Docker Hub image)
Use the prebuilt image with the provided `dockerhub-docker-compose.yml`. The image defaults to the Web UI.
Web (default):
```powershell
docker compose -f dockerhub-docker-compose.yml up -d
```
Pin to a version (edit the compose file image tag to `:X.Y.Z`) to avoid `latest` drift.
CLI mode (override default):
```powershell
docker compose -f dockerhub-docker-compose.yml run --rm `
-e APP_MODE=cli `
web
```
Notes:
- Volumes under `${PWD}` persist your exports/logs/configs locally next to the compose file.
- Health checks are built into the image for the Web UI. In CLI, `APP_MODE=cli` bypasses health.
#### Theme defaults and flags
- Enable the Theme selector with `ENABLE_THEMES=1` (on by default in `web` in compose).
- Set the initial default with `THEME=system|light|dark` (applies only when the browser has no prior preference saved):
- `THEME=system` (recommended): follows the OS theme; Light uses the consolidated Light (Blend) palette.
- `THEME=light`: maps to the consolidated Light (Blend) theme in the UI.
- `THEME=dark`: uses the dark theme.
- You can also force a one-off override via URL: `?theme=system|light|dark|high-contrast|cb-friendly`.
- Example: `http://localhost:8080/?theme=dark`
- The choice is saved in localStorage; the URL parameter is removed after applying.
- Reset: Use the “Reset theme” control in the header to clear your preference and re‑apply the server default (or system).
- YAML export strategy: By default the app now always regenerates per-theme YAML files even on fast refreshes to keep editorial artifacts synchronized with `theme_list.json`. To opt-out on incremental refreshes (rare), set `THEME_YAML_FAST_SKIP=1` (compose variable). This only skips export during fast-path (no retag) theme refreshes and never during full builds.
#### Theme catalog editorial pipeline
Build script: `python code/scripts/build_theme_catalog.py`
Key flags / env vars:
- `--limit N` (preview subset; guarded from overwriting canonical JSON unless `--allow-limit-write`)
- `--output path` (write catalog to alternate location; suppresses YAML backfill so tests don't mutate source files)
- `--backfill-yaml` or `EDITORIAL_BACKFILL_YAML=1` (write auto `description` + derived/pinned `popularity_bucket` into each YAML if missing)
- `--force-backfill-yaml` (overwrite existing description/popularity_bucket)
- `EDITORIAL_SEED=<int>` (deterministic ordering for inference and any randomized fallbacks)
- `EDITORIAL_AGGRESSIVE_FILL=1` (pad ultra-sparse themes with extra inferred synergies)
- `EDITORIAL_POP_BOUNDARIES="a,b,c,d"` (override popularity bucket thresholds)
- `EDITORIAL_POP_EXPORT=1` (emit `theme_popularity_metrics.json` summary)
Derived fields:
- `popularity_bucket` (Very Common / Common / Uncommon / Niche / Rare) based on summed color frequency; authors may pin a value in YAML.
- `description` auto-generated if absent using heuristics keyed to theme class (tribal, counters, graveyard, tokens, etc.).
Editorial quality & metadata info:
- Optional `editorial_quality: draft|reviewed|final` can be set in per-theme YAML (or force-backfilled) to track curation progress.
- Backfills now stamp a `metadata_info` block (formerly `provenance`) with keys such as `last_backfill`, `script`, and `version`. The legacy key `provenance` is still parsed for a short deprecation window; if both appear a warning is emitted (suppress via `SUPPRESS_PROVENANCE_DEPRECATION=1`). Planned removal of the legacy alias: version 2.4.0.
- A running specialization trend is appended to `config/themes/description_fallback_history.jsonl` whenever `EDITORIAL_INCLUDE_FALLBACK_SUMMARY=1` is set.
- When `EDITORIAL_INCLUDE_FALLBACK_SUMMARY=1`, the catalog additionally embeds a `description_fallback_summary` object exposing coverage KPIs (see below) to power regression tests.
Synergy pairs & clusters:
- `config/themes/synergy_pairs.yml` provides curated fallback synergies (applied only if a theme YAML lacks `curated_synergies`).
- `config/themes/theme_clusters.yml` groups themes (Tokens, Counters, Graveyard, etc.) for upcoming UI filters and analytics.
- Validator checks structure; UI integration planned in Phase E.
Description mapping externalization:
- `config/themes/description_mapping.yml` fully replaces internal mapping when present.
- Items: `triggers` (list of substrings, lowercase-insensitive) and `description` string.
- `{SYNERGIES}` placeholder expands to a short clause with top synergy names or is removed gracefully if none.
- Mapping validator (`scripts/validate_description_mapping.py`) reports duplicate triggers, suggests `{SYNERGIES}` usage, and validates synergy pairs & clusters.
KPI targets (initial schedule):
- Wave 1 (current): Maintain regression test thresholds (generic_pct < 52%).
- Wave 2: Ratchet to <45% after next mapping expansion.
- Wave 3: <38%; Wave 4: <30% (issue templates will adjust thresholds once prior target is achieved and history confirms downward trend).
Lint (`code/scripts/lint_theme_editorial.py`) enhancements:
- `--require-description` / env `EDITORIAL_REQUIRE_DESCRIPTION=1` enforces descriptions present.
- `--require-popularity` / env `EDITORIAL_REQUIRE_POPULARITY=1` enforces popularity bucket present.
- Strict mode also escalates missing cornerstone examples and missing description/popularity to errors.
Testing:
Determinism, boundary overrides, and backfill isolation validated in `code/tests/test_theme_catalog_generation.py` using the new `--output` flag.
### Commander catalog upkeep
- The Commander Browser and auto-select flows read from `csv_files/commander_cards.csv`; keep it version-controlled so web and CLI stay in sync.
- Regenerate the file after MTGJSON updates or commander policy tweaks:
```powershell
python -c "from file_setup.setup import regenerate_csvs_all; regenerate_csvs_all()"
```
- For quick fixes, you can refresh just the commander list:
```powershell
python -c "from file_setup.setup import determine_commanders; determine_commanders()"
```
- Full onboarding guidance (required columns, caching notes, validation steps) lives in `docs/commander_catalog.md`.
- Staging toggle: set `SHOW_COMMANDERS=0` to hide the sidebar link while testing privately; defaults to on.
### Theme Catalog API
Two new endpoints expose the merged catalog for the upcoming theme picker UI.
List themes:
`GET /themes/api/themes`
Parameters:
- `q` substring across theme name & synergies
- `archetype` filter by `deck_archetype`
- `bucket` popularity bucket (Very Common|Common|Uncommon|Niche|Rare)
- `colors` comma-separated color initials (e.g. `G,W`)
- `limit` (default 50, max 200) & `offset`
- `diagnostics=1` (requires env `WEB_THEME_PICKER_DIAGNOSTICS=1`) to expose `has_fallback_description` + `editorial_quality`
Sample response snippet:
```
{
"ok": true,
"count": 314,
"items": [
{
"id": "plus1-plus1-counters",
"theme": "+1/+1 Counters",
"primary_color": "Green",
"secondary_color": "White",
"popularity_bucket": "Very Common",
"deck_archetype": "Counters",
"description": "...",
"synergies": ["Adapt","Evolve","Proliferate","Counters Matter","Hydra Kindred"],
"synergy_count": 5
// diagnostics-only: has_fallback_description, editorial_quality
}
],
"next_offset": 50,
"generated_at": "2025-09-19T12:34:56",
"diagnostics": false
}
```
Theme detail:
`GET /themes/api/theme/{id}`
Parameters:
- `uncapped=1` (diagnostics only) returns `uncapped_synergies` (full curated+enforced+inferred ordered set)
- `diagnostics=1` enables extra fields (fallback + editorial quality)
Detail response excerpt:
```
{
"ok": true,
"theme": {
"id": "plus1-plus1-counters",
"theme": "+1/+1 Counters",
"description": "...",
"synergies": ["Adapt","Evolve","Proliferate","Counters Matter","Hydra Kindred"],
"curated_synergies": [...],
"enforced_synergies": [...],
"inferred_synergies": [...],
"example_commanders": [...],
"example_cards": [...],
"synergy_commanders": [...]
// diagnostics-only: uncapped_synergies, editorial_quality, has_fallback_description
}
}
```
Caching & freshness: ETag header reflects catalog size/mtime + newest YAML mtime. Use `/themes/status` for stale detection; POST `/themes/refresh` to rebuild in background.
Environment flag: `WEB_THEME_PICKER_DIAGNOSTICS=1` enables diagnostics extras and uncapped synergy toggle.
Performance & optimization:
- Fast list filtering uses precomputed lowercase search haystacks + memoized filtered slug cache (keyed by catalog ETag and query params) for sub‑50ms typical responses once warm.
- Skeleton loaders (picker shell, table list fragment, preview modal) improve perceived load time; real content swaps in via HTMX fragment requests.
- Metrics endpoint `/themes/metrics` (gated by `WEB_THEME_PICKER_DIAGNOSTICS=1`) exposes:
- Catalog filter requests, cache hits, cache entry count
- Preview requests, cache hits, average build time (ms), cache size, TTL
- Response headers: `X-ThemeCatalog-Filter-Duration-ms` and `ETag` let you observe server-side filter cost and enable conditional requests.
### Governance & Editorial
The theme catalog follows lightweight governance so curation quality scales:
1. Minimum Examples Threshold: Target 2 example cards and 1 example commander for every established theme. Enforcement flips from warning to required once coverage >90% (tracked via metrics). Until then, themes below threshold are flagged but not blocked.
2. Curation Ordering: Preview assembly order is deterministic — examples, curated synergy examples, sampled role cards (payoff/enabler/support/wildcard), then synthetic placeholders only if needed to reach target count.
3. Splash Relax Policy: Four- and five-color commanders may include a single off-color enabler (scored with a -0.3 penalty) to avoid over-pruning multi-color creative lines. This is documented to prevent accidental removal during future sampling refactors.
4. Popularity Buckets Are Non-Scoring: Popularity is for filtering and subtle UI hints only and must never directly influence sampling scores (avoids positive feedback loops that crowd out niche archetypes).
5. Taxonomy Expansion Criteria: Proposed new high-level themes (Combo, Storm, Extra Turns, Group Hug / Politics, Pillowfort, Toolbox / Tutors, Treasure Matters, Monarch / Initiative) must demonstrate: (a) a distinct strategic pattern, (b) at least 8 broadly played representative cards, (c) not a strict subset of another existing theme.
6. Editorial Quality Tracking: Optional `editorial_quality: draft|reviewed|final` in theme YAML guides review pass prioritization; metrics aggregate coverage to spot stagnation.
7. Deterministic Sampling: Seed = `theme|commander` hash; changes to sampling heuristics must preserve deterministic output for identical inputs (regression tests rely on this). Score contributors must append reasons for transparency (`reasons[]`).
For deeper rationale & checklist see `docs/theme_taxonomy_rationale.md`.
- Preview sampling caches results (TTL 600s) keyed by (slug, limit, colors, commander, ETag) ensuring catalog changes invalidate prior samples.
Governance tooling & experiments:
- Taxonomy snapshot: `python -m code.scripts.snapshot_taxonomy` writes `logs/taxonomy_snapshots/taxonomy_<timestamp>.json` with a SHA-256 `hash` for auditability. Identical content is skipped unless forced. Use this before tuning taxonomy-aware sampling heuristics.
- Adaptive splash penalty (optional): set `SPLASH_ADAPTIVE=1` to scale off-color enabler penalties based on commander color count; tune with `SPLASH_ADAPTIVE_SCALE` (e.g., `1:1.0,2:1.0,3:1.0,4:0.6,5:0.35`). Analytics recognize both static and adaptive reasons; experiments can compare counters when enabled vs disabled.
Preview endpoint:
`GET /themes/api/theme/{id}/preview?limit=12&colors=G,W&commander=Name`
Returns a deterministic sample (seeded by theme + optional commander) combining:
- Pinned curated `example_cards` (role `example`)
- Pinned curated `synergy_example_cards` (role `curated_synergy`)
- Heuristically sampled cards bucketed into payoff / enabler / support / wildcard with diversity quotas (~40/40/20 after pinned examples)
- Synthetic `[Synergy]` placeholders only if real sampling underfills the requested limit
Commander bias heuristics (color identity restriction + overlap/theme bonuses) influence scoring & ordering.
Preview sampling now includes curated examples, curated synergy layer, role-based heuristic sampling (payoff / enabler / support / wildcard) with diversity quotas, commander bias heuristics (color identity restriction + diminishing overlap bonuses), rarity diminishing weights, splash leniency (single off-color allowance with small penalty for 4–5 color commanders), role saturation penalty, refined overlap scaling curve, and tooltip-expanded heuristic reasons. Synthetic placeholders are only added if the sample underfills.
Edge considerations (Theme picker & preview):
- Empty dataset: graceful skeleton + 'No themes match your filters' message.
- High latency first load: skeleton shimmer + optional prewarm (when `WEB_THEME_FILTER_PREWARM=1`).
- Catalog reload or tagging completion: filter + preview caches bust; stale indicator triggers background refresh.
- Commander not found: commander bias reasons omitted; sampling still deterministic via seed.
- Color filter reduces pool below quota: synthetic `[Synergy]` placeholders pad to requested limit.
- Duplicate card names across tag shards: final defensive dedupe pass before payload.
- Oversized preview cache: FIFO eviction maintains <= THEME_PREVIEW_CACHE_MAX (default 400).
- Disabled diagnostics: uncapped synergies, editorial quality, fallback description markers, raw YAML & metrics all suppressed.
- Tooltip safety: JS errors are swallowed; preview remains functional without enhancements.
#### Editorial coverage metrics / specialization progress
- Set `EDITORIAL_INCLUDE_FALLBACK_SUMMARY=1` when running `build_theme_catalog.py` to embed a `description_fallback_summary` block in the resulting JSON. This includes:
- `generic_total`, `generic_with_synergies`, `generic_plain`, and `generic_pct`
- `top_generic_by_frequency` (capped list of the highest-frequency themes still using generic fallback phrasing)
- Used by `test_theme_description_fallback_regression.py` to enforce a shrinking ceiling on generic descriptions after each optimization wave.
- History persisted to `description_fallback_history.jsonl` for KPI trend visualization and automated threshold ratcheting.
#### External description mapping
- Extend or override auto-description heuristics without code edits via `config/themes/description_mapping.yml`.
- Each list item: `triggers: ["substring1", ...]`, `description: "Text with optional {SYNERGIES} placeholder"`
- First match wins; triggers are lowercase substring checks.
- `{SYNERGIES}` is replaced with a short clause (e.g., `Synergies like Tokens and Sacrifice reinforce the plan.`) when synergy context exists; omitted cleanly otherwise.
- If the file is absent, the internal mapping rules are used as a fallback.
### Quick guide (locks • replace • compare • permalinks)
- New Deck modal (Steps 1–3 unified): search for your commander, pick themes, choose bracket (defaults to 3), and optionally set Name. Submitting starts the build immediately. Name becomes the export filename stem and display name.
- Lock a card: click the card image or the lock control in Step 5. Locked cards are pinned for reruns. A small “Last action” chip confirms lock/unlock.
- Replace a card: toggle Replace in Step 5, then click a card to open Alternatives. Filter (including Owned-only) and pick a swap; the deck updates in place.
- When bracket violations exist, Step 5 shows them first and disables Continue/Rerun. Pick replacements inline; alternatives exclude commanders, locked, in-deck, and just-added cards and prefer role-consistent options.
- Permalink: use “Copy permalink” from Step 5 or a Finished deck to share/restore a run (commander, themes, bracket, ideals, flags). “Open Permalink…” lets you paste one to restore to review.
- Compare: from Finished Decks, open Compare and pick A and B (quick actions: Latest two, Swap A/B). Use Changed-only to focus diffs. Copy summary or Download .txt.
- Keyboard: In the New Deck modal, Enter selects the first commander result. Browser autofill is disabled for cleaner inputs. In Step 5, use L to lock/unlock the focused card, R to open Alternatives, and C to copy a permalink.
### Setup speed: parallel tagging (Web)
- On first run or when data is stale, the Web UI prepares and tags the card database.
- This tagging runs in parallel by default for faster setup.
- Tuning via env vars (set on the `web` service in `docker-compose.yml`):
- `WEB_TAG_PARALLEL=1|0` — enable/disable parallel workers (default: 1/enabled)
- `WEB_TAG_WORKERS=<N>` — number of worker processes (default: 4 in compose; omit to auto-pick)
- The UI shows progress and falls back to sequential tagging automatically if parallel init fails.
Virtualized lists and image polish (opt-in)
- Opt-in with `WEB_VIRTUALIZE=1` to enable virtualization in Step 5 lists/grids and the Owned library for smoother scrolling on large sets.
- Virtualization diagnostics overlay (for debugging): enable `SHOW_DIAGNOSTICS=1`, then press `v` to toggle an overlay per grid and a global summary bubble; shows visible range, total, rows, row height, render time, and counters.
- Thumbnails use lazy-loading with srcset/sizes; LQIP blur/fade-in is applied to Step 5 and Owned thumbnails, and the commander preview image.
- Respects reduced motion: when the OS prefers-reduced-motion, blur/fade transitions and smooth scrolling are disabled for accessibility.
### Build options (Web)
- Use only owned cards: builds strictly from your `owned_cards/` lists (commander exempt).
- Prefer owned cards: gently prioritizes owned cards across creatures and spells but allows unowned when they’re better fits.
- Implemented via a stable owned-first reorder and small weight boosts; preserves existing sort intent.
Notes:
- Steps 1–3 are consolidated into a single New Deck modal. Submitting starts the build immediately, skipping the old review page.
- The modal includes an optional Name; when set, it is used as the export filename stem and display name in the UI.
- The modal displays selected themes in order (1, 2, 3). The Bracket selector shows numbers (e.g., "Bracket 3: Upgraded") and defaults to 3 for new decks.
### Staged build visibility
- Step 5 can optionally “Show skipped stages” so you can see phases that added no cards with a clear annotation.
### Multi-copy archetypes (Web)
- When your commander + themes suggest a multi-copy strategy (e.g., petitioners, approach, apostles), the Web UI shows a one-time modal to add a package.
- Choose how many copies (bounded by the printed cap) and optionally include 1× Thrumming Stone when it synergizes.
- The package is applied as the first stage so later phases account for the volume.
- Targets adjust automatically (reduce creatures for creature packages, or spread reductions across spells). The UI shows an “Adjusted targets” note on that stage.
- A final safety clamp trims overflow from this stage to keep the deck at 100. A “Clamped N” chip appears when this happens.
- Suggestions won’t re-prompt repeatedly for the same commander+theme context unless you change selections; you can also dismiss the modal.
### Visual summaries (Web)
- Mana Curve bars with hover-to-highlight of matching cards in both list and thumbnail views
- Color Pips (requirements) and Sources (generation) bars with per-color tooltips listing cards
- Cross-highlighting from charts to the card views; list-mode highlights only the name span
- Sources detection includes non-land producers (artifacts/creatures/etc.) and colorless 'C'
- Fetch lands are not counted as mana sources; basic lands and Wastes are handled reliably
- Optional: Show colorless (C) toggle for Sources; persisted per browser
- Tooltips include a Copy button to copy the card list
### Combos & Synergies (Web)
- Detection: curated two-card combos/synergies are detected from the final deck (commander included) and shown with list version badges.
- UI: chip-style rows with badges (cheap/early, setup). Hover a row to preview both cards side-by-side; hover an individual name for single-card preview.
- Auto-Complete Combos: when enabled in the New Deck modal, the builder completes up to N pairs before theme fill/monolithic spells so partners stick.
- Settings: Prefer combos (on/off), How many combos (target), Balance (early/late/mix) to bias partner selection.
- Existing completed pairs are counted first; only missing partners are added.
- Color identity enforced via the filtered card pool; off-color/unavailable partners are skipped.
### Owned page (Web)
- View and manage your owned lists with:
- Export TXT/CSV, sort controls, and a live “N shown” counter
- Color identity dots and exact color-identity combo filters (including 4-color)
- Viewport-filling list with styled scrollbar
- Optional virtualization when `WEB_VIRTUALIZE=1` (improves performance on very large libraries)
- Uploading `.txt` or `.csv` lists enriches and deduplicates entries at upload-time and persists them, so page loads are fast.
### Finished Decks (Web)
- Theme filters are a dropdown with shareable state for easier browsing of saved builds.
- Each finished deck has a permalink you can copy and revisit; Compare mode lets you diff against another run. The Compare page has a Copy summary button to copy a plain-text diff.
- Locks and Replace flow: lock-in picks you like or replace a card with an alternative during iteration.
### Diagnostics (Web)
- Health endpoint: `GET /healthz` returns `{ status, version, uptime_seconds }`
- Responses include `X-Request-ID` for easier error correlation; unhandled errors return JSON with `request_id`
Logs and system tools:
- Logs page (`/logs`, enable with `SHOW_LOGS=1`):
- Auto-refresh toggle with adjustable interval
- Level filter (All/Error/Warning/Info/Debug) and keyword filter
- Copy button to copy the visible log tail
- System summary (`GET /status/sys`, shown on `/diagnostics` when `SHOW_DIAGNOSTICS=1`):
- Returns `{ version, uptime_seconds, server_time_utc, flags }`
- Shows resolved theme and stored preference; includes a Reset preference button.
Compose: enable diagnostics/logs (optional)
```yaml
services:
web:
environment:
- SHOW_LOGS=1
- SHOW_DIAGNOSTICS=1
# Random Modes (optional; alpha)
- RANDOM_MODES=1
- RANDOM_UI=1
- RANDOM_MAX_ATTEMPTS=5
- RANDOM_TIMEOUT_MS=5000
# RANDOM_BUILD_SUPPRESS_INITIAL_EXPORT: # (Default behavior auto-enables suppression; set to 0 to debug legacy double-export)
```
### Budget Mode and price/legal status
- Price and legality snippet integration is deferred to a dedicated Budget Mode initiative. This will centralize price sourcing, caching, legality checks, and UI surfaces. Track progress in `logs/roadmaps/roadmap_9_budget_mode.md`.
## 🤖 Headless mode (no prompts)
- Auto-headless: set `DECK_MODE=headless`
- Example (PowerShell):
```powershell
docker compose run --rm -e DECK_MODE=headless mtg-deckbuilder
```
- Use a JSON config: mount `./config` to `/app/config` and set `DECK_CONFIG=/app/config/deck.json`
- Example (PowerShell):
```powershell
docker compose run --rm `
-e DECK_MODE=headless `
-e DECK_CONFIG=/app/config/deck.json `
mtg-deckbuilder
```
- Override via env vars (subset): `DECK_COMMANDER`, `DECK_PRIMARY_CHOICE`, `DECK_SECONDARY_CHOICE`, `DECK_TERTIARY_CHOICE`, `DECK_ADD_LANDS`, `DECK_FETCH_COUNT`, `DECK_BRACKET_LEVEL`
- Precedence: CLI > env > JSON > defaults
### Enhanced CLI with Type Safety & Theme Names
The headless runner now features comprehensive CLI arguments with type indicators and intelligent theme selection:
```powershell
# Show all available options with type information
python code/headless_runner.py --help
# Build with theme names instead of index numbers
python code/headless_runner.py `
--commander "Aang, Airbending Master" `
--primary-tag "Airbending" `
--secondary-tag "Exile Matters" `
--bracket-level 4
# Override ideal deck composition counts
```
## 🧪 Theme Catalog Validation & Schema
The theme system now uses a merged catalog plus per-theme YAML files.
Validator usage:
```powershell
python code/scripts/validate_theme_catalog.py # Standard validation
python code/scripts/validate_theme_catalog.py --rebuild-pass # Ensures idempotent rebuild
python code/scripts/validate_theme_catalog.py --schema # Catalog JSON Schema
python code/scripts/validate_theme_catalog.py --yaml-schema # Per-file YAML JSON Schema
python code/scripts/validate_theme_catalog.py --strict-alias # Fail if alias display_name still present
```
CI runs the non-strict validator every push. A second step runs `--strict-alias` as an allowed failure until legacy alias YAMLs (e.g., `reanimator.yml`) are removed, after which the step will be enforced.
Per-file YAML structure (excerpt):
```yaml
id: plus1-plus1-counters
display_name: +1/+1 Counters
curated_synergies: [Proliferate, Counters Matter]
enforced_synergies: []
inferred_synergies: []
synergies: [Proliferate, Counters Matter, Adapt, Evolve]
primary_color: Green
secondary_color: White
notes: ''
```
Add or edit a theme by creating/updating a file in `config/themes/catalog/` and running the build script:
```powershell
python code/scripts/build_theme_catalog.py
```
Metadata info (formerly provenance) is written to `theme_list.json` and consumed by tests and future UI diagnostics. During the deprecation period the builder accepts either key; prefer `metadata_info` in all new tooling / docs.
### Duplicate suppression & example_cards quality controls
Example card generation (and rebuild / padding via `synergy_promote_fill.py`) includes safeguards to reduce noisy staples and over-represented picks:
- `--common-card-threshold <fraction>`: Excludes candidate example cards that already appear in more than this fraction of themes (default 0.18 = 18%). This curbs ubiquitous staples (e.g., Sol Ring) from crowding lists.
- `--print-dup-metrics`: After a run (dry-run or apply) prints global frequency distribution so thresholds can be tuned.
Typical loop (dry run with metrics):
```powershell
python code/scripts/synergy_promote_fill.py --fill-example-cards --common-card-threshold 0.18 --print-dup-metrics
```
Adjustment guidance:
- If many mid-frequency utility cards still leak through: lower the threshold (e.g., 0.15).
- If high-value, legitimately thematic cards are being suppressed prematurely: raise slightly (e.g., 0.22) and re-run with metrics to confirm net effect.
These controls complement existing staple suppression heuristics (`staples_block` list) and color-fallback restraint flags.
## 📝 Theme Editorial Enrichment
Phase D introduces automated editorial helpers to surface representative cards and synergy-informed commander suggestions per theme while preserving manual curation:
### Key scripts
- `code/scripts/generate_theme_editorial_suggestions.py`: Scans card & commander CSVs to propose `example_cards`, `example_commanders`, and a derived `synergy_commanders` list.
- Synergy commanders: up to 3 / 2 / 1 picks from the theme's top three synergies (3/2/1 pattern), falling back to legendary card hits if a synergy lacks direct commander-tagged entries.
- Promotion: ensures a minimum (default 5) example commanders by promoting synergy picks with annotations.
- Annotation format: `Name - Synergy (Synergy Theme)` for both promoted examples and residual synergy-only entries.
- Duplicate filtering: commanders already present (base name) in `example_commanders` are omitted from `synergy_commanders` to avoid redundancy.
- Augmentation (`--augment-synergies`): heuristically pads sparse `synergies` arrays (e.g., injects `Counters Matter`, `Proliferate` for counter variants) before deriving synergy picks.
- `code/scripts/lint_theme_editorial.py`: Validates editorial consistency (annotation correctness, min counts, deduplication, size bounds) and emits warnings (non-fatal by default).
### Usage examples
```powershell
# Dry run (first 25 themes)
python code/scripts/generate_theme_editorial_suggestions.py
# Apply to all themes with augmentation and minimum example commanders of 5
python code/scripts/generate_theme_editorial_suggestions.py --apply --augment-synergies --min-examples 5
# Lint results
python code/scripts/lint_theme_editorial.py
```
### Non-determinism & source control guidance
The editorial output depends on current CSV card data, ranks, and tagging heuristics. Regeneration on a different machine or after data refresh may yield a different ordering or composition. For that reason:
- Commit the tooling and schema changes.
- Prefer selective or manually reviewed YAML edits (especially cornerstone themes) instead of bulk committing the entire regenerated catalog.
- Treat full-catalog regeneration as an operational task (e.g., during a release preparation) rather than every feature branch.
### Schema fields
Per-theme YAML now may include:
```yaml
example_cards: [ ... ] # Representative non-commander cards
example_commanders: [ ... ] # Curated + promoted synergy commanders (annotated)
synergy_commanders: [ ... ] # Remaining annotated synergy picks not already promoted
```
### Lint expectations (default)
- example_commanders: <=12, aim for ≥5 (warnings below minimum)
- example_cards: ≤20
- synergy_commanders: ≤6 (3/2/1 pattern) after filtering duplicates
- Annotations must reference a synergy present in `synergies`.
Cornerstone themes (Landfall, Reanimate, Superfriends, Tokens Matter, +1/+1 Counters) should always have curated coverage; automated promotion fills gaps but manual review is encouraged.
python code/headless_runner.py `
--commander "Krenko, Mob Boss" `
--primary-tag "Goblin Kindred" `
--creature-count 35 `
--land-count 33 `
--ramp-count 12
# Include/exclude specific cards (semicolon for comma-containing names)
python code/headless_runner.py `
--commander "Jace, Vryn's Prodigy" `
--include-cards "Sol Ring;Jace, the Mind Sculptor" `
--exclude-cards "Chaos Orb;Shahrazad" `
--enforcement-mode strict
```
**New CLI Features:**
- **Type-safe help text**: All arguments show expected types (PATH, NAME, INT, BOOL)
- **Ideal count arguments**: `--ramp-count`, `--land-count`, `--creature-count`, etc.
- **Theme tag names**: `--primary-tag "Theme Name"` instead of `--primary-choice 1`
- **Include/exclude CLI**: `--include-cards`, `--exclude-cards` with semicolon support
- **Console diagnostics**: Detailed summary output for validation and results
Headless submenu notes:
- If one JSON exists in `config/`, it auto-runs it
- If multiple exist, they’re listed as "Commander - Theme1, Theme2, Theme3"; `deck.json` shows as "Default"
- CSV/TXT are exported as usual; JSON run-config is exported only in interactive runs
### Run locally (no Docker)
```powershell
# Show resolved settings (no run)
python code/headless_runner.py --config config/deck.json --dry-run
# Run with a specific config file
python code/headless_runner.py --config config/deck.json
# Point to a folder; if exactly one config exists, it's auto-used
python code/headless_runner.py --config config
# Override via CLI
python code/headless_runner.py --commander "Pantlaza, Sun-Favored" --primary-choice 2 --secondary-choice 0 --add-lands true --fetch-count 3
```
### CLI flag reference
Each flag below mirrors an environment variable (env vars override JSON, CLI overrides both). Types match `--help` output.
#### Core selection & flow
| Flag | Type | Description | JSON key / Env var |
| --- | --- | --- | --- |
| `--config PATH` | string | Path to JSON config file or directory (auto-detects a single file) | `DECK_CONFIG` |
| `--commander NAME` | string | Commander search term | `commander` / `DECK_COMMANDER` |
| `--primary-choice INT` | int | Primary theme menu index (1-based) | `primary_choice` / `DECK_PRIMARY_CHOICE` |
| `--secondary-choice INT` | optional int | Secondary theme index | `secondary_choice` / `DECK_SECONDARY_CHOICE` |
| `--tertiary-choice INT` | optional int | Tertiary theme index | `tertiary_choice` / `DECK_TERTIARY_CHOICE` |
| `--primary-tag NAME` | string | Primary theme name (auto-maps to index) | `primary_tag` / `DECK_PRIMARY_TAG` |
| `--secondary-tag NAME` | string | Secondary theme name | `secondary_tag` / `DECK_SECONDARY_TAG` |
| `--tertiary-tag NAME` | string | Tertiary theme name | `tertiary_tag` / `DECK_TERTIARY_TAG` |
| `--bracket-level 1-5` | int | Power bracket selection | `bracket_level` / `DECK_BRACKET_LEVEL` |
| `--dry-run` | flag | Print the resolved configuration and exit | — |
#### Deck composition & counts
| Flag | Type | Description | JSON key / Env var |
| --- | --- | --- | --- |
| `--ramp-count INT` | int | Target ramp spells | `ideal_counts.ramp` |
| `--land-count INT` | int | Total land target | `ideal_counts.lands` |
| `--basic-land-count INT` | int | Minimum basic lands | `ideal_counts.basic_lands` |
| `--creature-count INT` | int | Creature count target | `ideal_counts.creatures` |
| `--removal-count INT` | int | Spot removal target | `ideal_counts.removal` |
| `--wipe-count INT` | int | Board wipe target | `ideal_counts.wipes` |
| `--card-advantage-count INT` | int | Card advantage target | `ideal_counts.card_advantage` |
| `--protection-count INT` | int | Protection spell target | `ideal_counts.protection` |
| `--fetch-count INT` | optional int | Requested fetch land count | `fetch_count` / `DECK_FETCH_COUNT` |
| `--dual-count INT` | optional int | Dual land request | `dual_count` / `DECK_DUAL_COUNT` |
| `--triple-count INT` | optional int | Three-color land request | `triple_count` / `DECK_TRIPLE_COUNT` |
| `--utility-count INT` | optional int | Utility land request | `utility_count` / `DECK_UTILITY_COUNT` |
#### Card type toggles
| Flag | Type | Description | JSON key / Env var |
| --- | --- | --- | --- |
| `--add-lands BOOL` | bool | Run the land builder sequence | `add_lands` / `DECK_ADD_LANDS` |
| `--add-creatures BOOL` | bool | Add creatures | `add_creatures` / `DECK_ADD_CREATURES` |
| `--add-non-creature-spells BOOL` | bool | Bulk add non-creature spells (if available) | `add_non_creature_spells` / `DECK_ADD_NON_CREATURE_SPELLS` |
| `--add-ramp BOOL` | bool | Add ramp spells | `add_ramp` / `DECK_ADD_RAMP` |
| `--add-removal BOOL` | bool | Add removal spells | `add_removal` / `DECK_ADD_REMOVAL` |
| `--add-wipes BOOL` | bool | Add board wipes | `add_wipes` / `DECK_ADD_WIPES` |
| `--add-card-advantage BOOL` | bool | Add card draw | `add_card_advantage` / `DECK_ADD_CARD_ADVANTAGE` |
| `--add-protection BOOL` | bool | Add the protection suite | `add_protection` / `DECK_ADD_PROTECTION` |
#### Include / exclude controls
| Flag | Type | Description | JSON key / Env var |
| --- | --- | --- | --- |
| `--include-cards LIST` | string | Force-include cards (comma or semicolon separated) | `include_cards` |
| `--exclude-cards LIST` | string | Exclude cards | `exclude_cards` |
| `--enforcement-mode warn|strict` | string | Handling when includes cannot be satisfied | `enforcement_mode` |
| `--allow-illegal BOOL` | bool | Permit non-legal cards in include/exclude lists | `allow_illegal` |
| `--fuzzy-matching BOOL` | bool | Enable fuzzy card name matching | `fuzzy_matching` |
#### Random mode (web parity)
| Flag | Type | Description | Env var / JSON key |
| --- | --- | --- | --- |
| `--random-mode` | flag | Force the headless random builder path | `HEADLESS_RANDOM_MODE` / `random_mode` |
| `--random-theme NAME` | string | Legacy single-theme alias (maps to primary) | `RANDOM_THEME` / `random.theme` |
| `--random-primary-theme NAME` | string | Primary theme slug | `RANDOM_PRIMARY_THEME` / `random.primary_theme` |
| `--random-secondary-theme NAME` | string | Secondary theme slug | `RANDOM_SECONDARY_THEME` / `random.secondary_theme` |
| `--random-tertiary-theme NAME` | string | Tertiary theme slug | `RANDOM_TERTIARY_THEME` / `random.tertiary_theme` |
| `--random-auto-fill BOOL` | bool | Auto-fill missing theme slots | `RANDOM_AUTO_FILL` / `random.auto_fill` |
| `--random-auto-fill-secondary BOOL` | bool | Override secondary auto-fill | `RANDOM_AUTO_FILL_SECONDARY` / `random.auto_fill_secondary` |
| `--random-auto-fill-tertiary BOOL` | bool | Override tertiary auto-fill | `RANDOM_AUTO_FILL_TERTIARY` / `random.auto_fill_tertiary` |
| `--random-strict-theme-match BOOL` | bool | Require exact theme matches when selecting commanders | `RANDOM_STRICT_THEME_MATCH` / `random.strict_theme_match` |
| `--random-attempts INT` | int | Retry attempts before giving up | `RANDOM_MAX_ATTEMPTS` / `random.attempts` |
| `--random-timeout-ms INT` | int | Timeout per attempt (milliseconds) | `RANDOM_TIMEOUT_MS` / `random.timeout_ms` |
| `--random-seed VALUE` | int or string | Deterministic seed for reproducible runs | `RANDOM_SEED` / `random.seed` |
| `--random-constraints JSON_OR_PATH` | string | Inline JSON or path to constraint file | `RANDOM_CONSTRAINTS` / `RANDOM_CONSTRAINTS_PATH` / `random.constraints` |
| `--random-output-json PATH` | string | Write random build payload to file or directory | `RANDOM_OUTPUT_JSON` / `random.output_json` |
Booleans accept: 1/0, true/false, yes/no, on/off.
### Random mode parity (web → headless)
The headless runner now shares the web UI's random builder pipeline. Set `--random-mode` (or `HEADLESS_RANDOM_MODE=1`) to route through the Surprise/Reroll flow with multi-theme support, auto-fill assistance, constraints, and deterministic seeds. Combine flags as needed:
```powershell
python code/headless_runner.py `
--random-mode `
--random-primary-theme Artifacts `
--random-secondary-theme Tokens `
--random-auto-fill true `
--random-output-json deck_files/random/latest.json
```
The build prints a full summary (commander, seed, theme stack, fallback reasons, attempts vs timeout, auto-fill status) and writes the optional JSON payload when `--random-output-json`/`RANDOM_OUTPUT_JSON` is provided. Use `--random-constraints` (or `RANDOM_CONSTRAINTS[_PATH]`) to supply pool limits, and `--random-seed` for reproducible reruns. Pair any random flag with `--dry-run` to inspect the resolved configuration without building.
### JSON export in headless
- By default, headless runs do not export a JSON run-config to avoid duplicates.
- Opt-in with:
```powershell
$env:HEADLESS_EXPORT_JSON = "1"
python code/headless_runner.py --config config/deck.json
```
- Tip: when opting in, prefer using `--config` instead of a `DECK_CONFIG` file path to avoid creating both a stem-based JSON and a second explicit-path JSON.
Example JSON (`config/deck.json`):
```json
{
"commander": "Pantlaza",
"bracket_level": 3,
"primary_choice": 2,
"secondary_choice": 2,
"tertiary_choice": 2,
"tag_mode": "OR", // OR or AND; Web UI default is OR
"add_lands": true,
"fetch_count": 3,
"ideal_counts": { "ramp": 10, "lands": 36, "basic_lands": 16, "creatures": 28, "removal": 8, "wipes": 3, "card_advantage": 8, "protection": 3 }
}
```
Notes: headless honors `ideal_counts` (leave prompts blank to accept). Only `fetch_count` is tracked/exported for lands.
#### JSON fields for Combos (Web Configs)
When running from a JSON config in the Web UI, the following fields are supported and exported from interactive runs:
```json
{
"prefer_combos": true,
"combo_target_count": 3,
"combo_balance": "mix"
}
```
Auto-Complete Combos runs before theme fill/monolithic spells when `prefer_combos` is true.
## 🔒 Bracket policy & enforcement
- Policy source: `config/brackets.yml` defines per-bracket limits for categories like `game_changers`, `extra_turns`, `mass_land_denial`, `tutors_nonland`, and `two_card_combos`.
- Authoritative lists: JSON under `config/card_lists/` provides names for enforcement and reporting (`game_changers.json`, `extra_turns.json`, `mass_land_denial.json`, `tutors_nonland.json`). A `list_version` may be included for badges.
- Global safety prune: when a category has a limit of 0, the builder removes matching cards from the card pool up-front so they cannot be selected (Game Changers by name; others by tags when present). This runs in both Web and headless builds.
- Preemptive filters: spells and creatures phases also apply bracket-aware pre-filters.
- Inline enforcement (Web): if violations still occur, Step 5 shows them before the summary. You must replace or remove flagged cards before you can Continue or Rerun. Alternatives are role-consistent, exclude the replaced/in-deck/locked/commander cards, and bias toward owned when enabled.
- Game Changer fallback: if `config/card_lists/game_changers.json` is empty, enforcement and reporting fall back to the in-code `builder_constants.GAME_CHANGERS` list so low-bracket decks still exclude those cards.
- Auto-enforce (optional): set `WEB_AUTO_ENFORCE=1` to automatically apply the enforcement plan after build and re-export CSV/TXT when violations are detected.
Status levels
- PASS: within limits; panel remains collapsed by default.
- WARN: advisory thresholds met (from `<category>_warn` keys in `config/brackets.yml` or conservative defaults for low brackets on tutors/extra turns). Panel opens automatically and shows WARN tiles with a subtle amber border and badge; no gating or automatic enforcement.
- FAIL: over hard limits or disallowed categories for the selected bracket. Panel opens automatically, tiles show with red accents, and Continue/Rerun are disabled until resolved; enforcement actions are available.
Compliance report
- Every run writes `[stem]_compliance.json` next to your deck exports, including per-category counts/limits, status (PASS/FAIL), detected cheap/early two-card combos, and list versions.
- Headless JSON builds and Web builds use the same engine; summary and exports are consistent.
## 🧩 Theme combine mode (AND/OR)
- OR (default): Recommend cards that match any selected themes, prioritizing overlap.
- AND: Prioritize multi-theme intersections. For creatures, an AND pre-pass first picks "all selected themes" creatures up to a cap, then fills by weighted overlap.
UI tips:
- Step 2 includes AND/OR radios with a tooltip explaining trade-offs.
- In staged build view, "Creatures: All-Theme" shows which selected themes each card hits.
## 🕹️ Usage (interactive)
1) Start the app (Docker or from source)
2) Pick Build a New Deck
3) Search/confirm commander
4) Pick primary/secondary/tertiary themes (or stop at primary); choose AND/OR combine mode
5) Choose power bracket and review ideal counts
6) Deck builds; CSV/TXT export to `deck_files/`
## ⚙️ Environment variables (common)
- DECK_MODE=headless
- DECK_CONFIG=/app/config/deck.json
- DECK_COMMANDER, DECK_PRIMARY_CHOICE, DECK_SECONDARY_CHOICE, DECK_TERTIARY_CHOICE
- DECK_ADD_LANDS, DECK_FETCH_COUNT
- DECK_ADD_CREATURES, DECK_ADD_NON_CREATURE_SPELLS, DECK_ADD_RAMP, DECK_ADD_REMOVAL, DECK_ADD_WIPES, DECK_ADD_CARD_ADVANTAGE, DECK_ADD_PROTECTION
- DECK_BRACKET_LEVEL
- **ALLOW_MUST_HAVES=true** (enables include/exclude card lists feature with enhanced UI validation)
- Random Modes (alpha): core toggles `RANDOM_MODES=1`, `RANDOM_UI=1`, `RANDOM_MAX_ATTEMPTS=5`, `RANDOM_TIMEOUT_MS=5000` (see detailed list below for theme/auto-fill/seed overrides)
- `RANDOM_BUILD_SUPPRESS_INITIAL_EXPORT=0` — (optional) disable the default suppression of builder auto-export (will produce legacy duplicate artifacts; for debugging only)
- Testing/data overrides:
- `CSV_FILES_DIR=csv_files/testdata` — point the app/tests at a frozen dataset snapshot for determinism
Random Modes (alpha flags):
- `RANDOM_MODES=1` — enable random build endpoints and backend features
- `RANDOM_UI=1` — show Surprise/Reroll/Share controls in UI
- `RANDOM_MAX_ATTEMPTS=5` — cap retry attempts for random generation
- `RANDOM_TIMEOUT_MS=5000` — per-build timeout in milliseconds
- `RANDOM_THEME=`*slug* — legacy single-theme alias (maps to primary when others unset)
- `RANDOM_PRIMARY_THEME=`*slug* — primary theme override
- `RANDOM_SECONDARY_THEME=`*slug* — secondary theme override
- `RANDOM_TERTIARY_THEME=`*slug* — tertiary theme override
- `RANDOM_AUTO_FILL=0|1` — auto-fill missing secondary/tertiary slots when enabled
- `RANDOM_AUTO_FILL_SECONDARY=0|1` / `RANDOM_AUTO_FILL_TERTIARY=0|1` — explicit per-slot overrides
- `RANDOM_STRICT_THEME_MATCH=0|1` — require exact theme matches when rolling commanders
- `RANDOM_CONSTRAINTS=`*inline JSON* and/or `RANDOM_CONSTRAINTS_PATH=/app/config/random_constraints.json`
- `RANDOM_SEED=`*value* — deterministic run seed (int or string)
- `RANDOM_OUTPUT_JSON=/app/deck_files/random_build.json` — write random build payload to file/dir
- `HEADLESS_RANDOM_MODE=1` — force the headless runner to take the random pipeline
- `RANDOM_BUILD_SUPPRESS_INITIAL_EXPORT=0` — (optional) disable the default suppression of builder auto-export (useful for debugging legacy duplicate exports)
Testing/determinism helpers:
- CSV_FILES_DIR=csv_files/testdata — override CSV base dir for tests or pinned snapshots
Optional name-based tag overrides (mapped to indices for the chosen commander):
- DECK_PRIMARY_TAG, DECK_SECONDARY_TAG, DECK_TERTIARY_TAG
Combine mode in headless:
- JSON: set `"tag_mode": "AND" | "OR"`
- Env var: `DECK_TAG_MODE=AND|OR` (if configured in your environment)
Web UI performance tuning:
- WEB_TAG_PARALLEL=1|0
- WEB_TAG_WORKERS=<N>
- WEB_VIRTUALIZE=1 (enable list virtualization)
- SHOW_DIAGNOSTICS=1 (optional: diagnostics tools and virtualization overlay toggle using `v`)
- WEB_AUTO_ENFORCE=1 (optional; after building, auto-apply enforcement and re-export when the compliance report fails)
- SPLASH_ADAPTIVE=1 (optional: enable adaptive off-color penalty by commander color count)
- SPLASH_ADAPTIVE_SCALE="1:1.0,2:1.0,3:1.0,4:0.6,5:0.35" (optional: tuning for adaptive scaling)
Paths and data overrides:
- `DECK_EXPORTS=/app/deck_files` — where finished deck exports are read by the Web UI
- `OWNED_CARDS_DIR=/app/owned_cards` — preferred directory for owned inventory uploads
- `CARD_LIBRARY_DIR=/app/owned_cards` — back‑compat alias for OWNED_CARDS_DIR
- `DECK_CONFIG=/app/config/deck.json` — default headless config path
- `CSV_FILES_DIR=/app/csv_files` — override CSV base dir (tests, snapshots, or alternate datasets)
- `CARD_INDEX_EXTRA_CSV=path/to/synthetic_cards.csv` — inject an extra CSV into the card index for experiments/tests
Theme catalog scan & prewarm:
- `THEME_CATALOG_YAML_SCAN_INTERVAL_SEC=2.0` — poll for YAML changes (dev); omit in production
- `WEB_THEME_FILTER_PREWARM=1` — prewarm common filters for faster first renders
Theme preview cache & Redis (optional):
- `THEME_PREVIEW_CACHE_MAX=400` — max previews cached in memory
- `WEB_THEME_PREVIEW_LOG=0` — 1=verbose cache logs
- `THEME_PREVIEW_ADAPTIVE=0` — 1=adaptive cache policy
- `THEME_PREVIEW_EVICT_COST_THRESHOLDS=5,15,40` — cost tiers for eviction policy
- `THEME_PREVIEW_BG_REFRESH=0` — 1=background refresh worker; `THEME_PREVIEW_BG_REFRESH_INTERVAL=120` seconds
- TTL tuning: `THEME_PREVIEW_TTL_BASE=300`, `THEME_PREVIEW_TTL_MIN=60`, `THEME_PREVIEW_TTL_MAX=900`, `THEME_PREVIEW_TTL_BANDS=0.2,0.5,0.8`, `THEME_PREVIEW_TTL_STEPS=2,4,2,3,1`
- Redis: `THEME_PREVIEW_REDIS_URL=redis://localhost:6379/0`, `THEME_PREVIEW_REDIS_DISABLE=0` (1=disable redis even if URL set)
Rarity weighting and diversity (advanced):
- `RARITY_W_MYTHIC=1.2`, `RARITY_W_RARE=0.9`, `RARITY_W_UNCOMMON=0.65`, `RARITY_W_COMMON=0.4` — diminish or boost rarity selection
- `RARITY_DIVERSITY_TARGETS="mythic:0-1,rare:0-2,uncommon:0-4,common:0-6"` — soft targets per rarity band
- `RARITY_DIVERSITY_OVER_PENALTY=-0.5` — penalty when exceeding targets
Misc utility land tuning (Step 7):
- MISC_LAND_DEBUG=1 Write debug CSVs for misc land step (candidates/post-filter). Otherwise suppressed unless SHOW_DIAGNOSTICS=1.
- MISC_LAND_EDHREC_KEEP_PERCENT_MIN=0.75 Lower bound of dynamic EDHREC keep range (0–1). When MIN & MAX are both set, a random % in [MIN,MAX] is rolled per build.
- MISC_LAND_EDHREC_KEEP_PERCENT_MAX=1.0 Upper bound of dynamic EDHREC keep range (0–1).
- MISC_LAND_EDHREC_KEEP_PERCENT=0.80 Legacy fixed keep % used only if MIN/MAX not both present.
- MISC_LAND_THEME_MATCH_BASE=1.4 Base multiplier when at least one selected theme tag matches a candidate land.
- MISC_LAND_THEME_MATCH_PER_EXTRA=0.15 Incremental multiplier per additional matching theme tag beyond the first.
- MISC_LAND_THEME_MATCH_CAP=2.0 Cap for total theme multiplier after stacking base + extras.
Notes:
- Fetch lands are fully excluded from the misc step (handled earlier).
- Mono-color decks auto-filter broad rainbow/any-color lands except an explicit always-keep list.
- Land Alternatives endpoint: for land seeds, returns land-only suggestions; 12 random picks each request from a randomly sized window within the top 60–100 ranked candidates (per-card, uncached) for variety.
## 📁 Folders
- `deck_files/` — CSV/TXT exports
- `csv_files/` — card data
- `logs/` — logs
- `config/` — JSON configs (optional)
- `owned_cards/` — your owned cards lists (`.txt`/`.csv`); used for owned-only builds and Owned flagging
## 🧰 Troubleshooting
- Use `docker compose run --rm` (not `up`) for interactive sessions
- Ensure volumes are mounted so files persist (`deck_files`, `logs`, `csv_files`)
- For headless with config, mount `./config:/app/config` and set `DECK_CONFIG`
- Card data refresh: if `csv_files/cards.csv` is missing or older than 7 days, the app refreshes data and re-tags automatically. A `.tagging_complete.json` file in `csv_files/` indicates tagging completion.
- Web health: the header dot polls `/healthz` — green is OK, red is degraded. If red, check logs.
- Error details: HTMX errors show a toast with a “Copy details” button including X-Request-ID; include that when filing issues.
- Logs page: set `SHOW_LOGS=1` to enable `/logs` and `/status/logs?tail=200` (read‑only) for quick diagnostics.
- Diagnostics page: set `SHOW_DIAGNOSTICS=1` to enable the nav link and `/diagnostics` test tools.
Data integrity notes:
- Banned cards: per-color/guild CSVs now consistently respect the Commander banned list using exact, case‑insensitive name matching across `name` and `faceName`.
## 🧪 Development and tests
Set up a virtual environment, install dependencies, and run the test suite.
```powershell
# From repo root (PowerShell)
python -m venv .venv
& ".venv/Scripts/Activate.ps1"
pip install -r requirements.txt
pip install -r requirements-dev.txt
# Run tests (pytest is configured to look under code/tests)
python -m pytest -q
```
Notes:
- Test discovery is set to `code/tests` in `pytest.ini`.
- `pytest.ini` disables the built-in debugging plugin to avoid a stdlib module name clash with the project folder `code/`.
- Feature flags for local diagnostics: set `SHOW_DIAGNOSTICS=1` and/or `SHOW_LOGS=1` to expose `/diagnostics` and `/logs`.
- Index testing helper: set `CARD_INDEX_EXTRA_CSV=path\to\synthetic_cards.csv` to inject a temporary CSV during tests (e.g., color identity edge cases) without altering production shards.
### What’s new (quick summary)
- Faster browsing with optional virtualized grids/lists in Step 5 and Owned (`WEB_VIRTUALIZE=1`).
- Image polish: lazy-loading, responsive `srcset/sizes`, and LQIP blur/fade for Step 5, Owned, and commander preview.
- Diagnostics overlay (opt-in with `SHOW_DIAGNOSTICS=1`): press `v` to see visible range, totals, render time, and counters.
- Accessibility: respects reduced-motion (disables blur/fade and smooth scrolling).
- Small caching wins: short‑TTL fragment caching for summary partials and suggestions.
## 📦 Releases
- Release notes are maintained in `RELEASE_NOTES_TEMPLATE.md`. Automated workflows read from this file to populate Docker Hub and GitHub releases.
### Collecting diagnostics for an issue
- Note the Request-ID from the toast or error page.
- Copy logs from `/logs` (enable with `SHOW_LOGS=1`) or `/status/logs?tail=200`.
- Include your `/healthz` JSON and environment flags (SHOW_LOGS/SHOW_DIAGNOSTICS/SHOW_SETUP) when reporting.
## Performance Baselines
A warm preview performance baseline is committed at `logs/perf/theme_preview_warm_baseline.json`.
Tooling:
- Benchmark: `python -m code.scripts.preview_perf_benchmark --all --passes 2 --extract-warm-baseline logs/perf/theme_preview_warm_baseline.json`
- Compare (manual): `python -m code.scripts.preview_perf_compare --baseline logs/perf/theme_preview_warm_baseline.json --candidate logs/perf/new_run.json --warm-only --p95-threshold 5`
- Regression helper: `python -m code.scripts.preview_perf_ci_check --baseline logs/perf/theme_preview_warm_baseline.json --p95-threshold 5` (run locally or in ad-hoc CI when desired)
Policy:
- Treat warm-only p95 regressions beyond 5% as blockers even though enforcement is now manual.
- Update the baseline only after intentional performance improvements or systemic environment shifts.
Workflow to refresh baseline:
1. Ensure no active regressions (run wrapper locally, expect pass).
2. Run a multi-pass benchmark with at least two passes over all themes.
3. Replace `theme_preview_warm_baseline.json` with new warm pass output; commit in same PR with rationale in CHANGELOG.
Optional future enhancements:
- Reintroduce automated gating once preview perf stabilizes on shared runners.
- Separate p50 threshold (e.g., >7% regression) for early warning without failing build.

91
docs/headless_cli_guide.md Normal file
View file

@ -0,0 +1,91 @@
# Headless & CLI Guide
Leverage the shared deckbuilding engine from the command line, in headless mode, or within containers.
## Table of contents
- [Entry points](#entry-points)
- [Switching modes in Docker](#switching-modes-in-docker)
- [Headless JSON configs](#headless-json-configs)
- [Environment overrides](#environment-overrides)
- [CLI argument reference](#cli-argument-reference)
- [Include/exclude lists from the CLI](#includeexclude-lists-from-the-cli)
- [Practical examples](#practical-examples)
---
## Entry points
- Interactive menu: `python code/main.py`
- Headless runner: `python code/headless_runner.py --config config/deck.json`
- Both executables share the same builder core used by the Web UI.
## Switching modes in Docker
Override the container entrypoint to run the CLI or headless flows inside Docker Compose or plain `docker run`.
```powershell
# Compose example
docker compose run --rm -e APP_MODE=cli web
# Compose with headless automation
docker compose run --rm `
-e APP_MODE=cli `
-e DECK_MODE=headless `
-e DECK_CONFIG=/app/config/deck.json `
web
```
Set `APP_MODE=cli` to switch from the Web UI to the textual interface. Add `DECK_MODE=headless` to skip prompts and immediately run the configured deck.
## Headless JSON configs
- Drop JSON files into `config/` (e.g., `config/deck.json`).
- Headless mode auto-runs the lone JSON file; if multiple exist, the CLI lists them with summaries (commander + themes).
- Config fields cover commander, bracket, include/exclude lists, theme preferences, owned-mode toggles, and output naming.
## Environment overrides
When running in containers or automation, environment variables can override JSON settings. Typical variables include:
- `DECK_COMMANDER`
- `DECK_PRIMARY_CHOICE`, `DECK_SECONDARY_CHOICE`, `DECK_TERTIARY_CHOICE`
- `DECK_BRACKET_LEVEL`
- `DECK_ADD_LANDS`, `DECK_LAND_COUNT`, `DECK_CREATURE_COUNT`, `DECK_RAMP_COUNT`
Precedence order: **CLI flags > environment variables > JSON config > defaults**.
## CLI argument reference
Run `python code/headless_runner.py --help` to see the current argument surface. Highlights:
- Type indicators make expectations explicit (e.g., `PATH`, `NAME`, `INT`).
- Theme selection accepts human-readable names: `--primary-tag "Airbending"` instead of numeric indexes.
- Bracket selection via `--bracket-level`.
- Ideal counts such as `--land-count`, `--ramp-count`, `--creature-count`, and more.
## Include/exclude lists from the CLI
You can specify comma- or semicolon-separated lists directly through the CLI:
```powershell
python code/headless_runner.py `
--commander "Jace, Vryn's Prodigy" `
--include-cards "Sol Ring;Jace, the Mind Sculptor" `
--exclude-cards "Chaos Orb;Shahrazad" `
--enforcement-mode strict
```
Semicolons allow card names containing commas. Enforcement modes mirror the Web UI (`off`, `warn`, `strict`).
## Practical examples
```powershell
# Build a Goblins list with tuned counts
python code/headless_runner.py `
--commander "Krenko, Mob Boss" `
--primary-tag "Goblin Kindred" `
--creature-count 35 `
--land-count 33 `
--ramp-count 12
# Fire a headless run via Docker using an alternate config folder
docker compose run --rm `
-e APP_MODE=cli `
-e DECK_MODE=headless `
-e DECK_CONFIG=/app/config/custom_decks `
web
```
The CLI prints a detailed summary at the end of each run, including enforcement results, resolved themes, and export paths. All artifacts land in the same `deck_files/` folder used by the Web UI.

148
docs/theme_catalog_advanced.md Normal file
View file

@ -0,0 +1,148 @@
# Theme Catalog Advanced Guide
Additional details for developers and power users working with the theme catalog, editorial tooling, and diagnostics.
## Table of contents
- [HTMX API endpoints](#htmx-api-endpoints)
- [Caching, diagnostics, and metrics](#caching-diagnostics-and-metrics)
- [Governance principles](#governance-principles)
- [Operational tooling](#operational-tooling)
- [Refreshing catalogs](#refreshing-catalogs)
- [Snapshotting taxonomy](#snapshotting-taxonomy)
- [Adaptive splash penalty experiments](#adaptive-splash-penalty-experiments)
- [Editorial pipeline](#editorial-pipeline)
- [Script summary](#script-summary)
- [Example configuration](#example-configuration)
- [Duplicate suppression controls](#duplicate-suppression-controls)
- [Coverage metrics and KPIs](#coverage-metrics-and-kpis)
- [Description mapping overrides](#description-mapping-overrides)
- [Validation and schema tooling](#validation-and-schema-tooling)
---
## HTMX API endpoints
The upcoming theme picker UI is powered by two FastAPI endpoints.
### `GET /themes/api/themes`
Parameters:
- `q`: substring search across theme names and synergies.
- `archetype`: filter by `deck_archetype`.
- `bucket`: popularity bucket (Very Common, Common, Uncommon, Niche, Rare).
- `colors`: comma-separated color initials (e.g. `G,W`).
- `limit` / `offset`: pagination (limit defaults to 50, max 200).
- `diagnostics=1`: surfaces `has_fallback_description` and `editorial_quality` (requires `WEB_THEME_PICKER_DIAGNOSTICS=1`).
The response includes `count`, the filtered `items`, and `next_offset` for subsequent requests. Diagnostic mode adds extra telemetry fields.
### `GET /themes/api/theme/{id}`
Parameters:
- `uncapped=1`: (diagnostics) returns `uncapped_synergies`, combining curated, enforced, and inferred sets.
- `diagnostics=1`: exposes editorial metadata such as `editorial_quality` and `has_fallback_description`.
The payload merges curated data with editorial artifacts (`example_cards`, `example_commanders`, etc.) and respects the same diagnostic feature flag.
## Caching, diagnostics, and metrics
- Responses include an `ETag` header derived from catalog metadata so consumers can perform conditional GETs.
- `/themes/status` reports freshness and stale indicators; `/themes/refresh` (POST) triggers a background rebuild.
- When `WEB_THEME_PICKER_DIAGNOSTICS=1` is set, the app records:
- Filter cache hits/misses and duration (`X-ThemeCatalog-Filter-Duration-ms`).
- Preview cache metrics (`/themes/metrics` exposes counts, hit rates, TTL, and average build time).
- Skeleton loaders ship with the HTMX fragments to keep perceived latency low.
## Governance principles
To keep the catalog healthy, the project follows a lightweight governance checklist:
1. **Minimum examples** – target at least two example cards and one commander per established theme.
2. **Deterministic preview assembly** – curated examples first, then role-based samples (payoff/enabler/support/wildcard), then placeholders if needed.
3. **Splash relax policy** – four- and five-color commanders may include a single off-color enabler with a small penalty, preventing over-pruning.
4. **Popularity buckets are advisory** – they guide filters and UI hints but never directly influence scoring.
5. **Taxonomy expansion bar** – new high-level archetypes require a distinct pattern, at least eight representative cards, and no overlap with existing themes.
6. **Editorial quality tiers** – optional `editorial_quality: draft|reviewed|final` helps prioritize review passes.
7. **Deterministic sampling** – seeds derive from `theme|commander` hashes; scoring code should emit `reasons[]` to explain decisions and remain regression-test friendly.
See `docs/theme_taxonomy_rationale.md` for the underlying rationale and roadmap.
## Operational tooling
### Refreshing catalogs
- Primary builder: `python code/scripts/build_theme_catalog.py`
- Options:
- `--limit N`: preview a subset without overwriting canonical outputs (unless `--allow-limit-write`).
- `--output path`: write to an alternate path; suppresses YAML backfill to avoid mutating tracked files.
- `--backfill-yaml` or `EDITORIAL_BACKFILL_YAML=1`: fill missing descriptions and popularity buckets in YAML files.
- `--force-backfill-yaml`: overwrite existing description/popularity fields.
- `EDITORIAL_SEED=<int>`: force a deterministic ordering when heuristics use randomness.
- `EDITORIAL_AGGRESSIVE_FILL=1`: pad sparse themes with inferred synergies.
- `EDITORIAL_POP_BOUNDARIES="a,b,c,d"`: tune popularity thresholds.
- `EDITORIAL_POP_EXPORT=1`: emit `theme_popularity_metrics.json` summaries.
### Snapshotting taxonomy
`python -m code.scripts.snapshot_taxonomy` writes `logs/taxonomy_snapshots/taxonomy_<timestamp>.json` with a SHA-256 hash. Identical content is skipped unless you supply `--force`. Use snapshots before experimenting with taxonomy-aware sampling.
### Adaptive splash penalty experiments
Set `SPLASH_ADAPTIVE=1` to scale off-color enabler penalties based on commander color count. Tune with `SPLASH_ADAPTIVE_SCALE` (e.g. `1:1.0,2:1.0,3:1.0,4:0.6,5:0.35`). Analytics aggregate both static and adaptive reasons for comparison.
## Editorial pipeline
### Script summary
- `code/scripts/generate_theme_editorial_suggestions.py`
- Proposes `example_cards`, `example_commanders`, and `synergy_commanders` using card CSVs and tagging heuristics.
- `--augment-synergies` can pad sparse `synergies` arrays prior to suggestion.
- `--apply` writes results; dry runs print suggestions for review.
- `code/scripts/lint_theme_editorial.py`
- Validates annotation formats, min/max counts, and deduplication. Combine with environment toggles (`EDITORIAL_REQUIRE_DESCRIPTION`, `EDITORIAL_REQUIRE_POPULARITY`) for stricter gating.
### Example configuration
```powershell
# Dry run on the first 25 themes
python code/scripts/generate_theme_editorial_suggestions.py
# Apply across the catalog with augmentation and min example commanders set to 5
python code/scripts/generate_theme_editorial_suggestions.py --apply --augment-synergies --min-examples 5
# Lint results
python code/scripts/lint_theme_editorial.py
```
Editorial output depends on current CSV data. Expect ordering or composition changes after upstream dataset refreshes—treat full-catalog regeneration as an operational task and review diffs carefully.
### Duplicate suppression controls
`code/scripts/synergy_promote_fill.py` can rebalance example cards:
```powershell
python code/scripts/synergy_promote_fill.py --fill-example-cards --common-card-threshold 0.18 --print-dup-metrics
```
- `--common-card-threshold`: filters cards appearing in more than the specified fraction of themes (default `0.18`).
- Use metrics output to tune thresholds so staple utility cards stay in check without removing legitimate thematic cards.
### Coverage metrics and KPIs
- `EDITORIAL_INCLUDE_FALLBACK_SUMMARY=1` embeds a `description_fallback_summary` block in the generated catalog (`generic_total`, `generic_plain`, `generic_pct`, etc.).
- Regression tests use these metrics to ratchet down generic descriptions over time.
- Historical trends are appended to `config/themes/description_fallback_history.jsonl` for analysis.
### Description mapping overrides
Customize automatic descriptions without editing code:
- Add `config/themes/description_mapping.yml` with entries:
```yaml
- triggers: ["sacrifice", "aristocrat"]
description: "Leans on sacrifice loops and {SYNERGIES}."
```
- The first matching trigger wins (case-insensitive substring search).
- `{SYNERGIES}` expands to a short clause listing the top synergies when available, and disappears gracefully if not.
- Internal defaults remain as fallbacks when the mapping file is absent.
## Validation and schema tooling
Run validators to maintain catalog quality:
```powershell
python code/scripts/validate_theme_catalog.py
python code/scripts/validate_theme_catalog.py --rebuild-pass
python code/scripts/validate_theme_catalog.py --schema
python code/scripts/validate_theme_catalog.py --yaml-schema
python code/scripts/validate_theme_catalog.py --strict-alias
```
Per-theme YAML files (under `config/themes/catalog/`) are tracked in source control. Keys such as `metadata_info` replace the legacy `provenance`; the validator treats missing migrations as warnings until the deprecation completes.

103
docs/web_ui_deep_dive.md Normal file
View file

@ -0,0 +1,103 @@
# Web UI Deep Dive
A closer look at the rich interactions available in the MTG Python Deckbuilder Web UI. Use this guide after you are comfortable with the basic homepage flows described in the README.
## Table of contents
- [Unified New Deck modal](#unified-new-deck-modal)
- [Stage 5 tools: lock, replace, compare, permalinks](#stage-5-tools-lock-replace-compare-permalinks)
- [Multi-copy archetype packages](#multi-copy-archetype-packages)
- [Bracket compliance and skipped stages](#bracket-compliance-and-skipped-stages)
- [Build options: owned-only and prefer-owned](#build-options-owned-only-and-prefer-owned)
- [Visual summaries](#visual-summaries)
- [Combos & synergies](#combos--synergies)
- [Owned library page](#owned-library-page)
- [Finished decks workspace](#finished-decks-workspace)
- [Keyboard shortcuts](#keyboard-shortcuts)
- [Virtualization, tagging, and performance](#virtualization-tagging-and-performance)
- [Diagnostics and logs](#diagnostics-and-logs)
---
## Unified New Deck modal
The first three steps of deckbuilding live inside a single modal:
1. **Search for a commander** – autocomplete prioritizes color identity matches; press Enter to grab the top result.
2. **Pick primary/secondary/tertiary themes** – the modal displays your selections in order so you can revisit them quickly.
3. **Choose a bracket** – labels such as “Bracket 3: Upgraded” clarify power bands. Bracket 3 is the default tier for new builds.
Optional inputs:
- **Deck name** becomes the export filename stem and is reused in Finished Decks banners.
- **Combo auto-complete** and other preferences persist between runs.
Once you submit, the modal closes and the build starts immediately—no extra confirmation screen.
## Stage 5 tools: lock, replace, compare, permalinks
Stage 5 is the iterative workspace for tuning the deck:
- **Lock** a card by clicking the padlock or the card artwork. Locked cards persist across rerolls and show a “Last action” chip for quick confirmation.
- **Replace** opens the Alternatives drawer. Filters include Owned-only, role alignment, and bracket compliance. The system skips commanders, locked cards, just-added cards, and anything already in the list.
- **Permalink** buttons appear in Stage 5 and Finished Decks. Share a build (commander, themes, bracket, ideals, flags) or restore one by pasting a permalink back into the app.
- **Compare** mode lives in Finished Decks. Pick two builds (quick actions select the latest pair) and triage changes via Changed-only, Copy summary, or download the diff as TXT.
## Multi-copy archetype packages
When a commander + theme combination suggests a multi-copy strategy (e.g., Persistent Petitioners, Shadowborn Apostles), the UI offers an optional package:
- Choose the desired quantity (bounded by printed limits) and optionally add **Thrumming Stone** when it synergizes.
- Packages are inserted before other stages so target counts adjust appropriately.
- A safety clamp trims overflow to keep the deck at 100 cards; the stage displays a “Clamped N” indicator if it triggers.
- You can dismiss the modal, and we won’t re-prompt unless your selections change.
## Bracket compliance and skipped stages
- Bracket policy enforcement prunes disallowed categories before stage execution. Violations block reruns until you resolve them.
- Enforcement options: keep the panel collapsed when compliant, auto-open with a colored status chip (green/amber/red) when action is needed.
- Enable auto-enforcement by setting `WEB_AUTO_ENFORCE=1`.
- Toggle **Show skipped stages** to surface steps that added zero cards, making it easier to review the full pipeline.
## Build options: owned-only and prefer-owned
The modal includes toggles for **Use only owned cards** and **Prefer owned cards**:
- Owned-only builds pull strictly from the inventory in `owned_cards/` (commander exempt).
- Prefer-owned bumps owned cards slightly in the scoring pipeline but still allows unowned all-stars when necessary.
- Both modes respect the Owned Library filters and show Owned badges in the exported CSV (including the `Owned` column when you disable the mode).
## Visual summaries
Stage 5 displays multiple data visualizations that cross-link to the card list:
- **Mana curve** – hover a bar to highlight matching cards in list and thumbnail views.
- **Color requirements vs. sources** – pips show requirements; sources include non-land producers and an optional `C` (colorless) toggle.
- **Tooltips** – each tooltip lists contributing cards and offers a copy-to-clipboard action.
- Visual polish includes lazy-loaded thumbnails, blur-up transitions, and accessibility tweaks that respect `prefers-reduced-motion`.
## Combos & synergies
The builder detects curated two-card combos and synergy pairs in the final deck:
- Chips display badges such as “cheap” or “setup” with hover previews for each card and a split preview when hovering the entire row.
- Enable **Auto-complete combos** to add missing partners before theme filling. Configure target count, balance (early/late/mix), and preference weighting.
- Color identity restrictions keep the algorithm from suggesting off-color partners.
## Owned library page
Open the Owned tile to manage uploaded inventories:
- Upload `.txt` or `.csv` files with one card per line. The app enriches and deduplicates entries on ingestion.
- The page includes sortable columns, exact color-identity filters (including four-color combos), and an export button.
- Large collections benefit from virtualization when `WEB_VIRTUALIZE=1`.
## Finished decks workspace
- Browse historical builds with filterable theme chips.
- Each deck offers Download TXT, Copy summary, Open permalink, and Compare actions.
- Locks, replace history, and compliance metadata are stored per deck and surface alongside the exports.
## Keyboard shortcuts
- **Enter** selects the first commander suggestion while searching.
- Inside Stage 5 lists: **L** locks/unlocks the focused card, **R** opens the Replace drawer, and **C** copies the permalink.
- Browser autofill is disabled in the modal to keep searches clean.
## Virtualization, tagging, and performance
- `WEB_TAG_PARALLEL=1` with `WEB_TAG_WORKERS=4` (compose default) speeds up initial data preparation. The UI falls back to sequential tagging if workers fail to start.
- `WEB_VIRTUALIZE=1` enables virtualized grids in Stage 5 and the Owned library, smoothing large decks or libraries.
- Diagnostics overlays: enable `SHOW_DIAGNOSTICS=1`, then press **v** inside a virtualized grid to inspect render ranges, row counts, and paint timings.
## Diagnostics and logs
- `SHOW_DIAGNOSTICS=1` unlocks the `/diagnostics` page with system summaries (`/status/sys`), feature flags, and per-request `X-Request-ID` headers.
- `SHOW_LOGS=1` turns on the `/logs` viewer with level & keyword filters, auto-refresh, and copy-to-clipboard.
- Health probes live at `/healthz` and return `{status, version, uptime_seconds}` for integration with uptime monitors.