diff --git a/DOCKER.md b/DOCKER.md index 98ea6cc..c11e54a 100644 --- a/DOCKER.md +++ b/DOCKER.md @@ -1,228 +1,236 @@ # Docker Guide -Run the MTG Deckbuilder (CLI and Web UI) in Docker with persistent volumes and optional headless mode. +Spin up the MTG Python Deckbuilder inside containers. The image defaults to the Web UI; switch to the CLI/headless runner by flipping environment variables. All commands assume Windows PowerShell. -## Quick start +## Build a Deck (Web UI) -### PowerShell (recommended) -```powershell -docker compose build -docker compose run --rm mtg-deckbuilder -``` +- Build the image (first run only) and start the `web` service in detached mode: -### From Docker Hub (PowerShell) -```powershell -docker run -it --rm ` - -v "${PWD}/deck_files:/app/deck_files" ` - -v "${PWD}/logs:/app/logs" ` - -v "${PWD}/csv_files:/app/csv_files" ` - -v "${PWD}/owned_cards:/app/owned_cards" ` - -v "${PWD}/config:/app/config" ` - mwisnowski/mtg-python-deckbuilder:latest -``` - -## Web UI (new) - -The web UI runs the same deckbuilding logic behind a browser-based interface. - -### PowerShell (recommended) ```powershell docker compose up --build --no-deps -d web ``` -Then open http://localhost:8080 +- Open http://localhost:8080 to use the browser experience. First launch seeds data, downloads the latest card catalog, and tags cards automatically (`WEB_AUTO_SETUP=1`, `WEB_TAG_PARALLEL=1`, `WEB_TAG_WORKERS=4` in `docker-compose.yml`). -Volumes are the same as the CLI service, so deck exports/logs/configs persist in your working folder. -The app serves a favicon at `/favicon.ico` and exposes a health endpoint at `/healthz`. -Compare view offers a Copy summary button to copy a plain-text diff of two runs. The sidebar has a subtle depth shadow for clearer separation. - -Web UI feature highlights: -- Locks: Click a card or the lock control in Step 5; locks persist across reruns. -- Replace: Enable Replace in Step 5, click a card to open Alternatives (filters include Owned-only), then choose a swap. -- Permalinks: Copy a permalink from Step 5 or a Finished deck; paste via “Open Permalink…” to restore. -- Compare: Use the Compare page from Finished Decks; quick actions include Latest two and Swap A/B. - -Virtualized lists and lazy images (opt‑in) -- Set `WEB_VIRTUALIZE=1` to enable virtualization in Step 5 grids/lists and the Owned library for smoother scrolling on large sets. -- Example (Compose): - ```yaml - services: - web: - environment: - - WEB_VIRTUALIZE=1 - ``` -- Example (Docker Hub): - ```powershell - docker run --rm -p 8080:8080 ` - -e WEB_VIRTUALIZE=1 ` - -v "${PWD}/deck_files:/app/deck_files" ` - -v "${PWD}/logs:/app/logs" ` - -v "${PWD}/csv_files:/app/csv_files" ` - -v "${PWD}/owned_cards:/app/owned_cards" ` - -v "${PWD}/config:/app/config" ` - -e SHOW_DIAGNOSTICS=1 ` # optional: enables diagnostics tools and overlay - mwisnowski/mtg-python-deckbuilder:latest ` - bash -lc "cd /app && uvicorn code.web.app:app --host 0.0.0.0 --port 8080" - ``` - -### Diagnostics and logs (optional) -Enable internal diagnostics and a read-only logs viewer with environment flags. - -- `SHOW_DIAGNOSTICS=1` — adds a Diagnostics nav link and `/diagnostics` tools -- `SHOW_LOGS=1` — enables `/logs` and `/status/logs?tail=200` - -When enabled: -- `/logs` supports an auto-refresh toggle with interval, a level filter (All/Error/Warning/Info/Debug), and a Copy button to copy the visible tail. -- `/status/sys` returns a simple system summary (version, uptime, UTC server time, and feature flags) and is shown on the Diagnostics page when `SHOW_DIAGNOSTICS=1`. - - Virtualization overlay: press `v` on pages with virtualized grids to toggle per-grid overlays and a global summary bubble. - -Compose example (web service): -```yaml -environment: - - SHOW_LOGS=1 - - SHOW_DIAGNOSTICS=1 -``` - -Docker Hub (PowerShell) example: -```powershell -docker run --rm ` - -p 8080:8080 ` - -e SHOW_LOGS=1 -e SHOW_DIAGNOSTICS=1 -e ENABLE_THEMES=1 -e THEME=system ` - -e SPLASH_ADAPTIVE=1 -e SPLASH_ADAPTIVE_SCALE="1:1.0,2:1.0,3:1.0,4:0.6,5:0.35" ` # optional experiment - -e RANDOM_MODES=1 -e RANDOM_UI=1 -e RANDOM_MAX_ATTEMPTS=5 -e RANDOM_TIMEOUT_MS=5000 ` - -v "${PWD}/deck_files:/app/deck_files" ` - -v "${PWD}/logs:/app/logs" ` - -v "${PWD}/csv_files:/app/csv_files" ` - -v "${PWD}/owned_cards:/app/owned_cards" ` - -v "${PWD}/config:/app/config" ` - mwisnowski/mtg-python-deckbuilder:latest ` - bash -lc "cd /app && uvicorn code.web.app:app --host 0.0.0.0 --port 8080" -``` - -### Setup speed: parallel tagging (Web) -First-time setup or stale data triggers card tagging. The web service uses parallel workers by default. - -Configure via environment variables on the `web` service: -- `WEB_TAG_PARALLEL=1|0` — enable/disable parallel tagging (default: 1) -- `WEB_TAG_WORKERS=` — number of worker processes (default: 4 in compose) - -If parallel initialization fails, the service falls back to sequential tagging and continues. - -### From Docker Hub (PowerShell) -If you prefer not to build locally, pull `mwisnowski/mtg-python-deckbuilder:latest` and run uvicorn: -```powershell -docker run --rm ` - -p 8080:8080 ` - -v "${PWD}/deck_files:/app/deck_files" ` - -v "${PWD}/logs:/app/logs" ` - -v "${PWD}/csv_files:/app/csv_files" ` - -v "${PWD}/owned_cards:/app/owned_cards" ` - -v "${PWD}/config:/app/config" ` - mwisnowski/mtg-python-deckbuilder:latest ` - bash -lc "cd /app && uvicorn code.web.app:app --host 0.0.0.0 --port 8080" -``` - -Health check: -```text -GET http://localhost:8080/healthz -> { "status": "ok", "version": "dev", "uptime_seconds": 123 } -``` - -Theme preference reset (client-side): use the header’s Reset Theme control to clear the saved browser preference; the server default (THEME) applies on next paint. - -### Random Modes (alpha) and test dataset override - -Enable experimental Random Modes and UI controls in Web runs by setting: - -```yaml -services: - web: - environment: - - RANDOM_MODES=1 - - RANDOM_UI=1 - - RANDOM_MAX_ATTEMPTS=5 - - RANDOM_TIMEOUT_MS=5000 -``` - -For deterministic tests or development, you can point the app to a frozen dataset snapshot: - -```yaml -services: - web: - environment: - - CSV_FILES_DIR=/app/csv_files/testdata -``` - -### Taxonomy snapshot (maintainers) -Capture the current bracket taxonomy into an auditable JSON file inside the container: +- Stop or restart the service when you're done: ```powershell -docker compose run --rm web bash -lc "python -m code.scripts.snapshot_taxonomy" +docker compose stop web +docker compose start web ``` -Artifacts appear under `./logs/taxonomy_snapshots/` on your host via the mounted volume. -To force a new snapshot even when the content hash matches the latest, pass `--force` to the module. +- Prefer the public image? Pull and run it without building locally: -## Volumes -- `/app/deck_files` ↔ `./deck_files` -- `/app/logs` ↔ `./logs` -- `/app/csv_files` ↔ `./csv_files` -- `/app/owned_cards` ↔ `./owned_cards` (owned cards lists: .txt/.csv) -- Optional: `/app/config` ↔ `./config` (JSON configs for headless) - -## Interactive vs headless -- Interactive: attach a TTY (compose run or `docker run -it`) -- Headless auto-run: - ```powershell - docker compose run --rm -e DECK_MODE=headless mtg-deckbuilder - ``` -- Headless with JSON config: - ```powershell - docker compose run --rm ` - -e DECK_MODE=headless ` - -e DECK_CONFIG=/app/config/deck.json ` - mtg-deckbuilder - ``` - -### Common env vars -- DECK_MODE=headless -- DECK_CONFIG=/app/config/deck.json -- DECK_COMMANDER, DECK_PRIMARY_CHOICE -- DECK_ADD_LANDS, DECK_FETCH_COUNT - - DECK_TAG_MODE=AND|OR (combine mode used by the builder) - -### Web UI tuning env vars -- WEB_TAG_PARALLEL=1|0 (parallel tagging on/off) -- WEB_TAG_WORKERS= (process count; set based on CPU/memory) -- WEB_VIRTUALIZE=1 (enable virtualization) -- SHOW_DIAGNOSTICS=1 (enables diagnostics pages and overlay hotkey `v`) -- RANDOM_MODES=1 (enable random build endpoints) -- RANDOM_UI=1 (show Surprise/Theme/Reroll/Share controls) -- RANDOM_MAX_ATTEMPTS=5 (cap retry attempts) -- (Upcoming) Multi-theme inputs: once UI ships, Random Mode will accept `primary_theme`, `secondary_theme`, `tertiary_theme` fields; current backend already supports the cascade + diagnostics. -- RANDOM_TIMEOUT_MS=5000 (per-build timeout in ms) - -Testing/determinism helper (dev): -- CSV_FILES_DIR=csv_files/testdata — override CSV base dir to a frozen set for tests - -## Manual build/run ```powershell -docker build -t mtg-deckbuilder . -docker run -it --rm ` - -v "${PWD}/deck_files:/app/deck_files" ` - -v "${PWD}/logs:/app/logs" ` - -v "${PWD}/csv_files:/app/csv_files" ` - -v "${PWD}/owned_cards:/app/owned_cards" ` - -v "${PWD}/config:/app/config" ` - mtg-deckbuilder +docker run --rm -p 8080:8080 ` + -v "${PWD}/deck_files:/app/deck_files" ` + -v "${PWD}/logs:/app/logs" ` + -v "${PWD}/csv_files:/app/csv_files" ` + -v "${PWD}/config:/app/config" ` + -v "${PWD}/owned_cards:/app/owned_cards" ` + mwisnowski/mtg-python-deckbuilder:latest ``` +Shared volumes persist builds, logs, configs, and owned cards on the host. + +## Run a JSON Config + +Use the homepage “Run a JSON Config” button or run the same flow in-container: + +```powershell +docker compose run --rm ` + -e APP_MODE=cli ` + -e DECK_MODE=headless ` + -e DECK_CONFIG=/app/config/deck.json ` + web +``` + +- `APP_MODE=cli` routes the entrypoint to the CLI menu. +- `DECK_MODE=headless` skips prompts and calls `headless_runner`. +- Mount JSON configs under `config/` so both the UI and CLI can pick them up. + +Override counts, theme tags, or include/exclude lists by setting the matching environment variables before running the container (see “Environment variables” below). + +## Initial Setup + +The homepage “Initial Setup” tile appears when `SHOW_SETUP=1` (enabled in compose). It re-runs: + +1. Card downloads and color-filtered CSV generation. +2. Commander catalog rebuild (including multi-face merges). +3. Tagging and caching. + +To force a rebuild from the host: + +```powershell +docker compose run --rm --entrypoint bash web -lc "python -m code.file_setup.setup" +``` + +Add `--entrypoint bash ... "python -m code.scripts.refresh_commander_catalog"` when you only need the commander catalog (with MDFC merge and optional compatibility snapshot). + +## Owned Library + +Store `.txt` or `.csv` lists in `owned_cards/` (mounted to `/app/owned_cards`). The Web UI uses them for: + +- Owned-only or prefer-owned builds. +- The Owned Library management page (virtualized when `WEB_VIRTUALIZE=1`). +- Alternative suggestions that respect ownership. + +Use `/owned` to upload files and export enriched lists. These files persist through the `owned_cards` volume. + +## Browse Commanders + +`SHOW_COMMANDERS=1` exposes the commander browser tile. + +- Data lives in `csv_files/commander_cards.csv`. +- Refresh the catalog (including MDFC merges) from within the container: + +```powershell +docker compose run --rm --entrypoint bash web -lc "python -m code.scripts.refresh_commander_catalog" +``` + +Pass `--compat-snapshot` if you also need an unmerged compatibility CSV under `csv_files/compat_faces/`. + +## Finished Decks + +The Finished Decks page reads the `deck_files/` volume for completed builds: + +- Each run produces CSV, TXT, compliance JSON, and summary JSON sidecars. +- Locks and replace history persist per deck. +- Compare view can diff and export summaries. + +Ensure the deck exports volume remains mounted so these artifacts survive container restarts. + +## Browse Themes + +The Themes browser exposes the merged theme catalog with search, filters, and diagnostics. + +- `ENABLE_THEMES=1` keeps the selector visible. +- `WEB_THEME_PICKER_DIAGNOSTICS=1` unlocks uncapped synergies, extra metadata, and `/themes/metrics`. +- Regenerate the catalog manually: + +```powershell +docker compose run --rm --entrypoint bash web -lc "python -m code.scripts.build_theme_catalog" +``` + +Advanced options (e.g., `EDITORIAL_*` variables) live in `.env.example`. + +## Random Build + +Enable the Surprise/Reroll flow by setting: + +- `RANDOM_MODES=1` to expose backend random endpoints. +- `RANDOM_UI=1` to show the Random Build tile. +- Optional tunables: `RANDOM_MAX_ATTEMPTS`, `RANDOM_TIMEOUT_MS`, `RANDOM_PRIMARY_THEME`, `RANDOM_SEED`, and auto-fill flags. + +Headless parity is available by pairing `APP_MODE=cli` with `DECK_MODE=headless` and the same random variables. + +## Diagnostics + +`SHOW_DIAGNOSTICS=1` unlocks `/diagnostics` for system summaries, feature flags, and performance probes. Highlights: + +- `/healthz` returns `{status, version, uptime_seconds}` for external monitoring. +- Press `v` on pages with virtualized grids (when `WEB_VIRTUALIZE=1`) to toggle the range overlay. +- `WEB_AUTO_ENFORCE=1` (optional) applies bracket enforcement automatically after each build. + +## View Logs + +`SHOW_LOGS=1` enables the logs tile and `/logs` interface: + +- Tail the container log with filtering and copy-to-clipboard. +- `/status/logs?tail=200` offers a lightweight JSON endpoint. +- Raw files live under `logs/` on the host; rotate or archive them as needed. + +## Environment variables (Docker quick reference) + +See `.env.example` for the full catalog. Common knobs: + +### Core mode and networking + +| Variable | Default | Purpose | +| --- | --- | --- | +| `APP_MODE` | `web` | Switch between Web UI (`web`) and CLI (`cli`). | +| `DECK_MODE` | _(unset)_ | `headless` auto-runs the headless builder when the CLI starts. | +| `DECK_CONFIG` | `/app/config/deck.json` | JSON config file or directory (auto-discovery). | +| `HOST` / `PORT` / `WORKERS` | `0.0.0.0` / `8080` / `1` | Uvicorn binding when `APP_MODE=web`. | + +### Homepage visibility & UX + +| Variable | Default | Purpose | +| --- | --- | --- | +| `SHOW_SETUP` | `1` | Show the Initial Setup card. | +| `SHOW_LOGS` | `1` | Enable the View Logs tile and endpoints. | +| `SHOW_DIAGNOSTICS` | `1` | Enable Diagnostics tools and overlays. | +| `SHOW_COMMANDERS` | `1` | Expose the commander browser. | +| `ENABLE_THEMES` | `1` | Keep the theme selector and themes explorer visible. | +| `WEB_VIRTUALIZE` | `1` | Opt-in to virtualized lists/grids for large result sets. | +| `ALLOW_MUST_HAVES` | `1` | Enable include/exclude enforcement in Step 5. | +| `THEME` | `dark` | Initial UI theme (`system`, `light`, or `dark`). | + +### Random build controls + +| Variable | Default | Purpose | +| --- | --- | --- | +| `RANDOM_MODES` | _(unset)_ | Enable random build endpoints. | +| `RANDOM_UI` | _(unset)_ | Show the Random Build homepage tile. | +| `RANDOM_MAX_ATTEMPTS` | `5` | Retry budget for constrained random rolls. | +| `RANDOM_TIMEOUT_MS` | `5000` | Per-attempt timeout in milliseconds. | +| `RANDOM_PRIMARY_THEME` / `RANDOM_SECONDARY_THEME` / `RANDOM_TERTIARY_THEME` | _(blank)_ | Override theme slots for random runs. | +| `RANDOM_SEED` | _(blank)_ | Deterministic seed. | +| `RANDOM_AUTO_FILL` | `1` | Allow automatic backfill of missing theme slots. | + +### Automation & performance + +| Variable | Default | Purpose | +| --- | --- | --- | +| `WEB_AUTO_SETUP` | `1` | Auto-run data setup when artifacts are missing or stale. | +| `WEB_AUTO_REFRESH_DAYS` | `7` | Refresh `cards.csv` if older than N days. | +| `WEB_TAG_PARALLEL` | `1` | Use parallel workers during tagging. | +| `WEB_TAG_WORKERS` | `4` | Worker count for parallel tagging. | +| `WEB_AUTO_ENFORCE` | `0` | Re-export decks after auto-applying compliance fixes. | +| `WEB_THEME_PICKER_DIAGNOSTICS` | `1` | Enable theme diagnostics endpoints. | + +### Paths and data overrides + +| Variable | Default | Purpose | +| --- | --- | --- | +| `CSV_FILES_DIR` | `/app/csv_files` | Point the app at an alternate dataset (e.g., test snapshots). | +| `DECK_EXPORTS` | `/app/deck_files` | Override where the web UI looks for exports. | +| `OWNED_CARDS_DIR` / `CARD_LIBRARY_DIR` | `/app/owned_cards` | Override owned library directory. | +| `CARD_INDEX_EXTRA_CSV` | _(blank)_ | Inject a synthetic CSV into the card index for testing. | + +Advanced editorial and theme-catalog knobs (`EDITORIAL_*`, `SPLASH_ADAPTIVE`, etc.) are documented inline in `docker-compose.yml` and `.env.example`. + +## Shared volumes + +| Host path | Container path | Contents | +| --- | --- | --- | +| `deck_files/` | `/app/deck_files` | CSV/TXT exports, summary JSON, compliance reports. | +| `logs/` | `/app/logs` | Application logs and taxonomy snapshots. | +| `csv_files/` | `/app/csv_files` | Card datasets, commander catalog, tagging flags. | +| `config/` | `/app/config` | JSON configs, bracket policy, card list overrides. | +| `owned_cards/` | `/app/owned_cards` | Uploaded inventory files for owned-only flows. | + +## Maintenance commands + +Run ad-hoc tasks by overriding the entrypoint: + +```powershell +# Theme catalog rebuild +docker compose run --rm --entrypoint bash web -lc "python -m code.scripts.build_theme_catalog" + +# Snapshot taxonomy (writes logs/taxonomy_snapshots/) +docker compose run --rm --entrypoint bash web -lc "python -m code.scripts.snapshot_taxonomy" + +# Preview the MDFC commander diff +docker compose run --rm --entrypoint bash web -lc "python -m code.scripts.preview_dfc_catalog_diff" +``` + +Use the `--compat-snapshot` or other script arguments as needed. + ## Troubleshooting -- No prompts? Use `docker compose run --rm` (not `up`) or add `-it` to `docker run` -- Files not saving? Verify volume mounts and that folders exist -- Headless not picking config? Ensure `./config` is mounted to `/app/config` and `DECK_CONFIG` points to a JSON file -- Owned-cards prompt not seeing files? Ensure `./owned_cards` is mounted to `/app/owned_cards` -## Tips -- Use `docker compose run`, not `up`, for interactive mode -- Exported decks appear in `deck_files/` -- JSON run-config is exported only in interactive runs; headless skips it +- **Container starts but UI stays blank:** check `/healthz` and `/logs` (enable with `SHOW_LOGS=1`), then inspect the `logs/` volume. +- **Files missing on the host:** ensure the host directories exist before starting Compose; Windows will create empty folders if the path is invalid. +- **Long first boot:** dataset downloads and tagging can take several minutes the first time. Watch progress at `/setup`. +- **Random build hangs:** lower `RANDOM_MAX_ATTEMPTS` or raise `RANDOM_TIMEOUT_MS`, and confirm your theme overrides are valid slugs via `/themes/`. +- **Commander catalog outdated:** rerun the refresh command above or delete `csv_files/.tagging_complete.json` to force a full rebuild on next start. diff --git a/README.md b/README.md index 6024b28..112d7f4 100644 Binary files a/README.md and b/README.md differ diff --git a/docs/archive/DOCKER_2025-10-02.md b/docs/archive/DOCKER_2025-10-02.md new file mode 100644 index 0000000..98ea6cc --- /dev/null +++ b/docs/archive/DOCKER_2025-10-02.md @@ -0,0 +1,228 @@ +# Docker Guide + +Run the MTG Deckbuilder (CLI and Web UI) in Docker with persistent volumes and optional headless mode. + +## Quick start + +### PowerShell (recommended) +```powershell +docker compose build +docker compose run --rm mtg-deckbuilder +``` + +### From Docker Hub (PowerShell) +```powershell +docker run -it --rm ` + -v "${PWD}/deck_files:/app/deck_files" ` + -v "${PWD}/logs:/app/logs" ` + -v "${PWD}/csv_files:/app/csv_files" ` + -v "${PWD}/owned_cards:/app/owned_cards" ` + -v "${PWD}/config:/app/config" ` + mwisnowski/mtg-python-deckbuilder:latest +``` + +## Web UI (new) + +The web UI runs the same deckbuilding logic behind a browser-based interface. + +### PowerShell (recommended) +```powershell +docker compose up --build --no-deps -d web +``` + +Then open http://localhost:8080 + +Volumes are the same as the CLI service, so deck exports/logs/configs persist in your working folder. +The app serves a favicon at `/favicon.ico` and exposes a health endpoint at `/healthz`. +Compare view offers a Copy summary button to copy a plain-text diff of two runs. The sidebar has a subtle depth shadow for clearer separation. + +Web UI feature highlights: +- Locks: Click a card or the lock control in Step 5; locks persist across reruns. +- Replace: Enable Replace in Step 5, click a card to open Alternatives (filters include Owned-only), then choose a swap. +- Permalinks: Copy a permalink from Step 5 or a Finished deck; paste via “Open Permalink…” to restore. +- Compare: Use the Compare page from Finished Decks; quick actions include Latest two and Swap A/B. + +Virtualized lists and lazy images (opt‑in) +- Set `WEB_VIRTUALIZE=1` to enable virtualization in Step 5 grids/lists and the Owned library for smoother scrolling on large sets. +- Example (Compose): + ```yaml + services: + web: + environment: + - WEB_VIRTUALIZE=1 + ``` +- Example (Docker Hub): + ```powershell + docker run --rm -p 8080:8080 ` + -e WEB_VIRTUALIZE=1 ` + -v "${PWD}/deck_files:/app/deck_files" ` + -v "${PWD}/logs:/app/logs" ` + -v "${PWD}/csv_files:/app/csv_files" ` + -v "${PWD}/owned_cards:/app/owned_cards" ` + -v "${PWD}/config:/app/config" ` + -e SHOW_DIAGNOSTICS=1 ` # optional: enables diagnostics tools and overlay + mwisnowski/mtg-python-deckbuilder:latest ` + bash -lc "cd /app && uvicorn code.web.app:app --host 0.0.0.0 --port 8080" + ``` + +### Diagnostics and logs (optional) +Enable internal diagnostics and a read-only logs viewer with environment flags. + +- `SHOW_DIAGNOSTICS=1` — adds a Diagnostics nav link and `/diagnostics` tools +- `SHOW_LOGS=1` — enables `/logs` and `/status/logs?tail=200` + +When enabled: +- `/logs` supports an auto-refresh toggle with interval, a level filter (All/Error/Warning/Info/Debug), and a Copy button to copy the visible tail. +- `/status/sys` returns a simple system summary (version, uptime, UTC server time, and feature flags) and is shown on the Diagnostics page when `SHOW_DIAGNOSTICS=1`. + - Virtualization overlay: press `v` on pages with virtualized grids to toggle per-grid overlays and a global summary bubble. + +Compose example (web service): +```yaml +environment: + - SHOW_LOGS=1 + - SHOW_DIAGNOSTICS=1 +``` + +Docker Hub (PowerShell) example: +```powershell +docker run --rm ` + -p 8080:8080 ` + -e SHOW_LOGS=1 -e SHOW_DIAGNOSTICS=1 -e ENABLE_THEMES=1 -e THEME=system ` + -e SPLASH_ADAPTIVE=1 -e SPLASH_ADAPTIVE_SCALE="1:1.0,2:1.0,3:1.0,4:0.6,5:0.35" ` # optional experiment + -e RANDOM_MODES=1 -e RANDOM_UI=1 -e RANDOM_MAX_ATTEMPTS=5 -e RANDOM_TIMEOUT_MS=5000 ` + -v "${PWD}/deck_files:/app/deck_files" ` + -v "${PWD}/logs:/app/logs" ` + -v "${PWD}/csv_files:/app/csv_files" ` + -v "${PWD}/owned_cards:/app/owned_cards" ` + -v "${PWD}/config:/app/config" ` + mwisnowski/mtg-python-deckbuilder:latest ` + bash -lc "cd /app && uvicorn code.web.app:app --host 0.0.0.0 --port 8080" +``` + +### Setup speed: parallel tagging (Web) +First-time setup or stale data triggers card tagging. The web service uses parallel workers by default. + +Configure via environment variables on the `web` service: +- `WEB_TAG_PARALLEL=1|0` — enable/disable parallel tagging (default: 1) +- `WEB_TAG_WORKERS=` — number of worker processes (default: 4 in compose) + +If parallel initialization fails, the service falls back to sequential tagging and continues. + +### From Docker Hub (PowerShell) +If you prefer not to build locally, pull `mwisnowski/mtg-python-deckbuilder:latest` and run uvicorn: +```powershell +docker run --rm ` + -p 8080:8080 ` + -v "${PWD}/deck_files:/app/deck_files" ` + -v "${PWD}/logs:/app/logs" ` + -v "${PWD}/csv_files:/app/csv_files" ` + -v "${PWD}/owned_cards:/app/owned_cards" ` + -v "${PWD}/config:/app/config" ` + mwisnowski/mtg-python-deckbuilder:latest ` + bash -lc "cd /app && uvicorn code.web.app:app --host 0.0.0.0 --port 8080" +``` + +Health check: +```text +GET http://localhost:8080/healthz -> { "status": "ok", "version": "dev", "uptime_seconds": 123 } +``` + +Theme preference reset (client-side): use the header’s Reset Theme control to clear the saved browser preference; the server default (THEME) applies on next paint. + +### Random Modes (alpha) and test dataset override + +Enable experimental Random Modes and UI controls in Web runs by setting: + +```yaml +services: + web: + environment: + - RANDOM_MODES=1 + - RANDOM_UI=1 + - RANDOM_MAX_ATTEMPTS=5 + - RANDOM_TIMEOUT_MS=5000 +``` + +For deterministic tests or development, you can point the app to a frozen dataset snapshot: + +```yaml +services: + web: + environment: + - CSV_FILES_DIR=/app/csv_files/testdata +``` + +### Taxonomy snapshot (maintainers) +Capture the current bracket taxonomy into an auditable JSON file inside the container: + +```powershell +docker compose run --rm web bash -lc "python -m code.scripts.snapshot_taxonomy" +``` +Artifacts appear under `./logs/taxonomy_snapshots/` on your host via the mounted volume. + +To force a new snapshot even when the content hash matches the latest, pass `--force` to the module. + +## Volumes +- `/app/deck_files` ↔ `./deck_files` +- `/app/logs` ↔ `./logs` +- `/app/csv_files` ↔ `./csv_files` +- `/app/owned_cards` ↔ `./owned_cards` (owned cards lists: .txt/.csv) +- Optional: `/app/config` ↔ `./config` (JSON configs for headless) + +## Interactive vs headless +- Interactive: attach a TTY (compose run or `docker run -it`) +- Headless auto-run: + ```powershell + docker compose run --rm -e DECK_MODE=headless mtg-deckbuilder + ``` +- Headless with JSON config: + ```powershell + docker compose run --rm ` + -e DECK_MODE=headless ` + -e DECK_CONFIG=/app/config/deck.json ` + mtg-deckbuilder + ``` + +### Common env vars +- DECK_MODE=headless +- DECK_CONFIG=/app/config/deck.json +- DECK_COMMANDER, DECK_PRIMARY_CHOICE +- DECK_ADD_LANDS, DECK_FETCH_COUNT + - DECK_TAG_MODE=AND|OR (combine mode used by the builder) + +### Web UI tuning env vars +- WEB_TAG_PARALLEL=1|0 (parallel tagging on/off) +- WEB_TAG_WORKERS= (process count; set based on CPU/memory) +- WEB_VIRTUALIZE=1 (enable virtualization) +- SHOW_DIAGNOSTICS=1 (enables diagnostics pages and overlay hotkey `v`) +- RANDOM_MODES=1 (enable random build endpoints) +- RANDOM_UI=1 (show Surprise/Theme/Reroll/Share controls) +- RANDOM_MAX_ATTEMPTS=5 (cap retry attempts) +- (Upcoming) Multi-theme inputs: once UI ships, Random Mode will accept `primary_theme`, `secondary_theme`, `tertiary_theme` fields; current backend already supports the cascade + diagnostics. +- RANDOM_TIMEOUT_MS=5000 (per-build timeout in ms) + +Testing/determinism helper (dev): +- CSV_FILES_DIR=csv_files/testdata — override CSV base dir to a frozen set for tests + +## Manual build/run +```powershell +docker build -t mtg-deckbuilder . +docker run -it --rm ` + -v "${PWD}/deck_files:/app/deck_files" ` + -v "${PWD}/logs:/app/logs" ` + -v "${PWD}/csv_files:/app/csv_files" ` + -v "${PWD}/owned_cards:/app/owned_cards" ` + -v "${PWD}/config:/app/config" ` + mtg-deckbuilder +``` + +## Troubleshooting +- No prompts? Use `docker compose run --rm` (not `up`) or add `-it` to `docker run` +- Files not saving? Verify volume mounts and that folders exist +- Headless not picking config? Ensure `./config` is mounted to `/app/config` and `DECK_CONFIG` points to a JSON file +- Owned-cards prompt not seeing files? Ensure `./owned_cards` is mounted to `/app/owned_cards` + +## Tips +- Use `docker compose run`, not `up`, for interactive mode +- Exported decks appear in `deck_files/` +- JSON run-config is exported only in interactive runs; headless skips it diff --git a/docs/archive/README_2025-10-02.md b/docs/archive/README_2025-10-02.md new file mode 100644 index 0000000..f65426e --- /dev/null +++ b/docs/archive/README_2025-10-02.md @@ -0,0 +1,951 @@ +# 🃏 MTG Python Deckbuilder + +[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) +[![Python 3.11+](https://img.shields.io/badge/python-3.11+-blue.svg)](https://www.python.org/downloads/) +[![Docker](https://img.shields.io/badge/docker-supported-blue.svg)](https://www.docker.com/) + +A fast, menu-driven deckbuilder for MTG Commander/EDH with both CLI and Web UI. Supports interactive and headless runs, CSV/TXT exports, owned-card libraries, and Docker. + +## 🔹 Features +- Web UI and CLI, powered by the same builder +- Commander search with smart theme tagging (primary/secondary/tertiary) +- Power bracket selection and ideal count prompts +- **Include/Exclude Card Lists**: Must-include and must-exclude functionality with fuzzy matching, visual validation, and strict enforcement (set `ALLOW_MUST_HAVES=true`) +- CSV and TXT exports with stable filenames +- Headless mode (non-interactive) and a headless submenu in the main menu +- Config precedence: CLI > env > JSON > defaults + - Visual summaries in Web UI: Mana curve, Color pips and Sources with hover-to-highlight and copyable tooltips +- Web UI: Locks, Replace flow, Compare builds, and shareable permalinks + - Unified “New Deck” modal (steps 1–3): commander search, themes, bracket, options, and optional Name (used in export filenames) + - Multi-copy archetypes (opt-in modal): choose quantity for supported archetypes and optionally add Thrumming Stone; added early with target adjustments and a hard 100-card clamp + - Combos & Synergies: detect curated pairs, show chip-style lists with badges, and dual-card hover previews; optional auto-complete adds missing partners based on your plan. + +## ✨ Highlights +- Smart tagging and suggestions for commander + themes, with AND/OR combine modes +- Theme governance: generated theme catalog with whitelist normalization, enforced synergies, and capped synergy lists (top 5) for concise UI; roadmap includes YAML authoring export. +- Exports CSV and TXT decklists to `deck_files/` +- Random Full Build (alpha) now outputs a single authoritative artifact set (CSV, TXT, compliance JSON, summary JSON) without duplicate suffixed files; API returns paths for integration. Summary sidecars carry multi-theme metadata (`primary_theme`, `secondary_theme`, `tertiary_theme`, `resolved_themes`, fallback flags) plus legacy aliases for incremental consumer upgrades. +- Random Mode (alpha) multi-theme groundwork: curated pools now support manual exclusions via `config/random_theme_exclusions.yml` (documented in `docs/random_theme_exclusions.md` and exportable with `python code/scripts/report_random_theme_pool.py --write-exclusions`); UI/diagnostics surface fallback telemetry while the upcoming picker adds Primary/Secondary/Tertiary inputs with deterministic fallback (P+S+T → P+S → P+T → P → synergy overlap → full pool) and an inline “Clear themes” control quickly resets stored inputs. +- Random Mode instrumentation: `python code/scripts/profile_multi_theme_filter.py` captures mean/p95 filter timings, while `python code/scripts/check_random_theme_perf.py` guards against regressions relative to `config/random_theme_perf_baseline.json` (`--update-baseline` refreshes). +- Owned-cards mode: point to one or more `.txt`/`.csv` files in `owned_cards/` and choose "Use only owned cards?" +- If you opt out, final CSV marks owned cards with an `Owned` column +- Build options include Prefer-owned: bias picks toward owned cards while allowing unowned fallback +- Interactive menu with a headless submenu +- Works locally or in Docker (Windows PowerShell friendly) +- Card images and data via Scryfall (attribution shown in the Web UI) + - Web visual summaries: cross-highlight cards from charts; sources include non-lands and colorless 'C' with an on/off toggle + - New Deck modal keyboard UX: Enter selects the first suggestion; arrow navigation removed; browser autofill disabled + - Export naming: optional Name in the modal becomes the filename stem for CSV/TXT/JSON; decks include a sidecar `.summary.json` with `meta.name` + - Finished Decks and deck banners prefer your custom Name when present + - Compare page includes a Copy summary button to quickly share diffs + - New Deck modal shows your selected themes and their order (1 → 3) while picking; Bracket options are labeled (e.g., "Bracket 3: Upgraded"). Default bracket is 3. + - Bracket policy enforcement: disallowed categories (e.g., Game Changers in Brackets 1–2) are pruned from the pool up-front; UI blocks progression until violations are resolved, with inline Replace to pick compliant alternatives. + - Bracket compliance UX: the compliance panel auto-opens when non-compliant and shows a colored overall chip (green/WARN amber/red). WARN thresholds (e.g., tutors/extra turns) are advisory—tiles show with a subtle style and no gating; FAIL continues to gate and enables enforcement. + +### Commander eligibility rules (clarified) +The builder now applies stricter Commander legality filtering when generating `commander_cards.csv`: +1. Automatically eligible: + - Legendary Creatures (includes Artifact Creature / Enchantment Creature lines) + - Legendary Artifact Vehicles or Spacecraft that have printed power and toughness + - Any card whose rules text explicitly contains "can be your commander" +2. Not auto‑eligible (unless they have the explicit text above): + - Plain Legendary Planeswalkers + - Plain Legendary Enchantments without the Creature type + - Generic Legendary Artifacts (non‑Vehicle/Spacecraft or without P/T) +This prevents over-broad inclusion of non-creature legendary permanents. + +## 🚀 Quick start + +### Docker Hub (PowerShell) +```powershell +docker run -it --rm ` + -v "${PWD}/deck_files:/app/deck_files" ` + -v "${PWD}/logs:/app/logs" ` + -v "${PWD}/csv_files:/app/csv_files" ` + -v "${PWD}/owned_cards:/app/owned_cards" ` + -v "${PWD}/config:/app/config" ` + mwisnowski/mtg-python-deckbuilder:latest +``` + +### From source +```powershell +pip install -r requirements.txt +python code/main.py +``` + +### From this repo with Docker Compose (PowerShell) +```powershell +docker compose build +docker compose run --rm mtg-deckbuilder +``` + +## 🌐 Web UI + +Run the browser-based UI backed by the same deckbuilding engine. + +### From source +```powershell +pip install -r requirements.txt +uvicorn code.web.app:app --host 127.0.0.1 --port 8080 +``` + +Open http://127.0.0.1:8080 + +### Docker Compose (PowerShell) +```powershell +docker compose build web +docker compose up --no-deps web +``` + +Then open http://localhost:8080 + +Notes: +- Exports/logs/configs use the same folders and will appear under your mounted volumes (see compose file). +- The footer includes Scryfall attribution per their guidelines. + - Favicon is bundled; browsers load `/favicon.ico` (auto-falls back to PNG). + - Bracket enforcement is applied in both web and headless runs; see "Bracket policy & enforcement" below. + +### Run with Docker Compose (Docker Hub image) + +Use the prebuilt image with the provided `dockerhub-docker-compose.yml`. The image defaults to the Web UI. + +Web (default): +```powershell +docker compose -f dockerhub-docker-compose.yml up -d +``` + +Pin to a version (edit the compose file image tag to `:X.Y.Z`) to avoid `latest` drift. + +CLI mode (override default): +```powershell +docker compose -f dockerhub-docker-compose.yml run --rm ` + -e APP_MODE=cli ` + web +``` + +Notes: +- Volumes under `${PWD}` persist your exports/logs/configs locally next to the compose file. +- Health checks are built into the image for the Web UI. In CLI, `APP_MODE=cli` bypasses health. + +#### Theme defaults and flags +- Enable the Theme selector with `ENABLE_THEMES=1` (on by default in `web` in compose). +- Set the initial default with `THEME=system|light|dark` (applies only when the browser has no prior preference saved): + - `THEME=system` (recommended): follows the OS theme; Light uses the consolidated Light (Blend) palette. + - `THEME=light`: maps to the consolidated Light (Blend) theme in the UI. + - `THEME=dark`: uses the dark theme. +- You can also force a one-off override via URL: `?theme=system|light|dark|high-contrast|cb-friendly`. + - Example: `http://localhost:8080/?theme=dark` + - The choice is saved in localStorage; the URL parameter is removed after applying. + - Reset: Use the “Reset theme” control in the header to clear your preference and re‑apply the server default (or system). + - YAML export strategy: By default the app now always regenerates per-theme YAML files even on fast refreshes to keep editorial artifacts synchronized with `theme_list.json`. To opt-out on incremental refreshes (rare), set `THEME_YAML_FAST_SKIP=1` (compose variable). This only skips export during fast-path (no retag) theme refreshes and never during full builds. + +#### Theme catalog editorial pipeline +Build script: `python code/scripts/build_theme_catalog.py` + +Key flags / env vars: +- `--limit N` (preview subset; guarded from overwriting canonical JSON unless `--allow-limit-write`) +- `--output path` (write catalog to alternate location; suppresses YAML backfill so tests don't mutate source files) +- `--backfill-yaml` or `EDITORIAL_BACKFILL_YAML=1` (write auto `description` + derived/pinned `popularity_bucket` into each YAML if missing) +- `--force-backfill-yaml` (overwrite existing description/popularity_bucket) +- `EDITORIAL_SEED=` (deterministic ordering for inference and any randomized fallbacks) +- `EDITORIAL_AGGRESSIVE_FILL=1` (pad ultra-sparse themes with extra inferred synergies) +- `EDITORIAL_POP_BOUNDARIES="a,b,c,d"` (override popularity bucket thresholds) +- `EDITORIAL_POP_EXPORT=1` (emit `theme_popularity_metrics.json` summary) + +Derived fields: +- `popularity_bucket` (Very Common / Common / Uncommon / Niche / Rare) based on summed color frequency; authors may pin a value in YAML. +- `description` auto-generated if absent using heuristics keyed to theme class (tribal, counters, graveyard, tokens, etc.). + +Editorial quality & metadata info: +- Optional `editorial_quality: draft|reviewed|final` can be set in per-theme YAML (or force-backfilled) to track curation progress. +- Backfills now stamp a `metadata_info` block (formerly `provenance`) with keys such as `last_backfill`, `script`, and `version`. The legacy key `provenance` is still parsed for a short deprecation window; if both appear a warning is emitted (suppress via `SUPPRESS_PROVENANCE_DEPRECATION=1`). Planned removal of the legacy alias: version 2.4.0. +- A running specialization trend is appended to `config/themes/description_fallback_history.jsonl` whenever `EDITORIAL_INCLUDE_FALLBACK_SUMMARY=1` is set. +- When `EDITORIAL_INCLUDE_FALLBACK_SUMMARY=1`, the catalog additionally embeds a `description_fallback_summary` object exposing coverage KPIs (see below) to power regression tests. + +Synergy pairs & clusters: +- `config/themes/synergy_pairs.yml` provides curated fallback synergies (applied only if a theme YAML lacks `curated_synergies`). +- `config/themes/theme_clusters.yml` groups themes (Tokens, Counters, Graveyard, etc.) for upcoming UI filters and analytics. + - Validator checks structure; UI integration planned in Phase E. + +Description mapping externalization: +- `config/themes/description_mapping.yml` fully replaces internal mapping when present. + - Items: `triggers` (list of substrings, lowercase-insensitive) and `description` string. + - `{SYNERGIES}` placeholder expands to a short clause with top synergy names or is removed gracefully if none. + - Mapping validator (`scripts/validate_description_mapping.py`) reports duplicate triggers, suggests `{SYNERGIES}` usage, and validates synergy pairs & clusters. + +KPI targets (initial schedule): +- Wave 1 (current): Maintain regression test thresholds (generic_pct < 52%). +- Wave 2: Ratchet to <45% after next mapping expansion. +- Wave 3: <38%; Wave 4: <30% (issue templates will adjust thresholds once prior target is achieved and history confirms downward trend). + +Lint (`code/scripts/lint_theme_editorial.py`) enhancements: +- `--require-description` / env `EDITORIAL_REQUIRE_DESCRIPTION=1` enforces descriptions present. +- `--require-popularity` / env `EDITORIAL_REQUIRE_POPULARITY=1` enforces popularity bucket present. +- Strict mode also escalates missing cornerstone examples and missing description/popularity to errors. + +Testing: +Determinism, boundary overrides, and backfill isolation validated in `code/tests/test_theme_catalog_generation.py` using the new `--output` flag. + +### Commander catalog upkeep +- The Commander Browser and auto-select flows read from `csv_files/commander_cards.csv`; keep it version-controlled so web and CLI stay in sync. +- Regenerate the file after MTGJSON updates or commander policy tweaks: + ```powershell + python -c "from file_setup.setup import regenerate_csvs_all; regenerate_csvs_all()" + ``` +- For quick fixes, you can refresh just the commander list: + ```powershell + python -c "from file_setup.setup import determine_commanders; determine_commanders()" + ``` +- Full onboarding guidance (required columns, caching notes, validation steps) lives in `docs/commander_catalog.md`. +- Staging toggle: set `SHOW_COMMANDERS=0` to hide the sidebar link while testing privately; defaults to on. + +### Theme Catalog API +Two new endpoints expose the merged catalog for the upcoming theme picker UI. + +List themes: +`GET /themes/api/themes` +Parameters: +- `q` substring across theme name & synergies +- `archetype` filter by `deck_archetype` +- `bucket` popularity bucket (Very Common|Common|Uncommon|Niche|Rare) +- `colors` comma-separated color initials (e.g. `G,W`) +- `limit` (default 50, max 200) & `offset` +- `diagnostics=1` (requires env `WEB_THEME_PICKER_DIAGNOSTICS=1`) to expose `has_fallback_description` + `editorial_quality` + +Sample response snippet: +``` +{ + "ok": true, + "count": 314, + "items": [ + { + "id": "plus1-plus1-counters", + "theme": "+1/+1 Counters", + "primary_color": "Green", + "secondary_color": "White", + "popularity_bucket": "Very Common", + "deck_archetype": "Counters", + "description": "...", + "synergies": ["Adapt","Evolve","Proliferate","Counters Matter","Hydra Kindred"], + "synergy_count": 5 + // diagnostics-only: has_fallback_description, editorial_quality + } + ], + "next_offset": 50, + "generated_at": "2025-09-19T12:34:56", + "diagnostics": false +} +``` + +Theme detail: +`GET /themes/api/theme/{id}` +Parameters: +- `uncapped=1` (diagnostics only) returns `uncapped_synergies` (full curated+enforced+inferred ordered set) +- `diagnostics=1` enables extra fields (fallback + editorial quality) + +Detail response excerpt: +``` +{ + "ok": true, + "theme": { + "id": "plus1-plus1-counters", + "theme": "+1/+1 Counters", + "description": "...", + "synergies": ["Adapt","Evolve","Proliferate","Counters Matter","Hydra Kindred"], + "curated_synergies": [...], + "enforced_synergies": [...], + "inferred_synergies": [...], + "example_commanders": [...], + "example_cards": [...], + "synergy_commanders": [...] + // diagnostics-only: uncapped_synergies, editorial_quality, has_fallback_description + } +} +``` + +Caching & freshness: ETag header reflects catalog size/mtime + newest YAML mtime. Use `/themes/status` for stale detection; POST `/themes/refresh` to rebuild in background. + +Environment flag: `WEB_THEME_PICKER_DIAGNOSTICS=1` enables diagnostics extras and uncapped synergy toggle. + +Performance & optimization: +- Fast list filtering uses precomputed lowercase search haystacks + memoized filtered slug cache (keyed by catalog ETag and query params) for sub‑50ms typical responses once warm. +- Skeleton loaders (picker shell, table list fragment, preview modal) improve perceived load time; real content swaps in via HTMX fragment requests. +- Metrics endpoint `/themes/metrics` (gated by `WEB_THEME_PICKER_DIAGNOSTICS=1`) exposes: + - Catalog filter requests, cache hits, cache entry count + - Preview requests, cache hits, average build time (ms), cache size, TTL +- Response headers: `X-ThemeCatalog-Filter-Duration-ms` and `ETag` let you observe server-side filter cost and enable conditional requests. + +### Governance & Editorial +The theme catalog follows lightweight governance so curation quality scales: + +1. Minimum Examples Threshold: Target 2 example cards and 1 example commander for every established theme. Enforcement flips from warning to required once coverage >90% (tracked via metrics). Until then, themes below threshold are flagged but not blocked. +2. Curation Ordering: Preview assembly order is deterministic — examples, curated synergy examples, sampled role cards (payoff/enabler/support/wildcard), then synthetic placeholders only if needed to reach target count. +3. Splash Relax Policy: Four- and five-color commanders may include a single off-color enabler (scored with a -0.3 penalty) to avoid over-pruning multi-color creative lines. This is documented to prevent accidental removal during future sampling refactors. +4. Popularity Buckets Are Non-Scoring: Popularity is for filtering and subtle UI hints only and must never directly influence sampling scores (avoids positive feedback loops that crowd out niche archetypes). +5. Taxonomy Expansion Criteria: Proposed new high-level themes (Combo, Storm, Extra Turns, Group Hug / Politics, Pillowfort, Toolbox / Tutors, Treasure Matters, Monarch / Initiative) must demonstrate: (a) a distinct strategic pattern, (b) at least 8 broadly played representative cards, (c) not a strict subset of another existing theme. +6. Editorial Quality Tracking: Optional `editorial_quality: draft|reviewed|final` in theme YAML guides review pass prioritization; metrics aggregate coverage to spot stagnation. +7. Deterministic Sampling: Seed = `theme|commander` hash; changes to sampling heuristics must preserve deterministic output for identical inputs (regression tests rely on this). Score contributors must append reasons for transparency (`reasons[]`). + +For deeper rationale & checklist see `docs/theme_taxonomy_rationale.md`. +- Preview sampling caches results (TTL 600s) keyed by (slug, limit, colors, commander, ETag) ensuring catalog changes invalidate prior samples. + +Governance tooling & experiments: +- Taxonomy snapshot: `python -m code.scripts.snapshot_taxonomy` writes `logs/taxonomy_snapshots/taxonomy_.json` with a SHA-256 `hash` for auditability. Identical content is skipped unless forced. Use this before tuning taxonomy-aware sampling heuristics. +- Adaptive splash penalty (optional): set `SPLASH_ADAPTIVE=1` to scale off-color enabler penalties based on commander color count; tune with `SPLASH_ADAPTIVE_SCALE` (e.g., `1:1.0,2:1.0,3:1.0,4:0.6,5:0.35`). Analytics recognize both static and adaptive reasons; experiments can compare counters when enabled vs disabled. + +Preview endpoint: +`GET /themes/api/theme/{id}/preview?limit=12&colors=G,W&commander=Name` +Returns a deterministic sample (seeded by theme + optional commander) combining: +- Pinned curated `example_cards` (role `example`) +- Pinned curated `synergy_example_cards` (role `curated_synergy`) +- Heuristically sampled cards bucketed into payoff / enabler / support / wildcard with diversity quotas (~40/40/20 after pinned examples) +- Synthetic `[Synergy]` placeholders only if real sampling underfills the requested limit +Commander bias heuristics (color identity restriction + overlap/theme bonuses) influence scoring & ordering. + +Preview sampling now includes curated examples, curated synergy layer, role-based heuristic sampling (payoff / enabler / support / wildcard) with diversity quotas, commander bias heuristics (color identity restriction + diminishing overlap bonuses), rarity diminishing weights, splash leniency (single off-color allowance with small penalty for 4–5 color commanders), role saturation penalty, refined overlap scaling curve, and tooltip-expanded heuristic reasons. Synthetic placeholders are only added if the sample underfills. + +Edge considerations (Theme picker & preview): +- Empty dataset: graceful skeleton + 'No themes match your filters' message. +- High latency first load: skeleton shimmer + optional prewarm (when `WEB_THEME_FILTER_PREWARM=1`). +- Catalog reload or tagging completion: filter + preview caches bust; stale indicator triggers background refresh. +- Commander not found: commander bias reasons omitted; sampling still deterministic via seed. +- Color filter reduces pool below quota: synthetic `[Synergy]` placeholders pad to requested limit. +- Duplicate card names across tag shards: final defensive dedupe pass before payload. +- Oversized preview cache: FIFO eviction maintains <= THEME_PREVIEW_CACHE_MAX (default 400). +- Disabled diagnostics: uncapped synergies, editorial quality, fallback description markers, raw YAML & metrics all suppressed. +- Tooltip safety: JS errors are swallowed; preview remains functional without enhancements. + +#### Editorial coverage metrics / specialization progress +- Set `EDITORIAL_INCLUDE_FALLBACK_SUMMARY=1` when running `build_theme_catalog.py` to embed a `description_fallback_summary` block in the resulting JSON. This includes: + - `generic_total`, `generic_with_synergies`, `generic_plain`, and `generic_pct` + - `top_generic_by_frequency` (capped list of the highest-frequency themes still using generic fallback phrasing) +- Used by `test_theme_description_fallback_regression.py` to enforce a shrinking ceiling on generic descriptions after each optimization wave. + - History persisted to `description_fallback_history.jsonl` for KPI trend visualization and automated threshold ratcheting. + +#### External description mapping +- Extend or override auto-description heuristics without code edits via `config/themes/description_mapping.yml`. + - Each list item: `triggers: ["substring1", ...]`, `description: "Text with optional {SYNERGIES} placeholder"` + - First match wins; triggers are lowercase substring checks. + - `{SYNERGIES}` is replaced with a short clause (e.g., `Synergies like Tokens and Sacrifice reinforce the plan.`) when synergy context exists; omitted cleanly otherwise. +- If the file is absent, the internal mapping rules are used as a fallback. + +### Quick guide (locks • replace • compare • permalinks) +- New Deck modal (Steps 1–3 unified): search for your commander, pick themes, choose bracket (defaults to 3), and optionally set Name. Submitting starts the build immediately. Name becomes the export filename stem and display name. +- Lock a card: click the card image or the lock control in Step 5. Locked cards are pinned for reruns. A small “Last action” chip confirms lock/unlock. +- Replace a card: toggle Replace in Step 5, then click a card to open Alternatives. Filter (including Owned-only) and pick a swap; the deck updates in place. + - When bracket violations exist, Step 5 shows them first and disables Continue/Rerun. Pick replacements inline; alternatives exclude commanders, locked, in-deck, and just-added cards and prefer role-consistent options. +- Permalink: use “Copy permalink” from Step 5 or a Finished deck to share/restore a run (commander, themes, bracket, ideals, flags). “Open Permalink…” lets you paste one to restore to review. +- Compare: from Finished Decks, open Compare and pick A and B (quick actions: Latest two, Swap A/B). Use Changed-only to focus diffs. Copy summary or Download .txt. +- Keyboard: In the New Deck modal, Enter selects the first commander result. Browser autofill is disabled for cleaner inputs. In Step 5, use L to lock/unlock the focused card, R to open Alternatives, and C to copy a permalink. + +### Setup speed: parallel tagging (Web) +- On first run or when data is stale, the Web UI prepares and tags the card database. +- This tagging runs in parallel by default for faster setup. +- Tuning via env vars (set on the `web` service in `docker-compose.yml`): + - `WEB_TAG_PARALLEL=1|0` — enable/disable parallel workers (default: 1/enabled) + - `WEB_TAG_WORKERS=` — number of worker processes (default: 4 in compose; omit to auto-pick) +- The UI shows progress and falls back to sequential tagging automatically if parallel init fails. + +Virtualized lists and image polish (opt-in) +- Opt-in with `WEB_VIRTUALIZE=1` to enable virtualization in Step 5 lists/grids and the Owned library for smoother scrolling on large sets. +- Virtualization diagnostics overlay (for debugging): enable `SHOW_DIAGNOSTICS=1`, then press `v` to toggle an overlay per grid and a global summary bubble; shows visible range, total, rows, row height, render time, and counters. +- Thumbnails use lazy-loading with srcset/sizes; LQIP blur/fade-in is applied to Step 5 and Owned thumbnails, and the commander preview image. +- Respects reduced motion: when the OS prefers-reduced-motion, blur/fade transitions and smooth scrolling are disabled for accessibility. + +### Build options (Web) +- Use only owned cards: builds strictly from your `owned_cards/` lists (commander exempt). +- Prefer owned cards: gently prioritizes owned cards across creatures and spells but allows unowned when they’re better fits. + - Implemented via a stable owned-first reorder and small weight boosts; preserves existing sort intent. +Notes: +- Steps 1–3 are consolidated into a single New Deck modal. Submitting starts the build immediately, skipping the old review page. + - The modal includes an optional Name; when set, it is used as the export filename stem and display name in the UI. + - The modal displays selected themes in order (1, 2, 3). The Bracket selector shows numbers (e.g., "Bracket 3: Upgraded") and defaults to 3 for new decks. + +### Staged build visibility +- Step 5 can optionally “Show skipped stages” so you can see phases that added no cards with a clear annotation. + +### Multi-copy archetypes (Web) +- When your commander + themes suggest a multi-copy strategy (e.g., petitioners, approach, apostles), the Web UI shows a one-time modal to add a package. +- Choose how many copies (bounded by the printed cap) and optionally include 1× Thrumming Stone when it synergizes. +- The package is applied as the first stage so later phases account for the volume. +- Targets adjust automatically (reduce creatures for creature packages, or spread reductions across spells). The UI shows an “Adjusted targets” note on that stage. +- A final safety clamp trims overflow from this stage to keep the deck at 100. A “Clamped N” chip appears when this happens. +- Suggestions won’t re-prompt repeatedly for the same commander+theme context unless you change selections; you can also dismiss the modal. + +### Visual summaries (Web) +- Mana Curve bars with hover-to-highlight of matching cards in both list and thumbnail views +- Color Pips (requirements) and Sources (generation) bars with per-color tooltips listing cards +- Cross-highlighting from charts to the card views; list-mode highlights only the name span +- Sources detection includes non-land producers (artifacts/creatures/etc.) and colorless 'C' +- Fetch lands are not counted as mana sources; basic lands and Wastes are handled reliably +- Optional: Show colorless (C) toggle for Sources; persisted per browser +- Tooltips include a Copy button to copy the card list + +### Combos & Synergies (Web) +- Detection: curated two-card combos/synergies are detected from the final deck (commander included) and shown with list version badges. +- UI: chip-style rows with badges (cheap/early, setup). Hover a row to preview both cards side-by-side; hover an individual name for single-card preview. +- Auto-Complete Combos: when enabled in the New Deck modal, the builder completes up to N pairs before theme fill/monolithic spells so partners stick. + - Settings: Prefer combos (on/off), How many combos (target), Balance (early/late/mix) to bias partner selection. + - Existing completed pairs are counted first; only missing partners are added. + - Color identity enforced via the filtered card pool; off-color/unavailable partners are skipped. + +### Owned page (Web) +- View and manage your owned lists with: + - Export TXT/CSV, sort controls, and a live “N shown” counter + - Color identity dots and exact color-identity combo filters (including 4-color) + - Viewport-filling list with styled scrollbar + - Optional virtualization when `WEB_VIRTUALIZE=1` (improves performance on very large libraries) +- Uploading `.txt` or `.csv` lists enriches and deduplicates entries at upload-time and persists them, so page loads are fast. + +### Finished Decks (Web) +- Theme filters are a dropdown with shareable state for easier browsing of saved builds. + - Each finished deck has a permalink you can copy and revisit; Compare mode lets you diff against another run. The Compare page has a Copy summary button to copy a plain-text diff. + - Locks and Replace flow: lock-in picks you like or replace a card with an alternative during iteration. + +### Diagnostics (Web) +- Health endpoint: `GET /healthz` returns `{ status, version, uptime_seconds }` +- Responses include `X-Request-ID` for easier error correlation; unhandled errors return JSON with `request_id` + +Logs and system tools: +- Logs page (`/logs`, enable with `SHOW_LOGS=1`): + - Auto-refresh toggle with adjustable interval + - Level filter (All/Error/Warning/Info/Debug) and keyword filter + - Copy button to copy the visible log tail +- System summary (`GET /status/sys`, shown on `/diagnostics` when `SHOW_DIAGNOSTICS=1`): + - Returns `{ version, uptime_seconds, server_time_utc, flags }` + - Shows resolved theme and stored preference; includes a Reset preference button. + +Compose: enable diagnostics/logs (optional) + +```yaml +services: + web: + environment: + - SHOW_LOGS=1 + - SHOW_DIAGNOSTICS=1 + # Random Modes (optional; alpha) + - RANDOM_MODES=1 + - RANDOM_UI=1 + - RANDOM_MAX_ATTEMPTS=5 + - RANDOM_TIMEOUT_MS=5000 + # RANDOM_BUILD_SUPPRESS_INITIAL_EXPORT: # (Default behavior auto-enables suppression; set to 0 to debug legacy double-export) +``` + +### Budget Mode and price/legal status +- Price and legality snippet integration is deferred to a dedicated Budget Mode initiative. This will centralize price sourcing, caching, legality checks, and UI surfaces. Track progress in `logs/roadmaps/roadmap_9_budget_mode.md`. + +## 🤖 Headless mode (no prompts) + +- Auto-headless: set `DECK_MODE=headless` + - Example (PowerShell): + ```powershell + docker compose run --rm -e DECK_MODE=headless mtg-deckbuilder + ``` +- Use a JSON config: mount `./config` to `/app/config` and set `DECK_CONFIG=/app/config/deck.json` + - Example (PowerShell): + ```powershell + docker compose run --rm ` + -e DECK_MODE=headless ` + -e DECK_CONFIG=/app/config/deck.json ` + mtg-deckbuilder + ``` +- Override via env vars (subset): `DECK_COMMANDER`, `DECK_PRIMARY_CHOICE`, `DECK_SECONDARY_CHOICE`, `DECK_TERTIARY_CHOICE`, `DECK_ADD_LANDS`, `DECK_FETCH_COUNT`, `DECK_BRACKET_LEVEL` + - Precedence: CLI > env > JSON > defaults + +### Enhanced CLI with Type Safety & Theme Names + +The headless runner now features comprehensive CLI arguments with type indicators and intelligent theme selection: + +```powershell +# Show all available options with type information +python code/headless_runner.py --help + +# Build with theme names instead of index numbers +python code/headless_runner.py ` + --commander "Aang, Airbending Master" ` + --primary-tag "Airbending" ` + --secondary-tag "Exile Matters" ` + --bracket-level 4 + +# Override ideal deck composition counts +``` + +## 🧪 Theme Catalog Validation & Schema + +The theme system now uses a merged catalog plus per-theme YAML files. + +Validator usage: + +```powershell +python code/scripts/validate_theme_catalog.py # Standard validation +python code/scripts/validate_theme_catalog.py --rebuild-pass # Ensures idempotent rebuild +python code/scripts/validate_theme_catalog.py --schema # Catalog JSON Schema +python code/scripts/validate_theme_catalog.py --yaml-schema # Per-file YAML JSON Schema +python code/scripts/validate_theme_catalog.py --strict-alias # Fail if alias display_name still present +``` + +CI runs the non-strict validator every push. A second step runs `--strict-alias` as an allowed failure until legacy alias YAMLs (e.g., `reanimator.yml`) are removed, after which the step will be enforced. + +Per-file YAML structure (excerpt): + +```yaml +id: plus1-plus1-counters +display_name: +1/+1 Counters +curated_synergies: [Proliferate, Counters Matter] +enforced_synergies: [] +inferred_synergies: [] +synergies: [Proliferate, Counters Matter, Adapt, Evolve] +primary_color: Green +secondary_color: White +notes: '' +``` + +Add or edit a theme by creating/updating a file in `config/themes/catalog/` and running the build script: + +```powershell +python code/scripts/build_theme_catalog.py +``` + +Metadata info (formerly provenance) is written to `theme_list.json` and consumed by tests and future UI diagnostics. During the deprecation period the builder accepts either key; prefer `metadata_info` in all new tooling / docs. + +### Duplicate suppression & example_cards quality controls + +Example card generation (and rebuild / padding via `synergy_promote_fill.py`) includes safeguards to reduce noisy staples and over-represented picks: + +- `--common-card-threshold `: Excludes candidate example cards that already appear in more than this fraction of themes (default 0.18 = 18%). This curbs ubiquitous staples (e.g., Sol Ring) from crowding lists. +- `--print-dup-metrics`: After a run (dry-run or apply) prints global frequency distribution so thresholds can be tuned. + +Typical loop (dry run with metrics): + +```powershell +python code/scripts/synergy_promote_fill.py --fill-example-cards --common-card-threshold 0.18 --print-dup-metrics +``` + +Adjustment guidance: +- If many mid-frequency utility cards still leak through: lower the threshold (e.g., 0.15). +- If high-value, legitimately thematic cards are being suppressed prematurely: raise slightly (e.g., 0.22) and re-run with metrics to confirm net effect. + +These controls complement existing staple suppression heuristics (`staples_block` list) and color-fallback restraint flags. + +## 📝 Theme Editorial Enrichment + +Phase D introduces automated editorial helpers to surface representative cards and synergy-informed commander suggestions per theme while preserving manual curation: + +### Key scripts +- `code/scripts/generate_theme_editorial_suggestions.py`: Scans card & commander CSVs to propose `example_cards`, `example_commanders`, and a derived `synergy_commanders` list. + - Synergy commanders: up to 3 / 2 / 1 picks from the theme's top three synergies (3/2/1 pattern), falling back to legendary card hits if a synergy lacks direct commander-tagged entries. + - Promotion: ensures a minimum (default 5) example commanders by promoting synergy picks with annotations. + - Annotation format: `Name - Synergy (Synergy Theme)` for both promoted examples and residual synergy-only entries. + - Duplicate filtering: commanders already present (base name) in `example_commanders` are omitted from `synergy_commanders` to avoid redundancy. + - Augmentation (`--augment-synergies`): heuristically pads sparse `synergies` arrays (e.g., injects `Counters Matter`, `Proliferate` for counter variants) before deriving synergy picks. +- `code/scripts/lint_theme_editorial.py`: Validates editorial consistency (annotation correctness, min counts, deduplication, size bounds) and emits warnings (non-fatal by default). + +### Usage examples +```powershell +# Dry run (first 25 themes) +python code/scripts/generate_theme_editorial_suggestions.py + +# Apply to all themes with augmentation and minimum example commanders of 5 +python code/scripts/generate_theme_editorial_suggestions.py --apply --augment-synergies --min-examples 5 + +# Lint results +python code/scripts/lint_theme_editorial.py +``` + +### Non-determinism & source control guidance +The editorial output depends on current CSV card data, ranks, and tagging heuristics. Regeneration on a different machine or after data refresh may yield a different ordering or composition. For that reason: +- Commit the tooling and schema changes. +- Prefer selective or manually reviewed YAML edits (especially cornerstone themes) instead of bulk committing the entire regenerated catalog. +- Treat full-catalog regeneration as an operational task (e.g., during a release preparation) rather than every feature branch. + +### Schema fields +Per-theme YAML now may include: +```yaml +example_cards: [ ... ] # Representative non-commander cards +example_commanders: [ ... ] # Curated + promoted synergy commanders (annotated) +synergy_commanders: [ ... ] # Remaining annotated synergy picks not already promoted +``` + +### Lint expectations (default) +- example_commanders: <=12, aim for ≥5 (warnings below minimum) +- example_cards: ≤20 +- synergy_commanders: ≤6 (3/2/1 pattern) after filtering duplicates +- Annotations must reference a synergy present in `synergies`. + +Cornerstone themes (Landfall, Reanimate, Superfriends, Tokens Matter, +1/+1 Counters) should always have curated coverage; automated promotion fills gaps but manual review is encouraged. +python code/headless_runner.py ` + --commander "Krenko, Mob Boss" ` + --primary-tag "Goblin Kindred" ` + --creature-count 35 ` + --land-count 33 ` + --ramp-count 12 + +# Include/exclude specific cards (semicolon for comma-containing names) +python code/headless_runner.py ` + --commander "Jace, Vryn's Prodigy" ` + --include-cards "Sol Ring;Jace, the Mind Sculptor" ` + --exclude-cards "Chaos Orb;Shahrazad" ` + --enforcement-mode strict +``` + +**New CLI Features:** +- **Type-safe help text**: All arguments show expected types (PATH, NAME, INT, BOOL) +- **Ideal count arguments**: `--ramp-count`, `--land-count`, `--creature-count`, etc. +- **Theme tag names**: `--primary-tag "Theme Name"` instead of `--primary-choice 1` +- **Include/exclude CLI**: `--include-cards`, `--exclude-cards` with semicolon support +- **Console diagnostics**: Detailed summary output for validation and results + +Headless submenu notes: +- If one JSON exists in `config/`, it auto-runs it +- If multiple exist, they’re listed as "Commander - Theme1, Theme2, Theme3"; `deck.json` shows as "Default" +- CSV/TXT are exported as usual; JSON run-config is exported only in interactive runs + +### Run locally (no Docker) +```powershell +# Show resolved settings (no run) +python code/headless_runner.py --config config/deck.json --dry-run + +# Run with a specific config file +python code/headless_runner.py --config config/deck.json + +# Point to a folder; if exactly one config exists, it's auto-used +python code/headless_runner.py --config config + +# Override via CLI +python code/headless_runner.py --commander "Pantlaza, Sun-Favored" --primary-choice 2 --secondary-choice 0 --add-lands true --fetch-count 3 +``` + +### CLI flag reference + +Each flag below mirrors an environment variable (env vars override JSON, CLI overrides both). Types match `--help` output. + +#### Core selection & flow + +| Flag | Type | Description | JSON key / Env var | +| --- | --- | --- | --- | +| `--config PATH` | string | Path to JSON config file or directory (auto-detects a single file) | `DECK_CONFIG` | +| `--commander NAME` | string | Commander search term | `commander` / `DECK_COMMANDER` | +| `--primary-choice INT` | int | Primary theme menu index (1-based) | `primary_choice` / `DECK_PRIMARY_CHOICE` | +| `--secondary-choice INT` | optional int | Secondary theme index | `secondary_choice` / `DECK_SECONDARY_CHOICE` | +| `--tertiary-choice INT` | optional int | Tertiary theme index | `tertiary_choice` / `DECK_TERTIARY_CHOICE` | +| `--primary-tag NAME` | string | Primary theme name (auto-maps to index) | `primary_tag` / `DECK_PRIMARY_TAG` | +| `--secondary-tag NAME` | string | Secondary theme name | `secondary_tag` / `DECK_SECONDARY_TAG` | +| `--tertiary-tag NAME` | string | Tertiary theme name | `tertiary_tag` / `DECK_TERTIARY_TAG` | +| `--bracket-level 1-5` | int | Power bracket selection | `bracket_level` / `DECK_BRACKET_LEVEL` | +| `--dry-run` | flag | Print the resolved configuration and exit | — | + +#### Deck composition & counts + +| Flag | Type | Description | JSON key / Env var | +| --- | --- | --- | --- | +| `--ramp-count INT` | int | Target ramp spells | `ideal_counts.ramp` | +| `--land-count INT` | int | Total land target | `ideal_counts.lands` | +| `--basic-land-count INT` | int | Minimum basic lands | `ideal_counts.basic_lands` | +| `--creature-count INT` | int | Creature count target | `ideal_counts.creatures` | +| `--removal-count INT` | int | Spot removal target | `ideal_counts.removal` | +| `--wipe-count INT` | int | Board wipe target | `ideal_counts.wipes` | +| `--card-advantage-count INT` | int | Card advantage target | `ideal_counts.card_advantage` | +| `--protection-count INT` | int | Protection spell target | `ideal_counts.protection` | +| `--fetch-count INT` | optional int | Requested fetch land count | `fetch_count` / `DECK_FETCH_COUNT` | +| `--dual-count INT` | optional int | Dual land request | `dual_count` / `DECK_DUAL_COUNT` | +| `--triple-count INT` | optional int | Three-color land request | `triple_count` / `DECK_TRIPLE_COUNT` | +| `--utility-count INT` | optional int | Utility land request | `utility_count` / `DECK_UTILITY_COUNT` | + +#### Card type toggles + +| Flag | Type | Description | JSON key / Env var | +| --- | --- | --- | --- | +| `--add-lands BOOL` | bool | Run the land builder sequence | `add_lands` / `DECK_ADD_LANDS` | +| `--add-creatures BOOL` | bool | Add creatures | `add_creatures` / `DECK_ADD_CREATURES` | +| `--add-non-creature-spells BOOL` | bool | Bulk add non-creature spells (if available) | `add_non_creature_spells` / `DECK_ADD_NON_CREATURE_SPELLS` | +| `--add-ramp BOOL` | bool | Add ramp spells | `add_ramp` / `DECK_ADD_RAMP` | +| `--add-removal BOOL` | bool | Add removal spells | `add_removal` / `DECK_ADD_REMOVAL` | +| `--add-wipes BOOL` | bool | Add board wipes | `add_wipes` / `DECK_ADD_WIPES` | +| `--add-card-advantage BOOL` | bool | Add card draw | `add_card_advantage` / `DECK_ADD_CARD_ADVANTAGE` | +| `--add-protection BOOL` | bool | Add the protection suite | `add_protection` / `DECK_ADD_PROTECTION` | + +#### Include / exclude controls + +| Flag | Type | Description | JSON key / Env var | +| --- | --- | --- | --- | +| `--include-cards LIST` | string | Force-include cards (comma or semicolon separated) | `include_cards` | +| `--exclude-cards LIST` | string | Exclude cards | `exclude_cards` | +| `--enforcement-mode warn|strict` | string | Handling when includes cannot be satisfied | `enforcement_mode` | +| `--allow-illegal BOOL` | bool | Permit non-legal cards in include/exclude lists | `allow_illegal` | +| `--fuzzy-matching BOOL` | bool | Enable fuzzy card name matching | `fuzzy_matching` | + +#### Random mode (web parity) + +| Flag | Type | Description | Env var / JSON key | +| --- | --- | --- | --- | +| `--random-mode` | flag | Force the headless random builder path | `HEADLESS_RANDOM_MODE` / `random_mode` | +| `--random-theme NAME` | string | Legacy single-theme alias (maps to primary) | `RANDOM_THEME` / `random.theme` | +| `--random-primary-theme NAME` | string | Primary theme slug | `RANDOM_PRIMARY_THEME` / `random.primary_theme` | +| `--random-secondary-theme NAME` | string | Secondary theme slug | `RANDOM_SECONDARY_THEME` / `random.secondary_theme` | +| `--random-tertiary-theme NAME` | string | Tertiary theme slug | `RANDOM_TERTIARY_THEME` / `random.tertiary_theme` | +| `--random-auto-fill BOOL` | bool | Auto-fill missing theme slots | `RANDOM_AUTO_FILL` / `random.auto_fill` | +| `--random-auto-fill-secondary BOOL` | bool | Override secondary auto-fill | `RANDOM_AUTO_FILL_SECONDARY` / `random.auto_fill_secondary` | +| `--random-auto-fill-tertiary BOOL` | bool | Override tertiary auto-fill | `RANDOM_AUTO_FILL_TERTIARY` / `random.auto_fill_tertiary` | +| `--random-strict-theme-match BOOL` | bool | Require exact theme matches when selecting commanders | `RANDOM_STRICT_THEME_MATCH` / `random.strict_theme_match` | +| `--random-attempts INT` | int | Retry attempts before giving up | `RANDOM_MAX_ATTEMPTS` / `random.attempts` | +| `--random-timeout-ms INT` | int | Timeout per attempt (milliseconds) | `RANDOM_TIMEOUT_MS` / `random.timeout_ms` | +| `--random-seed VALUE` | int or string | Deterministic seed for reproducible runs | `RANDOM_SEED` / `random.seed` | +| `--random-constraints JSON_OR_PATH` | string | Inline JSON or path to constraint file | `RANDOM_CONSTRAINTS` / `RANDOM_CONSTRAINTS_PATH` / `random.constraints` | +| `--random-output-json PATH` | string | Write random build payload to file or directory | `RANDOM_OUTPUT_JSON` / `random.output_json` | + +Booleans accept: 1/0, true/false, yes/no, on/off. + +### Random mode parity (web → headless) + +The headless runner now shares the web UI's random builder pipeline. Set `--random-mode` (or `HEADLESS_RANDOM_MODE=1`) to route through the Surprise/Reroll flow with multi-theme support, auto-fill assistance, constraints, and deterministic seeds. Combine flags as needed: + +```powershell +python code/headless_runner.py ` + --random-mode ` + --random-primary-theme Artifacts ` + --random-secondary-theme Tokens ` + --random-auto-fill true ` + --random-output-json deck_files/random/latest.json +``` + +The build prints a full summary (commander, seed, theme stack, fallback reasons, attempts vs timeout, auto-fill status) and writes the optional JSON payload when `--random-output-json`/`RANDOM_OUTPUT_JSON` is provided. Use `--random-constraints` (or `RANDOM_CONSTRAINTS[_PATH]`) to supply pool limits, and `--random-seed` for reproducible reruns. Pair any random flag with `--dry-run` to inspect the resolved configuration without building. + +### JSON export in headless +- By default, headless runs do not export a JSON run-config to avoid duplicates. +- Opt-in with: + ```powershell + $env:HEADLESS_EXPORT_JSON = "1" + python code/headless_runner.py --config config/deck.json + ``` +- Tip: when opting in, prefer using `--config` instead of a `DECK_CONFIG` file path to avoid creating both a stem-based JSON and a second explicit-path JSON. + +Example JSON (`config/deck.json`): +```json +{ + "commander": "Pantlaza", + "bracket_level": 3, + "primary_choice": 2, + "secondary_choice": 2, + "tertiary_choice": 2, + "tag_mode": "OR", // OR or AND; Web UI default is OR + "add_lands": true, + "fetch_count": 3, + "ideal_counts": { "ramp": 10, "lands": 36, "basic_lands": 16, "creatures": 28, "removal": 8, "wipes": 3, "card_advantage": 8, "protection": 3 } +} +``` +Notes: headless honors `ideal_counts` (leave prompts blank to accept). Only `fetch_count` is tracked/exported for lands. + +#### JSON fields for Combos (Web Configs) +When running from a JSON config in the Web UI, the following fields are supported and exported from interactive runs: +```json +{ + "prefer_combos": true, + "combo_target_count": 3, + "combo_balance": "mix" +} +``` +Auto-Complete Combos runs before theme fill/monolithic spells when `prefer_combos` is true. + +## 🔒 Bracket policy & enforcement + +- Policy source: `config/brackets.yml` defines per-bracket limits for categories like `game_changers`, `extra_turns`, `mass_land_denial`, `tutors_nonland`, and `two_card_combos`. +- Authoritative lists: JSON under `config/card_lists/` provides names for enforcement and reporting (`game_changers.json`, `extra_turns.json`, `mass_land_denial.json`, `tutors_nonland.json`). A `list_version` may be included for badges. +- Global safety prune: when a category has a limit of 0, the builder removes matching cards from the card pool up-front so they cannot be selected (Game Changers by name; others by tags when present). This runs in both Web and headless builds. +- Preemptive filters: spells and creatures phases also apply bracket-aware pre-filters. +- Inline enforcement (Web): if violations still occur, Step 5 shows them before the summary. You must replace or remove flagged cards before you can Continue or Rerun. Alternatives are role-consistent, exclude the replaced/in-deck/locked/commander cards, and bias toward owned when enabled. +- Game Changer fallback: if `config/card_lists/game_changers.json` is empty, enforcement and reporting fall back to the in-code `builder_constants.GAME_CHANGERS` list so low-bracket decks still exclude those cards. +- Auto-enforce (optional): set `WEB_AUTO_ENFORCE=1` to automatically apply the enforcement plan after build and re-export CSV/TXT when violations are detected. + +Status levels +- PASS: within limits; panel remains collapsed by default. +- WARN: advisory thresholds met (from `_warn` keys in `config/brackets.yml` or conservative defaults for low brackets on tutors/extra turns). Panel opens automatically and shows WARN tiles with a subtle amber border and badge; no gating or automatic enforcement. +- FAIL: over hard limits or disallowed categories for the selected bracket. Panel opens automatically, tiles show with red accents, and Continue/Rerun are disabled until resolved; enforcement actions are available. + +Compliance report +- Every run writes `[stem]_compliance.json` next to your deck exports, including per-category counts/limits, status (PASS/FAIL), detected cheap/early two-card combos, and list versions. +- Headless JSON builds and Web builds use the same engine; summary and exports are consistent. + +## 🧩 Theme combine mode (AND/OR) + +- OR (default): Recommend cards that match any selected themes, prioritizing overlap. +- AND: Prioritize multi-theme intersections. For creatures, an AND pre-pass first picks "all selected themes" creatures up to a cap, then fills by weighted overlap. + +UI tips: +- Step 2 includes AND/OR radios with a tooltip explaining trade-offs. +- In staged build view, "Creatures: All-Theme" shows which selected themes each card hits. + +## 🕹️ Usage (interactive) +1) Start the app (Docker or from source) +2) Pick Build a New Deck +3) Search/confirm commander +4) Pick primary/secondary/tertiary themes (or stop at primary); choose AND/OR combine mode +5) Choose power bracket and review ideal counts +6) Deck builds; CSV/TXT export to `deck_files/` + +## ⚙️ Environment variables (common) +- DECK_MODE=headless +- DECK_CONFIG=/app/config/deck.json +- DECK_COMMANDER, DECK_PRIMARY_CHOICE, DECK_SECONDARY_CHOICE, DECK_TERTIARY_CHOICE +- DECK_ADD_LANDS, DECK_FETCH_COUNT +- DECK_ADD_CREATURES, DECK_ADD_NON_CREATURE_SPELLS, DECK_ADD_RAMP, DECK_ADD_REMOVAL, DECK_ADD_WIPES, DECK_ADD_CARD_ADVANTAGE, DECK_ADD_PROTECTION +- DECK_BRACKET_LEVEL +- **ALLOW_MUST_HAVES=true** (enables include/exclude card lists feature with enhanced UI validation) +- Random Modes (alpha): core toggles `RANDOM_MODES=1`, `RANDOM_UI=1`, `RANDOM_MAX_ATTEMPTS=5`, `RANDOM_TIMEOUT_MS=5000` (see detailed list below for theme/auto-fill/seed overrides) + - `RANDOM_BUILD_SUPPRESS_INITIAL_EXPORT=0` — (optional) disable the default suppression of builder auto-export (will produce legacy duplicate artifacts; for debugging only) + - Testing/data overrides: + - `CSV_FILES_DIR=csv_files/testdata` — point the app/tests at a frozen dataset snapshot for determinism + +Random Modes (alpha flags): +- `RANDOM_MODES=1` — enable random build endpoints and backend features +- `RANDOM_UI=1` — show Surprise/Reroll/Share controls in UI +- `RANDOM_MAX_ATTEMPTS=5` — cap retry attempts for random generation +- `RANDOM_TIMEOUT_MS=5000` — per-build timeout in milliseconds +- `RANDOM_THEME=`*slug* — legacy single-theme alias (maps to primary when others unset) +- `RANDOM_PRIMARY_THEME=`*slug* — primary theme override +- `RANDOM_SECONDARY_THEME=`*slug* — secondary theme override +- `RANDOM_TERTIARY_THEME=`*slug* — tertiary theme override +- `RANDOM_AUTO_FILL=0|1` — auto-fill missing secondary/tertiary slots when enabled +- `RANDOM_AUTO_FILL_SECONDARY=0|1` / `RANDOM_AUTO_FILL_TERTIARY=0|1` — explicit per-slot overrides +- `RANDOM_STRICT_THEME_MATCH=0|1` — require exact theme matches when rolling commanders +- `RANDOM_CONSTRAINTS=`*inline JSON* and/or `RANDOM_CONSTRAINTS_PATH=/app/config/random_constraints.json` +- `RANDOM_SEED=`*value* — deterministic run seed (int or string) +- `RANDOM_OUTPUT_JSON=/app/deck_files/random_build.json` — write random build payload to file/dir +- `HEADLESS_RANDOM_MODE=1` — force the headless runner to take the random pipeline +- `RANDOM_BUILD_SUPPRESS_INITIAL_EXPORT=0` — (optional) disable the default suppression of builder auto-export (useful for debugging legacy duplicate exports) + +Testing/determinism helpers: +- CSV_FILES_DIR=csv_files/testdata — override CSV base dir for tests or pinned snapshots + +Optional name-based tag overrides (mapped to indices for the chosen commander): +- DECK_PRIMARY_TAG, DECK_SECONDARY_TAG, DECK_TERTIARY_TAG + +Combine mode in headless: +- JSON: set `"tag_mode": "AND" | "OR"` +- Env var: `DECK_TAG_MODE=AND|OR` (if configured in your environment) + +Web UI performance tuning: +- WEB_TAG_PARALLEL=1|0 +- WEB_TAG_WORKERS= + - WEB_VIRTUALIZE=1 (enable list virtualization) + - SHOW_DIAGNOSTICS=1 (optional: diagnostics tools and virtualization overlay toggle using `v`) + - WEB_AUTO_ENFORCE=1 (optional; after building, auto-apply enforcement and re-export when the compliance report fails) + - SPLASH_ADAPTIVE=1 (optional: enable adaptive off-color penalty by commander color count) + - SPLASH_ADAPTIVE_SCALE="1:1.0,2:1.0,3:1.0,4:0.6,5:0.35" (optional: tuning for adaptive scaling) + +Paths and data overrides: +- `DECK_EXPORTS=/app/deck_files` — where finished deck exports are read by the Web UI +- `OWNED_CARDS_DIR=/app/owned_cards` — preferred directory for owned inventory uploads +- `CARD_LIBRARY_DIR=/app/owned_cards` — back‑compat alias for OWNED_CARDS_DIR +- `DECK_CONFIG=/app/config/deck.json` — default headless config path +- `CSV_FILES_DIR=/app/csv_files` — override CSV base dir (tests, snapshots, or alternate datasets) +- `CARD_INDEX_EXTRA_CSV=path/to/synthetic_cards.csv` — inject an extra CSV into the card index for experiments/tests + +Theme catalog scan & prewarm: +- `THEME_CATALOG_YAML_SCAN_INTERVAL_SEC=2.0` — poll for YAML changes (dev); omit in production +- `WEB_THEME_FILTER_PREWARM=1` — prewarm common filters for faster first renders + +Theme preview cache & Redis (optional): +- `THEME_PREVIEW_CACHE_MAX=400` — max previews cached in memory +- `WEB_THEME_PREVIEW_LOG=0` — 1=verbose cache logs +- `THEME_PREVIEW_ADAPTIVE=0` — 1=adaptive cache policy +- `THEME_PREVIEW_EVICT_COST_THRESHOLDS=5,15,40` — cost tiers for eviction policy +- `THEME_PREVIEW_BG_REFRESH=0` — 1=background refresh worker; `THEME_PREVIEW_BG_REFRESH_INTERVAL=120` seconds +- TTL tuning: `THEME_PREVIEW_TTL_BASE=300`, `THEME_PREVIEW_TTL_MIN=60`, `THEME_PREVIEW_TTL_MAX=900`, `THEME_PREVIEW_TTL_BANDS=0.2,0.5,0.8`, `THEME_PREVIEW_TTL_STEPS=2,4,2,3,1` +- Redis: `THEME_PREVIEW_REDIS_URL=redis://localhost:6379/0`, `THEME_PREVIEW_REDIS_DISABLE=0` (1=disable redis even if URL set) + +Rarity weighting and diversity (advanced): +- `RARITY_W_MYTHIC=1.2`, `RARITY_W_RARE=0.9`, `RARITY_W_UNCOMMON=0.65`, `RARITY_W_COMMON=0.4` — diminish or boost rarity selection +- `RARITY_DIVERSITY_TARGETS="mythic:0-1,rare:0-2,uncommon:0-4,common:0-6"` — soft targets per rarity band +- `RARITY_DIVERSITY_OVER_PENALTY=-0.5` — penalty when exceeding targets + +Misc utility land tuning (Step 7): +- MISC_LAND_DEBUG=1 Write debug CSVs for misc land step (candidates/post-filter). Otherwise suppressed unless SHOW_DIAGNOSTICS=1. +- MISC_LAND_EDHREC_KEEP_PERCENT_MIN=0.75 Lower bound of dynamic EDHREC keep range (0–1). When MIN & MAX are both set, a random % in [MIN,MAX] is rolled per build. +- MISC_LAND_EDHREC_KEEP_PERCENT_MAX=1.0 Upper bound of dynamic EDHREC keep range (0–1). +- MISC_LAND_EDHREC_KEEP_PERCENT=0.80 Legacy fixed keep % used only if MIN/MAX not both present. +- MISC_LAND_THEME_MATCH_BASE=1.4 Base multiplier when at least one selected theme tag matches a candidate land. +- MISC_LAND_THEME_MATCH_PER_EXTRA=0.15 Incremental multiplier per additional matching theme tag beyond the first. +- MISC_LAND_THEME_MATCH_CAP=2.0 Cap for total theme multiplier after stacking base + extras. + +Notes: +- Fetch lands are fully excluded from the misc step (handled earlier). +- Mono-color decks auto-filter broad rainbow/any-color lands except an explicit always-keep list. +- Land Alternatives endpoint: for land seeds, returns land-only suggestions; 12 random picks each request from a randomly sized window within the top 60–100 ranked candidates (per-card, uncached) for variety. + +## 📁 Folders +- `deck_files/` — CSV/TXT exports +- `csv_files/` — card data +- `logs/` — logs +- `config/` — JSON configs (optional) +- `owned_cards/` — your owned cards lists (`.txt`/`.csv`); used for owned-only builds and Owned flagging + +## 🧰 Troubleshooting +- Use `docker compose run --rm` (not `up`) for interactive sessions +- Ensure volumes are mounted so files persist (`deck_files`, `logs`, `csv_files`) +- For headless with config, mount `./config:/app/config` and set `DECK_CONFIG` +- Card data refresh: if `csv_files/cards.csv` is missing or older than 7 days, the app refreshes data and re-tags automatically. A `.tagging_complete.json` file in `csv_files/` indicates tagging completion. + - Web health: the header dot polls `/healthz` — green is OK, red is degraded. If red, check logs. + - Error details: HTMX errors show a toast with a “Copy details” button including X-Request-ID; include that when filing issues. + - Logs page: set `SHOW_LOGS=1` to enable `/logs` and `/status/logs?tail=200` (read‑only) for quick diagnostics. + - Diagnostics page: set `SHOW_DIAGNOSTICS=1` to enable the nav link and `/diagnostics` test tools. + +Data integrity notes: +- Banned cards: per-color/guild CSVs now consistently respect the Commander banned list using exact, case‑insensitive name matching across `name` and `faceName`. + +## 🧪 Development and tests + +Set up a virtual environment, install dependencies, and run the test suite. + +```powershell +# From repo root (PowerShell) +python -m venv .venv +& ".venv/Scripts/Activate.ps1" +pip install -r requirements.txt +pip install -r requirements-dev.txt + +# Run tests (pytest is configured to look under code/tests) +python -m pytest -q +``` + +Notes: +- Test discovery is set to `code/tests` in `pytest.ini`. +- `pytest.ini` disables the built-in debugging plugin to avoid a stdlib module name clash with the project folder `code/`. +- Feature flags for local diagnostics: set `SHOW_DIAGNOSTICS=1` and/or `SHOW_LOGS=1` to expose `/diagnostics` and `/logs`. + - Index testing helper: set `CARD_INDEX_EXTRA_CSV=path\to\synthetic_cards.csv` to inject a temporary CSV during tests (e.g., color identity edge cases) without altering production shards. + +### What’s new (quick summary) +- Faster browsing with optional virtualized grids/lists in Step 5 and Owned (`WEB_VIRTUALIZE=1`). +- Image polish: lazy-loading, responsive `srcset/sizes`, and LQIP blur/fade for Step 5, Owned, and commander preview. +- Diagnostics overlay (opt-in with `SHOW_DIAGNOSTICS=1`): press `v` to see visible range, totals, render time, and counters. +- Accessibility: respects reduced-motion (disables blur/fade and smooth scrolling). +- Small caching wins: short‑TTL fragment caching for summary partials and suggestions. + +## 📦 Releases +- Release notes are maintained in `RELEASE_NOTES_TEMPLATE.md`. Automated workflows read from this file to populate Docker Hub and GitHub releases. + +### Collecting diagnostics for an issue +- Note the Request-ID from the toast or error page. +- Copy logs from `/logs` (enable with `SHOW_LOGS=1`) or `/status/logs?tail=200`. +- Include your `/healthz` JSON and environment flags (SHOW_LOGS/SHOW_DIAGNOSTICS/SHOW_SETUP) when reporting. + +## Performance Baselines + +A warm preview performance baseline is committed at `logs/perf/theme_preview_warm_baseline.json`. + +Tooling: +- Benchmark: `python -m code.scripts.preview_perf_benchmark --all --passes 2 --extract-warm-baseline logs/perf/theme_preview_warm_baseline.json` +- Compare (manual): `python -m code.scripts.preview_perf_compare --baseline logs/perf/theme_preview_warm_baseline.json --candidate logs/perf/new_run.json --warm-only --p95-threshold 5` +- Regression helper: `python -m code.scripts.preview_perf_ci_check --baseline logs/perf/theme_preview_warm_baseline.json --p95-threshold 5` (run locally or in ad-hoc CI when desired) + +Policy: +- Treat warm-only p95 regressions beyond 5% as blockers even though enforcement is now manual. +- Update the baseline only after intentional performance improvements or systemic environment shifts. + +Workflow to refresh baseline: +1. Ensure no active regressions (run wrapper locally, expect pass). +2. Run a multi-pass benchmark with at least two passes over all themes. +3. Replace `theme_preview_warm_baseline.json` with new warm pass output; commit in same PR with rationale in CHANGELOG. + +Optional future enhancements: +- Reintroduce automated gating once preview perf stabilizes on shared runners. +- Separate p50 threshold (e.g., >7% regression) for early warning without failing build. diff --git a/docs/headless_cli_guide.md b/docs/headless_cli_guide.md new file mode 100644 index 0000000..a3a6641 --- /dev/null +++ b/docs/headless_cli_guide.md @@ -0,0 +1,91 @@ +# Headless & CLI Guide + +Leverage the shared deckbuilding engine from the command line, in headless mode, or within containers. + +## Table of contents +- [Entry points](#entry-points) +- [Switching modes in Docker](#switching-modes-in-docker) +- [Headless JSON configs](#headless-json-configs) +- [Environment overrides](#environment-overrides) +- [CLI argument reference](#cli-argument-reference) +- [Include/exclude lists from the CLI](#includeexclude-lists-from-the-cli) +- [Practical examples](#practical-examples) + +--- + +## Entry points +- Interactive menu: `python code/main.py` +- Headless runner: `python code/headless_runner.py --config config/deck.json` +- Both executables share the same builder core used by the Web UI. + +## Switching modes in Docker +Override the container entrypoint to run the CLI or headless flows inside Docker Compose or plain `docker run`. + +```powershell +# Compose example +docker compose run --rm -e APP_MODE=cli web + +# Compose with headless automation +docker compose run --rm ` + -e APP_MODE=cli ` + -e DECK_MODE=headless ` + -e DECK_CONFIG=/app/config/deck.json ` + web +``` + +Set `APP_MODE=cli` to switch from the Web UI to the textual interface. Add `DECK_MODE=headless` to skip prompts and immediately run the configured deck. + +## Headless JSON configs +- Drop JSON files into `config/` (e.g., `config/deck.json`). +- Headless mode auto-runs the lone JSON file; if multiple exist, the CLI lists them with summaries (commander + themes). +- Config fields cover commander, bracket, include/exclude lists, theme preferences, owned-mode toggles, and output naming. + +## Environment overrides +When running in containers or automation, environment variables can override JSON settings. Typical variables include: +- `DECK_COMMANDER` +- `DECK_PRIMARY_CHOICE`, `DECK_SECONDARY_CHOICE`, `DECK_TERTIARY_CHOICE` +- `DECK_BRACKET_LEVEL` +- `DECK_ADD_LANDS`, `DECK_LAND_COUNT`, `DECK_CREATURE_COUNT`, `DECK_RAMP_COUNT` + +Precedence order: **CLI flags > environment variables > JSON config > defaults**. + +## CLI argument reference +Run `python code/headless_runner.py --help` to see the current argument surface. Highlights: + +- Type indicators make expectations explicit (e.g., `PATH`, `NAME`, `INT`). +- Theme selection accepts human-readable names: `--primary-tag "Airbending"` instead of numeric indexes. +- Bracket selection via `--bracket-level`. +- Ideal counts such as `--land-count`, `--ramp-count`, `--creature-count`, and more. + +## Include/exclude lists from the CLI +You can specify comma- or semicolon-separated lists directly through the CLI: + +```powershell +python code/headless_runner.py ` + --commander "Jace, Vryn's Prodigy" ` + --include-cards "Sol Ring;Jace, the Mind Sculptor" ` + --exclude-cards "Chaos Orb;Shahrazad" ` + --enforcement-mode strict +``` + +Semicolons allow card names containing commas. Enforcement modes mirror the Web UI (`off`, `warn`, `strict`). + +## Practical examples +```powershell +# Build a Goblins list with tuned counts +python code/headless_runner.py ` + --commander "Krenko, Mob Boss" ` + --primary-tag "Goblin Kindred" ` + --creature-count 35 ` + --land-count 33 ` + --ramp-count 12 + +# Fire a headless run via Docker using an alternate config folder +docker compose run --rm ` + -e APP_MODE=cli ` + -e DECK_MODE=headless ` + -e DECK_CONFIG=/app/config/custom_decks ` + web +``` + +The CLI prints a detailed summary at the end of each run, including enforcement results, resolved themes, and export paths. All artifacts land in the same `deck_files/` folder used by the Web UI. diff --git a/docs/theme_catalog_advanced.md b/docs/theme_catalog_advanced.md new file mode 100644 index 0000000..2d74cee --- /dev/null +++ b/docs/theme_catalog_advanced.md @@ -0,0 +1,148 @@ +# Theme Catalog Advanced Guide + +Additional details for developers and power users working with the theme catalog, editorial tooling, and diagnostics. + +## Table of contents +- [HTMX API endpoints](#htmx-api-endpoints) +- [Caching, diagnostics, and metrics](#caching-diagnostics-and-metrics) +- [Governance principles](#governance-principles) +- [Operational tooling](#operational-tooling) + - [Refreshing catalogs](#refreshing-catalogs) + - [Snapshotting taxonomy](#snapshotting-taxonomy) + - [Adaptive splash penalty experiments](#adaptive-splash-penalty-experiments) +- [Editorial pipeline](#editorial-pipeline) + - [Script summary](#script-summary) + - [Example configuration](#example-configuration) + - [Duplicate suppression controls](#duplicate-suppression-controls) + - [Coverage metrics and KPIs](#coverage-metrics-and-kpis) + - [Description mapping overrides](#description-mapping-overrides) +- [Validation and schema tooling](#validation-and-schema-tooling) + +--- + +## HTMX API endpoints +The upcoming theme picker UI is powered by two FastAPI endpoints. + +### `GET /themes/api/themes` +Parameters: +- `q`: substring search across theme names and synergies. +- `archetype`: filter by `deck_archetype`. +- `bucket`: popularity bucket (Very Common, Common, Uncommon, Niche, Rare). +- `colors`: comma-separated color initials (e.g. `G,W`). +- `limit` / `offset`: pagination (limit defaults to 50, max 200). +- `diagnostics=1`: surfaces `has_fallback_description` and `editorial_quality` (requires `WEB_THEME_PICKER_DIAGNOSTICS=1`). + +The response includes `count`, the filtered `items`, and `next_offset` for subsequent requests. Diagnostic mode adds extra telemetry fields. + +### `GET /themes/api/theme/{id}` +Parameters: +- `uncapped=1`: (diagnostics) returns `uncapped_synergies`, combining curated, enforced, and inferred sets. +- `diagnostics=1`: exposes editorial metadata such as `editorial_quality` and `has_fallback_description`. + +The payload merges curated data with editorial artifacts (`example_cards`, `example_commanders`, etc.) and respects the same diagnostic feature flag. + +## Caching, diagnostics, and metrics +- Responses include an `ETag` header derived from catalog metadata so consumers can perform conditional GETs. +- `/themes/status` reports freshness and stale indicators; `/themes/refresh` (POST) triggers a background rebuild. +- When `WEB_THEME_PICKER_DIAGNOSTICS=1` is set, the app records: + - Filter cache hits/misses and duration (`X-ThemeCatalog-Filter-Duration-ms`). + - Preview cache metrics (`/themes/metrics` exposes counts, hit rates, TTL, and average build time). +- Skeleton loaders ship with the HTMX fragments to keep perceived latency low. + +## Governance principles +To keep the catalog healthy, the project follows a lightweight governance checklist: + +1. **Minimum examples** – target at least two example cards and one commander per established theme. +2. **Deterministic preview assembly** – curated examples first, then role-based samples (payoff/enabler/support/wildcard), then placeholders if needed. +3. **Splash relax policy** – four- and five-color commanders may include a single off-color enabler with a small penalty, preventing over-pruning. +4. **Popularity buckets are advisory** – they guide filters and UI hints but never directly influence scoring. +5. **Taxonomy expansion bar** – new high-level archetypes require a distinct pattern, at least eight representative cards, and no overlap with existing themes. +6. **Editorial quality tiers** – optional `editorial_quality: draft|reviewed|final` helps prioritize review passes. +7. **Deterministic sampling** – seeds derive from `theme|commander` hashes; scoring code should emit `reasons[]` to explain decisions and remain regression-test friendly. + +See `docs/theme_taxonomy_rationale.md` for the underlying rationale and roadmap. + +## Operational tooling + +### Refreshing catalogs +- Primary builder: `python code/scripts/build_theme_catalog.py` +- Options: + - `--limit N`: preview a subset without overwriting canonical outputs (unless `--allow-limit-write`). + - `--output path`: write to an alternate path; suppresses YAML backfill to avoid mutating tracked files. + - `--backfill-yaml` or `EDITORIAL_BACKFILL_YAML=1`: fill missing descriptions and popularity buckets in YAML files. + - `--force-backfill-yaml`: overwrite existing description/popularity fields. + - `EDITORIAL_SEED=`: force a deterministic ordering when heuristics use randomness. + - `EDITORIAL_AGGRESSIVE_FILL=1`: pad sparse themes with inferred synergies. + - `EDITORIAL_POP_BOUNDARIES="a,b,c,d"`: tune popularity thresholds. + - `EDITORIAL_POP_EXPORT=1`: emit `theme_popularity_metrics.json` summaries. + +### Snapshotting taxonomy +`python -m code.scripts.snapshot_taxonomy` writes `logs/taxonomy_snapshots/taxonomy_.json` with a SHA-256 hash. Identical content is skipped unless you supply `--force`. Use snapshots before experimenting with taxonomy-aware sampling. + +### Adaptive splash penalty experiments +Set `SPLASH_ADAPTIVE=1` to scale off-color enabler penalties based on commander color count. Tune with `SPLASH_ADAPTIVE_SCALE` (e.g. `1:1.0,2:1.0,3:1.0,4:0.6,5:0.35`). Analytics aggregate both static and adaptive reasons for comparison. + +## Editorial pipeline + +### Script summary +- `code/scripts/generate_theme_editorial_suggestions.py` + - Proposes `example_cards`, `example_commanders`, and `synergy_commanders` using card CSVs and tagging heuristics. + - `--augment-synergies` can pad sparse `synergies` arrays prior to suggestion. + - `--apply` writes results; dry runs print suggestions for review. +- `code/scripts/lint_theme_editorial.py` + - Validates annotation formats, min/max counts, and deduplication. Combine with environment toggles (`EDITORIAL_REQUIRE_DESCRIPTION`, `EDITORIAL_REQUIRE_POPULARITY`) for stricter gating. + +### Example configuration +```powershell +# Dry run on the first 25 themes +python code/scripts/generate_theme_editorial_suggestions.py + +# Apply across the catalog with augmentation and min example commanders set to 5 +python code/scripts/generate_theme_editorial_suggestions.py --apply --augment-synergies --min-examples 5 + +# Lint results +python code/scripts/lint_theme_editorial.py +``` + +Editorial output depends on current CSV data. Expect ordering or composition changes after upstream dataset refreshes—treat full-catalog regeneration as an operational task and review diffs carefully. + +### Duplicate suppression controls +`code/scripts/synergy_promote_fill.py` can rebalance example cards: + +```powershell +python code/scripts/synergy_promote_fill.py --fill-example-cards --common-card-threshold 0.18 --print-dup-metrics +``` + +- `--common-card-threshold`: filters cards appearing in more than the specified fraction of themes (default `0.18`). +- Use metrics output to tune thresholds so staple utility cards stay in check without removing legitimate thematic cards. + +### Coverage metrics and KPIs +- `EDITORIAL_INCLUDE_FALLBACK_SUMMARY=1` embeds a `description_fallback_summary` block in the generated catalog (`generic_total`, `generic_plain`, `generic_pct`, etc.). +- Regression tests use these metrics to ratchet down generic descriptions over time. +- Historical trends are appended to `config/themes/description_fallback_history.jsonl` for analysis. + +### Description mapping overrides +Customize automatic descriptions without editing code: + +- Add `config/themes/description_mapping.yml` with entries: + ```yaml + - triggers: ["sacrifice", "aristocrat"] + description: "Leans on sacrifice loops and {SYNERGIES}." + ``` +- The first matching trigger wins (case-insensitive substring search). +- `{SYNERGIES}` expands to a short clause listing the top synergies when available, and disappears gracefully if not. +- Internal defaults remain as fallbacks when the mapping file is absent. + +## Validation and schema tooling + +Run validators to maintain catalog quality: + +```powershell +python code/scripts/validate_theme_catalog.py +python code/scripts/validate_theme_catalog.py --rebuild-pass +python code/scripts/validate_theme_catalog.py --schema +python code/scripts/validate_theme_catalog.py --yaml-schema +python code/scripts/validate_theme_catalog.py --strict-alias +``` + +Per-theme YAML files (under `config/themes/catalog/`) are tracked in source control. Keys such as `metadata_info` replace the legacy `provenance`; the validator treats missing migrations as warnings until the deprecation completes. diff --git a/docs/web_ui_deep_dive.md b/docs/web_ui_deep_dive.md new file mode 100644 index 0000000..051b4f3 --- /dev/null +++ b/docs/web_ui_deep_dive.md @@ -0,0 +1,103 @@ +# Web UI Deep Dive + +A closer look at the rich interactions available in the MTG Python Deckbuilder Web UI. Use this guide after you are comfortable with the basic homepage flows described in the README. + +## Table of contents +- [Unified New Deck modal](#unified-new-deck-modal) +- [Stage 5 tools: lock, replace, compare, permalinks](#stage-5-tools-lock-replace-compare-permalinks) +- [Multi-copy archetype packages](#multi-copy-archetype-packages) +- [Bracket compliance and skipped stages](#bracket-compliance-and-skipped-stages) +- [Build options: owned-only and prefer-owned](#build-options-owned-only-and-prefer-owned) +- [Visual summaries](#visual-summaries) +- [Combos & synergies](#combos--synergies) +- [Owned library page](#owned-library-page) +- [Finished decks workspace](#finished-decks-workspace) +- [Keyboard shortcuts](#keyboard-shortcuts) +- [Virtualization, tagging, and performance](#virtualization-tagging-and-performance) +- [Diagnostics and logs](#diagnostics-and-logs) + +--- + +## Unified New Deck modal +The first three steps of deckbuilding live inside a single modal: + +1. **Search for a commander** – autocomplete prioritizes color identity matches; press Enter to grab the top result. +2. **Pick primary/secondary/tertiary themes** – the modal displays your selections in order so you can revisit them quickly. +3. **Choose a bracket** – labels such as “Bracket 3: Upgraded” clarify power bands. Bracket 3 is the default tier for new builds. + +Optional inputs: +- **Deck name** becomes the export filename stem and is reused in Finished Decks banners. +- **Combo auto-complete** and other preferences persist between runs. + +Once you submit, the modal closes and the build starts immediately—no extra confirmation screen. + +## Stage 5 tools: lock, replace, compare, permalinks +Stage 5 is the iterative workspace for tuning the deck: + +- **Lock** a card by clicking the padlock or the card artwork. Locked cards persist across rerolls and show a “Last action” chip for quick confirmation. +- **Replace** opens the Alternatives drawer. Filters include Owned-only, role alignment, and bracket compliance. The system skips commanders, locked cards, just-added cards, and anything already in the list. +- **Permalink** buttons appear in Stage 5 and Finished Decks. Share a build (commander, themes, bracket, ideals, flags) or restore one by pasting a permalink back into the app. +- **Compare** mode lives in Finished Decks. Pick two builds (quick actions select the latest pair) and triage changes via Changed-only, Copy summary, or download the diff as TXT. + +## Multi-copy archetype packages +When a commander + theme combination suggests a multi-copy strategy (e.g., Persistent Petitioners, Shadowborn Apostles), the UI offers an optional package: + +- Choose the desired quantity (bounded by printed limits) and optionally add **Thrumming Stone** when it synergizes. +- Packages are inserted before other stages so target counts adjust appropriately. +- A safety clamp trims overflow to keep the deck at 100 cards; the stage displays a “Clamped N” indicator if it triggers. +- You can dismiss the modal, and we won’t re-prompt unless your selections change. + +## Bracket compliance and skipped stages +- Bracket policy enforcement prunes disallowed categories before stage execution. Violations block reruns until you resolve them. +- Enforcement options: keep the panel collapsed when compliant, auto-open with a colored status chip (green/amber/red) when action is needed. +- Enable auto-enforcement by setting `WEB_AUTO_ENFORCE=1`. +- Toggle **Show skipped stages** to surface steps that added zero cards, making it easier to review the full pipeline. + +## Build options: owned-only and prefer-owned +The modal includes toggles for **Use only owned cards** and **Prefer owned cards**: + +- Owned-only builds pull strictly from the inventory in `owned_cards/` (commander exempt). +- Prefer-owned bumps owned cards slightly in the scoring pipeline but still allows unowned all-stars when necessary. +- Both modes respect the Owned Library filters and show Owned badges in the exported CSV (including the `Owned` column when you disable the mode). + +## Visual summaries +Stage 5 displays multiple data visualizations that cross-link to the card list: + +- **Mana curve** – hover a bar to highlight matching cards in list and thumbnail views. +- **Color requirements vs. sources** – pips show requirements; sources include non-land producers and an optional `C` (colorless) toggle. +- **Tooltips** – each tooltip lists contributing cards and offers a copy-to-clipboard action. +- Visual polish includes lazy-loaded thumbnails, blur-up transitions, and accessibility tweaks that respect `prefers-reduced-motion`. + +## Combos & synergies +The builder detects curated two-card combos and synergy pairs in the final deck: + +- Chips display badges such as “cheap” or “setup” with hover previews for each card and a split preview when hovering the entire row. +- Enable **Auto-complete combos** to add missing partners before theme filling. Configure target count, balance (early/late/mix), and preference weighting. +- Color identity restrictions keep the algorithm from suggesting off-color partners. + +## Owned library page +Open the Owned tile to manage uploaded inventories: + +- Upload `.txt` or `.csv` files with one card per line. The app enriches and deduplicates entries on ingestion. +- The page includes sortable columns, exact color-identity filters (including four-color combos), and an export button. +- Large collections benefit from virtualization when `WEB_VIRTUALIZE=1`. + +## Finished decks workspace +- Browse historical builds with filterable theme chips. +- Each deck offers Download TXT, Copy summary, Open permalink, and Compare actions. +- Locks, replace history, and compliance metadata are stored per deck and surface alongside the exports. + +## Keyboard shortcuts +- **Enter** selects the first commander suggestion while searching. +- Inside Stage 5 lists: **L** locks/unlocks the focused card, **R** opens the Replace drawer, and **C** copies the permalink. +- Browser autofill is disabled in the modal to keep searches clean. + +## Virtualization, tagging, and performance +- `WEB_TAG_PARALLEL=1` with `WEB_TAG_WORKERS=4` (compose default) speeds up initial data preparation. The UI falls back to sequential tagging if workers fail to start. +- `WEB_VIRTUALIZE=1` enables virtualized grids in Stage 5 and the Owned library, smoothing large decks or libraries. +- Diagnostics overlays: enable `SHOW_DIAGNOSTICS=1`, then press **v** inside a virtualized grid to inspect render ranges, row counts, and paint timings. + +## Diagnostics and logs +- `SHOW_DIAGNOSTICS=1` unlocks the `/diagnostics` page with system summaries (`/status/sys`), feature flags, and per-request `X-Request-ID` headers. +- `SHOW_LOGS=1` turns on the `/logs` viewer with level & keyword filters, auto-refresh, and copy-to-clipboard. +- Health probes live at `/healthz` and return `{status, version, uptime_seconds}` for integration with uptime monitors.