mtg_python_deckbuilder/README.md

# 🃏 MTG Python Deckbuilder

[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Python 3.11+](https://img.shields.io/badge/python-3.11+-blue.svg)](https://www.python.org/downloads/)
[![Docker](https://img.shields.io/badge/docker-supported-blue.svg)](https://www.docker.com/)

A fast, menu-driven deckbuilder for MTG Commander/EDH with both CLI and Web UI. Supports interactive and headless runs, CSV/TXT exports, owned-card libraries, and Docker.

## 🔹 Features
- Web UI and CLI, powered by the same builder
- Commander search with smart theme tagging (primary/secondary/tertiary)
- Power bracket selection and ideal count prompts
- CSV and TXT exports with stable filenames
- Headless mode (non-interactive) and a headless submenu in the main menu
- Config precedence: CLI > env > JSON > defaults
 - Visual summaries in Web UI: Mana curve, Color pips and Sources with hover-to-highlight and copyable tooltips
- Web UI: Locks, Replace flow, Compare builds, and shareable permalinks
 - Unified “New Deck” modal (steps 1–3): commander search, themes, bracket, options, and optional Name (used in export filenames)

## ✨ Highlights
- Smart tagging and suggestions for commander + themes, with AND/OR combine modes
- Exports CSV and TXT decklists to `deck_files/`
- Owned-cards mode: point to one or more `.txt`/`.csv` files in `owned_cards/` and choose "Use only owned cards?"
- If you opt out, final CSV marks owned cards with an `Owned` column
- Build options include Prefer-owned: bias picks toward owned cards while allowing unowned fallback
- Interactive menu with a headless submenu
- Works locally or in Docker (Windows PowerShell friendly)
- Card images and data via Scryfall (attribution shown in the Web UI)
 - Web visual summaries: cross-highlight cards from charts; sources include non-lands and colorless 'C' with an on/off toggle
 - New Deck modal keyboard UX: Enter selects the first suggestion; arrow navigation removed; browser autofill disabled
 - Export naming: optional Name in the modal becomes the filename stem for CSV/TXT/JSON; decks include a sidecar `.summary.json` with `meta.name`
 - Finished Decks and deck banners prefer your custom Name when present
 - Compare page includes a Copy summary button to quickly share diffs
 - New Deck modal shows your selected themes and their order (1 → 3) while picking; Bracket options are labeled (e.g., "Bracket 3: Upgraded"). Default bracket is 3.

## 🚀 Quick start

### Docker Hub (PowerShell)
```powershell
docker run -it --rm `
  -v "${PWD}/deck_files:/app/deck_files" `
  -v "${PWD}/logs:/app/logs" `
  -v "${PWD}/csv_files:/app/csv_files" `
  -v "${PWD}/owned_cards:/app/owned_cards" `
  -v "${PWD}/config:/app/config" `
  mwisnowski/mtg-python-deckbuilder:latest
```

### From source
```powershell
pip install -r requirements.txt
python code/main.py
```

### From this repo with Docker Compose (PowerShell)
```powershell
docker compose build
docker compose run --rm mtg-deckbuilder
```

## 🌐 Web UI

Run the browser-based UI backed by the same deckbuilding engine.

### From source
```powershell
pip install -r requirements.txt
uvicorn code.web.app:app --host 127.0.0.1 --port 8080
```

Open http://127.0.0.1:8080

### Docker Compose (PowerShell)
```powershell
docker compose build web
docker compose up --no-deps web
```

Then open http://localhost:8080

Notes:
- Exports/logs/configs use the same folders and will appear under your mounted volumes (see compose file).
- The footer includes Scryfall attribution per their guidelines.
 - Favicon is bundled; browsers load `/favicon.ico` (auto-falls back to PNG).

### Quick guide (locks • replace • compare • permalinks)
- New Deck modal (Steps 1–3 unified): search for your commander, pick themes, choose bracket (defaults to 3), and optionally set Name. Submitting starts the build immediately. Name becomes the export filename stem and display name.
- Lock a card: click the card image or the lock control in Step 5. Locked cards are pinned for reruns. A small “Last action” chip confirms lock/unlock.
- Replace a card: toggle Replace in Step 5, then click a card to open Alternatives. Filter (including Owned-only) and pick a swap; the deck updates in place.
- Permalink: use “Copy permalink” from Step 5 or a Finished deck to share/restore a run (commander, themes, bracket, ideals, flags). “Open Permalink…” lets you paste one to restore to review.
- Compare: from Finished Decks, open Compare and pick A and B (quick actions: Latest two, Swap A/B). Use Changed-only to focus diffs. Copy summary or Download .txt.
- Keyboard: In the New Deck modal, Enter selects the first commander result. Browser autofill is disabled for cleaner inputs. In Step 5, use L to lock/unlock the focused card, R to open Alternatives, and C to copy a permalink.

### Setup speed: parallel tagging (Web)
- On first run or when data is stale, the Web UI prepares and tags the card database.
- This tagging runs in parallel by default for faster setup.
- Tuning via env vars (set on the `web` service in `docker-compose.yml`):
  - `WEB_TAG_PARALLEL=1|0` — enable/disable parallel workers (default: 1/enabled)
  - `WEB_TAG_WORKERS=<N>` — number of worker processes (default: 4 in compose; omit to auto-pick)
- The UI shows progress and falls back to sequential tagging automatically if parallel init fails.

Virtualized lists and image polish (opt-in)
- Opt-in with `WEB_VIRTUALIZE=1` to enable virtualization in Step 5 lists/grids and the Owned library for smoother scrolling on large sets.
- Virtualization diagnostics overlay (for debugging): enable `SHOW_DIAGNOSTICS=1`, then press `v` to toggle an overlay per grid and a global summary bubble; shows visible range, total, rows, row height, render time, and counters.
- Thumbnails use lazy-loading with srcset/sizes; LQIP blur/fade-in is applied to Step 5 and Owned thumbnails, and the commander preview image.
- Respects reduced motion: when the OS prefers-reduced-motion, blur/fade transitions and smooth scrolling are disabled for accessibility.

### Build options (Web)
- Use only owned cards: builds strictly from your `owned_cards/` lists (commander exempt).
- Prefer owned cards: gently prioritizes owned cards across creatures and spells but allows unowned when they’re better fits.
  - Implemented via a stable owned-first reorder and small weight boosts; preserves existing sort intent.
Notes:
- Steps 1–3 are consolidated into a single New Deck modal. Submitting starts the build immediately, skipping the old review page.
 - The modal includes an optional Name; when set, it is used as the export filename stem and display name in the UI.
 - The modal displays selected themes in order (1, 2, 3). The Bracket selector shows numbers (e.g., "Bracket 3: Upgraded") and defaults to 3 for new decks.

### Staged build visibility
- Step 5 can optionally “Show skipped stages” so you can see phases that added no cards with a clear annotation.

### Visual summaries (Web)
- Mana Curve bars with hover-to-highlight of matching cards in both list and thumbnail views
- Color Pips (requirements) and Sources (generation) bars with per-color tooltips listing cards
- Cross-highlighting from charts to the card views; list-mode highlights only the name span
- Sources detection includes non-land producers (artifacts/creatures/etc.) and colorless 'C'
- Fetch lands are not counted as mana sources; basic lands and Wastes are handled reliably
- Optional: Show colorless (C) toggle for Sources; persisted per browser
- Tooltips include a Copy button to copy the card list

### Owned page (Web)
- View and manage your owned lists with:
  - Export TXT/CSV, sort controls, and a live “N shown” counter
  - Color identity dots and exact color-identity combo filters (including 4-color)
  - Viewport-filling list with styled scrollbar
  - Optional virtualization when `WEB_VIRTUALIZE=1` (improves performance on very large libraries)
- Uploading `.txt` or `.csv` lists enriches and deduplicates entries at upload-time and persists them, so page loads are fast.

### Finished Decks (Web)
- Theme filters are a dropdown with shareable state for easier browsing of saved builds.
 - Each finished deck has a permalink you can copy and revisit; Compare mode lets you diff against another run. The Compare page has a Copy summary button to copy a plain-text diff.
 - Locks and Replace flow: lock-in picks you like or replace a card with an alternative during iteration.

### Diagnostics (Web)
- Health endpoint: `GET /healthz` returns `{ status, version, uptime_seconds }`
- Responses include `X-Request-ID` for easier error correlation; unhandled errors return JSON with `request_id`

Logs and system tools:
- Logs page (`/logs`, enable with `SHOW_LOGS=1`):
  - Auto-refresh toggle with adjustable interval
  - Level filter (All/Error/Warning/Info/Debug) and keyword filter
  - Copy button to copy the visible log tail
- System summary (`GET /status/sys`, shown on `/diagnostics` when `SHOW_DIAGNOSTICS=1`):
  - Returns `{ version, uptime_seconds, server_time_utc, flags }`

Compose: enable diagnostics/logs (optional)

```yaml
services:
  web:
    environment:
      - SHOW_LOGS=1
      - SHOW_DIAGNOSTICS=1
```

## 🤖 Headless mode (no prompts)

- Auto-headless: set `DECK_MODE=headless`
  - Example (PowerShell):
    ```powershell
    docker compose run --rm -e DECK_MODE=headless mtg-deckbuilder
    ```
- Use a JSON config: mount `./config` to `/app/config` and set `DECK_CONFIG=/app/config/deck.json`
  - Example (PowerShell):
    ```powershell
    docker compose run --rm `
      -e DECK_MODE=headless `
      -e DECK_CONFIG=/app/config/deck.json `
      mtg-deckbuilder
    ```
- Override via env vars (subset): `DECK_COMMANDER`, `DECK_PRIMARY_CHOICE`, `DECK_SECONDARY_CHOICE`, `DECK_TERTIARY_CHOICE`, `DECK_ADD_LANDS`, `DECK_FETCH_COUNT`, `DECK_BRACKET_LEVEL`
  - Precedence: CLI > env > JSON > defaults

Headless submenu notes:
- If one JSON exists in `config/`, it auto-runs it
- If multiple exist, they’re listed as "Commander - Theme1, Theme2, Theme3"; `deck.json` shows as "Default"
- CSV/TXT are exported as usual; JSON run-config is exported only in interactive runs

### Run locally (no Docker)
```powershell
# Show resolved settings (no run)
python code/headless_runner.py --config config/deck.json --dry-run

# Run with a specific config file
python code/headless_runner.py --config config/deck.json

# Point to a folder; if exactly one config exists, it's auto-used
python code/headless_runner.py --config config

# Override via CLI
python code/headless_runner.py --commander "Pantlaza, Sun-Favored" --primary-choice 2 --secondary-choice 0 --add-lands true --fetch-count 3
```

### CLI flags
- --config <path>                Path to JSON config file or a folder to discover configs (uses DECK_CONFIG by default)
- --commander <name>             Commander name to search and select
- --primary-choice <n>           Primary theme index
- --secondary-choice <n|none>    Secondary theme index or omit with "none"
- --tertiary-choice <n|none>     Tertiary theme index or omit with "none"
- --add-lands <bool>             Include land building steps (true/false)
- --fetch-count <n>              Requested number of fetch lands
- --dual-count <n>               Requested number of dual lands (optional; not exported)
- --triple-count <n>             Requested number of triple lands (optional; not exported)
- --utility-count <n>            Requested number of utility lands (optional; not exported)
- --add-creatures <bool>         Add creatures
- --add-non-creature-spells <bool>  Add non-creature spells orchestrator
- --add-ramp <bool>              Add ramp (when not using orchestrator)
- --add-removal <bool>           Add removal (when not using orchestrator)
- --add-wipes <bool>             Add board wipes (when not using orchestrator)
- --add-card-advantage <bool>    Add card draw (when not using orchestrator)
- --add-protection <bool>        Add protection (when not using orchestrator)
- --bracket-level <1-5>          Power bracket selection (or use DECK_BRACKET_LEVEL)
- --dry-run                      Print resolved config and exit


Booleans accept: 1/0, true/false, yes/no, on/off.

### JSON export in headless
- By default, headless runs do not export a JSON run-config to avoid duplicates.
- Opt-in with:
  ```powershell
  $env:HEADLESS_EXPORT_JSON = "1"
  python code/headless_runner.py --config config/deck.json
  ```
- Tip: when opting in, prefer using `--config` instead of a `DECK_CONFIG` file path to avoid creating both a stem-based JSON and a second explicit-path JSON.

Example JSON (`config/deck.json`):
```json
{
  "commander": "Pantlaza",
  "bracket_level": 3,
  "primary_choice": 2,
  "secondary_choice": 2,
  "tertiary_choice": 2,
  "tag_mode": "OR", // OR or AND; Web UI default is OR
  "add_lands": true,
  "fetch_count": 3,
  "ideal_counts": { "ramp": 10, "lands": 36, "basic_lands": 16, "creatures": 28, "removal": 8, "wipes": 3, "card_advantage": 8, "protection": 3 }
}
```
Notes: headless honors `ideal_counts` (leave prompts blank to accept). Only `fetch_count` is tracked/exported for lands.

## 🧩 Theme combine mode (AND/OR)

- OR (default): Recommend cards that match any selected themes, prioritizing overlap.
- AND: Prioritize multi-theme intersections. For creatures, an AND pre-pass first picks "all selected themes" creatures up to a cap, then fills by weighted overlap.

UI tips:
- Step 2 includes AND/OR radios with a tooltip explaining trade-offs.
- In staged build view, "Creatures: All-Theme" shows which selected themes each card hits.

## 🕹️ Usage (interactive)
1) Start the app (Docker or from source)
2) Pick Build a New Deck
3) Search/confirm commander
4) Pick primary/secondary/tertiary themes (or stop at primary); choose AND/OR combine mode
5) Choose power bracket and review ideal counts
6) Deck builds; CSV/TXT export to `deck_files/`

## ⚙️ Environment variables (common)
- DECK_MODE=headless
- DECK_CONFIG=/app/config/deck.json
- DECK_COMMANDER, DECK_PRIMARY_CHOICE, DECK_SECONDARY_CHOICE, DECK_TERTIARY_CHOICE
- DECK_ADD_LANDS, DECK_FETCH_COUNT
- DECK_ADD_CREATURES, DECK_ADD_NON_CREATURE_SPELLS, DECK_ADD_RAMP, DECK_ADD_REMOVAL, DECK_ADD_WIPES, DECK_ADD_CARD_ADVANTAGE, DECK_ADD_PROTECTION
- DECK_BRACKET_LEVEL

Optional name-based tag overrides (mapped to indices for the chosen commander):
- DECK_PRIMARY_TAG, DECK_SECONDARY_TAG, DECK_TERTIARY_TAG

Combine mode in headless:
- JSON: set `"tag_mode": "AND" | "OR"`
- Env var: `DECK_TAG_MODE=AND|OR` (if configured in your environment)

Web UI performance tuning:
- WEB_TAG_PARALLEL=1|0
- WEB_TAG_WORKERS=<N>
 - WEB_VIRTUALIZE=1 (enable list virtualization)
 - SHOW_DIAGNOSTICS=1 (optional: diagnostics tools and virtualization overlay toggle using `v`)

## 📁 Folders
- `deck_files/` — CSV/TXT exports
- `csv_files/` — card data
- `logs/` — logs
- `config/` — JSON configs (optional)
- `owned_cards/` — your owned cards lists (`.txt`/`.csv`); used for owned-only builds and Owned flagging

## 🧰 Troubleshooting
- Use `docker compose run --rm` (not `up`) for interactive sessions
- Ensure volumes are mounted so files persist (`deck_files`, `logs`, `csv_files`)
- For headless with config, mount `./config:/app/config` and set `DECK_CONFIG`
- Card data refresh: if `csv_files/cards.csv` is missing or older than 7 days, the app refreshes data and re-tags automatically. A `.tagging_complete.json` file in `csv_files/` indicates tagging completion.
 - Web health: the header dot polls `/healthz` — green is OK, red is degraded. If red, check logs.
 - Error details: HTMX errors show a toast with a “Copy details” button including X-Request-ID; include that when filing issues.
 - Logs page: set `SHOW_LOGS=1` to enable `/logs` and `/status/logs?tail=200` (read‑only) for quick diagnostics.
  - Diagnostics page: set `SHOW_DIAGNOSTICS=1` to enable the nav link and `/diagnostics` test tools.

Data integrity notes:
- Banned cards: per-color/guild CSVs now consistently respect the Commander banned list using exact, case‑insensitive name matching across `name` and `faceName`.

## 🧪 Development and tests

Set up a virtual environment, install dependencies, and run the test suite.

```powershell
# From repo root (PowerShell)
python -m venv .venv
& ".venv/Scripts/Activate.ps1"
pip install -r requirements.txt
pip install -r requirements-dev.txt

# Run tests (pytest is configured to look under code/tests)
python -m pytest -q
```

Notes:
- Test discovery is set to `code/tests` in `pytest.ini`.
- `pytest.ini` disables the built-in debugging plugin to avoid a stdlib module name clash with the project folder `code/`.
- Feature flags for local diagnostics: set `SHOW_DIAGNOSTICS=1` and/or `SHOW_LOGS=1` to expose `/diagnostics` and `/logs`.

### What’s new (quick summary)
- Faster browsing with optional virtualized grids/lists in Step 5 and Owned (`WEB_VIRTUALIZE=1`).
- Image polish: lazy-loading, responsive `srcset/sizes`, and LQIP blur/fade for Step 5, Owned, and commander preview.
- Diagnostics overlay (opt-in with `SHOW_DIAGNOSTICS=1`): press `v` to see visible range, totals, render time, and counters.
- Accessibility: respects reduced-motion (disables blur/fade and smooth scrolling).
- Small caching wins: short‑TTL fragment caching for summary partials and suggestions.

## 📦 Releases
- Release notes are maintained in `RELEASE_NOTES_TEMPLATE.md`. Automated workflows read from this file to populate Docker Hub and GitHub releases.

### Collecting diagnostics for an issue
- Note the Request-ID from the toast or error page.
- Copy logs from `/logs` (enable with `SHOW_LOGS=1`) or `/status/logs?tail=200`.
- Include your `/healthz` JSON and environment flags (SHOW_LOGS/SHOW_DIAGNOSTICS/SHOW_SETUP) when reporting.

## 🧩 Owned cards library
- Place `.txt` or `.csv` lists in `owned_cards/` (one card name per line for `.txt`; any `.csv` with a `name` column works).
- After commander selection, you’ll be prompted: "Use only owned cards?"
  - Yes: deck builds from owned cards only; if fewer than 100 cards, it stays incomplete and prints a note.
  - No: the build uses the full pool, but the final CSV marks owned cards with an `Owned` column.
- If an owned-only build is incomplete, a recommendations file is exported: `deck_files/[stem]_recommendations.csv` and `.txt`. The app prints: "Recommended but unowned cards in deck_files/[stem]_recommendations.csv".
- Web uploads: owned lists are parsed/enriched at upload-time, header rows are skipped, and duplicates are deduped. The enriched result is stored so subsequent page loads don’t re-parse your files.

## 📄 License & help
- License: MIT (see `LICENSE`)
- Issues/Requests: GitHub Issues/Discussions
- Docker details: see `DOCKER.md`