Merge branch 'main' into maintenance/documentation-update

docs/authoring/cards.md

@@ -0,0 +1,25 @@
+# Card Authoring Guide
+
+This guide captures the conventions used by the deckbuilder when new cards are added to the CSV inputs. Always validate your edits by running the fast tagging tests or a local build before committing changes.
+
+## Modal double-faced & transform cards
+
+The tagging and reporting pipeline expects one row per face for any multi-faced card (modal double-faced, transform, split, or adventure). Use the checklist below when adding or updating these entries:
+
+1. **Canonical name** — Keep the `name` column identical for every face (e.g., `Valakut Awakening // Valakut Stoneforge`). Individual faces should instead set `face_name` when available; the merger preserves front-face copy for downstream consumers.
+2. **Layout & side** — Populate `layout` with the value emitted by Scryfall (`modal_dfc`, `transform`, `split`, `adventure`, etc.) and include a `side` column (`a`, `b`, …). The merger uses `side` ordering when reconstructing per-face metadata.
+3. **Mana details** — Supply `mana_cost`, `mana_value`, and `produces_mana` for every face. The per-face land snapshot and deck summary badges rely on these fields to surface the “DFC land” chip and annotated mana production.
+4. **Type line accuracy** — Ensure `type_line` includes `Land` for any land faces. The builder counts a card toward land totals when at least one face includes `Land`.
+5. **Tags & roles** — Tag every face with the appropriate `themeTags`, `roleTags`, and `card_tags`. The merge stage unions these sets so the finished card retains all relevant metadata.
+6. **Commander eligibility** — Only the primary (`side == 'a'`) face is considered for commander legality. If you add a new MDFC commander, double-check that the front face satisfies the Commander rules text; otherwise the record is filtered during catalog refresh.
+7. **Cross-check exports** — After the card is added, run a local build and confirm the deck exports include the new `DFCNote` column entry for the card. The annotation summarizes each land face so offline reviewers see the same guidance as the web UI.
+
+### Diagnostics snapshot (optional)
+
+When validating a large batch of MDFCs, enable the snapshot helper to inspect the merged faces:
+
+- Set `DFC_PER_FACE_SNAPSHOT=1` (and optionally `DFC_PER_FACE_SNAPSHOT_PATH`) before running the tagging pipeline.
+- Disable parallel tagging (`WEB_TAG_PARALLEL=0`) while the snapshot is active; the helper only writes output during sequential runs.
+- Once tagging completes, review `logs/dfc_per_face_snapshot.json` for the card you added to verify mana fields, `produces_mana`, and land detection flags.
+
+Following these guidelines keeps the deck summary badges, exporter annotations, and diagnostics snapshots in sync for every new double-faced card.

docs/commander_catalog.md

@@ -35,17 +35,16 @@ Additional columns are preserved but ignored by the browser; feel free to keep u
 ## Recommended refresh workflow
 
 1. Ensure dependencies are installed: `pip install -r requirements.txt`.
-2. Regenerate the commander CSV using the setup module:
+2. Regenerate the commander catalog with the MDFC-aware helper (multi-face merge always on):
    ```powershell
-   python -c "from file_setup.setup import regenerate_csvs_all; regenerate_csvs_all()"
+   python -m code.scripts.refresh_commander_catalog
    ```
-   This downloads the latest MTGJSON card dump (if needed), reapplies commander eligibility rules, and rewrites `commander_cards.csv`.
-3. (Optional) If you only need a fresh commander list and already have up-to-date `cards.csv`, run:
-   ```powershell
-   python -c "from file_setup.setup import determine_commanders; determine_commanders()"
-   ```
-4. Restart the web server (or your desktop app) so the cache reloads the new file.
-5. Validate with the targeted test:
+   - Pass `--compat-snapshot` when you need both `csv_files/commander_cards.csv` and `csv_files/compat_faces/commander_cards_unmerged.csv` so downstream consumers can diff the historical row-per-face layout.
+   - The legacy `--mode` argument is deprecated; it no longer disables the merge but still maps `--mode compat` to `--compat-snapshot` for older automation. Use `--skip-setup` if `determine_commanders()` has already been run and you simply need to reapply tagging.
+   - When running the web service during staging, set `DFC_COMPAT_SNAPSHOT=1` if you need the compatibility snapshot written on each rebuild. The merge itself no longer requires a feature flag.
+   - Use the staging QA checklist (`docs/qa/mdfc_staging_checklist.md`) to validate commander flows and downstream consumers before promoting the flag in production.
+3. Restart the web server (or your desktop app) so the cache reloads the new file.
+4. Validate with the targeted test:
    ```powershell
    python -m pytest -q code/tests/test_commander_catalog_loader.py
    ```

docs/qa/mdfc_staging_checklist.md

@@ -0,0 +1,63 @@
+# MDFC Staging QA Checklist
+
+Use this checklist when validating the MDFC merge in staging. The merge now runs unconditionally; set `DFC_COMPAT_SNAPSHOT=1` when you also need the legacy unmerged snapshots for downstream validation.
+
+_Last updated: 2025-10-02_
+
+## Prerequisites
+- Staging environment (Docker Compose or infrastructure equivalent) can override environment variables for the web service.
+- Latest code synced with the MDFC merge helper (`code/scripts/refresh_commander_catalog.py`).
+- Virtualenv or container image contains current project dependencies (`pip install -r requirements.txt`).
+
+## Configuration Steps
+1. Set the staging web service environment as needed:
+   - `DFC_COMPAT_SNAPSHOT=1` when downstream teams still require the compatibility snapshot.
+   - Optional diagnostics helpers: `SHOW_DIAGNOSTICS=1`, `SHOW_LOGS=1` (helps confirm telemetry output during smoke testing).
+2. Inside the staging container (or server), regenerate commander data:
+   ```powershell
+   python -m code.scripts.refresh_commander_catalog
+   ```
+   - Verify the script reports both the merged output (`csv_files/commander_cards.csv`) and the compatibility snapshot (`csv_files/compat_faces/commander_cards_unmerged.csv`).
+3. Restart the web service so the refreshed files (and optional compatibility snapshot setting) take effect.
+
+## Smoke QA
+| Area | Steps | Pass Criteria |
+| --- | --- | --- |
+| Commander Browser | Load `/commanders`, search for a known MDFC commander (e.g., "Elmar, Ulvenwald Informant"), flip faces, paginate results. | No duplicate rows per face, flip control works, pagination remains responsive. |
+| Deck Builder | Run a New Deck build with a commander that adds MDFC lands (e.g., "Atraxa, Grand Unifier" with MDFC swap option). | Deck summary shows "Lands: X (Y with DFC)" copy, MDFC notes render, CLI summary matches web copy (check download/export). |
+| Commander Exclusions | Attempt to search for a commander that should be excluded because only the back face is legal (e.g., "Withengar Unbound"). | UI surfaces exclusion guidance; the commander is not selectable. |
+| Diagnostics | Open `/diagnostics` with `SHOW_DIAGNOSTICS=1`. Confirm MDFC telemetry panel shows merged counts. | `dfc_merge_summary` card present with non-zero merged totals; land telemetry includes MDFC contribution counts. |
+| Logs | Tail application logs via `/logs` or container logs during a build. | No errors related to tag merging or commander loading. |
+
+## Automated Checks
+Run the targeted test suite to ensure MDFC regressions are caught:
+```powershell
+c:/Users/Matt/mtg_python/mtg_python_deckbuilder/.venv/Scripts/python.exe -m pytest -q ^
+  code/tests/test_land_summary_totals.py ^
+  code/tests/test_commander_primary_face_filter.py ^
+  code/tests/test_commander_exclusion_warnings.py
+```
+- All tests should pass. Investigate any failures before promoting the flag.
+
+## Downstream Sign-off
+1. Provide consumers with:
+   - Merged file: `csv_files/commander_cards.csv`
+   - Compatibility snapshot: `csv_files/compat_faces/commander_cards_unmerged.csv`
+2. Share expected merge metrics (`logs/dfc_merge_summary.json`) to help validate MDFC counts.
+3. Collect acknowledgements that downstream pipelines work with the merged file (or have cut over) before retiring the compatibility flag.
+
+## Rollback Plan
+- Disable `DFC_COMPAT_SNAPSHOT` (or leave it unset) and rerun `python -m code.scripts.refresh_commander_catalog` if compatibility snapshots are no longer required.
+- Revert to the previous committed commander CSV if needed (`git checkout -- csv_files/commander_cards.csv`).
+- Document the issue in the roadmap and schedule the fix before reattempting the staging rollout.
+
+## Latest Run (2025-10-02)
+- Environment: staging compose updated (temporarily set `ENABLE_DFC_MERGE=compat`, now retired) and reconfigured with optional `DFC_COMPAT_SNAPSHOT=1` for compatibility checks.
+- Scripts executed:
+   - `python -m code.scripts.refresh_commander_catalog --compat-snapshot`
+   - `python -m code.scripts.preview_dfc_catalog_diff --compat-snapshot --output logs/dfc_catalog_diff.json`
+- Automated tests passed:
+   - `code/tests/test_land_summary_totals.py`
+   - `code/tests/test_commander_primary_face_filter.py`
+   - `code/tests/test_commander_exclusion_warnings.py`
+- Downstream sign-off: `logs/dfc_catalog_diff.json` shared with catalog consumers alongside `csv_files/compat_faces/commander_cards_unmerged.csv`; acknowledgements recorded in `docs/releases/dfc_merge_rollout.md`.

docs/releases/dfc_merge_rollout.md

@@ -0,0 +1,31 @@
+# MDFC Merge Rollout (2025-10-02)
+
+## Summary
+- Staging environment refreshed with the MDFC merge permanently enabled; compatibility snapshot retained via `DFC_COMPAT_SNAPSHOT=1` during validation.
+- Commander catalog rebuilt with `python -m code.scripts.refresh_commander_catalog --compat-snapshot`, generating both the merged output and `csv_files/compat_faces/commander_cards_unmerged.csv` for downstream comparison.
+- Diff artifact `logs/dfc_catalog_diff.json` captured via `python -m code.scripts.preview_dfc_catalog_diff --compat-snapshot --output logs/dfc_catalog_diff.json` and shared with downstream consumers.
+- `ENABLE_DFC_MERGE` guard removed across the codebase; documentation updated to reflect the always-on merge and optional compatibility snapshot workflow.
+
+## QA Artifacts
+| Artifact | Description |
+| --- | --- |
+| `docs/qa/mdfc_staging_checklist.md` | Latest run log documents the staging enablement procedure and verification steps. |
+| `logs/dfc_catalog_diff.json` | JSON diff summarising merged vs. unmerged commander/catalog rows for parity review. |
+| `csv_files/commander_cards.csv` | Merged commander catalog generated after guard removal. |
+| `csv_files/compat_faces/commander_cards_unmerged.csv` | Legacy snapshot retained for downstream validation during the final review window. |
+
+## Automated Verification
+| Check | Command | Result |
+| --- | --- | --- |
+| MDFC land accounting | `python -m pytest -q code/tests/test_land_summary_totals.py` | ✅ Passed |
+| Commander primary-face filter | `python -m pytest -q code/tests/test_commander_primary_face_filter.py` | ✅ Passed |
+| Commander exclusion warnings | `python -m pytest -q code/tests/test_commander_exclusion_warnings.py` | ✅ Passed |
+
+## Downstream Sign-off
+| Consumer / Surface | Validation | Status |
+| --- | --- | --- |
+| Web UI (builder + diagnostics) | MDFC staging checklist smoke QA | ✅ Complete |
+| CLI / Headless workflows | Targeted pytest suite confirmations (see above) | ✅ Complete |
+| Data exports & analytics | `logs/dfc_catalog_diff.json` review against `commander_cards_unmerged.csv` | ✅ Complete |
+
+All downstream teams confirmed parity with the merged catalog and agreed to proceed without the `ENABLE_DFC_MERGE` guard. Compatibility snapshots remain available via `DFC_COMPAT_SNAPSHOT=1` for any follow-up spot checks.