# Ghost Node — MD Update Playbook

## Purpose
This file is a reusable, copy-paste workflow for the task:
- Deep semantic audit of **every** `*.md` file in the repo (no exceptions)
- Update **only** the Markdown files that are missing or contradicting current program behavior
- Keep edits minimal, accuracy-first, and safe

Use this when you want to avoid manually re-verifying documentation for “browser visibility override” and “keyword batching + retry tracking” mechanics.

---

## Mechanics that this playbook must cover
### Per-site browser visibility override
- `TargetSite.custom_visible_browser` (0/1)
- Precedence rule:
  - When global `show_browser=true`, visible mode is forced for all sites
  - In that case, per-site `custom_visible_browser` is ignored
  - When global `show_browser=false`, a site becomes visible only if `custom_visible_browser=1`

### Keyword batching + persistent retry tracking
- `keyword_batch_enabled`
- `scrape_rounds` and `scrape_round_items`
- `ScrapeRoundItem` statuses: `pending | in_progress | done | failed`
- Pending retry window: **4 hours**
- Hourly warning bookkeeping:
  - `last_hour_warn_at`
  - dashboard “due” logic (doc must not contradict UI)

### Dashboard behavior
- Pinned/section UI named **“Keyword Retry Tracking”**
- Pinned content is sourced from:
  - `GET /api/scrape/progress`

---

## Strict editing constraints
1. Only edit Markdown files (`*.md`).
2. Do NOT change any code files.
3. Do NOT “regenerate from scratch”.
4. Only edit files when the semantic audit finds missing or contradictory content.
5. Prefer minimal wording changes (small patch, keep structure).
6. Use a verification pass by searching for required terms/endpoint names.

---

## Required output format (when you run this job)
1. **Doc audit findings**
   - For each `*.md` file:
     - `OK` if already correct
     - otherwise 1–3 bullets describing what was missing or wrong
2. **Approval gate**
   - Ask for approval before applying edits:
     - “Reply `YES` to apply the listed changes”
3. **Applied edits** (only after approval)
   - For each modified file, include small excerpt(s) of the updated sections
4. **Verification checklist**
   - Confirm these terms/behaviors are present and consistent in modified files:
     - `custom_visible_browser`
     - `show_browser=true` precedence vs per-site override
     - `keyword_batch_enabled`
     - `scrape_round_items`
     - “Keyword Retry Tracking”
     - `GET /api/scrape/progress`

---

## Copy-paste prompt (TEXT)
You are a documentation editor for the Ghost Node repo. I only want updates to Markdown files (`*.md`) and ONLY when the Markdown is missing or contradicting current program behavior.

### Goal
Perform a deep semantic audit across **all Markdown files in the repo** and update ONLY the MD files that are missing relevant documentation for the latest mechanics:
- Per-site browser visibility override: `TargetSite.custom_visible_browser` (0/1)
- Precedence rule: **global `show_browser=true` forces visible mode for all sites** and **ignores** per-site `custom_visible_browser`
- Keyword batching + persistent retry tracking across cycles:
  - `keyword_batch_enabled`
  - `scrape_rounds`, `scrape_round_items`
  - pending/in_progress/done/failed keyword items
  - 4-hour retry window for pending items
  - hourly warnings bookkeeping (`last_hour_warn_at`, `warn_due`)
- Dashboard behavior:
  - there is a pinned/section UI named **“Keyword Retry Tracking”**
  - pinned content is sourced from **`GET /api/scrape/progress`**

### Strict rules
1. Enumerate every `*.md` file in the repo recursively and check for relevance.
2. If a given MD file already contains correct info, do not change it.
3. If it is missing sections about the mechanics above, add short accurate documentation.
4. If it contains outdated/contradicting text, update the wording to match current behavior.
5. Do not “regenerate from scratch”. Use minimal edits.
6. Do not edit code files. Only `*.md`.
7. After completing edits, update only the affected MD files.
8. After edits, run a verification pass using keyword searches to confirm:
   - `custom_visible_browser`
   - `show_browser=true` precedence vs per-site override
   - `keyword_batch_enabled`
   - `scrape_round_items`
   - “Keyword Retry Tracking”
   - `GET /api/scrape/progress`

### Output format
1. First, list “Doc audit findings”:
   - For each changed file: what was missing or wrong (1–3 bullets)
   - For each unchanged file that was checked but already correct: mention “OK”
2. Second, list “Applied edits”:
   - For each modified file: show the updated section as a small excerpt
3. Third, list “Verification checklist”:
   - confirm that all key terms/behaviors appear where expected.

### Work constraints
- Prefer reading authoritative docs first (`docs/CLAUDE.md`, `docs/PROGRESS.md`, `docs/MEMORY.md`).
- Only read `worker.py` sections if needed to resolve ambiguity.
- Only proceed with edits after identifying which MD files are actually missing/incorrect.

---

## Copy-paste prompt (JSON)
```json
{
  "task": "Deep semantic audit and minimal update of ALL Markdown docs for Ghost Node",
  "scope": {
    "include_glob": ["**/*.md"],
    "no_exceptions": true
  },
  "edits": {
    "allowed_file_types": ["*.md"],
    "disallowed": ["any code files", "non-markdown files"],
    "minimal_changes": true,
    "no_regeneration_from_scratch": true,
    "only_change_when_needed": true,
    "do_not_touch_correct_files": true
  },
  "mechanics_semantic_targets": [
    "TargetSite.custom_visible_browser (0/1) per-site visibility override",
    "Precedence: global show_browser=true forces visible mode for all sites and ignores custom_visible_browser",
    "keyword_batch_enabled config behavior",
    "scrape_rounds + scrape_round_items persistent retry tracking",
    "ScrapeRoundItem statuses: pending/in_progress/done/failed",
    "Retry window: 4-hour limit for pending keyword retries",
    "Hourly warnings: last_hour_warn_at and warn_due logic documented",
    "Dashboard pinned UI: 'Keyword Retry Tracking' sourced from GET /api/scrape/progress"
  ],
  "approval_gate": {
    "required_before_edits": true,
    "approval_keyword": "YES"
  },
  "verification_must_appear_in_modified_files": [
    "custom_visible_browser",
    "show_browser=true precedence",
    "keyword_batch_enabled",
    "scrape_round_items",
    "Keyword Retry Tracking",
    "GET /api/scrape/progress"
  ],
  "required_output_format": [
    "Doc audit findings (OK vs missing/wrong)",
    "Ask for approval",
    "Applied edits (small excerpts only)",
    "Verification checklist"
  ]
}
```

