Ghost Node — MD Update Playbook

Purpose

This file is a reusable, copy-paste workflow for the task:

Deep semantic audit of every *.md file in the repo (no exceptions)
Update only the Markdown files that are missing or contradicting current program behavior
Keep edits minimal, accuracy-first, and safe

Use this when you want to avoid manually re-verifying documentation for “browser visibility override” and “keyword batching + retry tracking” mechanics.

Mechanics that this playbook must cover

Per-site browser visibility override

TargetSite.custom_visible_browser (0/1)
Precedence rule:
- When global show_browser=true, visible mode is forced for all sites
- In that case, per-site custom_visible_browser is ignored
- When global show_browser=false, a site becomes visible only if custom_visible_browser=1

Keyword batching + persistent retry tracking

keyword_batch_enabled
scrape_rounds and scrape_round_items
ScrapeRoundItem statuses: pending | in_progress | done | failed
Pending retry window: 4 hours
Hourly warning bookkeeping:
- last_hour_warn_at
- dashboard “due” logic (doc must not contradict UI)

Dashboard behavior

Pinned/section UI named “Keyword Retry Tracking”
Pinned content is sourced from:
- GET /api/scrape/progress

Strict editing constraints

Only edit Markdown files (*.md).
Do NOT change any code files.
Do NOT “regenerate from scratch”.
Only edit files when the semantic audit finds missing or contradictory content.
Prefer minimal wording changes (small patch, keep structure).
Use a verification pass by searching for required terms/endpoint names.

Required output format (when you run this job)

Doc audit findings
- For each *.md file:
  - OK if already correct
  - otherwise 1–3 bullets describing what was missing or wrong
Approval gate
- Ask for approval before applying edits:
  - “Reply YES to apply the listed changes”
Applied edits (only after approval)
- For each modified file, include small excerpt(s) of the updated sections
Verification checklist
- Confirm these terms/behaviors are present and consistent in modified files:
  - custom_visible_browser
  - show_browser=true precedence vs per-site override
  - keyword_batch_enabled
  - scrape_round_items
  - “Keyword Retry Tracking”
  - GET /api/scrape/progress

Copy-paste prompt (TEXT)

You are a documentation editor for the Ghost Node repo. I only want updates to Markdown files (*.md) and ONLY when the Markdown is missing or contradicting current program behavior.

Goal

Perform a deep semantic audit across all Markdown files in the repo and update ONLY the MD files that are missing relevant documentation for the latest mechanics:

Per-site browser visibility override: TargetSite.custom_visible_browser (0/1)
Precedence rule: global show_browser=true forces visible mode for all sites and ignores per-site custom_visible_browser
Keyword batching + persistent retry tracking across cycles:
- keyword_batch_enabled
- scrape_rounds, scrape_round_items
- pending/in_progress/done/failed keyword items
- 4-hour retry window for pending items
- hourly warnings bookkeeping (last_hour_warn_at, warn_due)
Dashboard behavior:
- there is a pinned/section UI named “Keyword Retry Tracking”
- pinned content is sourced from GET /api/scrape/progress

Strict rules

Enumerate every *.md file in the repo recursively and check for relevance.
If a given MD file already contains correct info, do not change it.
If it is missing sections about the mechanics above, add short accurate documentation.
If it contains outdated/contradicting text, update the wording to match current behavior.
Do not “regenerate from scratch”. Use minimal edits.
Do not edit code files. Only *.md.
After completing edits, update only the affected MD files.
After edits, run a verification pass using keyword searches to confirm:
- custom_visible_browser
- show_browser=true precedence vs per-site override
- keyword_batch_enabled
- scrape_round_items
- “Keyword Retry Tracking”
- GET /api/scrape/progress

Output format

First, list “Doc audit findings”:
- For each changed file: what was missing or wrong (1–3 bullets)
- For each unchanged file that was checked but already correct: mention “OK”
Second, list “Applied edits”:
- For each modified file: show the updated section as a small excerpt
Third, list “Verification checklist”:
- confirm that all key terms/behaviors appear where expected.

Work constraints

Prefer reading authoritative docs first (docs/CLAUDE.md, docs/PROGRESS.md, docs/MEMORY.md).
Only read worker.py sections if needed to resolve ambiguity.
Only proceed with edits after identifying which MD files are actually missing/incorrect.

Copy-paste prompt (JSON)

{
  "task": "Deep semantic audit and minimal update of ALL Markdown docs for Ghost Node",
  "scope": {
    "include_glob": ["**/*.md"],
    "no_exceptions": true
  },
  "edits": {
    "allowed_file_types": ["*.md"],
    "disallowed": ["any code files", "non-markdown files"],
    "minimal_changes": true,
    "no_regeneration_from_scratch": true,
    "only_change_when_needed": true,
    "do_not_touch_correct_files": true
  },
  "mechanics_semantic_targets": [
    "TargetSite.custom_visible_browser (0/1) per-site visibility override",
    "Precedence: global show_browser=true forces visible mode for all sites and ignores custom_visible_browser",
    "keyword_batch_enabled config behavior",
    "scrape_rounds + scrape_round_items persistent retry tracking",
    "ScrapeRoundItem statuses: pending/in_progress/done/failed",
    "Retry window: 4-hour limit for pending keyword retries",
    "Hourly warnings: last_hour_warn_at and warn_due logic documented",
    "Dashboard pinned UI: 'Keyword Retry Tracking' sourced from GET /api/scrape/progress"
  ],
  "approval_gate": {
    "required_before_edits": true,
    "approval_keyword": "YES"
  },
  "verification_must_appear_in_modified_files": [
    "custom_visible_browser",
    "show_browser=true precedence",
    "keyword_batch_enabled",
    "scrape_round_items",
    "Keyword Retry Tracking",
    "GET /api/scrape/progress"
  ],
  "required_output_format": [
    "Doc audit findings (OK vs missing/wrong)",
    "Ask for approval",
    "Applied edits (small excerpts only)",
    "Verification checklist"
  ]
}

6.5 KiB Raw Permalink Blame History Unescape Escape