# Ghost Node — Session Memory

> **What belongs here:** Current architecture state, key gotchas, reusable patterns, dev paths.
> **What does NOT belong here:** Session history (→ PROGRESS.md or ARCHIVE.md).
>
> ### ⚠️ MD Update Rule — DYNAMIC, NOT STATIC
> Only update the MD file(s) whose content was actually touched this session.
> Do NOT update all files after every task. Each file gets only what is relevant to it.
> - Update **this file** only when a new reusable gotcha, pattern, or architecture decision was made.
> - Update **PROGRESS.md** whenever any code file was changed.
> - Update **CLAUDE.md** only if API endpoints, DB schema, config keys, or architecture changed.
> - Move old PROGRESS.md session entries to **ARCHIVE.md** when they're no longer active context.

---

## Project Identity
- Ghost Node International Auction Sniper v2.7
- Owner: Abbas, Baghdad, Iraq
- Path: `C:\Users\Abbas\Documents\Downloads\ClaudeAuction2\`
- Backend: Python FastAPI (worker.py, 5000+ lines), port 7000
- NOT a git repository

---

## Frontend Stack (Current — Session 16, fully migrated)

- Location: `frontend/` directory
- Next.js 16.1.6, React 19, TypeScript, Tailwind CSS v4 (**NOT v3**)
- **Tailwind v4**: CSS variables via `@theme {}` in `globals.css` — NO `tailwind.config.ts`
- **Session 27 redesign**: CSS prefix changed from `ghost-*` to `g-*`. Colors: `--color-g-base` (#050510), `--color-g-green` (#00e87b), `--color-g-cyan` (#06b6d4), etc. Classes: `bg-g-base`, `text-g-green`. Components: `.g-card`, `.g-card-glow`, `.g-btn`, `.g-btn-primary`, `.g-badge`, `.g-table`, `.glass`, `.glass-strong`, `.gradient-text`, `.gradient-accent`. Backward compat aliases (`.card-panel`, `.btn-ghost`, `.btn-danger`, `.page-title`, `.page-subtitle`) still work.
- `output: 'export'` in next.config.ts → static build at `frontend/out/`
- Node.js: default installer `C:\Program Files\nodejs\` (on PATH). Alternative: portable at `C:\Users\Abbas\AppData\Local\nodejs-portable\node-v22.14.0-win-x64\`

### Key Architecture Decisions
- SSE route disabled for static build (`app/api/stream/route.ts.disabled`)
- All API files use `http://localhost:7000` as BASE (full URL, not relative)
- FastAPI serves Next.js static build when `frontend/out/` exists
- `if _frontend_out.exists():` guard is safe — `dashboard.html` still works without out/ dir
- **SPA routing**: `app.mount("/_next", ...)` for assets only + `@app.get("/{full_path:path}")` catch-all for SPA routing. Never use `app.mount("/", StaticFiles(html=True))` — it shadows all explicit routes

### File Structure
```
frontend/
├── app/
│   ├── ai-log/page.tsx
│   ├── dashboard/page.tsx
│   ├── keywords/page.tsx
│   ├── listings/page.tsx
│   ├── settings/page.tsx
│   ├── sites/page.tsx
│   ├── globals.css (Tailwind v4 @theme, cyberpunk palette, btn-ghost, btn-danger)
│   ├── layout.tsx
│   ├── page.tsx (redirects to /dashboard)
│   └── providers.tsx ('use client' QueryClientProvider)
├── components/
│   ├── ai-log/ (AILogCard, AILogFeed)
│   ├── dashboard/ (StatsGrid, ActivityLog)
│   ├── keywords/ (KeywordRow, KeywordsTable — dnd-kit, ScoringRulesPanel)
│   ├── layout/ (Header, Nav, StatusBar)
│   ├── listings/ (ListingRow, ListingsTable, ListingDetailPanel, ImageGallery)
│   └── sites/ (SiteRow, SitesTable — dnd-kit)
├── hooks/ (useSSE, useListings, useCountdown, useKeywords, useSites)
├── lib/
│   ├── api/ (listings, keywords, sites, config, engine, system, ai, scoring-rules)
│   ├── types.ts
│   └── utils.ts (cn, formatUptime)
├── store/ (engineStore, settingsStore)
├── out/ (static build — served by FastAPI)
└── vitest.config.ts (has @/ alias resolution)
```

### Tests (21 passing)
- theme.test.ts (2), types.test.ts (4), engineStore.test.ts (3)
- listings-api.test.ts (3), useCountdown.test.ts (2)
- StatsGrid.test.tsx (1), StatusBar.test.tsx (3), ListingRow.test.tsx (3)

---

## Dev Paths & Commands

### Node.js
- **Default installer** (`C:\Program Files\nodejs\`): usually on system PATH — open new terminal and run `node --version`; no manual PATH needed.
- **Portable** (if used): set PATH in each shell before `npx`/`npm`:
  - PowerShell: `$env:PATH = "C:\Users\Abbas\AppData\Local\nodejs-portable\node-v22.14.0-win-x64;$env:PATH"`
  - Bash: `export PATH="/c/Users/Abbas/AppData/Local/nodejs-portable/node-v22.14.0-win-x64:$PATH"`

### Common Commands
```bash
# Start backend
python worker.py                          # port 7000

# Frontend build (from project root)
export PATH="..." && npm run build --prefix frontend

# Run tests (from project root)
export PATH="..." && cd frontend && npx vitest run

# Combined build + test
export PATH="..." && cd frontend && npx vitest run && npm run build
```

---

## Key Gotchas (Never Forget These)

### Python / Backend
- **`threading.Event` not `asyncio.Event`** for cross-thread signalling — asyncio.Event is not thread-safe
- **SQLite `enabled` field**: Store as `int` (1/0), never `bool` — `filter(enabled==1)` breaks on Python `True`
- **JS in f-strings**: Use backtick template literals, never apostrophes — breaks Python f-string parsing
- **`db.flush()` before `db.commit()`**: SQLite WAL locking requires this
- **`calculate_attribute_score()` opens own DB session** — don't call inside an already-open session loop
- **Keyword batching retry warnings**: the dashboard pins only retry candidates with `attempt_count > 0` via `/api/scrape/progress` (queued-but-never-attempted items stay hidden until first failure).
- **Per-site visible override precedence**: `show_browser=true` forces visible mode for all sites and ignores per-site `custom_visible_browser`; `show_browser=false` applies per-site visibility.
- **Retry-tracking state model**: `scrape_round_items` moves through `pending | in_progress | done | failed`; each active round has a 4-hour retry window, and `/api/scrape/progress` computes hourly `warn_due` from `last_hour_warn_at` fallback logic (`first_pending_at`, then round start).
- **`closing_alerts_sent`**: JSON list — always `json.loads` + append, never overwrite

### FastAPI Routing
- **NEVER `app.mount("/", StaticFiles(html=True))`** with explicit routes you want to preserve — the root mount captures ALL paths. SPA fallback returns 200+index.html for unknown paths, shadowing `@app.get("/legacy")` etc.
- **Fix**: `app.mount("/_next", ...)` for assets + `@app.get("/{full_path:path}")` explicit catch-all

### API Format
- **`/api/config` GET** returns flat `{key: value}` dict, NOT `Config[]` array
- **`/api/config` POST** expects flat dict — not `[{key, value}]` array
- Frontend `fetchConfig()` must call `.then(data => setConfig(data))` directly — no `.map()`

### Frontend
- **SSE disabled in static export** — `EventSource('/api/stream')` always 404s. Use HTTP polling instead
- **`createPortal` needs SSR guard** — `document.body` unavailable during Next.js SSR. Use `useEffect(() => setMounted(true), [])` before rendering portal

### HiBid
- **Apollo cache**: Search results pages have 0 Lot entries — images only from detail pages
- **Image dedup**: Use full URL including query string (HiBid CDN uses same path `img.axd` for all images, differentiated only by query params)

---

## Redis Layer (added Session 32)

- Enabled via `REDIS_URL` env var — app runs unchanged without it (graceful fallback)
- `_redis_publish("new_listing", {...})` fires on every alert; channel: `ghostnode:events`
- `_redis_set_stats(_stats)` — call after every meaningful `_stats` write (cycle end, pause, resume, running)
- `GET /api/redis/status` — connectivity check + cached stats hash
- Install locally: `pip install redis` (auto-installed in Docker)

## N18 — Lot Description (added Session 32)

- `JS_DETAIL_TEXT` runs on the already-open detail page (same visit as images — zero extra HTTP cost)
- `description` column: `Text`, max 1500 chars, nullable — in `Listing` ORM + `_migrate_schema()` for both SQLite and PostgreSQL
- `_build_ai_prompt(title, ai_target, description="")` — description appended as `Lot description:` block when non-empty
- `_ai_analyze(title, ai_target, description="")` — forwards to prompt builder
- Re-analysis: after detail fetch, `ai_match=1` listings with `ai_target` are re-evaluated with description; verdict updated in DB

## Docker (added Session 32)

- `Dockerfile` — Python 3.11-slim + Playwright Chromium; serves port 7000
- `docker-compose.yml` — ghostnode + postgres:16-alpine + redis:7-alpine; health checks before app starts
- `DATABASE_URL` + `REDIS_URL` wired in compose env
- `shm_size: 512mb` on ghostnode service — Chromium requires shared memory

## Pending Work (Priority Order)

1. **Docker Testing** — run `docker compose up --build`; verify PostgreSQL migration, Redis pub/sub, Playwright headless in container
2. **Frontend: show description** — add `description` field to `ListingDetailPanel` (already in `to_dict()`, just needs UI)
3. **vLLM production inference** — replace Groq free tier for production scale