39246-vm/docs/MEMORY.md

Ghost Node — Session Memory

What belongs here: Current architecture state, key gotchas, reusable patterns, dev paths. What does NOT belong here: Session history (→ PROGRESS.md or ARCHIVE.md).

⚠️ MD Update Rule — DYNAMIC, NOT STATIC

Only update the MD file(s) whose content was actually touched this session. Do NOT update all files after every task. Each file gets only what is relevant to it.

• Update this file only when a new reusable gotcha, pattern, or architecture decision was made.
• Update PROGRESS.md whenever any code file was changed.
• Update CLAUDE.md only if API endpoints, DB schema, config keys, or architecture changed.
• Move old PROGRESS.md session entries to ARCHIVE.md when they're no longer active context.

---

Project Identity

• Ghost Node International Auction Sniper v2.7
• Owner: Abbas, Baghdad, Iraq
• Path: C:\Users\Abbas\Documents\Downloads\ClaudeAuction2\
• Backend: Python FastAPI (worker.py, 5000+ lines), port 7000
• NOT a git repository

---

Frontend Stack (Current — Session 16, fully migrated)

• Location: frontend/ directory
• Next.js 16.1.6, React 19, TypeScript, Tailwind CSS v4 (NOT v3)
• Tailwind v4: CSS variables via @theme {} in globals.css — NO tailwind.config.ts
• Session 27 redesign: CSS prefix changed from ghost-* to g-*. Colors: --color-g-base (#050510), --color-g-green (#00e87b), --color-g-cyan (#06b6d4), etc. Classes: bg-g-base, text-g-green. Components: .g-card, .g-card-glow, .g-btn, .g-btn-primary, .g-badge, .g-table, .glass, .glass-strong, .gradient-text, .gradient-accent. Backward compat aliases (.card-panel, .btn-ghost, .btn-danger, .page-title, .page-subtitle) still work.
• output: 'export' in next.config.ts → static build at frontend/out/
• Node.js: default installer C:\Program Files\nodejs\ (on PATH). Alternative: portable at C:\Users\Abbas\AppData\Local\nodejs-portable\node-v22.14.0-win-x64\

Key Architecture Decisions

• SSE route disabled for static build (app/api/stream/route.ts.disabled)
• All API files use http://localhost:7000 as BASE (full URL, not relative)
• FastAPI serves Next.js static build when frontend/out/ exists
• if _frontend_out.exists(): guard is safe — dashboard.html still works without out/ dir
• SPA routing: app.mount("/_next", ...) for assets only + @app.get("/{full_path:path}") catch-all for SPA routing. Never use app.mount("/", StaticFiles(html=True)) — it shadows all explicit routes

File Structure

frontend/
├── app/
│   ├── ai-log/page.tsx
│   ├── dashboard/page.tsx
│   ├── keywords/page.tsx
│   ├── listings/page.tsx
│   ├── settings/page.tsx
│   ├── sites/page.tsx
│   ├── globals.css (Tailwind v4 @theme, cyberpunk palette, btn-ghost, btn-danger)
│   ├── layout.tsx
│   ├── page.tsx (redirects to /dashboard)
│   └── providers.tsx ('use client' QueryClientProvider)
├── components/
│   ├── ai-log/ (AILogCard, AILogFeed)
│   ├── dashboard/ (StatsGrid, ActivityLog)
│   ├── keywords/ (KeywordRow, KeywordsTable — dnd-kit, ScoringRulesPanel)
│   ├── layout/ (Header, Nav, StatusBar)
│   ├── listings/ (ListingRow, ListingsTable, ListingDetailPanel, ImageGallery)
│   └── sites/ (SiteRow, SitesTable — dnd-kit)
├── hooks/ (useSSE, useListings, useCountdown, useKeywords, useSites)
├── lib/
│   ├── api/ (listings, keywords, sites, config, engine, system, ai, scoring-rules)
│   ├── types.ts
│   └── utils.ts (cn, formatUptime)
├── store/ (engineStore, settingsStore)
├── out/ (static build — served by FastAPI)
└── vitest.config.ts (has @/ alias resolution)

Tests (21 passing)

• theme.test.ts (2), types.test.ts (4), engineStore.test.ts (3)
• listings-api.test.ts (3), useCountdown.test.ts (2)
• StatsGrid.test.tsx (1), StatusBar.test.tsx (3), ListingRow.test.tsx (3)

---

Dev Paths & Commands

Node.js

• Default installer (C:\Program Files\nodejs\): usually on system PATH — open new terminal and run node --version; no manual PATH needed.
• Portable (if used): set PATH in each shell before npx/npm:
  • PowerShell: $env:PATH = "C:\Users\Abbas\AppData\Local\nodejs-portable\node-v22.14.0-win-x64;$env:PATH"
  • Bash: export PATH="/c/Users/Abbas/AppData/Local/nodejs-portable/node-v22.14.0-win-x64:$PATH"

Common Commands

# Start backend
python worker.py                          # port 7000

# Frontend build (from project root)
export PATH="..." && npm run build --prefix frontend

# Run tests (from project root)
export PATH="..." && cd frontend && npx vitest run

# Combined build + test
export PATH="..." && cd frontend && npx vitest run && npm run build

---

Key Gotchas (Never Forget These)

Python / Backend

• threading.Event not asyncio.Event for cross-thread signalling — asyncio.Event is not thread-safe
• SQLite enabled field: Store as int (1/0), never bool — filter(enabled==1) breaks on Python True
• JS in f-strings: Use backtick template literals, never apostrophes — breaks Python f-string parsing
• db.flush() before db.commit(): SQLite WAL locking requires this
• calculate_attribute_score() opens own DB session — don't call inside an already-open session loop
• Keyword batching retry warnings: the dashboard pins only retry candidates with attempt_count > 0 via /api/scrape/progress (queued-but-never-attempted items stay hidden until first failure).
• Per-site visible override precedence: show_browser=true forces visible mode for all sites and ignores per-site custom_visible_browser; show_browser=false applies per-site visibility.
• Retry-tracking state model: scrape_round_items moves through pending | in_progress | done | failed; each active round has a 4-hour retry window, and /api/scrape/progress computes hourly warn_due from last_hour_warn_at fallback logic (first_pending_at, then round start).
• closing_alerts_sent: JSON list — always json.loads + append, never overwrite

FastAPI Routing

• NEVER app.mount("/", StaticFiles(html=True)) with explicit routes you want to preserve — the root mount captures ALL paths. SPA fallback returns 200+index.html for unknown paths, shadowing @app.get("/legacy") etc.
• Fix: app.mount("/_next", ...) for assets + @app.get("/{full_path:path}") explicit catch-all

API Format

• /api/config GET returns flat {key: value} dict, NOT Config[] array
• /api/config POST expects flat dict — not [{key, value}] array
• Frontend fetchConfig() must call .then(data => setConfig(data)) directly — no .map()

Frontend

• SSE disabled in static export — EventSource('/api/stream') always 404s. Use HTTP polling instead
• createPortal needs SSR guard — document.body unavailable during Next.js SSR. Use useEffect(() => setMounted(true), []) before rendering portal

HiBid

• Apollo cache: Search results pages have 0 Lot entries — images only from detail pages
• Image dedup: Use full URL including query string (HiBid CDN uses same path img.axd for all images, differentiated only by query params)

---

Redis Layer (added Session 32)

• Enabled via REDIS_URL env var — app runs unchanged without it (graceful fallback)
• _redis_publish("new_listing", {...}) fires on every alert; channel: ghostnode:events
• _redis_set_stats(_stats) — call after every meaningful _stats write (cycle end, pause, resume, running)
• GET /api/redis/status — connectivity check + cached stats hash
• Install locally: pip install redis (auto-installed in Docker)

N18 — Lot Description (added Session 32)

• JS_DETAIL_TEXT runs on the already-open detail page (same visit as images — zero extra HTTP cost)
• description column: Text, max 1500 chars, nullable — in Listing ORM + _migrate_schema() for both SQLite and PostgreSQL
• _build_ai_prompt(title, ai_target, description="") — description appended as Lot description: block when non-empty
• _ai_analyze(title, ai_target, description="") — forwards to prompt builder
• Re-analysis: after detail fetch, ai_match=1 listings with ai_target are re-evaluated with description; verdict updated in DB

Docker (added Session 32)

• Dockerfile — Python 3.11-slim + Playwright Chromium; serves port 7000
• docker-compose.yml — ghostnode + postgres:16-alpine + redis:7-alpine; health checks before app starts
• DATABASE_URL + REDIS_URL wired in compose env
• shm_size: 512mb on ghostnode service — Chromium requires shared memory

Pending Work (Priority Order)

1. Docker Testing — run docker compose up --build; verify PostgreSQL migration, Redis pub/sub, Playwright headless in container
2. Frontend: show description — add description field to ListingDetailPanel (already in to_dict(), just needs UI)
3. vLLM production inference — replace Groq free tier for production scale