# Ghost Node — Frontend Migration Design Spec

**Date:** 2026-03-11
**Author:** Claude (Session 16)
**Status:** Approved by Abbas — Rev 4 (pragmatic: only use features that solve real problems)

---

## Overview

Replace `dashboard.html` (3,515-line single-file vanilla JS dashboard) with a Next.js +
React + TypeScript + Tailwind frontend. Next.js is used for its full capability set —
not just routing. The Python FastAPI backend (`worker.py`, all endpoints, port 8000) is
untouched during migration. The new frontend lives in `frontend/` alongside `dashboard.html`
and is built tab-by-tab. When all 6 tabs are complete, FastAPI serves the Next.js build at `/`.

---

## Goals

- Preserve cyberpunk terminal aesthetic while modernizing with Framer Motion animations
- Replace global JS vars + `setInterval` polling with Zustand stores + TanStack Query
- Use Next.js features only where they solve a real problem: image lazy-loading, font
  optimization, SSE for real-time engine state
- Structure for SaaS multi-user expansion (auth route group + middleware placeholder) at
  zero cost now — no actual auth code until it's needed
- Zero backend changes during migration — all FastAPI endpoints consumed as-is

---

## Tech Stack

| Layer | Technology | Purpose |
|---|---|---|
| Framework | Next.js 14 (App Router) | Routing, API routes, SSE, Server Actions, image/font optimization |
| Language | TypeScript | Type safety across all components |
| Styling | Tailwind CSS | Cyberpunk theme as utility classes |
| Components | shadcn/ui | Accessible Modal, Badge, Button, Toast primitives |
| State | Zustand | Engine status + settings cache |
| Data fetching | TanStack Query | Auto-refetch, caching, mutations |
| Animations | Framer Motion | Entrance animations, price pulse, modal springs |
| Drag-drop | @dnd-kit/core + @dnd-kit/sortable | Keywords and sites table reordering |
| Icons | Lucide React | Replaces emoji indicators |
| Fonts | next/font (Inter + JetBrains Mono) | Zero layout shift, self-hosted, 2 lines of code |
| Images | next/image | Lazy-load 100+ listing thumbnails from external CDNs |
| Real-time | Next.js API Route (SSE) | Pushes engine state to browser — eliminates all setInterval polling |
| Auth (future) | NextAuth.js — deferred | Added when SaaS multi-user launches, slot reserved |
| PWA, i18n | Deferred to SaaS phase | Not needed until multi-user hosting — adds complexity now for no gain |

---

## How Next.js Is Used (Only Where It Solves a Real Problem)

### 1. next/image — Lot Thumbnail Optimization
Every `ListingRow` and `ImageGallery` uses `<Image>` from `next/image` instead of `<img>`.
Benefits for Ghost Node:
- **Lazy loading** — only loads images when they scroll into view (critical: listings table can have 100+ rows each with images from external auction CDNs)
- **Blur placeholder** — shows a blurred ghost-panel rectangle while the CDN image loads
- **Size optimization** — Next.js resizes images to the exact rendered size (48×48 for thumbnails), saving bandwidth from HiBid/eBay CDNs
- **Error handling** — `onError` falls back gracefully without crashing

```tsx
<Image
  src={listing.images[0]}
  alt={listing.title}
  width={48} height={48}
  className="object-cover rounded"
  placeholder="blur"
  blurDataURL="data:image/png;base64,..."
  onError={(e) => e.currentTarget.style.display = 'none'}
/>
```

### 2. next/font — Zero Layout Shift
Fonts loaded via `next/font/google` are self-hosted by Next.js at build time — no
external Google Fonts request, no layout shift, no flash of unstyled text:

```ts
// app/layout.tsx
import { Inter, JetBrains_Mono } from 'next/font/google'
const inter = Inter({ subsets: ['latin'], variable: '--font-inter' })
const mono  = JetBrains_Mono({ subsets: ['latin'], variable: '--font-mono' })
```

Inter is used for labels, headings, nav. JetBrains Mono for prices, scores, timestamps,
lot IDs — anything that looks better monospaced in a terminal-style dashboard.

### 3. Next.js API Routes — SSE Real-Time Stream
`app/api/stream/route.ts` is a Next.js API route that opens a Server-Sent Events
connection to the browser. It polls FastAPI's `/api/stats` and `/api/listings/countdown-sync`
internally and pushes updates to connected clients — eliminating client-side polling loops.

```ts
// app/api/stream/route.ts
export async function GET() {
  const encoder = new TextEncoder()
  const stream = new ReadableStream({
    async start(controller) {
      while (true) {
        const stats = await fetch('http://localhost:8000/api/stats').then(r => r.json())
        controller.enqueue(encoder.encode(`data: ${JSON.stringify(stats)}\n\n`))
        await new Promise(r => setTimeout(r, 3000))
      }
    }
  })
  return new Response(stream, { headers: { 'Content-Type': 'text/event-stream' } })
}
```

Components subscribe via `EventSource('/api/stream')`. When Redis pub/sub is added,
this route becomes a true push stream — no code changes needed in the components.

### 4. Next.js Middleware — Auth Guard Placeholder
`middleware.ts` is a no-op now (passes all requests through). When NextAuth.js is added
for multi-user SaaS, the auth guard drops in here — no component restructuring needed.
Zero cost to include it now, saves a refactor later.

**Deferred (added when actually needed, not before):**
- **next-pwa** — push notifications + offline mode. Needs a push server. Add in SaaS phase.
- **next-intl** — Arabic/English i18n. Add when multi-user hosting launches.
- **BFF API routes** — TanStack Query handles parallel fetches cleanly. A BFF middleman adds debugging complexity for no gain at single-user scale.
- **Server Actions** — TanStack mutations already give optimistic updates + error handling. Two mutation patterns in one codebase is inconsistent.
- **Metadata API** — meaningless for a local dashboard. Add when public marketing site exists.

---

## Cyberpunk Theme Palette

```ts
// tailwind.config.ts
colors: {
  ghost: {
    bg:      '#0a0e17',
    panel:   '#111827',
    border:  '#1f2937',
    accent:  '#00ff88',
    gold:    '#fbbf24',
    danger:  '#ef4444',
    dim:     '#6b7280',
    text:    '#e5e7eb',
  }
}
```

**shadcn/ui note:** Override shadcn's CSS variables in `globals.css` to use ghost palette.
**RTL note:** All layout uses Tailwind's `rtl:` variant for Arabic direction support.

---

## Project Structure

```
ClaudeAuction2/
├── worker.py / models.py / database.py / dashboard.html   ← untouched during migration
└── frontend/
    ├── app/
    │   ├── providers.tsx                ← QueryClientProvider + Zustand (client component)
    │   ├── layout.tsx                   ← root layout: Header + Nav + StatusBar + fonts
    │   ├── page.tsx                     ← redirect → /dashboard
    │   ├── dashboard/page.tsx
    │   ├── listings/page.tsx
    │   ├── keywords/page.tsx
    │   ├── sites/page.tsx
    │   ├── settings/page.tsx
    │   ├── ai-log/page.tsx
    │   ├── (auth)/                      ← NextAuth.js route group (placeholder)
    │   │   ├── login/page.tsx
    │   │   └── register/page.tsx
    │   └── api/
    │       ├── stream/route.ts          ← SSE real-time stream (engine state → browser)
    │       └── auth/[...nextauth]/      ← NextAuth.js placeholder (empty until SaaS)
    │
    ├── components/
    │   ├── layout/
    │   │   ├── Header.tsx
    │   │   ├── Nav.tsx
    │   │   └── StatusBar.tsx
    │   ├── listings/
    │   │   ├── ListingsTable.tsx
    │   │   ├── ListingRow.tsx
    │   │   ├── ListingDetailPanel.tsx
    │   │   └── ImageGallery.tsx
    │   ├── keywords/
    │   │   ├── KeywordsTable.tsx
    │   │   └── KeywordRow.tsx
    │   ├── sites/
    │   │   ├── SitesTable.tsx
    │   │   └── SiteRow.tsx
    │   └── ui/                          ← shadcn/ui primitives
    │
    ├── store/
    │   ├── engineStore.ts
    │   └── settingsStore.ts
    │
    ├── hooks/
    │   ├── useListings.ts
    │   ├── useKeywords.ts
    │   ├── useSites.ts
    │   ├── useStats.ts
    │   ├── useCountdown.ts
    │   └── useSSE.ts                    ← EventSource('/api/stream') hook
    │
    ├── lib/
    │   ├── api/
    │   │   ├── listings.ts
    │   │   ├── keywords.ts
    │   │   ├── sites.ts
    │   │   ├── config.ts
    │   │   ├── engine.ts
    │   │   ├── ai.ts
    │   │   └── system.ts
    │   └── types.ts
    │
    ├── public/
    │   └── icons/                       ← Ghost Node favicon + app icon
    │
    ├── middleware.ts                     ← auth guard (no-op now → NextAuth when SaaS)
    ├── next.config.ts                   ← rewrites + PWA + i18n config
    ├── tailwind.config.ts
    └── package.json
```

---

## Architecture

### Client Component Boundary
`providers.tsx` is the client boundary. It wraps `QueryClientProvider` and any Zustand
hydration. All interactive components are Client Components (`"use client"`). `layout.tsx`
stays a Server Component that imports `providers.tsx` as a child.

### Data Flow
```
FastAPI (port 8000)
    ↓  lib/api/*.ts wrappers (parse JSON fields before returning)
TanStack Query hooks + Next.js BFF routes (with next/cache)
    ↓  typed, parsed data
React components (all Client Components)
    ↓  mutations via Server Actions (forms) or TanStack mutations (real-time)
FastAPI endpoints
    ↓  invalidate query cache / revalidatePath
Components re-render
```

### Real-Time Data Flow (SSE)
```
FastAPI /api/stats (polling inside Next.js API route)
    ↓  app/api/stream/route.ts (SSE)
useSSE() hook (EventSource)
    ↓  writes to engineStore (Zustand)
StatusBar + Dashboard tab re-render instantly
```

### State Management
| State | Where | Why |
|---|---|---|
| Listings | TanStack Query | Server state, background refetch |
| Keywords | TanStack Query | Server state |
| Sites | TanStack Query | Server state |
| Engine status | Zustand engineStore (fed by SSE) | Shared across Header + StatusBar |
| Config | Zustand settingsStore | No refetch on tab switch |
| Countdown | useCountdown hook | Pure client-side ticker |

---

## API Coverage (lib/api/ mapping)

| File | Endpoints |
|---|---|
| `listings.ts` | GET/DELETE /api/listings, countdown-sync, refresh-status. Export endpoints via `window.location`. |
| `keywords.ts` | GET/POST/PUT/DELETE /api/keywords, reorder |
| `sites.ts` | GET/POST/PUT/DELETE /api/sites, reorder, login, adapt, selectors |
| `config.ts` | GET/POST /api/config |
| `engine.ts` | pause/resume/restart/kill |
| `ai.ts` | POST /api/ai/test, GET/DELETE /api/ai/debug/log |
| `system.ts` | telegram test, backup download (`window.location`), backup restore (FormData), debug/db |

Export endpoints (`/api/export/csv|json|html`) and backup download are triggered via
`window.location.href` — browser downloads directly from FastAPI, bypassing Next.js proxy.

---

## TypeScript Types (lib/types.ts)

```ts
interface Listing {
  id: number
  title: string
  price: number | null
  currency: string
  price_raw: string
  price_usd: number | null
  time_left: string
  time_left_mins: number | null
  link: string
  score: number
  keyword: string
  site_name: string
  timestamp: string
  price_updated_at: string | null
  ai_match: 1 | 0 | null
  ai_reason: string | null
  location: string | null
  images: string[]           // parsed from JSON string in lib/api/listings.ts
  closing_alerts_sent: number[]  // parsed from JSON string in lib/api/listings.ts
}

interface Keyword {
  id: number
  term: string
  weight: number
  ai_target: string | null
  min_price: number | null
  max_price: number | null
  sort_order: number
}

interface TargetSite {
  id: number
  name: string
  url_template: string
  search_selector: string
  enabled: 0 | 1
  max_pages: number
  last_error: string | null
  error_count: number
  consecutive_failures: number
  last_success_at: string | null
  cooldown_until: string | null
  requires_login: 0 | 1
  login_url: string | null
  login_check_selector: string | null
  login_enabled: 0 | 1
  sort_order: number
}

// Fetched separately via GET /api/sites/{id}/selectors
interface SiteSelectors {
  site_id: number
  confidence: number
  container_sel: string | null
  title_sel: string | null
  price_sel: string | null
  time_sel: string | null
  link_sel: string | null
  next_page_sel: string | null
  stale: boolean
  provider: 'groq' | 'ollama' | null
  generated_at: string | null
}

// Real fields from worker.py _stats + /api/stats handler
interface Stats {
  total_scanned: number
  total_alerts: number
  last_cycle: string
  engine_status: 'Idle' | 'Running' | 'Paused'
  uptime_start: number
  uptime_seconds: number
}

interface Config {
  key: string
  value: string
}
```

---

## useCountdown Hook

**State shape:**
```ts
const [offsets, setOffsets] = useState<Record<number, number>>({})
const [syncedAt, setSyncedAt] = useState<number>(Date.now())
```

**Behaviour:**
1. Mount: fetch `/api/listings/countdown-sync` → populate offsets, record syncedAt
2. Every 1s: `currentMins = offsets[id] - (Date.now() - syncedAt) / 60000`
3. Every 60s: re-fetch sync endpoint → refresh offsets + syncedAt
4. New listings from `useListings` refetch that aren't in offsets map → return `null`, component falls back to raw `time_left` string
5. Returns `(id: number) => number | null`

---

## Loading and Error States

| State | UI |
|---|---|
| Loading | Skeleton rows — ghost-panel bg, shimmer pulse animation |
| Backend unreachable | Red banner: "ENGINE OFFLINE — cannot reach localhost:8000" + retry |
| Empty | Dim: "NO LISTINGS CAPTURED YET" (per tab equivalent) |
| Mutation failure | shadcn Toast: `[OPERATION] failed — [error]` in ghost-danger color |

StatusBar shows "OFFLINE" in red if `/api/stream` SSE disconnects.

---

## Key Components

### StatusBar.tsx — Engine state, uptime, scanned, alerts. Fed by SSE via useSSE → engineStore.
### ListingRow.tsx — Thumbnail (next/image), countdown, price, AI badge, score. Click → detail panel.
### ImageGallery.tsx — Up to 10 images, next/image, click → new tab. Hidden when empty.
### KeywordRow.tsx — dnd-kit drag, inline edit, 💰 price filter, 🤖 AI target modal.
### SiteRow.tsx — Health badge, confidence (useSiteSelectors), adapt button, full edit modal.

---

## Animations (Framer Motion)

| Interaction | Animation |
|---|---|
| Listings load | Stagger slide-in from bottom (0.05s per row) |
| New listing | Pulse green border 2s |
| Price update | Gold flash on price cell |
| Modal | Scale + fade spring 0.3s |
| Tab switch | Cross-fade 0.15s |
| Drag row | Spring physics + shadow lift |
| Closing-soon | Red pulse on time_left badge |

---

## Migration Strategy

| Phase | Build | Acceptance Criteria |
|---|---|---|
| 0 | Scaffold: Next.js, Tailwind theme, providers, layout shell, SSE route, PWA, i18n skeleton | Port 3000 starts, nav renders in cyberpunk, StatusBar shows live engine state via SSE |
| 1 | Dashboard tab | Stats cards show live totals, activity log auto-scrolls, engine controls work |
| 2 | Listings tab | Table with thumbnails (next/image), countdown ticks, detail panel, delete works |
| 3 | Keywords tab | CRUD, inline edit, drag-drop reorder, AI modal with live test, price filter |
| 4 | Sites tab | CRUD, adapt triggers, confidence badge, health badge, edit modal |
| 5 | Settings tab | All config keys load/save, telegram test, backup download/restore |
| 6 | AI Log tab | Log cards stream via SSE, filter buttons, clear works |
| 7 | FastAPI serves Next.js build at `/`, dashboard.html retired | All tabs functional via :8000 |

### Phase 7 — Production Serving
`output: 'export'` produces static build in `frontend/out/`. FastAPI mounts it:

```python
# worker.py — Phase 7 addition
from fastapi.staticfiles import StaticFiles
app.mount("/", StaticFiles(directory="frontend/out", html=True), name="static")
```

When multi-user auth is added, switch to Next.js as a Node server (remove `output: 'export'`,
run `next start` on port 3000, Nginx proxies `/api` to FastAPI and `/` to Next.js).

---

## Future-Proofing Notes (Deferred — Add When Actually Needed)

- **WebSocket** — swap `app/api/stream/route.ts` from SSE to WebSocket; components don't change
- **PWA + push notifications** — add `next-pwa` when push server infrastructure exists (SaaS phase)
- **Arabic i18n** — add `next-intl` when multi-user SaaS launches; Tailwind `rtl:` variant handles layout
- **Auth** — `middleware.ts` guard + `(auth)/` login/register pages; zero restructuring needed
- **Docker** — `docker-compose.yml` with `python:3.12-slim` + `node:20-alpine` services

---

*Spec written: 2026-03-11 — Session 16 (Rev 3 — Next.js full capabilities)*