39246-vm/docs/superpowers/specs/2026-03-11-frontend-migration-design.md

Ghost Node — Frontend Migration Design Spec

Date: 2026-03-11 Author: Claude (Session 16) Status: Approved by Abbas — Rev 4 (pragmatic: only use features that solve real problems)

---

Overview

Replace dashboard.html (3,515-line single-file vanilla JS dashboard) with a Next.js + React + TypeScript + Tailwind frontend. Next.js is used for its full capability set — not just routing. The Python FastAPI backend (worker.py, all endpoints, port 8000) is untouched during migration. The new frontend lives in frontend/ alongside dashboard.html and is built tab-by-tab. When all 6 tabs are complete, FastAPI serves the Next.js build at /.

---

Goals

• Preserve cyberpunk terminal aesthetic while modernizing with Framer Motion animations
• Replace global JS vars + setInterval polling with Zustand stores + TanStack Query
• Use Next.js features only where they solve a real problem: image lazy-loading, font optimization, SSE for real-time engine state
• Structure for SaaS multi-user expansion (auth route group + middleware placeholder) at zero cost now — no actual auth code until it's needed
• Zero backend changes during migration — all FastAPI endpoints consumed as-is

---

Tech Stack

Layer	Technology	Purpose
Framework	Next.js 14 (App Router)	Routing, API routes, SSE, Server Actions, image/font optimization
Language	TypeScript	Type safety across all components
Styling	Tailwind CSS	Cyberpunk theme as utility classes
Components	shadcn/ui	Accessible Modal, Badge, Button, Toast primitives
State	Zustand	Engine status + settings cache
Data fetching	TanStack Query	Auto-refetch, caching, mutations
Animations	Framer Motion	Entrance animations, price pulse, modal springs
Drag-drop	@dnd-kit/core + @dnd-kit/sortable	Keywords and sites table reordering
Icons	Lucide React	Replaces emoji indicators
Fonts	next/font (Inter + JetBrains Mono)	Zero layout shift, self-hosted, 2 lines of code
Images	next/image	Lazy-load 100+ listing thumbnails from external CDNs
Real-time	Next.js API Route (SSE)	Pushes engine state to browser — eliminates all setInterval polling
Auth (future)	NextAuth.js — deferred	Added when SaaS multi-user launches, slot reserved
PWA, i18n	Deferred to SaaS phase	Not needed until multi-user hosting — adds complexity now for no gain

---

How Next.js Is Used (Only Where It Solves a Real Problem)

1. next/image — Lot Thumbnail Optimization

Every ListingRow and ImageGallery uses <Image> from next/image instead of <img>. Benefits for Ghost Node:

• Lazy loading — only loads images when they scroll into view (critical: listings table can have 100+ rows each with images from external auction CDNs)
• Blur placeholder — shows a blurred ghost-panel rectangle while the CDN image loads
• Size optimization — Next.js resizes images to the exact rendered size (48×48 for thumbnails), saving bandwidth from HiBid/eBay CDNs
• Error handling — onError falls back gracefully without crashing

<Image
  src={listing.images[0]}
  alt={listing.title}
  width={48} height={48}
  className="object-cover rounded"
  placeholder="blur"
  blurDataURL="data:image/png;base64,..."
  onError={(e) => e.currentTarget.style.display = 'none'}
/>

2. next/font — Zero Layout Shift

Fonts loaded via next/font/google are self-hosted by Next.js at build time — no external Google Fonts request, no layout shift, no flash of unstyled text:

// app/layout.tsx
import { Inter, JetBrains_Mono } from 'next/font/google'
const inter = Inter({ subsets: ['latin'], variable: '--font-inter' })
const mono  = JetBrains_Mono({ subsets: ['latin'], variable: '--font-mono' })

Inter is used for labels, headings, nav. JetBrains Mono for prices, scores, timestamps, lot IDs — anything that looks better monospaced in a terminal-style dashboard.

3. Next.js API Routes — SSE Real-Time Stream

app/api/stream/route.ts is a Next.js API route that opens a Server-Sent Events connection to the browser. It polls FastAPI's /api/stats and /api/listings/countdown-sync internally and pushes updates to connected clients — eliminating client-side polling loops.

// app/api/stream/route.ts
export async function GET() {
  const encoder = new TextEncoder()
  const stream = new ReadableStream({
    async start(controller) {
      while (true) {
        const stats = await fetch('http://localhost:8000/api/stats').then(r => r.json())
        controller.enqueue(encoder.encode(`data: ${JSON.stringify(stats)}\n\n`))
        await new Promise(r => setTimeout(r, 3000))
      }
    }
  })
  return new Response(stream, { headers: { 'Content-Type': 'text/event-stream' } })
}

Components subscribe via EventSource('/api/stream'). When Redis pub/sub is added, this route becomes a true push stream — no code changes needed in the components.

4. Next.js Middleware — Auth Guard Placeholder

middleware.ts is a no-op now (passes all requests through). When NextAuth.js is added for multi-user SaaS, the auth guard drops in here — no component restructuring needed. Zero cost to include it now, saves a refactor later.

Deferred (added when actually needed, not before):

• next-pwa — push notifications + offline mode. Needs a push server. Add in SaaS phase.
• next-intl — Arabic/English i18n. Add when multi-user hosting launches.
• BFF API routes — TanStack Query handles parallel fetches cleanly. A BFF middleman adds debugging complexity for no gain at single-user scale.
• Server Actions — TanStack mutations already give optimistic updates + error handling. Two mutation patterns in one codebase is inconsistent.
• Metadata API — meaningless for a local dashboard. Add when public marketing site exists.

---

Cyberpunk Theme Palette

// tailwind.config.ts
colors: {
  ghost: {
    bg:      '#0a0e17',
    panel:   '#111827',
    border:  '#1f2937',
    accent:  '#00ff88',
    gold:    '#fbbf24',
    danger:  '#ef4444',
    dim:     '#6b7280',
    text:    '#e5e7eb',
  }
}

shadcn/ui note: Override shadcn's CSS variables in globals.css to use ghost palette. RTL note: All layout uses Tailwind's rtl: variant for Arabic direction support.

---

Project Structure

ClaudeAuction2/
├── worker.py / models.py / database.py / dashboard.html   ← untouched during migration
└── frontend/
    ├── app/
    │   ├── providers.tsx                ← QueryClientProvider + Zustand (client component)
    │   ├── layout.tsx                   ← root layout: Header + Nav + StatusBar + fonts
    │   ├── page.tsx                     ← redirect → /dashboard
    │   ├── dashboard/page.tsx
    │   ├── listings/page.tsx
    │   ├── keywords/page.tsx
    │   ├── sites/page.tsx
    │   ├── settings/page.tsx
    │   ├── ai-log/page.tsx
    │   ├── (auth)/                      ← NextAuth.js route group (placeholder)
    │   │   ├── login/page.tsx
    │   │   └── register/page.tsx
    │   └── api/
    │       ├── stream/route.ts          ← SSE real-time stream (engine state → browser)
    │       └── auth/[...nextauth]/      ← NextAuth.js placeholder (empty until SaaS)
    │
    ├── components/
    │   ├── layout/
    │   │   ├── Header.tsx
    │   │   ├── Nav.tsx
    │   │   └── StatusBar.tsx
    │   ├── listings/
    │   │   ├── ListingsTable.tsx
    │   │   ├── ListingRow.tsx
    │   │   ├── ListingDetailPanel.tsx
    │   │   └── ImageGallery.tsx
    │   ├── keywords/
    │   │   ├── KeywordsTable.tsx
    │   │   └── KeywordRow.tsx
    │   ├── sites/
    │   │   ├── SitesTable.tsx
    │   │   └── SiteRow.tsx
    │   └── ui/                          ← shadcn/ui primitives
    │
    ├── store/
    │   ├── engineStore.ts
    │   └── settingsStore.ts
    │
    ├── hooks/
    │   ├── useListings.ts
    │   ├── useKeywords.ts
    │   ├── useSites.ts
    │   ├── useStats.ts
    │   ├── useCountdown.ts
    │   └── useSSE.ts                    ← EventSource('/api/stream') hook
    │
    ├── lib/
    │   ├── api/
    │   │   ├── listings.ts
    │   │   ├── keywords.ts
    │   │   ├── sites.ts
    │   │   ├── config.ts
    │   │   ├── engine.ts
    │   │   ├── ai.ts
    │   │   └── system.ts
    │   └── types.ts
    │
    ├── public/
    │   └── icons/                       ← Ghost Node favicon + app icon
    │
    ├── middleware.ts                     ← auth guard (no-op now → NextAuth when SaaS)
    ├── next.config.ts                   ← rewrites + PWA + i18n config
    ├── tailwind.config.ts
    └── package.json

---

Architecture

Client Component Boundary

providers.tsx is the client boundary. It wraps QueryClientProvider and any Zustand hydration. All interactive components are Client Components ("use client"). layout.tsx stays a Server Component that imports providers.tsx as a child.

Data Flow

FastAPI (port 8000)
    ↓  lib/api/*.ts wrappers (parse JSON fields before returning)
TanStack Query hooks + Next.js BFF routes (with next/cache)
    ↓  typed, parsed data
React components (all Client Components)
    ↓  mutations via Server Actions (forms) or TanStack mutations (real-time)
FastAPI endpoints
    ↓  invalidate query cache / revalidatePath
Components re-render

Real-Time Data Flow (SSE)

FastAPI /api/stats (polling inside Next.js API route)
    ↓  app/api/stream/route.ts (SSE)
useSSE() hook (EventSource)
    ↓  writes to engineStore (Zustand)
StatusBar + Dashboard tab re-render instantly

State Management

State	Where	Why
Listings	TanStack Query	Server state, background refetch
Keywords	TanStack Query	Server state
Sites	TanStack Query	Server state
Engine status	Zustand engineStore (fed by SSE)	Shared across Header + StatusBar
Config	Zustand settingsStore	No refetch on tab switch
Countdown	useCountdown hook	Pure client-side ticker

---

API Coverage (lib/api/ mapping)

File	Endpoints
listings.ts	GET/DELETE /api/listings, countdown-sync, refresh-status. Export endpoints via window.location.
keywords.ts	GET/POST/PUT/DELETE /api/keywords, reorder
sites.ts	GET/POST/PUT/DELETE /api/sites, reorder, login, adapt, selectors
config.ts	GET/POST /api/config
engine.ts	pause/resume/restart/kill
ai.ts	POST /api/ai/test, GET/DELETE /api/ai/debug/log
system.ts	telegram test, backup download (window.location), backup restore (FormData), debug/db

Export endpoints (/api/export/csv|json|html) and backup download are triggered via window.location.href — browser downloads directly from FastAPI, bypassing Next.js proxy.

---

TypeScript Types (lib/types.ts)

interface Listing {
  id: number
  title: string
  price: number | null
  currency: string
  price_raw: string
  price_usd: number | null
  time_left: string
  time_left_mins: number | null
  link: string
  score: number
  keyword: string
  site_name: string
  timestamp: string
  price_updated_at: string | null
  ai_match: 1 | 0 | null
  ai_reason: string | null
  location: string | null
  images: string[]           // parsed from JSON string in lib/api/listings.ts
  closing_alerts_sent: number[]  // parsed from JSON string in lib/api/listings.ts
}

interface Keyword {
  id: number
  term: string
  weight: number
  ai_target: string | null
  min_price: number | null
  max_price: number | null
  sort_order: number
}

interface TargetSite {
  id: number
  name: string
  url_template: string
  search_selector: string
  enabled: 0 | 1
  max_pages: number
  last_error: string | null
  error_count: number
  consecutive_failures: number
  last_success_at: string | null
  cooldown_until: string | null
  requires_login: 0 | 1
  login_url: string | null
  login_check_selector: string | null
  login_enabled: 0 | 1
  sort_order: number
}

// Fetched separately via GET /api/sites/{id}/selectors
interface SiteSelectors {
  site_id: number
  confidence: number
  container_sel: string | null
  title_sel: string | null
  price_sel: string | null
  time_sel: string | null
  link_sel: string | null
  next_page_sel: string | null
  stale: boolean
  provider: 'groq' | 'ollama' | null
  generated_at: string | null
}

// Real fields from worker.py _stats + /api/stats handler
interface Stats {
  total_scanned: number
  total_alerts: number
  last_cycle: string
  engine_status: 'Idle' | 'Running' | 'Paused'
  uptime_start: number
  uptime_seconds: number
}

interface Config {
  key: string
  value: string
}

---

useCountdown Hook

State shape:

const [offsets, setOffsets] = useState<Record<number, number>>({})
const [syncedAt, setSyncedAt] = useState<number>(Date.now())

Behaviour:

1. Mount: fetch /api/listings/countdown-sync → populate offsets, record syncedAt
2. Every 1s: currentMins = offsets[id] - (Date.now() - syncedAt) / 60000
3. Every 60s: re-fetch sync endpoint → refresh offsets + syncedAt
4. New listings from useListings refetch that aren't in offsets map → return null, component falls back to raw time_left string
5. Returns (id: number) => number | null

---

Loading and Error States

State	UI
Loading	Skeleton rows — ghost-panel bg, shimmer pulse animation
Backend unreachable	Red banner: "ENGINE OFFLINE — cannot reach localhost:8000" + retry
Empty	Dim: "NO LISTINGS CAPTURED YET" (per tab equivalent)
Mutation failure	shadcn Toast: [OPERATION] failed — [error] in ghost-danger color

StatusBar shows "OFFLINE" in red if /api/stream SSE disconnects.

---

Key Components

StatusBar.tsx — Engine state, uptime, scanned, alerts. Fed by SSE via useSSE → engineStore.

ListingRow.tsx — Thumbnail (next/image), countdown, price, AI badge, score. Click → detail panel.

ImageGallery.tsx — Up to 10 images, next/image, click → new tab. Hidden when empty.

KeywordRow.tsx — dnd-kit drag, inline edit, 💰 price filter, 🤖 AI target modal.

SiteRow.tsx — Health badge, confidence (useSiteSelectors), adapt button, full edit modal.

---

Animations (Framer Motion)

Interaction	Animation
Listings load	Stagger slide-in from bottom (0.05s per row)
New listing	Pulse green border 2s
Price update	Gold flash on price cell
Modal	Scale + fade spring 0.3s
Tab switch	Cross-fade 0.15s
Drag row	Spring physics + shadow lift
Closing-soon	Red pulse on time_left badge

---

Migration Strategy

Phase	Build	Acceptance Criteria
0	Scaffold: Next.js, Tailwind theme, providers, layout shell, SSE route, PWA, i18n skeleton	Port 3000 starts, nav renders in cyberpunk, StatusBar shows live engine state via SSE
1	Dashboard tab	Stats cards show live totals, activity log auto-scrolls, engine controls work
2	Listings tab	Table with thumbnails (next/image), countdown ticks, detail panel, delete works
3	Keywords tab	CRUD, inline edit, drag-drop reorder, AI modal with live test, price filter
4	Sites tab	CRUD, adapt triggers, confidence badge, health badge, edit modal
5	Settings tab	All config keys load/save, telegram test, backup download/restore
6	AI Log tab	Log cards stream via SSE, filter buttons, clear works
7	FastAPI serves Next.js build at /, dashboard.html retired	All tabs functional via :8000

Phase 7 — Production Serving

output: 'export' produces static build in frontend/out/. FastAPI mounts it:

# worker.py — Phase 7 addition
from fastapi.staticfiles import StaticFiles
app.mount("/", StaticFiles(directory="frontend/out", html=True), name="static")

When multi-user auth is added, switch to Next.js as a Node server (remove output: 'export', run next start on port 3000, Nginx proxies /api to FastAPI and / to Next.js).

---

Future-Proofing Notes (Deferred — Add When Actually Needed)

• WebSocket — swap app/api/stream/route.ts from SSE to WebSocket; components don't change
• PWA + push notifications — add next-pwa when push server infrastructure exists (SaaS phase)
• Arabic i18n — add next-intl when multi-user SaaS launches; Tailwind rtl: variant handles layout
• Auth — middleware.ts guard + (auth)/ login/register pages; zero restructuring needed
• Docker — docker-compose.yml with python:3.12-slim + node:20-alpine services

---

Spec written: 2026-03-11 — Session 16 (Rev 3 — Next.js full capabilities)