462 lines
17 KiB
Plaintext
462 lines
17 KiB
Plaintext
# Ghost Node — Frontend Migration Design Spec
|
||
|
||
**Date:** 2026-03-11
|
||
**Author:** Claude (Session 16)
|
||
**Status:** Approved by Abbas — Rev 4 (pragmatic: only use features that solve real problems)
|
||
|
||
---
|
||
|
||
## Overview
|
||
|
||
Replace `dashboard.html` (3,515-line single-file vanilla JS dashboard) with a Next.js +
|
||
React + TypeScript + Tailwind frontend. Next.js is used for its full capability set —
|
||
not just routing. The Python FastAPI backend (`worker.py`, all endpoints, port 8000) is
|
||
untouched during migration. The new frontend lives in `frontend/` alongside `dashboard.html`
|
||
and is built tab-by-tab. When all 6 tabs are complete, FastAPI serves the Next.js build at `/`.
|
||
|
||
---
|
||
|
||
## Goals
|
||
|
||
- Preserve cyberpunk terminal aesthetic while modernizing with Framer Motion animations
|
||
- Replace global JS vars + `setInterval` polling with Zustand stores + TanStack Query
|
||
- Use Next.js features only where they solve a real problem: image lazy-loading, font
|
||
optimization, SSE for real-time engine state
|
||
- Structure for SaaS multi-user expansion (auth route group + middleware placeholder) at
|
||
zero cost now — no actual auth code until it's needed
|
||
- Zero backend changes during migration — all FastAPI endpoints consumed as-is
|
||
|
||
---
|
||
|
||
## Tech Stack
|
||
|
||
| Layer | Technology | Purpose |
|
||
|---|---|---|
|
||
| Framework | Next.js 14 (App Router) | Routing, API routes, SSE, Server Actions, image/font optimization |
|
||
| Language | TypeScript | Type safety across all components |
|
||
| Styling | Tailwind CSS | Cyberpunk theme as utility classes |
|
||
| Components | shadcn/ui | Accessible Modal, Badge, Button, Toast primitives |
|
||
| State | Zustand | Engine status + settings cache |
|
||
| Data fetching | TanStack Query | Auto-refetch, caching, mutations |
|
||
| Animations | Framer Motion | Entrance animations, price pulse, modal springs |
|
||
| Drag-drop | @dnd-kit/core + @dnd-kit/sortable | Keywords and sites table reordering |
|
||
| Icons | Lucide React | Replaces emoji indicators |
|
||
| Fonts | next/font (Inter + JetBrains Mono) | Zero layout shift, self-hosted, 2 lines of code |
|
||
| Images | next/image | Lazy-load 100+ listing thumbnails from external CDNs |
|
||
| Real-time | Next.js API Route (SSE) | Pushes engine state to browser — eliminates all setInterval polling |
|
||
| Auth (future) | NextAuth.js — deferred | Added when SaaS multi-user launches, slot reserved |
|
||
| PWA, i18n | Deferred to SaaS phase | Not needed until multi-user hosting — adds complexity now for no gain |
|
||
|
||
---
|
||
|
||
## How Next.js Is Used (Only Where It Solves a Real Problem)
|
||
|
||
### 1. next/image — Lot Thumbnail Optimization
|
||
Every `ListingRow` and `ImageGallery` uses `<Image>` from `next/image` instead of `<img>`.
|
||
Benefits for Ghost Node:
|
||
- **Lazy loading** — only loads images when they scroll into view (critical: listings table can have 100+ rows each with images from external auction CDNs)
|
||
- **Blur placeholder** — shows a blurred ghost-panel rectangle while the CDN image loads
|
||
- **Size optimization** — Next.js resizes images to the exact rendered size (48×48 for thumbnails), saving bandwidth from HiBid/eBay CDNs
|
||
- **Error handling** — `onError` falls back gracefully without crashing
|
||
|
||
```tsx
|
||
<Image
|
||
src={listing.images[0]}
|
||
alt={listing.title}
|
||
width={48} height={48}
|
||
className="object-cover rounded"
|
||
placeholder="blur"
|
||
blurDataURL="data:image/png;base64,..."
|
||
onError={(e) => e.currentTarget.style.display = 'none'}
|
||
/>
|
||
```
|
||
|
||
### 2. next/font — Zero Layout Shift
|
||
Fonts loaded via `next/font/google` are self-hosted by Next.js at build time — no
|
||
external Google Fonts request, no layout shift, no flash of unstyled text:
|
||
|
||
```ts
|
||
// app/layout.tsx
|
||
import { Inter, JetBrains_Mono } from 'next/font/google'
|
||
const inter = Inter({ subsets: ['latin'], variable: '--font-inter' })
|
||
const mono = JetBrains_Mono({ subsets: ['latin'], variable: '--font-mono' })
|
||
```
|
||
|
||
Inter is used for labels, headings, nav. JetBrains Mono for prices, scores, timestamps,
|
||
lot IDs — anything that looks better monospaced in a terminal-style dashboard.
|
||
|
||
### 3. Next.js API Routes — SSE Real-Time Stream
|
||
`app/api/stream/route.ts` is a Next.js API route that opens a Server-Sent Events
|
||
connection to the browser. It polls FastAPI's `/api/stats` and `/api/listings/countdown-sync`
|
||
internally and pushes updates to connected clients — eliminating client-side polling loops.
|
||
|
||
```ts
|
||
// app/api/stream/route.ts
|
||
export async function GET() {
|
||
const encoder = new TextEncoder()
|
||
const stream = new ReadableStream({
|
||
async start(controller) {
|
||
while (true) {
|
||
const stats = await fetch('http://localhost:8000/api/stats').then(r => r.json())
|
||
controller.enqueue(encoder.encode(`data: ${JSON.stringify(stats)}\n\n`))
|
||
await new Promise(r => setTimeout(r, 3000))
|
||
}
|
||
}
|
||
})
|
||
return new Response(stream, { headers: { 'Content-Type': 'text/event-stream' } })
|
||
}
|
||
```
|
||
|
||
Components subscribe via `EventSource('/api/stream')`. When Redis pub/sub is added,
|
||
this route becomes a true push stream — no code changes needed in the components.
|
||
|
||
### 4. Next.js Middleware — Auth Guard Placeholder
|
||
`middleware.ts` is a no-op now (passes all requests through). When NextAuth.js is added
|
||
for multi-user SaaS, the auth guard drops in here — no component restructuring needed.
|
||
Zero cost to include it now, saves a refactor later.
|
||
|
||
**Deferred (added when actually needed, not before):**
|
||
- **next-pwa** — push notifications + offline mode. Needs a push server. Add in SaaS phase.
|
||
- **next-intl** — Arabic/English i18n. Add when multi-user hosting launches.
|
||
- **BFF API routes** — TanStack Query handles parallel fetches cleanly. A BFF middleman adds debugging complexity for no gain at single-user scale.
|
||
- **Server Actions** — TanStack mutations already give optimistic updates + error handling. Two mutation patterns in one codebase is inconsistent.
|
||
- **Metadata API** — meaningless for a local dashboard. Add when public marketing site exists.
|
||
|
||
---
|
||
|
||
## Cyberpunk Theme Palette
|
||
|
||
```ts
|
||
// tailwind.config.ts
|
||
colors: {
|
||
ghost: {
|
||
bg: '#0a0e17',
|
||
panel: '#111827',
|
||
border: '#1f2937',
|
||
accent: '#00ff88',
|
||
gold: '#fbbf24',
|
||
danger: '#ef4444',
|
||
dim: '#6b7280',
|
||
text: '#e5e7eb',
|
||
}
|
||
}
|
||
```
|
||
|
||
**shadcn/ui note:** Override shadcn's CSS variables in `globals.css` to use ghost palette.
|
||
**RTL note:** All layout uses Tailwind's `rtl:` variant for Arabic direction support.
|
||
|
||
---
|
||
|
||
## Project Structure
|
||
|
||
```
|
||
ClaudeAuction2/
|
||
├── worker.py / models.py / database.py / dashboard.html ← untouched during migration
|
||
└── frontend/
|
||
├── app/
|
||
│ ├── providers.tsx ← QueryClientProvider + Zustand (client component)
|
||
│ ├── layout.tsx ← root layout: Header + Nav + StatusBar + fonts
|
||
│ ├── page.tsx ← redirect → /dashboard
|
||
│ ├── dashboard/page.tsx
|
||
│ ├── listings/page.tsx
|
||
│ ├── keywords/page.tsx
|
||
│ ├── sites/page.tsx
|
||
│ ├── settings/page.tsx
|
||
│ ├── ai-log/page.tsx
|
||
│ ├── (auth)/ ← NextAuth.js route group (placeholder)
|
||
│ │ ├── login/page.tsx
|
||
│ │ └── register/page.tsx
|
||
│ └── api/
|
||
│ ├── stream/route.ts ← SSE real-time stream (engine state → browser)
|
||
│ └── auth/[...nextauth]/ ← NextAuth.js placeholder (empty until SaaS)
|
||
│
|
||
├── components/
|
||
│ ├── layout/
|
||
│ │ ├── Header.tsx
|
||
│ │ ├── Nav.tsx
|
||
│ │ └── StatusBar.tsx
|
||
│ ├── listings/
|
||
│ │ ├── ListingsTable.tsx
|
||
│ │ ├── ListingRow.tsx
|
||
│ │ ├── ListingDetailPanel.tsx
|
||
│ │ └── ImageGallery.tsx
|
||
│ ├── keywords/
|
||
│ │ ├── KeywordsTable.tsx
|
||
│ │ └── KeywordRow.tsx
|
||
│ ├── sites/
|
||
│ │ ├── SitesTable.tsx
|
||
│ │ └── SiteRow.tsx
|
||
│ └── ui/ ← shadcn/ui primitives
|
||
│
|
||
├── store/
|
||
│ ├── engineStore.ts
|
||
│ └── settingsStore.ts
|
||
│
|
||
├── hooks/
|
||
│ ├── useListings.ts
|
||
│ ├── useKeywords.ts
|
||
│ ├── useSites.ts
|
||
│ ├── useStats.ts
|
||
│ ├── useCountdown.ts
|
||
│ └── useSSE.ts ← EventSource('/api/stream') hook
|
||
│
|
||
├── lib/
|
||
│ ├── api/
|
||
│ │ ├── listings.ts
|
||
│ │ ├── keywords.ts
|
||
│ │ ├── sites.ts
|
||
│ │ ├── config.ts
|
||
│ │ ├── engine.ts
|
||
│ │ ├── ai.ts
|
||
│ │ └── system.ts
|
||
│ └── types.ts
|
||
│
|
||
├── public/
|
||
│ └── icons/ ← Ghost Node favicon + app icon
|
||
│
|
||
├── middleware.ts ← auth guard (no-op now → NextAuth when SaaS)
|
||
├── next.config.ts ← rewrites + PWA + i18n config
|
||
├── tailwind.config.ts
|
||
└── package.json
|
||
```
|
||
|
||
---
|
||
|
||
## Architecture
|
||
|
||
### Client Component Boundary
|
||
`providers.tsx` is the client boundary. It wraps `QueryClientProvider` and any Zustand
|
||
hydration. All interactive components are Client Components (`"use client"`). `layout.tsx`
|
||
stays a Server Component that imports `providers.tsx` as a child.
|
||
|
||
### Data Flow
|
||
```
|
||
FastAPI (port 8000)
|
||
↓ lib/api/*.ts wrappers (parse JSON fields before returning)
|
||
TanStack Query hooks + Next.js BFF routes (with next/cache)
|
||
↓ typed, parsed data
|
||
React components (all Client Components)
|
||
↓ mutations via Server Actions (forms) or TanStack mutations (real-time)
|
||
FastAPI endpoints
|
||
↓ invalidate query cache / revalidatePath
|
||
Components re-render
|
||
```
|
||
|
||
### Real-Time Data Flow (SSE)
|
||
```
|
||
FastAPI /api/stats (polling inside Next.js API route)
|
||
↓ app/api/stream/route.ts (SSE)
|
||
useSSE() hook (EventSource)
|
||
↓ writes to engineStore (Zustand)
|
||
StatusBar + Dashboard tab re-render instantly
|
||
```
|
||
|
||
### State Management
|
||
| State | Where | Why |
|
||
|---|---|---|
|
||
| Listings | TanStack Query | Server state, background refetch |
|
||
| Keywords | TanStack Query | Server state |
|
||
| Sites | TanStack Query | Server state |
|
||
| Engine status | Zustand engineStore (fed by SSE) | Shared across Header + StatusBar |
|
||
| Config | Zustand settingsStore | No refetch on tab switch |
|
||
| Countdown | useCountdown hook | Pure client-side ticker |
|
||
|
||
---
|
||
|
||
## API Coverage (lib/api/ mapping)
|
||
|
||
| File | Endpoints |
|
||
|---|---|
|
||
| `listings.ts` | GET/DELETE /api/listings, countdown-sync, refresh-status. Export endpoints via `window.location`. |
|
||
| `keywords.ts` | GET/POST/PUT/DELETE /api/keywords, reorder |
|
||
| `sites.ts` | GET/POST/PUT/DELETE /api/sites, reorder, login, adapt, selectors |
|
||
| `config.ts` | GET/POST /api/config |
|
||
| `engine.ts` | pause/resume/restart/kill |
|
||
| `ai.ts` | POST /api/ai/test, GET/DELETE /api/ai/debug/log |
|
||
| `system.ts` | telegram test, backup download (`window.location`), backup restore (FormData), debug/db |
|
||
|
||
Export endpoints (`/api/export/csv|json|html`) and backup download are triggered via
|
||
`window.location.href` — browser downloads directly from FastAPI, bypassing Next.js proxy.
|
||
|
||
---
|
||
|
||
## TypeScript Types (lib/types.ts)
|
||
|
||
```ts
|
||
interface Listing {
|
||
id: number
|
||
title: string
|
||
price: number | null
|
||
currency: string
|
||
price_raw: string
|
||
price_usd: number | null
|
||
time_left: string
|
||
time_left_mins: number | null
|
||
link: string
|
||
score: number
|
||
keyword: string
|
||
site_name: string
|
||
timestamp: string
|
||
price_updated_at: string | null
|
||
ai_match: 1 | 0 | null
|
||
ai_reason: string | null
|
||
location: string | null
|
||
images: string[] // parsed from JSON string in lib/api/listings.ts
|
||
closing_alerts_sent: number[] // parsed from JSON string in lib/api/listings.ts
|
||
}
|
||
|
||
interface Keyword {
|
||
id: number
|
||
term: string
|
||
weight: number
|
||
ai_target: string | null
|
||
min_price: number | null
|
||
max_price: number | null
|
||
sort_order: number
|
||
}
|
||
|
||
interface TargetSite {
|
||
id: number
|
||
name: string
|
||
url_template: string
|
||
search_selector: string
|
||
enabled: 0 | 1
|
||
max_pages: number
|
||
last_error: string | null
|
||
error_count: number
|
||
consecutive_failures: number
|
||
last_success_at: string | null
|
||
cooldown_until: string | null
|
||
requires_login: 0 | 1
|
||
login_url: string | null
|
||
login_check_selector: string | null
|
||
login_enabled: 0 | 1
|
||
sort_order: number
|
||
}
|
||
|
||
// Fetched separately via GET /api/sites/{id}/selectors
|
||
interface SiteSelectors {
|
||
site_id: number
|
||
confidence: number
|
||
container_sel: string | null
|
||
title_sel: string | null
|
||
price_sel: string | null
|
||
time_sel: string | null
|
||
link_sel: string | null
|
||
next_page_sel: string | null
|
||
stale: boolean
|
||
provider: 'groq' | 'ollama' | null
|
||
generated_at: string | null
|
||
}
|
||
|
||
// Real fields from worker.py _stats + /api/stats handler
|
||
interface Stats {
|
||
total_scanned: number
|
||
total_alerts: number
|
||
last_cycle: string
|
||
engine_status: 'Idle' | 'Running' | 'Paused'
|
||
uptime_start: number
|
||
uptime_seconds: number
|
||
}
|
||
|
||
interface Config {
|
||
key: string
|
||
value: string
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
## useCountdown Hook
|
||
|
||
**State shape:**
|
||
```ts
|
||
const [offsets, setOffsets] = useState<Record<number, number>>({})
|
||
const [syncedAt, setSyncedAt] = useState<number>(Date.now())
|
||
```
|
||
|
||
**Behaviour:**
|
||
1. Mount: fetch `/api/listings/countdown-sync` → populate offsets, record syncedAt
|
||
2. Every 1s: `currentMins = offsets[id] - (Date.now() - syncedAt) / 60000`
|
||
3. Every 60s: re-fetch sync endpoint → refresh offsets + syncedAt
|
||
4. New listings from `useListings` refetch that aren't in offsets map → return `null`, component falls back to raw `time_left` string
|
||
5. Returns `(id: number) => number | null`
|
||
|
||
---
|
||
|
||
## Loading and Error States
|
||
|
||
| State | UI |
|
||
|---|---|
|
||
| Loading | Skeleton rows — ghost-panel bg, shimmer pulse animation |
|
||
| Backend unreachable | Red banner: "ENGINE OFFLINE — cannot reach localhost:8000" + retry |
|
||
| Empty | Dim: "NO LISTINGS CAPTURED YET" (per tab equivalent) |
|
||
| Mutation failure | shadcn Toast: `[OPERATION] failed — [error]` in ghost-danger color |
|
||
|
||
StatusBar shows "OFFLINE" in red if `/api/stream` SSE disconnects.
|
||
|
||
---
|
||
|
||
## Key Components
|
||
|
||
### StatusBar.tsx — Engine state, uptime, scanned, alerts. Fed by SSE via useSSE → engineStore.
|
||
### ListingRow.tsx — Thumbnail (next/image), countdown, price, AI badge, score. Click → detail panel.
|
||
### ImageGallery.tsx — Up to 10 images, next/image, click → new tab. Hidden when empty.
|
||
### KeywordRow.tsx — dnd-kit drag, inline edit, 💰 price filter, 🤖 AI target modal.
|
||
### SiteRow.tsx — Health badge, confidence (useSiteSelectors), adapt button, full edit modal.
|
||
|
||
---
|
||
|
||
## Animations (Framer Motion)
|
||
|
||
| Interaction | Animation |
|
||
|---|---|
|
||
| Listings load | Stagger slide-in from bottom (0.05s per row) |
|
||
| New listing | Pulse green border 2s |
|
||
| Price update | Gold flash on price cell |
|
||
| Modal | Scale + fade spring 0.3s |
|
||
| Tab switch | Cross-fade 0.15s |
|
||
| Drag row | Spring physics + shadow lift |
|
||
| Closing-soon | Red pulse on time_left badge |
|
||
|
||
---
|
||
|
||
## Migration Strategy
|
||
|
||
| Phase | Build | Acceptance Criteria |
|
||
|---|---|---|
|
||
| 0 | Scaffold: Next.js, Tailwind theme, providers, layout shell, SSE route, PWA, i18n skeleton | Port 3000 starts, nav renders in cyberpunk, StatusBar shows live engine state via SSE |
|
||
| 1 | Dashboard tab | Stats cards show live totals, activity log auto-scrolls, engine controls work |
|
||
| 2 | Listings tab | Table with thumbnails (next/image), countdown ticks, detail panel, delete works |
|
||
| 3 | Keywords tab | CRUD, inline edit, drag-drop reorder, AI modal with live test, price filter |
|
||
| 4 | Sites tab | CRUD, adapt triggers, confidence badge, health badge, edit modal |
|
||
| 5 | Settings tab | All config keys load/save, telegram test, backup download/restore |
|
||
| 6 | AI Log tab | Log cards stream via SSE, filter buttons, clear works |
|
||
| 7 | FastAPI serves Next.js build at `/`, dashboard.html retired | All tabs functional via :8000 |
|
||
|
||
### Phase 7 — Production Serving
|
||
`output: 'export'` produces static build in `frontend/out/`. FastAPI mounts it:
|
||
|
||
```python
|
||
# worker.py — Phase 7 addition
|
||
from fastapi.staticfiles import StaticFiles
|
||
app.mount("/", StaticFiles(directory="frontend/out", html=True), name="static")
|
||
```
|
||
|
||
When multi-user auth is added, switch to Next.js as a Node server (remove `output: 'export'`,
|
||
run `next start` on port 3000, Nginx proxies `/api` to FastAPI and `/` to Next.js).
|
||
|
||
---
|
||
|
||
## Future-Proofing Notes (Deferred — Add When Actually Needed)
|
||
|
||
- **WebSocket** — swap `app/api/stream/route.ts` from SSE to WebSocket; components don't change
|
||
- **PWA + push notifications** — add `next-pwa` when push server infrastructure exists (SaaS phase)
|
||
- **Arabic i18n** — add `next-intl` when multi-user SaaS launches; Tailwind `rtl:` variant handles layout
|
||
- **Auth** — `middleware.ts` guard + `(auth)/` login/register pages; zero restructuring needed
|
||
- **Docker** — `docker-compose.yml` with `python:3.12-slim` + `node:20-alpine` services
|
||
|
||
---
|
||
|
||
*Spec written: 2026-03-11 — Session 16 (Rev 3 — Next.js full capabilities)*
|