123 lines
4.1 KiB
Markdown
123 lines
4.1 KiB
Markdown
# π» Ghost Node β Auction Sniper
|
||
|
||
A modular, multi-threaded auction sniper for Windows 11 with human-mimicry,
|
||
Telegram C2 command, and a cyber-terminal dashboard.
|
||
|
||
---
|
||
|
||
## Architecture
|
||
|
||
```
|
||
worker.py
|
||
βββ Thread A β FastAPI + Dashboard (port 7000)
|
||
βββ Thread B β Async Playwright Scraper (nuclear_engine)
|
||
βββ Thread C β Telegram C2 Command Listener
|
||
```
|
||
|
||
## File Structure
|
||
|
||
```
|
||
ghost_node/
|
||
βββ database.py # SQLAlchemy engine & session factory
|
||
βββ models.py # ORM models, heuristic scoring, DB seeder
|
||
βββ worker.py # FastAPI routes + scraper + Telegram C2
|
||
βββ dashboard.html # Cyberpunk terminal UI (5 tabs)
|
||
βββ requirements.txt
|
||
βββ README.md
|
||
```
|
||
|
||
---
|
||
|
||
## Installation (Windows 11, Python 3.11+)
|
||
|
||
```powershell
|
||
# 1. Create virtual environment
|
||
python -m venv .venv
|
||
.venv\Scripts\activate
|
||
|
||
# 2. Install dependencies
|
||
pip install -r requirements.txt
|
||
|
||
# 3. Install Chromium for Playwright
|
||
playwright install chromium
|
||
|
||
# 4. Launch Ghost Node
|
||
python worker.py
|
||
```
|
||
|
||
Then open: **http://localhost:7000**
|
||
|
||
---
|
||
|
||
## Quick Setup via Dashboard
|
||
|
||
1. **Settings tab** β Enter your Telegram Bot Token and Chat ID, set cycle timer.
|
||
2. **Keywords tab** β Your top search terms are pre-seeded (RTX 4090, Tab S10, etc.).
|
||
3. **Target Sites tab** β eBay UK and eBay US are pre-seeded with `{keyword}` templates.
|
||
4. The engine starts automatically β watch the Dashboard for live stats.
|
||
5. Optional resilience features β per-site visible override (`custom_visible_browser`, ignored when global `show_browser=true`) plus keyword batching with pinned βKeyword Retry Trackingβ in the dashboard.
|
||
|
||
---
|
||
|
||
## Retry Tracking Mechanics
|
||
|
||
- Per-site visibility precedence:
|
||
- `show_browser=true` forces visible mode for all sites and ignores per-site `custom_visible_browser`.
|
||
- `show_browser=false` applies per-site visibility (`custom_visible_browser=1` visible, `0` headless).
|
||
- Keyword batching is controlled by `keyword_batch_enabled`.
|
||
- Persistent retry state is stored in `scrape_rounds` + `scrape_round_items` with statuses:
|
||
`pending`, `in_progress`, `done`, `failed`.
|
||
- Pending retries have a 4-hour window per active round; expired pending/in-progress items are marked failed.
|
||
- The dashboard "Keyword Retry Tracking" section is sourced from `GET /api/scrape/progress`.
|
||
- Pinned items are pending retries with `attempt_count > 0`.
|
||
- Hourly warning due-state is returned as `warn_due` using `last_hour_warn_at` fallback logic (`first_pending_at`, then round start).
|
||
|
||
---
|
||
|
||
## Telegram C2 Commands
|
||
|
||
| Command | Action |
|
||
|-------------|---------------------------------------------|
|
||
| `/status` | Node health report (engine, uptime, alerts) |
|
||
| `/pause` | Pause the scraper engine |
|
||
| `/resume` | Resume the scraper engine |
|
||
| `/listings` | Last 5 captured listings |
|
||
|
||
---
|
||
|
||
## Heuristic Scoring
|
||
|
||
**Positive (+10):** GB, RAM, Unlocked, SSD, RTX, GPU, S10, NVMe, OLED
|
||
**Negative (β10):** Cover, Case, Sleeve, Box Only, Broken, For Parts, Faulty
|
||
|
||
Only listings with **score β₯ 0** trigger Telegram alerts.
|
||
Keyword weight multiplies the base score.
|
||
|
||
---
|
||
|
||
## Baghdad Network Optimisations
|
||
|
||
- 60-second global page timeout per navigation
|
||
- Per-site try/except β one timeout never crashes the full cycle
|
||
- Resource blocking (images/fonts/videos) for faster loads
|
||
- 5β15s randomised jitter between keywords
|
||
|
||
---
|
||
|
||
## Adding New Sites
|
||
|
||
In the **Target Sites** tab:
|
||
- **Name**: Human label (e.g. `Gumtree UK`)
|
||
- **URL Template**: Full search URL with `{keyword}` placeholder
|
||
e.g. `https://www.gumtree.com/search?search_category=all&q={keyword}`
|
||
- **Search Selector**: CSS selector for the search box (leave blank to use URL directly)
|
||
|
||
---
|
||
|
||
## Notes
|
||
|
||
- `sniper.db` is created automatically in the working directory on first run.
|
||
- All config (Telegram token, timer) is persisted to SQLite on every Save.
|
||
- Playwright-stealth is implemented via `add_init_script` to avoid detection.
|
||
- Rotating user-agents are applied per browser context (one context per site per cycle).
|