297 lines
12 KiB
Markdown
297 lines
12 KiB
Markdown
# VM Deployment (Flatlogic Executor)
|
|
|
|
This is how Flatlogic production/preview works: no Docker — PM2 + local PostgreSQL +
|
|
Cloudflare tunnel. The end of this document contains a reference **VM file structure**
|
|
(executor, pm2, project).
|
|
|
|
> **Docker deployment** (compose / single-image / staging) is covered in a separate
|
|
> document: [`deployment-docker.md`](./deployment-docker.md).
|
|
|
|
The project consists of two applications:
|
|
|
|
- `frontend/` — Vite + React + TypeScript (SPA). Build output → `frontend/dist/`.
|
|
- `backend/` — Express + Sequelize on TypeScript/ESM. Build output → `backend/dist/`.
|
|
|
|
The project lives in `~/executor/workspace`, processes are managed by PM2, DB is local
|
|
PostgreSQL, external access via Cloudflare tunnel (`cloudflared`). Docker is not used here.
|
|
|
|
## 1.1. Topology
|
|
|
|
```
|
|
Browser
|
|
│ https://<subdomain>.dev.flatlogic.app
|
|
▼
|
|
cloudflared (tunnel)
|
|
│
|
|
▼
|
|
nginx :8080
|
|
├── / → frontend :3001 (Vite)
|
|
├── /api → backend :3000 (Express)
|
|
└── /api-docs → backend :3000
|
|
│
|
|
▼
|
|
PostgreSQL :5432 (local)
|
|
```
|
|
|
|
- In `NODE_ENV=dev_stage` the backend listens on **3000** (`config.serverPort`), frontend on **3001**.
|
|
- The browser opens the tunnel domain; the frontend calls the API via relative path `/api`
|
|
(see `frontend/src/shared/constants/api.ts`), nginx proxies it to the backend —
|
|
same origin, so CORS/CSRF are not a problem.
|
|
|
|
## 1.2. Environment Variables
|
|
|
|
**Injected by the platform** into the pm2 environment of the backend process (values are
|
|
secrets, not committed to the repository):
|
|
|
|
| Variable | Purpose |
|
|
|---|---|
|
|
| `NODE_ENV=dev_stage` | mode (production-like, but without strict production checks) |
|
|
| `SECRET_KEY` | JWT signature (required — otherwise backend won't start) |
|
|
| `DB_HOST`, `DB_PORT`, `DB_NAME`, `DB_USER`, `DB_PASS` | connection to local Postgres (`DB_PASS` = project UUID) |
|
|
| `GOOGLE_CLIENT_ID/SECRET`, `MS_CLIENT_ID/SECRET` | OAuth (optional) |
|
|
| `SMTP_*`, `EMAIL_*`, `MAIL_*` | email (optional) |
|
|
| `CF_TUNNEL_*` | Cloudflare tunnel (for `cloudflared`, not for the application) |
|
|
|
|
**From committed `backend/.env`** (not production-level secrets):
|
|
|
|
- `PORT` (optional, defaults to 8080).
|
|
- Seed passwords and DB credentials for development are hardcoded in `shared/constants/app.ts`
|
|
and `seeders/*`; `.env` is only needed for overrides.
|
|
- `backend/src/shared/config/load-env.ts` finds `backend/.env` the same way for both `tsx` and
|
|
compiled `dist`.
|
|
|
|
> `ALLOWED_ORIGINS` is NOT set by the platform. In `dev_stage` this is acceptable: the
|
|
> backend doesn't crash and reflects the request origin (`config.auth.allowAllOrigins`).
|
|
> Strict `ALLOWED_ORIGINS` check only applies in strict `NODE_ENV=production`.
|
|
|
|
## 1.3. PM2 Processes
|
|
|
|
`pm2 status` on the VM shows (names are fixed during provisioning):
|
|
|
|
| Process | Command (cwd) | Port |
|
|
|---|---|---|
|
|
| `frontend-dev` | `npm run dev -- --hostname 0.0.0.0 --port 3001` (`workspace/frontend`) | 3001 |
|
|
| `backend-dev` | `NODE_ENV=dev_stage npm run start` (`workspace/backend`) | 3000 |
|
|
| `fl-executor` | `executor.js` — Flatlogic agent (cable, git, AI commands) | — |
|
|
| `fl-telemetry` | `telemetry-daemon.js` | — |
|
|
|
|
> ⚠️ `--hostname` is a Next.js flag; Vite CLI rejects it (`Unknown option`).
|
|
> Therefore `dev`/`start` for the frontend run through the wrapper `frontend/scripts/serve.mjs`,
|
|
> which translates `--hostname`→`--host` and starts Vite. Without it, the frontend won't
|
|
> start on the VM.
|
|
|
|
## 1.4. Deployment After `git pull`
|
|
|
|
PM2 commands **do not install dependencies**. After pulling new code:
|
|
|
|
```bash
|
|
cd ~/executor/workspace/backend && npm ci
|
|
cd ~/executor/workspace/frontend && npm ci
|
|
```
|
|
|
|
The DB schema is created by the initial migration. On an existing DB, `npm run start`
|
|
runs `db:migrate` (idempotent, `CREATE TABLE IF NOT EXISTS`) + `db:seed` automatically.
|
|
For a guaranteed clean state (recommended after major migrations —
|
|
**will delete data**):
|
|
|
|
```bash
|
|
cd ~/executor/workspace/backend && npm run db:reset # drop all tables → migrate → seed
|
|
```
|
|
|
|
> ⚠️ **On VM**: `npm run db:reset` uses local dev credentials by default, which won't work.
|
|
> You must install dependencies and pass the platform-injected DB credentials explicitly:
|
|
>
|
|
> ```bash
|
|
> # 1. Install dependencies first
|
|
> cd ~/executor/workspace/backend && npm ci
|
|
> cd ~/executor/workspace/frontend && npm ci
|
|
>
|
|
> # 2. Get DB credentials
|
|
> pm2 env 1 | grep DB_PASS # DB_PASS from PM2 environment
|
|
> cat ~/executor/.env | grep DB_ # DB_NAME, DB_USER, DB_HOST, DB_PORT
|
|
>
|
|
> # 3. Run reset with correct credentials
|
|
> cd ~/executor/workspace/backend && \
|
|
> DB_HOST=127.0.0.1 DB_PORT=5432 \
|
|
> DB_NAME=app_<id> DB_USER=app_<id> DB_PASS=<uuid> \
|
|
> npm run db:reset
|
|
>
|
|
> # 4. Restart PM2 (has platform credentials)
|
|
> pm2 restart backend-dev frontend-dev --update-env
|
|
> ```
|
|
|
|
Restart processes (or the executor does this after pull):
|
|
|
|
```bash
|
|
pm2 restart backend-dev frontend-dev --update-env
|
|
```
|
|
|
|
### Scheduled maintenance: refresh-token cleanup
|
|
|
|
Expired refresh-token rows accumulate (the table is not paranoid). Schedule the
|
|
cleanup command (e.g. a daily cron) to delete rows past the retention window
|
|
(`AUTH_REFRESH_TOKEN_RETENTION_MS`, default 7 days):
|
|
|
|
```bash
|
|
# crontab -e — daily at 03:30
|
|
30 3 * * * cd ~/executor/workspace/backend && npm run db:cleanup-tokens >> ~/token-cleanup.log 2>&1
|
|
```
|
|
|
|
It is idempotent and never touches valid sessions. See
|
|
`backend/docs/cookie-auth.md` (Operational maintenance).
|
|
|
|
### What the Startup Commands Do
|
|
|
|
- `backend-dev` → `npm run start` = `db:migrate` (initial migration → schema) +
|
|
`db:seed` + `watch` (server via `tsx` + nodemon, port 3000).
|
|
- `frontend-dev` → `npm run dev` (via `serve.mjs`) — Vite dev server on 3001,
|
|
`allowedHosts: true` allows the tunnel domain.
|
|
|
|
## 1.5. nginx
|
|
|
|
If nginx is a system service, its config should match `nginx.conf` from the
|
|
repository (`/` → 3001, `/api` and `/api-docs` → 3000):
|
|
|
|
```bash
|
|
sudo cp ~/executor/workspace/nginx.conf /etc/nginx/nginx.conf
|
|
sudo nginx -t && sudo nginx -s reload
|
|
```
|
|
|
|
## 1.6. Verification
|
|
|
|
```bash
|
|
curl -s -o /dev/null -w "front %{http_code}\n" http://127.0.0.1:3001/
|
|
curl -s -o /dev/null -w "api %{http_code}\n" http://127.0.0.1:3000/api-docs/
|
|
pm2 status
|
|
pm2 logs backend-dev --lines 50
|
|
```
|
|
|
|
## 1.7. Production Mode on VM (Compiled Builds)
|
|
|
|
"Production mode" = **compiled builds while keeping `NODE_ENV=dev_stage`**
|
|
(frontend minification, `node dist` for backend, source maps). Strict `NODE_ENV=production`
|
|
cannot be used — it requires `ALLOWED_ORIGINS`, which the platform doesn't set,
|
|
and the backend will crash on startup.
|
|
|
|
To switch to production, change the pm2 commands (on the executor side):
|
|
|
|
```bash
|
|
# Frontend
|
|
cd ~/executor/workspace/frontend && npm ci && npm run build
|
|
pm2 delete frontend-dev 2>/dev/null || true
|
|
FRONT_PORT=3001 pm2 start npm --name frontend --update-env -- run start
|
|
|
|
# Backend (keep NODE_ENV=dev_stage)
|
|
cd ~/executor/workspace/backend && npm ci && npm run build
|
|
pm2 delete backend-dev 2>/dev/null || true
|
|
NODE_ENV=dev_stage pm2 start npm --name backend --update-env -- run start:production
|
|
|
|
pm2 save
|
|
```
|
|
|
|
- `frontend npm run start` = `vite preview` on `FRONT_PORT` (3001), serves `dist/`.
|
|
- `backend npm run start:production` = `db:migrate:prod` + `db:seed:prod` +
|
|
`node --enable-source-maps dist/index.js` (port 3000).
|
|
|
|
To have the executor automatically rebuild production on each code update — add
|
|
`npm ci && npm run build` to its service restart step (see `vcs.js`, ~line 412,
|
|
array `const services = ['backend-dev', 'frontend-dev']`).
|
|
|
|
## 1.8. Troubleshooting
|
|
|
|
| Symptom | Cause / Solution |
|
|
|---|---|
|
|
| Frontend won't start, `Unknown option --hostname` | old code without `serve.mjs`; update workspace (`git pull` + `npm ci`) |
|
|
| Backend crashes: `ALLOWED_ORIGINS must be configured` | running with `NODE_ENV=production`; on VM it should be `dev_stage` |
|
|
| Backend crashes: `SECRET_KEY` required | platform didn't pass `SECRET_KEY` to pm2-env (required in production/dev_stage) |
|
|
| Backend crashes: `Missing required database credentials` | DB_* not set in production/dev_stage (dev defaults don't apply) |
|
|
| `tsx: not found` / `vite: not found` | `npm ci` not run after pull |
|
|
| 502 on domain | backend/frontend not listening on 3000/3001, or nginx routing mismatch |
|
|
|
|
---
|
|
|
|
# Part 2. VM File Structure (Reference)
|
|
|
|
Snapshot of a production VM (`pool-saas-*`). Useful for understanding where things are located.
|
|
|
|
## 2.1. Home Directory `~`
|
|
|
|
```
|
|
~/
|
|
├── executor/ # Flatlogic agent + the project itself (see below)
|
|
├── .pm2/ # PM2: processes, logs, pids
|
|
│ ├── logs/ # *-out.log, *-error.log per process
|
|
│ ├── pids/
|
|
│ └── dump.pm2 # saved process list (pm2 save)
|
|
├── .bun/ .yarn/ .npm/ .cache/ # toolchains/caches
|
|
├── .codex/ .gemini/ .config/ # AI CLI configs
|
|
├── .ssh/ .pki/
|
|
├── recipes.md
|
|
└── google-gemini-cli-0.17.1.tgz
|
|
```
|
|
|
|
## 2.2. `~/executor/` — Flatlogic Agent
|
|
|
|
```
|
|
~/executor/
|
|
├── .env # agent and project config (PROJECT_UUID, PROJECT_ID,
|
|
│ # CABLE_URL, DB_NAME=app_<id>, SUBDOMAIN, FRONT_PORT, ...)
|
|
├── executor.js # main agent process (pm2: fl-executor):
|
|
│ # WebSocket cable to flatlogic.com, receiving commands,
|
|
│ # launching AI runners (gemini/codex), fs/git operations
|
|
├── gemini.js / gemini-proc.js / opencode.js / opencode-proc.js # AI runners
|
|
├── vcs.js and vcs/vcs.js # git: init/pull/push/commit, Gitea mirror,
|
|
│ # service restart (array ['backend-dev','frontend-dev'])
|
|
├── vm-tools.js # VM tools (commands from platform)
|
|
├── activity-tracker.js # runner activity tracking
|
|
├── telemetry-daemon.js / telemetry-server.js / telemetry-file-watcher.js # telemetry (pm2: fl-telemetry)
|
|
├── sentry.js # Sentry
|
|
├── config.js # WORKSPACE_ROOT etc.
|
|
├── index.php # static preloader page ("Analyzing your requirements...")
|
|
├── setup_postgres_project.sh # create role/DB for project in local Postgres
|
|
├── setup_mariadb_project.sh # same for MariaDB
|
|
├── setup_workspace_permissions.sh
|
|
├── cleanup_vm.sh
|
|
├── AGENTS.md / README.md # instructions for AI agent (describe project template)
|
|
├── otel-local.yaml / schema.json / proto/
|
|
├── node_modules/ package.json package-lock.json
|
|
├── workspace/ # ◀── THE PROJECT ITSELF (git repository = this repository)
|
|
├── workspace_baseline.tar.gz # baseline workspace snapshot
|
|
├── workspace_codegen/ # code generation workspace
|
|
└── templates/ # templates for new projects
|
|
├── app-templates/
|
|
└── frontend-tailwind-backend-nodejs/ # Next.js+Node template (NOT our stack)
|
|
```
|
|
|
|
> Files in `executor/` (agent) and `templates/` are not related to running **our** project —
|
|
> they don't need to be modified. `index.php` is just a preloader during generation.
|
|
|
|
## 2.3. `~/executor/workspace/` — The Project
|
|
|
|
This is the git repository (`40227-vm`):
|
|
|
|
```
|
|
workspace/
|
|
├── frontend/ # Vite + React + TS → builds to frontend/dist, served on :3001
|
|
├── backend/ # Express + Sequelize TS/ESM → backend/dist, served on :3000
|
|
│ ├── .env # PORT (committed)
|
|
│ └── src/db/migrations/ # initial migration (schema)
|
|
├── nginx.conf # routing / → 3001, /api → 3000 (for system nginx)
|
|
├── Dockerfile / Dockerfile.dev / docker/ # alternative Docker path
|
|
├── 502.html
|
|
└── docs/ # including this file
|
|
```
|
|
|
|
## 2.4. `~/executor/.env` — Keys (Values are Secret/Individual)
|
|
|
|
```
|
|
PROJECT_UUID, PROJECT_ID, CABLE_URL, GEMINI_MODEL, TELEMETRY_*, OTEL_*,
|
|
MAIL_*, SMTP_*, SUBDOMAIN, BASE_DOMAIN, FULL_DOMAIN, HOST_FQDN,
|
|
DB_NAME=app_<id>, DB_USER=app_<id>, DB_HOST=127.0.0.1, DB_PORT=5432,
|
|
FRONT_PORT=3001, SENTRY_DSN
|
|
```
|
|
|
|
Application secrets (`SECRET_KEY`, `DB_PASS`, OAuth/SMTP, git tokens, `CF_TUNNEL_*`)
|
|
are placed by the platform directly into the **pm2 environment** of `backend-dev`/`fl-executor`
|
|
processes, not in `backend/.env`.
|