# Backend Architecture Documentation This document provides a comprehensive analysis of the Tour Builder Platform backend architecture, including design patterns, layers, and implementation details. ## Overview - **Runtime**: Node.js 24 LTS - **Framework**: Express.js 4.x - **ORM**: Sequelize 6.x with PostgreSQL - **Authentication**: Passport.js (JWT, Google OAuth, Microsoft OAuth) - **Documentation**: Swagger/OpenAPI 3.0 - **Logging**: Pino (structured JSON logging) - **TypeScript/ESM**: the backend package is `"type": "module"` and active backend source is strict TypeScript ESM. - **File Storage**: AWS S3 / Local filesystem (Strategy Pattern, provider-based) --- ## Architecture Diagram ``` ┌──────────────────────────────────────────────────────────────────────────┐ │ Express Application │ │ (src/index.ts) │ └──────────────────────────────────────────────────────────────────────────┘ │ ┌─────────────────┼─────────────────┐ ▼ ▼ ▼ ┌────────────────────────┐ ┌────────────────┐ ┌────────────────────────┐ │ Middleware │ │ Rate Limiter │ │ Request Logger │ │ • runtimeContext │ │ • auth (10/15m)│ │ • Pino structured │ │ • runtimePublic │ │ • api (100/1m) │ │ • Request ID tracking │ │ • checkPermissions │ │ • upload (10/m)│ │ • Duration metrics │ │ • passport JWT │ │ • download(200)│ └────────────────────────┘ └────────────────────────┘ │ • search (30/m)│ │ • AI (20/1m) │ └────────────────┘ │ ┌──────────────────────────────────────────────────────────────────────────┐ │ Routes Layer │ │ (src/routes/*.ts) │ │ ┌────────────────────────────────────────────────────────────────────┐ │ │ │ Factory-generated routes: router.factory.js │ │ │ │ • Standard CRUD: POST /, PUT /:id, DELETE /:id, GET /, GET /:id │ │ │ │ • Extra: GET /count, GET /autocomplete, POST /deleteByIds │ │ │ │ • CSV export: GET /?filetype=csv │ │ │ └────────────────────────────────────────────────────────────────────┘ │ │ ┌────────────────────────────────────────────────────────────────────┐ │ │ │ Custom routes: auth, file, publish, search, sql, runtime │ │ │ └────────────────────────────────────────────────────────────────────┘ │ └──────────────────────────────────────────────────────────────────────────┘ │ ▼ ┌──────────────────────────────────────────────────────────────────────────┐ │ Service Layer │ │ (src/services/*.js) │ │ ┌────────────────────────────────────────────────────────────────────┐ │ │ │ Factory-generated services: service.factory.js │ │ │ │ • Transaction-wrapped CRUD operations │ │ │ │ • Delegates to DB API layer │ │ │ └────────────────────────────────────────────────────────────────────┘ │ │ ┌────────────────────────────────────────────────────────────────────┐ │ │ │ Custom services: │ │ │ │ • auth.js - Authentication logic (signin, signup, password) │ │ │ │ • publish.ts - Publishing workflow service (dev→stage→production)│ │ │ │ • file.ts - File storage operations │ │ │ │ • email/index.js - Email sending via Nodemailer/SES │ │ │ │ • search.ts - Full-text search route │ │ │ │ • pwa_manifest.js - PWA offline manifest generation │ │ │ └────────────────────────────────────────────────────────────────────┘ │ └──────────────────────────────────────────────────────────────────────────┘ │ ▼ ┌──────────────────────────────────────────────────────────────────────────┐ │ Database API Layer │ │ (src/db/api/*.js) │ │ ┌────────────────────────────────────────────────────────────────────┐ │ │ │ GenericDBApi (base.api.js) - Template Method Pattern │ │ │ │ • Configurable: MODEL, SEARCHABLE_FIELDS, RANGE_FIELDS, etc. │ │ │ │ • CRUD: create, update, remove, deleteByIds │ │ │ │ • Query: findBy, findAll, findAllAutocomplete │ │ │ │ • Data Transform: getFieldMapping(), JSON_FIELDS, FIELD_DEFAULTS │ │ │ └────────────────────────────────────────────────────────────────────┘ │ │ ┌────────────────────────────────────────────────────────────────────┐ │ │ │ Entity APIs extend GenericDBApi: │ │ │ │ UsersDBApi, ProjectsDBApi, AssetsDBApi, TourPagesDBApi, etc. │ │ │ └────────────────────────────────────────────────────────────────────┘ │ └──────────────────────────────────────────────────────────────────────────┘ │ ▼ ┌──────────────────────────────────────────────────────────────────────────┐ │ Sequelize Models │ │ (src/db/models/*.ts) │ │ Entity models + file model, loaded dynamically via loader.ts │ └──────────────────────────────────────────────────────────────────────────┘ │ ▼ ┌──────────────────────────────────────────────────────────────────────────┐ │ PostgreSQL │ └──────────────────────────────────────────────────────────────────────────┘ ``` --- ## Directory Structure ``` backend/src/ ├── index.ts # Application entry point ├── config.ts # Environment configuration ├── helpers.ts # Utility functions (wrapAsync, JWT, UUID validation) ├── types/ # Reusable strict TypeScript contracts for migrated code │ ├── auth/ │ └── auth.ts # Passport strategies (JWT, Google, Microsoft) │ ├── middlewares/ │ ├── check-permissions.ts # RBAC permission checking │ ├── runtime-context.ts # Runtime environment context (dev/stage/production) │ ├── runtime-public.ts # Public runtime access control & field sanitization │ ├── rateLimiter.ts # Rate limiting (auth, API, upload, download) │ └── upload.ts # File upload handling (multer) │ ├── routes/ │ ├── auth.ts # Authentication routes (custom) │ ├── file.ts # File upload/download routes (custom) │ ├── publish.ts # Publishing workflow routes (custom) │ ├── search.ts # Full-text search routes (custom) │ ├── runtime-context.ts # Runtime context detection (custom) │ └── [entity].ts # Entity CRUD routes │ ├── services/ │ ├── auth.ts # Authentication service │ ├── publish.ts # Publishing workflow (dev→stage→production) │ ├── file.ts # Unified file storage service │ ├── search.ts # Full-text search service │ ├── file/ # File storage providers (S3, Local) │ │ ├── index.ts # Module exports & provider factory │ │ ├── BaseStorageProvider.ts # Abstract base class │ │ ├── S3StorageProvider.ts # AWS S3 implementation │ │ ├── LocalStorageProvider.ts # Local filesystem implementation │ │ └── UploadSessionManager.ts # Chunked upload session management │ ├── email/ │ │ ├── index.ts # Email sender (Nodemailer/SES) │ │ └── list/ # Email templates │ ├── notifications/ │ │ ├── helpers.ts # Notification helpers │ │ └── errors/ # Error classes (ValidationError, ForbiddenError) │ └── [entity].ts # Entity services │ ├── factories/ │ ├── router.factory.ts # Route generator (createEntityRouter) │ └── service.factory.ts # Service generator (createEntityService) │ ├── db/ │ ├── db-config.ts # Database configuration (env-based) │ ├── umzug.ts # Migration and seeder runner │ ├── models/ │ │ ├── index.ts # Model registry entrypoint │ │ ├── loader.ts # Model loader │ │ └── [entity].ts # Sequelize models │ ├── api/ │ │ ├── base.api.ts # GenericDBApi base class │ │ ├── runtime-context.ts # Runtime filtering helpers │ │ └── [entity].ts # Entity DB APIs │ ├── migrations/ # Applied migration history │ └── seeders/ # Typed seed data files │ └── utils/ ├── index.ts # Utils barrel export ├── errors.ts # Error classes (AppError, NotFoundError, etc.) ├── logger.ts # Pino logger configuration └── env-validation.ts # Environment variable validation ``` --- ## Design Patterns ### 1. Factory Pattern **Router Factory** (`factories/router.factory.js`) Generates standardized CRUD routes for entities: ```javascript const { createEntityRouter } = require('../factories/router.factory'); // Creates routes: POST /, PUT /:id, DELETE /:id, GET /, GET /:id, GET /count, GET /autocomplete module.exports = createEntityRouter('assets', AssetsService, AssetsDBApi, { permissionEntity: 'assets', csvFields: ['id', 'name', 'asset_type', 'createdAt'], }); ``` **Service Factory** (`factories/service.factory.js`) Generates transaction-wrapped service classes: ```javascript const { createEntityService } = require('../factories/service.factory'); // Creates: create(), update(), remove(), deleteByIds(), bulkImport() module.exports = createEntityService(AssetsDBApi, { entityName: 'Asset' }); ``` ### 2. Template Method Pattern **GenericDBApi** (`db/api/base.api.js`) Base class with configurable hooks for entity-specific behavior: ```javascript class AssetsDBApi extends GenericDBApi { // Required: Define the Sequelize model static get MODEL() { return db.assets; } // Configurable behavior via static getters static get SEARCHABLE_FIELDS() { return ['name', 'cdn_url']; } static get RANGE_FIELDS() { return ['size_mb', 'width_px']; } static get ENUM_FIELDS() { return ['asset_type', 'is_public']; } static get JSON_FIELDS() { return ['settings_json']; } static get FIELD_DEFAULTS() { return { type: { default: 'general' } }; } static get ASSOCIATIONS() { return [{ field: 'project', setter: 'setProject' }]; } static get FIND_BY_INCLUDES() { return [{ association: 'project' }]; } static get FIND_ALL_INCLUDES() { return [{ model: db.projects, as: 'project' }]; } // Custom field transformation static getFieldMapping(data) { return { name: data.name || null, asset_type: data.asset_type || null, type: data.type || 'general', // ... }; } } ``` ### 3. Strategy Pattern **File Storage Providers** (`services/file/`) Two concrete implementations with pluggable architecture: ``` BaseStorageProvider (abstract) │ ├── S3StorageProvider # AWS S3 implementation (with timeout/retry) └── LocalStorageProvider # Local filesystem implementation ``` The storage provider base, S3 provider, and local provider are migrated TS/ESM modules. The S3 implementation uses official AWS SDK v3 types; shared provider-domain contracts are in `src/types/file.ts`. Interface: - `upload(key, data, options)` → `{ key, url }` - `download(key)` → `{ body, contentType }` - `delete(key)` → `void` - `deleteMany(keys)` → `void` - `exists(key)` → `boolean` - `list(prefix)` → `string[]` - `getSignedUrl(key, expiresIn)` → `string` ### 4. Middleware Chain Pattern Request flow through middleware stack: ``` Request → requestLogger → runtimeContextMiddleware → rateLimiter → passport.authenticate('jwt') → checkCrudPermissions → Route Handler → Service → DB API → Database → Response ``` --- ## Layers in Detail ### Entry Point (`src/index.ts`) Application bootstrap: 1. **Security**: Helmet, CORS configuration 2. **Logging**: Request logger middleware (Pino) 3. **Authentication**: Passport JWT initialization 4. **Rate Limiting**: Per-route rate limiters 5. **Body Parsing**: JSON (1mb limit), applied after file routes 6. **Runtime Context**: Environment detection middleware 7. **Route Mounting**: Entity routes with auth/permissions 8. **Error Handling**: Generic error handler 9. **Static Files**: Public directory serving ```javascript // Key route mounting patterns app.use('/api/auth', authRoutes); // No JWT required app.use('/api/users', jwtAuth, usersRoutes); // JWT required // Runtime public routes (production content accessible without auth) const mountRuntimeEntityRoute = (path, entityName, router) => { app.use(path, requireRuntimeReadOrAuth, // JWT or public production blockNonPublicRuntimeListEndpoints, // Block non-list endpoints sanitizePublicRuntimeListResponse(entityName), // Filter sensitive fields router ); }; mountRuntimeEntityRoute('/api/projects', 'projects', projectsRoutes); mountRuntimeEntityRoute('/api/tour_pages', 'tour_pages', tour_pagesRoutes); ``` ### Routes Layer **Factory-Generated Routes** provide standard CRUD: | Method | Path | Description | |--------|------|-------------| | POST | `/` | Create record | | POST | `/bulk-import` | Bulk import from CSV | | PUT | `/:id` | Update record | | DELETE | `/:id` | Delete record | | POST | `/deleteByIds` | Bulk delete | | GET | `/` | List with pagination & filters | | GET | `/count` | Count only | | GET | `/autocomplete` | Autocomplete search | | GET | `/:id` | Get single record | **Custom Routes** (auth, file, publish, search, runtime-context): | Route | Endpoints | |-------|-----------| | `/api/auth` | signin, signup, me, password-reset, verify-email, Google/Microsoft OAuth | | `/api/file` | upload, download, presign, upload-sessions (chunked) | | `/api/publish` | publish (stage→production), save-to-stage (dev→stage) | | `/api/search` | Global full-text search | | `/api/runtime-context` | Runtime environment detection | ### Service Layer **Transaction Management**: Services wrap operations in transactions: ```javascript static async create({ data, currentUser, transaction: externalTransaction, runtimeContext }) { const transaction = externalTransaction || await db.sequelize.transaction(); const ownsTransaction = !externalTransaction; try { const record = await DBApi.create({ data, currentUser, transaction, runtimeContext }); if (ownsTransaction) await transaction.commit(); return record; } catch (error) { if (ownsTransaction) await transaction.rollback(); throw error; } } ``` **Publish Service** (`services/publish.ts`): Implements the dev→stage→production workflow with: - Transaction locking to prevent concurrent publishes - Source key tracking for content lineage - Bulk copy operations for pages and audio tracks ### Database API Layer **Query Building** in `findAll()`: | Filter Type | Example | SQL | |-------------|---------|-----| | Text search | `?name=foo` | `name ILIKE '%foo%'` | | Range | `?size_mbRange=[0,100]` | `size_mb >= 0 AND size_mb <= 100` | | Enum | `?asset_type=image` | `asset_type = 'image'` | | Relation | `?project=uuid` | JOIN with projects table | | Sort | `?field=name&sort=asc` | `ORDER BY name ASC` | | Pagination | `?page=1&limit=10` | `OFFSET 0 LIMIT 10` | --- ## Authentication & Authorization ### Authentication Flow ``` ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │ POST /signin │ │ Passport JWT │ │ Protected │ │ → JWT Token │────▶│ Middleware │────▶│ Route │ └─────────────────┘ └─────────────────┘ └─────────────────┘ │ ▼ req.currentUser = { id, email, app_role, custom_permissions } ``` ### Authorization (RBAC) **Permission Check Flow** (`middlewares/check-permissions.ts`): 1. Self-access bypass (user accessing own resource) 2. Check custom permissions (user-specific) 3. Check role permissions (from app_role) 4. Fallback to Public role for unauthenticated **Permission Naming Convention**: - `CREATE_` - Create records - `READ_` - Read records - `UPDATE_` - Modify records - `DELETE_` - Delete records ```javascript // Auto-generated from HTTP method const permissionName = `${METHOD_MAP[req.method]}_${name.toUpperCase()}`; // POST /api/assets → CREATE_ASSETS // GET /api/assets → READ_ASSETS ``` ### Runtime Public Access For production content accessible without authentication: ```javascript const requireRuntimeReadOrAuth = (req, res, next) => { const isPublicEnvironment = req.runtimeContext?.headerEnvironment === 'production'; const isReadOnlyRequest = ['GET', 'OPTIONS'].includes(req.method); if (isPublicEnvironment && isReadOnlyRequest && !hasAuthHeader) { req.isRuntimePublicRequest = true; return next(); // Allow without JWT } return jwtAuth(req, res, next); // Require JWT }; ``` --- ## Rate Limiting Pre-configured limiters (`middlewares/rateLimiter.ts`): | Limiter | Window | Max Requests | Use Case | |---------|--------|--------------|----------| | `authLimiter` | 15 min | 10 | Authentication endpoints | | `passwordResetLimiter` | 1 hour | 5 | Password reset | | `apiLimiter` | 1 min | 100 | General API | | `uploadLimiter` | 1 min | 10 | File uploads | | `downloadLimiter` | 1 min | 200 | File downloads | | `searchLimiter` | 1 min | 30 | Search queries | Headers returned: - `X-RateLimit-Limit`: Maximum requests - `X-RateLimit-Remaining`: Remaining requests - `X-RateLimit-Reset`: Reset time (ISO timestamp) - `Retry-After`: Seconds until reset (when limited) --- ## File Storage **Storage Provider Selection**: ```javascript const provider = config.fileStorage.provider || (hasS3Credentials ? 's3' : hasGCloudCredentials ? 'gcloud' : 'local'); ``` **S3 Operations**: | Operation | Method | Description | |-----------|--------|-------------| | Upload | `upload(key, data, options)` | Put object with metadata | | Download | `download(key)` | Get object stream | | Presign | `getSignedUrl(key, expiresIn)` | Generate presigned URL | | Delete | `delete(key)` / `deleteMany(keys)` | Remove objects | | Check | `exists(key)` | Head object | | List | `list(prefix)` | List objects with prefix | **Chunked Uploads** (`UploadSessionManager`): For large files, supports multipart upload sessions: 1. `POST /upload-sessions/init` - Create session 2. `POST /upload-sessions/:id/chunk` - Upload chunk 3. `POST /upload-sessions/:id/finalize` - Complete upload --- ## Error Handling **Error Classes** (`utils/errors.js`): ```javascript class AppError extends Error { constructor(message, statusCode = 500, details = null) { super(message); this.statusCode = statusCode; this.details = details; this.isOperational = true; } } class NotFoundError extends AppError { statusCode = 404 } class ValidationError extends AppError { statusCode = 400 } class ForbiddenError extends AppError { statusCode = 403 } class UnauthorizedError extends AppError { statusCode = 401 } class ConflictError extends AppError { statusCode = 409 } ``` **Async Handler** (`helpers.ts`): ```javascript // Wraps async route handlers to catch errors static wrapAsync(fn) { return function(req, res, next) { fn(req, res, next).catch(next); }; } ``` **Common Error Handler** (`helpers.ts`): ```javascript static commonErrorHandler(error, req, res, _next) { const statusCode = error.code || error.status; if ([400, 401, 403, 404, 409, 422].includes(statusCode)) { return res.status(statusCode).send(error.message); } console.error(error); return res.status(500).send('Internal server error'); } ``` --- ## Logging **Pino Logger** (`utils/logger.js`): Logger initialization is a bootstrap exception: it may read `process.env` directly because importing `config.ts` would create an initialization cycle. ```javascript const logger = pino({ level: process.env.LOG_LEVEL || 'info', transport: isDevelopment ? { target: 'pino-pretty', options: { colorize: true } } : undefined, base: { service: 'tour-builder-api', env: process.env.NODE_ENV || 'development', }, }); ``` **Request Logging**: ```javascript function requestLogger(req, res, next) { const requestId = req.headers['x-request-id'] || crypto.randomUUID(); req.log = logger.child({ requestId }); req.requestId = requestId; res.setHeader('X-Request-Id', requestId); res.on('finish', () => { req.log.info({ method: req.method, url: req.originalUrl, status: res.statusCode, duration: Date.now() - start, }, 'Request completed'); }); } ``` --- ## Configuration **Environment Variables** (`config.ts`): | Variable | Description | Default | |----------|-------------|---------| | `SECRET_KEY` | JWT signing key | UUID-based default | | `ADMIN_EMAIL` | Admin user email | `admin@flatlogic.com` | | `ADMIN_PASS` | Admin user password | Generated | | `AWS_S3_BUCKET` | S3 bucket name | - | | `AWS_S3_REGION` | S3 region | `us-east-1` | | `AWS_ACCESS_KEY_ID` | AWS access key | - | | `AWS_SECRET_ACCESS_KEY` | AWS secret key | - | | `GOOGLE_CLIENT_ID` | Google OAuth client ID | - | | `GOOGLE_CLIENT_SECRET` | Google OAuth client secret | - | | `MS_CLIENT_ID` | Microsoft OAuth client ID | - | | `MS_CLIENT_SECRET` | Microsoft OAuth client secret | - | | `EMAIL_USER` | SMTP username | - | | `EMAIL_PASS` | SMTP password | - | | `LOG_LEVEL` | Logging level | `info` | **Database Configuration** (`db/db-config.ts`): | Environment | Database | Logging | |-------------|----------|---------| | `production` | `DB_*` env vars | Disabled | | `development` | `db_tour_builder_platform` | Console | | `dev_stage` | `DB_*` env vars | Console | --- ## API Documentation Swagger/OpenAPI documentation is available at `/api-docs`. The served document is centralized in `backend/src/openapi/document.ts`. The OpenAPI module defines shared schemas, common parameters, reusable responses, and generated standard CRUD paths for every `createEntityRouter` resource. This keeps the documented factory contract aligned with the route factory endpoints: create, bulk import, update, delete, bulk delete, list, count, autocomplete, and get by ID. ```javascript const specs = createOpenApiDocument({ serverUrl: config.server.swaggerServerUrl, }); ``` When adding a new route, update `backend/src/openapi/document.ts` in the same change. For new factory-backed entities, add the entity schema and `CrudResource` entry; for custom routes, add an explicit path item. --- ## Health Check ```javascript GET /api/health { "status": "ok", // or "degraded" "timestamp": "2026-03-29T...", "uptime": 12345.678, "environment": "production", "database": "connected" // or "disconnected" } ``` --- ## Key Implementation Files | File | Purpose | |------|---------| | `src/index.ts` | Application entry, middleware setup, route mounting | | `src/config.ts` | Environment configuration | | `src/helpers.ts` | wrapAsync, commonErrorHandler, jwtSign, isUuidV4 | | `src/auth/auth.ts` | Passport strategies (JWT, Google, Microsoft) | | `src/factories/router.factory.ts` | Route generator for entities | | `src/factories/service.factory.ts` | Service generator for entities | | `src/db/api/base.api.ts` | GenericDBApi base class | | `src/middlewares/check-permissions.ts` | RBAC permission checking | | `src/middlewares/rateLimiter.ts` | Rate limiting configuration | | `src/middlewares/runtime-context.ts` | Runtime environment detection | | `src/middlewares/runtime-public.ts` | Public runtime access control & field sanitization | | `src/services/publish.ts` | Publishing workflow service | | `src/services/file/S3StorageProvider.ts` | S3 storage implementation using official AWS SDK v3 types | | `src/utils/logger.ts` | Pino logger configuration | | `src/utils/errors.ts` | Error class definitions | --- ## Best Practices Implemented 1. **Factory Patterns** - Reduce boilerplate for CRUD operations 2. **Template Method** - Configurable base class for DB operations 3. **Strategy Pattern** - Pluggable storage providers 4. **Middleware Chain** - Composable request processing 5. **Transaction Management** - Consistent rollback on errors 6. **Rate Limiting** - Protection against abuse 7. **Structured Logging** - JSON logs with request IDs 8. **Environment-Based Config** - Secure credential handling 9. **Soft Deletes** - Paranoid models for data recovery 10. **RBAC** - Fine-grained permission control