# Core Module Documentation The Core module provides the foundational components of the backend application: the entry point, configuration management, and utility functions. ## Overview | File | Purpose | Lines | |------|---------|-------| | `src/index.ts` | Application entry point, Express setup, middleware, route mounting | varies | | `src/config.ts` | Environment configuration and settings | varies | | `src/helpers.js` | Utility functions (wrapAsync, JWT, validation) | 32 | | `src/types/` | Shared strict TypeScript contracts for migrated backend code | varies | | `src/load-env.ts` | Central backend `.env` bootstrap for app and DB entrypoints | varies | --- ## 1. Application Entry Point (`index.ts`) ### Purpose The main entry point that bootstraps the Express application, configures middleware, mounts routes, and starts the HTTP server. ### Dependencies ```typescript import bodyParser from 'body-parser'; import cors from 'cors'; import express from 'express'; import helmet from 'helmet'; import * as swaggerUI from 'swagger-ui-express'; import { authenticateJwt, authenticateJwtWithCallback } from './auth/passport-middleware.ts'; import config from './config.ts'; import { wrapAsync } from './helpers.ts'; import { runtimeContextMiddleware } from './middlewares/runtime-context.ts'; import { downloadLimiter, searchLimiter, uploadLimiter } from './middlewares/rateLimiter.ts'; import { createOpenApiDocument } from './openapi/document.ts'; import { exitAfterLogging, logger, registerProcessErrorHandlers, requestLogger, } from './utils/logger.ts'; ``` > TypeScript migration note: `index.ts`, `config.ts`, `helpers.ts`, `middlewares/validate-request.ts`, `middlewares/runtime-context.ts`, `middlewares/runtime-public.ts`, `routes/runtime-context.ts`, `validators/request-schemas.ts`, `services/notifications/list.ts`, `services/notifications/helpers.ts`, `utils/logger.ts`, and `utils/env-validation.ts` are now migrated runtime TypeScript modules. New migrated backend TypeScript files use `backend/tsconfig.json` with `strict: true`, reusable named types from `src/types/`, and lint rules that reject `any`, non-null assertions, and unsafe type assertions. > Environment bootstrap note: `src/load-env.ts` is loaded by app config and DB entrypoints so short backend scripts can see `backend/.env` without repeating `NODE_OPTIONS=-r dotenv/config`. If `NODE_ENV` is absent, it defaults to `dev_stage`, matching the standard VM backend flow. ### Application Bootstrap Sequence ``` ┌─────────────────────────────────────────────────────────────────┐ │ APPLICATION BOOTSTRAP │ └─────────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────────┐ │ 1. EXPRESS INITIALIZATION │ │ • Create Express app │ │ • Enable trust proxy (for reverse proxies) │ └─────────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────────┐ │ 2. SECURITY MIDDLEWARE │ │ • Helmet (CSP disabled, COEP disabled) │ │ • CORS (origin: true - allow all origins) │ └─────────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────────┐ │ 3. AUTHENTICATION SETUP │ │ • Load Passport strategies (JWT, Google, Microsoft) │ │ • Create jwtAuth middleware │ └─────────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────────┐ │ 4. LOGGING │ │ • Register process-level error handlers │ │ • Apply requestLogger middleware (early for full coverage) │ └─────────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────────┐ │ 5. FILE ROUTES (BEFORE BODY PARSER) │ │ • Mount file routes without JSON parsing │ │ • Apply rate limiters (download: 200/min, upload: 10/min) │ └─────────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────────┐ │ 6. BODY PARSING │ │ • JSON parser (1MB limit) │ │ • URL-encoded parser (1MB limit) │ └─────────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────────┐ │ 7. RUNTIME CONTEXT │ │ • Apply runtimeContextMiddleware │ │ • Detect environment from X-Runtime-Environment header │ └─────────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────────┐ │ 8. ROUTE MOUNTING │ │ • Public routes (health, auth, runtime-context) │ │ • Protected routes (JWT required) │ │ • Runtime public routes (production content without auth) │ └─────────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────────┐ │ 9. STATIC FILES & ERROR HANDLING │ │ • Serve public directory if exists │ │ • Generic error handler │ └─────────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────────┐ │ 10. SERVER START │ │ • Listen on PORT (8080 or 3000 for dev_stage) │ │ • Log startup message │ └─────────────────────────────────────────────────────────────────┘ ``` ### Middleware Stack ```javascript // Order matters - applied in this sequence: 1. swaggerUI.serve // API documentation at /api-docs 2. helmet() // Security headers 3. cors({ origin: true }) // Cross-origin requests 4. requestLogger // Request logging (Pino) 5. downloadLimiter // Rate limit for /api/file/download, /api/file/presign 6. uploadLimiter // Rate limit for /api/file/upload* 7. bodyParser.json() // JSON parsing (1MB limit) 8. bodyParser.urlencoded() // Form data parsing 9. runtimeContextMiddleware // Environment detection 10. passport.authenticate() // JWT authentication (per-route) 11. checkPermissions // RBAC (per-route) 12. errorHandler // Generic error handling ``` ### Route Mounting #### Public Routes (No Authentication) ```javascript app.get('/api/health', ...) // Health check app.use('/api/auth', authRoutes) // Authentication app.use('/api/runtime-context', ...) // Runtime context app.use('/api/file', fileRoutes) // File download/presign (partial) ``` #### Protected Routes (JWT Required) ```javascript app.use('/api/users', jwtAuth, usersRoutes) app.use('/api/roles', jwtAuth, rolesRoutes) app.use('/api/permissions', jwtAuth, permissionsRoutes) app.use('/api/project_memberships', jwtAuth, project_membershipsRoutes) app.use('/api/assets', jwtAuth, assetsRoutes) app.use('/api/asset_variants', jwtAuth, asset_variantsRoutes) app.use('/api/presigned_url_requests', jwtAuth, presigned_url_requestsRoutes) app.use('/api/publish_events', jwtAuth, publish_eventsRoutes) app.use('/api/pwa_caches', jwtAuth, pwa_cachesRoutes) app.use('/api/access_logs', jwtAuth, access_logsRoutes) app.use('/api/element-type-defaults', jwtAuth, element_type_defaultsRoutes) app.use('/api/ui-elements', jwtAuth, element_type_defaultsRoutes) // Alias app.use('/api/project-element-defaults', jwtAuth, project_element_defaultsRoutes) app.use('/api/publish', jwtAuth, publishRoutes) app.use('/api/search', jwtAuth, searchLimiter, searchRoutes) ``` #### Runtime Public Routes (Production Content Without Auth) ```javascript // These routes use requireRuntimeReadOrAuth middleware // Allows unauthenticated GET requests in production environment mountRuntimeEntityRoute('/api/projects', 'projects', projectsRoutes) mountRuntimeEntityRoute('/api/tour_pages', 'tour_pages', tour_pagesRoutes) mountRuntimeEntityRoute('/api/project_audio_tracks', 'project_audio_tracks', project_audio_tracksRoutes) ``` ### Key Functions #### `requireRuntimeReadOrAuth` Middleware that allows public read access for production content: ```javascript const requireRuntimeReadOrAuth = (req, res, next) => { const headerEnvironment = req.runtimeContext?.headerEnvironment; const isReadOnlyRequest = ['GET', 'OPTIONS'].includes(req.method); const hasAuthHeader = Boolean(req.headers.authorization); // Only production is public. Stage requires authentication. const isPublicEnvironment = headerEnvironment === 'production'; if (isPublicEnvironment && isReadOnlyRequest && !hasAuthHeader) { req.isRuntimePublicRequest = true; return next(); // Allow without JWT } req.isRuntimePublicRequest = false; return jwtAuth(req, res, next); // Require JWT }; ``` #### `mountRuntimeEntityRoute` Helper to mount routes with runtime public access middleware stack: ```javascript const mountRuntimeEntityRoute = (path, entityName, router) => { app.use( path, requireRuntimeReadOrAuth, // JWT or public production blockNonPublicRuntimeListEndpoints, // Block non-list for public sanitizePublicRuntimeListResponse(entityName), // Filter sensitive fields router, ); }; ``` #### `getBaseUrl` Utility to extract base URL for Swagger: ```javascript const getBaseUrl = (url) => { if (!url) return ''; return url.endsWith('/api') ? url.slice(0, -4) : url; }; ``` ### Health Check Endpoint ```javascript GET /api/health // Response (200 - healthy): { "status": "ok", "timestamp": "2026-03-30T12:00:00.000Z", "uptime": 12345.678, "environment": "production", "database": "connected" } // Response (503 - degraded): { "status": "degraded", "timestamp": "2026-03-30T12:00:00.000Z", "uptime": 12345.678, "environment": "production", "database": "disconnected", "databaseError": "Connection refused" } ``` ### Swagger/OpenAPI Configuration ```javascript const specs = createOpenApiDocument({ serverUrl: config.server.swaggerServerUrl, }); app.use('/api-docs', swaggerUI.serve, swaggerUI.setup(specs)); ``` The canonical OpenAPI source is `backend/src/openapi/document.ts`. It defines shared schemas, common responses and parameters, generated factory CRUD paths, and explicit custom-route paths. Route-local JSDoc comments are not the source of truth for Swagger UI. ### Error Handler ```javascript app.use((err, req, res, _next) => { if (!res.headersSent) { const requestLog = getRequestLogger(req) ?? logger; requestLog.error( { err, url: req.url, method: req.method }, 'Express error middleware caught unhandled error', ); res.status(safeStatusCode).json({ message: safeStatusCode === 503 ? 'Service temporarily unavailable' : 'Internal server error', }); } }); ``` Route-level `commonErrorHandler` also uses the request-scoped logger when available and logs unexpected route failures as `Route handler failed`. Circuit breaker rejections use status `503` instead of being collapsed to `500`. ### Server Configuration ```javascript const PORT = config.server.port; const server = app.listen(PORT, () => { logger.info( { port: PORT, env: config.server.env }, 'Server started', ); }); server.on('error', (err) => { logger.error( { err, port: PORT, env: config.server.env }, 'Server failed to start', ); exitAfterLogging(); }); ``` Startup errors such as `EADDRINUSE` happen on the Node `Server`, outside the Express request lifecycle, so the generic Express error middleware cannot catch them. The server-level `error` listener logs the failure and exits with a non-zero status for nodemon/PM2 supervision. --- ## 2. Configuration (`config.ts`) ### Purpose Centralized configuration management with environment variable support and sensible defaults. ### Configuration Structure ```javascript const config = { // Google Cloud Storage gcloud: { bucket: 'fldemo-files', hash: 'afeefb9d49f5b7977577876b99532ac7', projectId: env.GC_PROJECT_ID, clientEmail: env.GC_CLIENT_EMAIL, privateKey: env.GC_PRIVATE_KEY, }, fileStorage: { provider: env.FILE_STORAGE_PROVIDER, }, // AWS S3 s3: { bucket: env.AWS_S3_BUCKET, region: env.AWS_S3_REGION, accessKeyId: env.AWS_ACCESS_KEY_ID, secretAccessKey: env.AWS_SECRET_ACCESS_KEY, prefix: env.AWS_S3_PREFIX, connectionTimeout: env.AWS_S3_CONNECTION_TIMEOUT, requestTimeout: env.AWS_S3_REQUEST_TIMEOUT, maxAttempts: env.AWS_S3_MAX_ATTEMPTS, maxSockets: env.AWS_S3_MAX_SOCKETS, keepAlive: env.AWS_S3_KEEP_ALIVE !== 'false', presignExpirySeconds: env.AWS_S3_PRESIGN_EXPIRY, }, resilience: { ffmpeg: { reverseTimeoutMs: env.FFMPEG_REVERSE_TIMEOUT_MS, ffprobeTimeoutMs: env.FFPROBE_TIMEOUT_MS, breaker: { failureThreshold: env.FFMPEG_BREAKER_FAILURE_THRESHOLD, cooldownMs: env.FFMPEG_BREAKER_COOLDOWN_MS, successThreshold: env.FFMPEG_BREAKER_SUCCESS_THRESHOLD, }, }, fileStorage: { breaker: { failureThreshold: env.FILE_STORAGE_BREAKER_FAILURE_THRESHOLD, cooldownMs: env.FILE_STORAGE_BREAKER_COOLDOWN_MS, successThreshold: env.FILE_STORAGE_BREAKER_SUCCESS_THRESHOLD, }, }, }, // Password hashing bcrypt: { saltRounds: 12, }, // Default credentials admin_pass: env.ADMIN_PASS, user_pass: env.USER_PASS, admin_email: env.ADMIN_EMAIL, // Authentication providers providers: { LOCAL: 'local', GOOGLE: 'google', MICROSOFT: 'microsoft', }, // JWT secret_key: env.SECRET_KEY, // Server URLs remote: '', port, hostUI, portUI, // Swagger swaggerUI, swaggerPort, // OAuth google: { clientId: env.GOOGLE_CLIENT_ID, clientSecret: env.GOOGLE_CLIENT_SECRET, }, microsoft: { clientId: env.MS_CLIENT_ID, clientSecret: env.MS_CLIENT_SECRET, }, // File uploads uploadDir: os.tmpdir(), // Email (AWS SES) email: { from: 'Tour Builder Platform ', host: 'email-smtp.us-east-1.amazonaws.com', port: 587, auth: { user: env.EMAIL_USER, pass: env.EMAIL_PASS, }, tls: { rejectUnauthorized: env.EMAIL_TLS_REJECT_UNAUTHORIZED !== 'false', }, }, // Default roles roles: { admin: 'Administrator', user: 'Analytics Viewer', }, server: { env: env.NODE_ENV, port: serverPort, swaggerServerUrl, }, }; ``` ### Environment Variables Reference | Variable | Type | Default | Description | |----------|------|---------|-------------| | `NODE_ENV` | string | `development` | Environment: `development`, `production`, `dev_stage`, `test` | | `PORT` | number | `8080` | Server port | | `SECRET_KEY` | string | UUID | JWT signing key (min 16 chars) | | `ADMIN_EMAIL` | string | `admin@flatlogic.com` | Admin user email | | `ADMIN_PASS` | string | Generated | Admin user password | | `USER_PASS` | string | Generated | Default user password | | `AWS_S3_BUCKET` | string | - | S3 bucket name | | `AWS_S3_REGION` | string | `us-east-1` | S3 region | | `AWS_ACCESS_KEY_ID` | string | - | AWS access key | | `AWS_SECRET_ACCESS_KEY` | string | - | AWS secret key | | `AWS_S3_PREFIX` | string | Hash | S3 key prefix | | `GOOGLE_CLIENT_ID` | string | - | Google OAuth client ID | | `GOOGLE_CLIENT_SECRET` | string | - | Google OAuth client secret | | `MS_CLIENT_ID` | string | - | Microsoft OAuth client ID | | `MS_CLIENT_SECRET` | string | - | Microsoft OAuth client secret | | `EMAIL_USER` | string | - | SMTP username | | `EMAIL_PASS` | string | - | SMTP password | | `EMAIL_TLS_REJECT_UNAUTHORIZED` | string | `true` | TLS validation | | `LOG_LEVEL` | string | `info` | Pino log level | ### Environment Validation The configuration uses Joi schema validation via `utils/env-validation.js`: ```javascript const Joi = require('joi'); const envSchema = Joi.object({ NODE_ENV: Joi.string() .valid('development', 'test', 'production', 'dev_stage') .default('development'), PORT: Joi.number().default(8080), DB_HOST: Joi.string().default('localhost'), DB_PORT: Joi.number().default(5432), DB_NAME: Joi.string().default('db_tour_builder_platform'), DB_USER: Joi.string().default('postgres'), DB_PASS: Joi.string().allow('').default(''), SECRET_KEY: Joi.string() .min(16) .default('88dbeaf8-e906-405e-9e41-c3baadeda5c6'), // ... more validations }).unknown(true); function validateEnv() { const { error, value } = envSchema.validate(process.env, { abortEarly: false, stripUnknown: false, }); if (error) { const messages = error.details.map((d) => ` - ${d.message}`); logger.error({ errors: messages }, 'Environment validation failed'); if (process.env.NODE_ENV === 'production') { process.exit(1); // Fatal in production } else { logger.warn('Continuing with default values in non-production mode'); } } return value; } ``` --- ## 3. Helpers (`helpers.js`) ### Purpose Utility class providing common functions used across the application. ### Class Definition ```javascript const jwt = require('jsonwebtoken'); const config = require('./config'); module.exports = class Helpers { /** * Wraps async route handlers to catch errors and pass to next() * @param {Function} fn - Async function (req, res, next) => Promise * @returns {Function} Wrapped function */ static wrapAsync(fn) { return function (req, res, next) { fn(req, res, next).catch(next); }; } /** * Common error handler middleware * @param {Error} error - Error object with code/status property * @param {Request} req - Express request * @param {Response} res - Express response * @param {Function} _next - Next middleware (unused) */ static commonErrorHandler(error, req, res, _next) { const statusCode = error.code || error.status; // Known HTTP error codes - return error message if ([400, 401, 403, 404, 409, 422, 503].includes(statusCode)) { return res.status(statusCode).send(error.message); } // Unknown errors - log and return generic message const requestLog = getRequestLogger(req) ?? logger; requestLog.error({ err: error }, 'Route handler failed'); return res.status(500).send('Internal server error'); } /** * Sign JWT token with 6-hour expiration * @param {Object} data - Payload to sign * @returns {string} JWT token */ static jwtSign(data) { return jwt.sign(data, config.secret_key, { expiresIn: '6h' }); } /** * Validate UUID v4 format * @param {string} value - String to validate * @returns {boolean} True if valid UUID v4 */ static isUuidV4(value) { return /^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i.test( value, ); } }; ``` ### Usage Examples #### wrapAsync ```javascript // Without wrapAsync - need try/catch router.get('/', async (req, res, next) => { try { const data = await Service.findAll(); res.json(data); } catch (error) { next(error); } }); // With wrapAsync - cleaner code const wrapAsync = require('../helpers').wrapAsync; router.get('/', wrapAsync(async (req, res) => { const data = await Service.findAll(); res.json(data); })); ``` #### commonErrorHandler ```javascript const { commonErrorHandler } = require('../helpers'); // At end of route file router.use('/', commonErrorHandler); // Errors with code/status are returned as-is const error = new Error('Not found'); error.code = 404; throw error; // → 404 "Not found" // Unknown errors return 500 throw new Error('Database connection failed'); // → 500 "Internal server error" ``` #### jwtSign ```javascript const { jwtSign } = require('./helpers'); // Create JWT token const token = jwtSign({ user: { id: user.id, email: user.email, }, }); // Token expires in 6 hours // Token is signed with config.secret_key ``` #### isUuidV4 ```javascript const { isUuidV4 } = require('./helpers'); // Validate UUID format isUuidV4('550e8400-e29b-41d4-a716-446655440000'); // true isUuidV4('550e8400-e29b-31d4-a716-446655440000'); // false (version 3) isUuidV4('not-a-uuid'); // false isUuidV4(''); // false ``` --- ## Module Dependencies ``` ┌─────────────────────────────────────────────────────────────────┐ │ index.ts │ │ (Application Entry) │ └─────────────────────────────────────────────────────────────────┘ │ ┌───────────────────┼───────────────────┐ ▼ ▼ ▼ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │ config.ts │ │ helpers.ts │ │ utils/logger │ │ (Configuration) │ │ (Utilities) │ │ (Logging) │ └─────────────────┘ └─────────────────┘ └─────────────────┘ │ │ ▼ ▼ ┌─────────────────┐ ┌─────────────────┐ │ env-validation │ │ jsonwebtoken │ │ (Joi) │ │ (JWT lib) │ └─────────────────┘ └─────────────────┘ External Dependencies: • express (web framework) • cors (CORS middleware) • helmet (security headers) • passport (authentication) • body-parser (request parsing) • swagger-ui-express (API docs) • jsonwebtoken (JWT signing) • joi (schema validation) • dotenv (environment loading) • pino (logging) ``` --- ## Best Practices Implemented ### 1. Environment Validation - Joi schema validates all environment variables at startup - Fails fast in production, warns in development - Provides sensible defaults for optional variables ### 2. Security - Helmet middleware for security headers - CORS configured for cross-origin requests - JWT authentication with 6-hour expiration - Trust proxy enabled for reverse proxy support ### 3. Error Handling - `wrapAsync` wrapper for async route handlers - Centralized error handler with status code mapping - Generic error handler logs and returns safe message ### 4. Middleware Ordering - File routes mounted before body parser (binary uploads) - Rate limiters applied before routes - Authentication applied per-route (not globally) - Error handler at the end of middleware stack ### 5. API Documentation - Swagger/OpenAPI 3.0 generated from `backend/src/openapi/document.ts` - Available at `/api-docs` endpoint - Security scheme documented (Bearer JWT) - `backend/tests/openapi-document.test.ts` checks key paths, factory CRUD coverage, internal `$ref` resolution, and Swagger UI handler smoke behavior ### 6. Logging - Pino structured logging - Request logger for all routes - Error logging with context (URL, method) --- ## Configuration Precedence ``` 1. Environment Variables (process.env.*) ↓ (fallback) 2. .env file (loaded by dotenv) ↓ (fallback) 3. Joi schema defaults ↓ (fallback) 4. Hardcoded defaults in config.ts ``` --- ## Server Modes | NODE_ENV | Port | Database | Swagger | Description | |----------|------|----------|---------|-------------| | `development` | 8080 | Local | localhost:8080 | Legacy local development | | `dev_stage` | 3000 | Remote | localhost:3000 | Staging preview | | `production` | 8080 | Remote | Disabled | Production deployment | | `test` | 8080 | Test DB | Disabled | Automated testing | **Standard VM note:** the VM PM2 setup runs the backend with `NODE_ENV=dev_stage`, so the backend listens on port `3000`. The frontend runs separately on port `3001`, and Apache proxies public traffic from port `80`. Do not use `8080` as the VM backend health check unless the PM2 definition has been changed. See [deployment-vm.md](../../../documentation/deployment-vm.md).