39948-vm/backend/docs/modules/core.md

Core Module Documentation

The Core module provides the foundational components of the backend application: the entry point, configuration management, and utility functions.

Overview

File	Purpose	Lines
src/index.ts	Application entry point, Express setup, middleware, route mounting	varies
src/config.ts	Environment configuration and settings	varies
src/helpers.js	Utility functions (wrapAsync, JWT, validation)	32
src/types/	Shared strict TypeScript contracts for migrated backend code	varies
src/load-env.ts	Central backend .env bootstrap for app and DB entrypoints	varies

---

1. Application Entry Point (index.ts)

Purpose

The main entry point that bootstraps the Express application, configures middleware, mounts routes, and starts the HTTP server.

Dependencies

import bodyParser from 'body-parser';
import cors from 'cors';
import express from 'express';
import helmet from 'helmet';
import * as swaggerUI from 'swagger-ui-express';

import { authenticateJwt, authenticateJwtWithCallback } from './auth/passport-middleware.ts';
import config from './config.ts';
import { wrapAsync } from './helpers.ts';
import { runtimeContextMiddleware } from './middlewares/runtime-context.ts';
import { downloadLimiter, searchLimiter, uploadLimiter } from './middlewares/rateLimiter.ts';
import { createOpenApiDocument } from './openapi/document.ts';
import {
  exitAfterLogging,
  logger,
  registerProcessErrorHandlers,
  requestLogger,
} from './utils/logger.ts';

TypeScript migration note: index.ts, config.ts, helpers.ts, middlewares/validate-request.ts, middlewares/runtime-context.ts, middlewares/runtime-public.ts, routes/runtime-context.ts, validators/request-schemas.ts, services/notifications/list.ts, services/notifications/helpers.ts, utils/logger.ts, and utils/env-validation.ts are now migrated runtime TypeScript modules. New migrated backend TypeScript files use backend/tsconfig.json with strict: true, reusable named types from src/types/, and lint rules that reject any, non-null assertions, and unsafe type assertions.

Environment bootstrap note: src/load-env.ts is loaded by app config and DB entrypoints so short backend scripts can see backend/.env without repeating NODE_OPTIONS=-r dotenv/config. If NODE_ENV is absent, it defaults to dev_stage, matching the standard VM backend flow.

Application Bootstrap Sequence

┌─────────────────────────────────────────────────────────────────┐
│                    APPLICATION BOOTSTRAP                        │
└─────────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────────┐
│ 1. EXPRESS INITIALIZATION                                       │
│    • Create Express app                                         │
│    • Enable trust proxy (for reverse proxies)                   │
└─────────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────────┐
│ 2. SECURITY MIDDLEWARE                                          │
│    • Helmet (CSP disabled, COEP disabled)                       │
│    • CORS (origin: true - allow all origins)                    │
└─────────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────────┐
│ 3. AUTHENTICATION SETUP                                         │
│    • Load Passport strategies (JWT, Google, Microsoft)          │
│    • Create jwtAuth middleware                                  │
└─────────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────────┐
│ 4. LOGGING                                                      │
│    • Register process-level error handlers                      │
│    • Apply requestLogger middleware (early for full coverage)   │
└─────────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────────┐
│ 5. FILE ROUTES (BEFORE BODY PARSER)                             │
│    • Mount file routes without JSON parsing                     │
│    • Apply rate limiters (download: 200/min, upload: 10/min)    │
└─────────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────────┐
│ 6. BODY PARSING                                                 │
│    • JSON parser (1MB limit)                                    │
│    • URL-encoded parser (1MB limit)                             │
└─────────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────────┐
│ 7. RUNTIME CONTEXT                                              │
│    • Apply runtimeContextMiddleware                             │
│    • Detect environment from X-Runtime-Environment header       │
└─────────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────────┐
│ 8. ROUTE MOUNTING                                               │
│    • Public routes (health, auth, runtime-context)              │
│    • Protected routes (JWT required)                            │
│    • Runtime public routes (production content without auth)    │
└─────────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────────┐
│ 9. STATIC FILES & ERROR HANDLING                                │
│    • Serve public directory if exists                           │
│    • Generic error handler                                      │
└─────────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────────┐
│ 10. SERVER START                                                │
│    • Listen on PORT (8080 or 3000 for dev_stage)                │
│    • Log startup message                                        │
└─────────────────────────────────────────────────────────────────┘

Middleware Stack

// Order matters - applied in this sequence:

1. swaggerUI.serve          // API documentation at /api-docs
2. helmet()                 // Security headers
3. cors({ origin: true })   // Cross-origin requests
4. requestLogger            // Request logging (Pino)
5. downloadLimiter          // Rate limit for /api/file/download, /api/file/presign
6. uploadLimiter            // Rate limit for /api/file/upload*
7. bodyParser.json()        // JSON parsing (1MB limit)
8. bodyParser.urlencoded()  // Form data parsing
9. runtimeContextMiddleware // Environment detection
10. passport.authenticate() // JWT authentication (per-route)
11. checkPermissions        // RBAC (per-route)
12. errorHandler            // Generic error handling

Route Mounting

Public Routes (No Authentication)

app.get('/api/health', ...)           // Health check
app.use('/api/auth', authRoutes)      // Authentication
app.use('/api/runtime-context', ...)  // Runtime context
app.use('/api/file', fileRoutes)      // File download/presign (partial)

Protected Routes (JWT Required)

app.use('/api/users', jwtAuth, usersRoutes)
app.use('/api/roles', jwtAuth, rolesRoutes)
app.use('/api/permissions', jwtAuth, permissionsRoutes)
app.use('/api/project_memberships', jwtAuth, project_membershipsRoutes)
app.use('/api/assets', jwtAuth, assetsRoutes)
app.use('/api/asset_variants', jwtAuth, asset_variantsRoutes)
app.use('/api/presigned_url_requests', jwtAuth, presigned_url_requestsRoutes)
app.use('/api/publish_events', jwtAuth, publish_eventsRoutes)
app.use('/api/pwa_caches', jwtAuth, pwa_cachesRoutes)
app.use('/api/access_logs', jwtAuth, access_logsRoutes)
app.use('/api/element-type-defaults', jwtAuth, element_type_defaultsRoutes)
app.use('/api/ui-elements', jwtAuth, element_type_defaultsRoutes)  // Alias
app.use('/api/project-element-defaults', jwtAuth, project_element_defaultsRoutes)
app.use('/api/publish', jwtAuth, publishRoutes)
app.use('/api/search', jwtAuth, searchLimiter, searchRoutes)

Runtime Public Routes (Production Content Without Auth)

// These routes use requireRuntimeReadOrAuth middleware
// Allows unauthenticated GET requests in production environment

mountRuntimeEntityRoute('/api/projects', 'projects', projectsRoutes)
mountRuntimeEntityRoute('/api/tour_pages', 'tour_pages', tour_pagesRoutes)
mountRuntimeEntityRoute('/api/project_audio_tracks', 'project_audio_tracks', project_audio_tracksRoutes)

Key Functions

requireRuntimeReadOrAuth

Middleware that allows public read access for production content:

const requireRuntimeReadOrAuth = (req, res, next) => {
  const headerEnvironment = req.runtimeContext?.headerEnvironment;
  const isReadOnlyRequest = ['GET', 'OPTIONS'].includes(req.method);
  const hasAuthHeader = Boolean(req.headers.authorization);

  // Only production is public. Stage requires authentication.
  const isPublicEnvironment = headerEnvironment === 'production';

  if (isPublicEnvironment && isReadOnlyRequest && !hasAuthHeader) {
    req.isRuntimePublicRequest = true;
    return next();  // Allow without JWT
  }

  req.isRuntimePublicRequest = false;
  return jwtAuth(req, res, next);  // Require JWT
};

mountRuntimeEntityRoute

Helper to mount routes with runtime public access middleware stack:

const mountRuntimeEntityRoute = (path, entityName, router) => {
  app.use(
    path,
    requireRuntimeReadOrAuth,              // JWT or public production
    blockNonPublicRuntimeListEndpoints,    // Block non-list for public
    sanitizePublicRuntimeListResponse(entityName),  // Filter sensitive fields
    router,
  );
};

getBaseUrl

Utility to extract base URL for Swagger:

const getBaseUrl = (url) => {
  if (!url) return '';
  return url.endsWith('/api') ? url.slice(0, -4) : url;
};

Health Check Endpoint

GET /api/health

// Response (200 - healthy):
{
  "status": "ok",
  "timestamp": "2026-03-30T12:00:00.000Z",
  "uptime": 12345.678,
  "environment": "production",
  "database": "connected"
}

// Response (503 - degraded):
{
  "status": "degraded",
  "timestamp": "2026-03-30T12:00:00.000Z",
  "uptime": 12345.678,
  "environment": "production",
  "database": "disconnected",
  "databaseError": "Connection refused"
}

Swagger/OpenAPI Configuration

const specs = createOpenApiDocument({
  serverUrl: config.server.swaggerServerUrl,
});

app.use('/api-docs', swaggerUI.serve, swaggerUI.setup(specs));

The canonical OpenAPI source is backend/src/openapi/document.ts. It defines shared schemas, common responses and parameters, generated factory CRUD paths, and explicit custom-route paths. Route-local JSDoc comments are not the source of truth for Swagger UI.

Error Handler

app.use((err, req, res, _next) => {
  if (!res.headersSent) {
    const requestLog = getRequestLogger(req) ?? logger;
    requestLog.error(
      { err, url: req.url, method: req.method },
      'Express error middleware caught unhandled error',
    );
    res.status(safeStatusCode).json({
      message:
        safeStatusCode === 503
          ? 'Service temporarily unavailable'
          : 'Internal server error',
    });
  }
});

Route-level commonErrorHandler also uses the request-scoped logger when available and logs unexpected route failures as Route handler failed. Circuit breaker rejections use status 503 instead of being collapsed to 500.

Server Configuration

const PORT = config.server.port;

const server = app.listen(PORT, () => {
  logger.info(
    { port: PORT, env: config.server.env },
    'Server started',
  );
});

server.on('error', (err) => {
  logger.error(
    { err, port: PORT, env: config.server.env },
    'Server failed to start',
  );
  exitAfterLogging();
});

Startup errors such as EADDRINUSE happen on the Node Server, outside the Express request lifecycle, so the generic Express error middleware cannot catch them. The server-level error listener logs the failure and exits with a non-zero status for nodemon/PM2 supervision.

---

2. Configuration (config.ts)

Purpose

Centralized configuration management with environment variable support and sensible defaults.

Configuration Structure

const config = {
  // Google Cloud Storage
  gcloud: {
    bucket: 'fldemo-files',
    hash: 'afeefb9d49f5b7977577876b99532ac7',
    projectId: env.GC_PROJECT_ID,
    clientEmail: env.GC_CLIENT_EMAIL,
    privateKey: env.GC_PRIVATE_KEY,
  },
  fileStorage: {
    provider: env.FILE_STORAGE_PROVIDER,
  },

  // AWS S3
  s3: {
    bucket: env.AWS_S3_BUCKET,
    region: env.AWS_S3_REGION,
    accessKeyId: env.AWS_ACCESS_KEY_ID,
    secretAccessKey: env.AWS_SECRET_ACCESS_KEY,
    prefix: env.AWS_S3_PREFIX,
    connectionTimeout: env.AWS_S3_CONNECTION_TIMEOUT,
    requestTimeout: env.AWS_S3_REQUEST_TIMEOUT,
    maxAttempts: env.AWS_S3_MAX_ATTEMPTS,
    maxSockets: env.AWS_S3_MAX_SOCKETS,
    keepAlive: env.AWS_S3_KEEP_ALIVE !== 'false',
    presignExpirySeconds: env.AWS_S3_PRESIGN_EXPIRY,
  },

  resilience: {
    ffmpeg: {
      reverseTimeoutMs: env.FFMPEG_REVERSE_TIMEOUT_MS,
      ffprobeTimeoutMs: env.FFPROBE_TIMEOUT_MS,
      breaker: {
        failureThreshold: env.FFMPEG_BREAKER_FAILURE_THRESHOLD,
        cooldownMs: env.FFMPEG_BREAKER_COOLDOWN_MS,
        successThreshold: env.FFMPEG_BREAKER_SUCCESS_THRESHOLD,
      },
    },
    fileStorage: {
      breaker: {
        failureThreshold: env.FILE_STORAGE_BREAKER_FAILURE_THRESHOLD,
        cooldownMs: env.FILE_STORAGE_BREAKER_COOLDOWN_MS,
        successThreshold: env.FILE_STORAGE_BREAKER_SUCCESS_THRESHOLD,
      },
    },
  },

  // Password hashing
  bcrypt: {
    saltRounds: 12,
  },

  // Default credentials
  admin_pass: env.ADMIN_PASS,
  user_pass: env.USER_PASS,
  admin_email: env.ADMIN_EMAIL,

  // Authentication providers
  providers: {
    LOCAL: 'local',
    GOOGLE: 'google',
    MICROSOFT: 'microsoft',
  },

  // JWT
  secret_key: env.SECRET_KEY,

  // Server URLs
  remote: '',
  port,
  hostUI,
  portUI,

  // Swagger
  swaggerUI,
  swaggerPort,

  // OAuth
  google: {
    clientId: env.GOOGLE_CLIENT_ID,
    clientSecret: env.GOOGLE_CLIENT_SECRET,
  },
  microsoft: {
    clientId: env.MS_CLIENT_ID,
    clientSecret: env.MS_CLIENT_SECRET,
  },

  // File uploads
  uploadDir: os.tmpdir(),

  // Email (AWS SES)
  email: {
    from: 'Tour Builder Platform <app@flatlogic.app>',
    host: 'email-smtp.us-east-1.amazonaws.com',
    port: 587,
    auth: {
      user: env.EMAIL_USER,
      pass: env.EMAIL_PASS,
    },
    tls: {
      rejectUnauthorized: env.EMAIL_TLS_REJECT_UNAUTHORIZED !== 'false',
    },
  },

  // Default roles
  roles: {
    admin: 'Administrator',
    user: 'Analytics Viewer',
  },
  server: {
    env: env.NODE_ENV,
    port: serverPort,
    swaggerServerUrl,
  },

};

Environment Variables Reference

Variable	Type	Default	Description
NODE_ENV	string	development	Environment: development, production, dev_stage, test
PORT	number	8080	Server port
SECRET_KEY	string	UUID	JWT signing key (min 16 chars)
ADMIN_EMAIL	string	admin@flatlogic.com	Admin user email
ADMIN_PASS	string	Generated	Admin user password
USER_PASS	string	Generated	Default user password
AWS_S3_BUCKET	string	-	S3 bucket name
AWS_S3_REGION	string	us-east-1	S3 region
AWS_ACCESS_KEY_ID	string	-	AWS access key
AWS_SECRET_ACCESS_KEY	string	-	AWS secret key
AWS_S3_PREFIX	string	Hash	S3 key prefix
GOOGLE_CLIENT_ID	string	-	Google OAuth client ID
GOOGLE_CLIENT_SECRET	string	-	Google OAuth client secret
MS_CLIENT_ID	string	-	Microsoft OAuth client ID
MS_CLIENT_SECRET	string	-	Microsoft OAuth client secret
EMAIL_USER	string	-	SMTP username
EMAIL_PASS	string	-	SMTP password
EMAIL_TLS_REJECT_UNAUTHORIZED	string	true	TLS validation
LOG_LEVEL	string	info	Pino log level

Environment Validation

The configuration uses Joi schema validation via utils/env-validation.js:

const Joi = require('joi');

const envSchema = Joi.object({
  NODE_ENV: Joi.string()
    .valid('development', 'test', 'production', 'dev_stage')
    .default('development'),

  PORT: Joi.number().default(8080),

  DB_HOST: Joi.string().default('localhost'),
  DB_PORT: Joi.number().default(5432),
  DB_NAME: Joi.string().default('db_tour_builder_platform'),
  DB_USER: Joi.string().default('postgres'),
  DB_PASS: Joi.string().allow('').default(''),

  SECRET_KEY: Joi.string()
    .min(16)
    .default('88dbeaf8-e906-405e-9e41-c3baadeda5c6'),

  // ... more validations
}).unknown(true);

function validateEnv() {
  const { error, value } = envSchema.validate(process.env, {
    abortEarly: false,
    stripUnknown: false,
  });

  if (error) {
    const messages = error.details.map((d) => `  - ${d.message}`);
    logger.error({ errors: messages }, 'Environment validation failed');

    if (process.env.NODE_ENV === 'production') {
      process.exit(1);  // Fatal in production
    } else {
      logger.warn('Continuing with default values in non-production mode');
    }
  }

  return value;
}

---

3. Helpers (helpers.js)

Purpose

Utility class providing common functions used across the application.

Class Definition

const jwt = require('jsonwebtoken');
const config = require('./config');

module.exports = class Helpers {
  /**
   * Wraps async route handlers to catch errors and pass to next()
   * @param {Function} fn - Async function (req, res, next) => Promise
   * @returns {Function} Wrapped function
   */
  static wrapAsync(fn) {
    return function (req, res, next) {
      fn(req, res, next).catch(next);
    };
  }

  /**
   * Common error handler middleware
   * @param {Error} error - Error object with code/status property
   * @param {Request} req - Express request
   * @param {Response} res - Express response
   * @param {Function} _next - Next middleware (unused)
   */
  static commonErrorHandler(error, req, res, _next) {
    const statusCode = error.code || error.status;

    // Known HTTP error codes - return error message
    if ([400, 401, 403, 404, 409, 422, 503].includes(statusCode)) {
      return res.status(statusCode).send(error.message);
    }

    // Unknown errors - log and return generic message
    const requestLog = getRequestLogger(req) ?? logger;
    requestLog.error({ err: error }, 'Route handler failed');
    return res.status(500).send('Internal server error');
  }

  /**
   * Sign JWT token with 6-hour expiration
   * @param {Object} data - Payload to sign
   * @returns {string} JWT token
   */
  static jwtSign(data) {
    return jwt.sign(data, config.secret_key, { expiresIn: '6h' });
  }

  /**
   * Validate UUID v4 format
   * @param {string} value - String to validate
   * @returns {boolean} True if valid UUID v4
   */
  static isUuidV4(value) {
    return /^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i.test(
      value,
    );
  }
};

Usage Examples

wrapAsync

// Without wrapAsync - need try/catch
router.get('/', async (req, res, next) => {
  try {
    const data = await Service.findAll();
    res.json(data);
  } catch (error) {
    next(error);
  }
});

// With wrapAsync - cleaner code
const wrapAsync = require('../helpers').wrapAsync;

router.get('/', wrapAsync(async (req, res) => {
  const data = await Service.findAll();
  res.json(data);
}));

commonErrorHandler

const { commonErrorHandler } = require('../helpers');

// At end of route file
router.use('/', commonErrorHandler);

// Errors with code/status are returned as-is
const error = new Error('Not found');
error.code = 404;
throw error;  // → 404 "Not found"

// Unknown errors return 500
throw new Error('Database connection failed');  // → 500 "Internal server error"

jwtSign

const { jwtSign } = require('./helpers');

// Create JWT token
const token = jwtSign({
  user: {
    id: user.id,
    email: user.email,
  },
});

// Token expires in 6 hours
// Token is signed with config.secret_key

isUuidV4

const { isUuidV4 } = require('./helpers');

// Validate UUID format
isUuidV4('550e8400-e29b-41d4-a716-446655440000');  // true
isUuidV4('550e8400-e29b-31d4-a716-446655440000');  // false (version 3)
isUuidV4('not-a-uuid');                             // false
isUuidV4('');                                       // false

---

Module Dependencies

┌─────────────────────────────────────────────────────────────────┐
│                        index.ts                                 │
│                    (Application Entry)                          │
└─────────────────────────────────────────────────────────────────┘
                              │
          ┌───────────────────┼───────────────────┐
          ▼                   ▼                   ▼
┌─────────────────┐  ┌─────────────────┐  ┌─────────────────┐
│   config.ts     │  │   helpers.ts    │  │  utils/logger   │
│ (Configuration) │  │  (Utilities)    │  │   (Logging)     │
└─────────────────┘  └─────────────────┘  └─────────────────┘
          │                   │
          ▼                   ▼
┌─────────────────┐  ┌─────────────────┐
│ env-validation  │  │  jsonwebtoken   │
│    (Joi)        │  │   (JWT lib)     │
└─────────────────┘  └─────────────────┘

External Dependencies:
  • express (web framework)
  • cors (CORS middleware)
  • helmet (security headers)
  • passport (authentication)
  • body-parser (request parsing)
  • swagger-ui-express (API docs)
  • jsonwebtoken (JWT signing)
  • joi (schema validation)
  • dotenv (environment loading)
  • pino (logging)

---

Best Practices Implemented

1. Environment Validation

• Joi schema validates all environment variables at startup
• Fails fast in production, warns in development
• Provides sensible defaults for optional variables

2. Security

• Helmet middleware for security headers
• CORS configured for cross-origin requests
• JWT authentication with 6-hour expiration
• Trust proxy enabled for reverse proxy support

3. Error Handling

• wrapAsync wrapper for async route handlers
• Centralized error handler with status code mapping
• Generic error handler logs and returns safe message

4. Middleware Ordering

• File routes mounted before body parser (binary uploads)
• Rate limiters applied before routes
• Authentication applied per-route (not globally)
• Error handler at the end of middleware stack

5. API Documentation

• Swagger/OpenAPI 3.0 generated from backend/src/openapi/document.ts
• Available at /api-docs endpoint
• Security scheme documented (Bearer JWT)
• backend/tests/openapi-document.test.ts checks key paths, factory CRUD coverage, internal $ref resolution, and Swagger UI handler smoke behavior

6. Logging

• Pino structured logging
• Request logger for all routes
• Error logging with context (URL, method)

---

Configuration Precedence

1. Environment Variables (process.env.*)
   ↓ (fallback)
2. .env file (loaded by dotenv)
   ↓ (fallback)
3. Joi schema defaults
   ↓ (fallback)
4. Hardcoded defaults in config.ts

---

Server Modes

NODE_ENV	Port	Database	Swagger	Description
development	8080	Local	localhost:8080	Legacy local development
dev_stage	3000	Remote	localhost:3000	Staging preview
production	8080	Remote	Disabled	Production deployment
test	8080	Test DB	Disabled	Automated testing

Standard VM note: the VM PM2 setup runs the backend with NODE_ENV=dev_stage, so the backend listens on port 3000. The frontend runs separately on port 3001, and Apache proxies public traffic from port 80. Do not use 8080 as the VM backend health check unless the PM2 definition has been changed. See deployment-vm.md.