2026-07-03 16:11:24 +02:00

782 lines
28 KiB
Markdown

# Core Module Documentation
The Core module provides the foundational components of the backend application: the entry point, configuration management, and utility functions.
## Overview
| File | Purpose | Lines |
|------|---------|-------|
| `src/index.ts` | Application entry point, Express setup, middleware, route mounting | varies |
| `src/config.ts` | Environment configuration and settings | varies |
| `src/helpers.js` | Utility functions (wrapAsync, JWT, validation) | 32 |
| `src/types/` | Shared strict TypeScript contracts for migrated backend code | varies |
| `src/load-env.ts` | Central backend `.env` bootstrap for app and DB entrypoints | varies |
---
## 1. Application Entry Point (`index.ts`)
### Purpose
The main entry point that bootstraps the Express application, configures middleware, mounts routes, and starts the HTTP server.
### Dependencies
```typescript
import bodyParser from 'body-parser';
import cors from 'cors';
import express from 'express';
import helmet from 'helmet';
import * as swaggerUI from 'swagger-ui-express';
import { authenticateJwt, authenticateJwtWithCallback } from './auth/passport-middleware.ts';
import config from './config.ts';
import { wrapAsync } from './helpers.ts';
import { runtimeContextMiddleware } from './middlewares/runtime-context.ts';
import { downloadLimiter, searchLimiter, uploadLimiter } from './middlewares/rateLimiter.ts';
import { createOpenApiDocument } from './openapi/document.ts';
import {
exitAfterLogging,
logger,
registerProcessErrorHandlers,
requestLogger,
} from './utils/logger.ts';
```
> TypeScript migration note: `index.ts`, `config.ts`, `helpers.ts`, `middlewares/validate-request.ts`, `middlewares/runtime-context.ts`, `middlewares/runtime-public.ts`, `routes/runtime-context.ts`, `validators/request-schemas.ts`, `services/notifications/list.ts`, `services/notifications/helpers.ts`, `utils/logger.ts`, and `utils/env-validation.ts` are now migrated runtime TypeScript modules. New migrated backend TypeScript files use `backend/tsconfig.json` with `strict: true`, reusable named types from `src/types/`, and lint rules that reject `any`, non-null assertions, and unsafe type assertions.
> Environment bootstrap note: `src/load-env.ts` is loaded by app config and DB entrypoints so short backend scripts can see `backend/.env` without repeating `NODE_OPTIONS=-r dotenv/config`. If `NODE_ENV` is absent, it defaults to `dev_stage`, matching the standard VM backend flow.
### Application Bootstrap Sequence
```
┌─────────────────────────────────────────────────────────────────┐
│ APPLICATION BOOTSTRAP │
└─────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────┐
│ 1. EXPRESS INITIALIZATION │
│ • Create Express app │
│ • Enable trust proxy (for reverse proxies) │
└─────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────┐
│ 2. SECURITY MIDDLEWARE │
│ • Helmet (CSP disabled, COEP disabled) │
│ • CORS (origin: true - allow all origins) │
└─────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────┐
│ 3. AUTHENTICATION SETUP │
│ • Load Passport strategies (JWT, Google, Microsoft) │
│ • Create jwtAuth middleware │
└─────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────┐
│ 4. LOGGING │
│ • Register process-level error handlers │
│ • Apply requestLogger middleware (early for full coverage) │
└─────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────┐
│ 5. FILE ROUTES (BEFORE BODY PARSER) │
│ • Mount file routes without JSON parsing │
│ • Apply rate limiters (download: 200/min, upload: 10/min) │
└─────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────┐
│ 6. BODY PARSING │
│ • JSON parser (1MB limit) │
│ • URL-encoded parser (1MB limit) │
└─────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────┐
│ 7. RUNTIME CONTEXT │
│ • Apply runtimeContextMiddleware │
│ • Detect environment from X-Runtime-Environment header │
└─────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────┐
│ 8. ROUTE MOUNTING │
│ • Public routes (health, auth, runtime-context) │
│ • Protected routes (JWT required) │
│ • Runtime public routes (production content without auth) │
└─────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────┐
│ 9. STATIC FILES & ERROR HANDLING │
│ • Serve public directory if exists │
│ • Generic error handler │
└─────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────┐
│ 10. SERVER START │
│ • Listen on PORT (8080 or 3000 for dev_stage) │
│ • Log startup message │
└─────────────────────────────────────────────────────────────────┘
```
### Middleware Stack
```javascript
// Order matters - applied in this sequence:
1. swaggerUI.serve // API documentation at /api-docs
2. helmet() // Security headers
3. cors({ origin: true }) // Cross-origin requests
4. requestLogger // Request logging (Pino)
5. downloadLimiter // Rate limit for /api/file/download, /api/file/presign
6. uploadLimiter // Rate limit for /api/file/upload*
7. bodyParser.json() // JSON parsing (1MB limit)
8. bodyParser.urlencoded() // Form data parsing
9. runtimeContextMiddleware // Environment detection
10. passport.authenticate() // JWT authentication (per-route)
11. checkPermissions // RBAC (per-route)
12. errorHandler // Generic error handling
```
### Route Mounting
#### Public Routes (No Authentication)
```javascript
app.get('/api/health', ...) // Health check
app.use('/api/auth', authRoutes) // Authentication
app.use('/api/runtime-context', ...) // Runtime context
app.use('/api/file', fileRoutes) // File download/presign (partial)
```
#### Protected Routes (JWT Required)
```javascript
app.use('/api/users', jwtAuth, usersRoutes)
app.use('/api/roles', jwtAuth, rolesRoutes)
app.use('/api/permissions', jwtAuth, permissionsRoutes)
app.use('/api/project_memberships', jwtAuth, project_membershipsRoutes)
app.use('/api/assets', jwtAuth, assetsRoutes)
app.use('/api/asset_variants', jwtAuth, asset_variantsRoutes)
app.use('/api/presigned_url_requests', jwtAuth, presigned_url_requestsRoutes)
app.use('/api/publish_events', jwtAuth, publish_eventsRoutes)
app.use('/api/pwa_caches', jwtAuth, pwa_cachesRoutes)
app.use('/api/access_logs', jwtAuth, access_logsRoutes)
app.use('/api/element-type-defaults', jwtAuth, element_type_defaultsRoutes)
app.use('/api/ui-elements', jwtAuth, element_type_defaultsRoutes) // Alias
app.use('/api/project-element-defaults', jwtAuth, project_element_defaultsRoutes)
app.use('/api/publish', jwtAuth, publishRoutes)
app.use('/api/search', jwtAuth, searchLimiter, searchRoutes)
```
#### Runtime Public Routes (Production Content Without Auth)
```javascript
// These routes use requireRuntimeReadOrAuth middleware
// Allows unauthenticated GET requests in production environment
mountRuntimeEntityRoute('/api/projects', 'projects', projectsRoutes)
mountRuntimeEntityRoute('/api/tour_pages', 'tour_pages', tour_pagesRoutes)
mountRuntimeEntityRoute('/api/project_audio_tracks', 'project_audio_tracks', project_audio_tracksRoutes)
```
### Key Functions
#### `requireRuntimeReadOrAuth`
Middleware that allows public read access for production content:
```javascript
const requireRuntimeReadOrAuth = (req, res, next) => {
const headerEnvironment = req.runtimeContext?.headerEnvironment;
const isReadOnlyRequest = ['GET', 'OPTIONS'].includes(req.method);
const hasAuthHeader = Boolean(req.headers.authorization);
// Only production is public. Stage requires authentication.
const isPublicEnvironment = headerEnvironment === 'production';
if (isPublicEnvironment && isReadOnlyRequest && !hasAuthHeader) {
req.isRuntimePublicRequest = true;
return next(); // Allow without JWT
}
req.isRuntimePublicRequest = false;
return jwtAuth(req, res, next); // Require JWT
};
```
#### `mountRuntimeEntityRoute`
Helper to mount routes with runtime public access middleware stack:
```javascript
const mountRuntimeEntityRoute = (path, entityName, router) => {
app.use(
path,
requireRuntimeReadOrAuth, // JWT or public production
blockNonPublicRuntimeListEndpoints, // Block non-list for public
sanitizePublicRuntimeListResponse(entityName), // Filter sensitive fields
router,
);
};
```
#### `getBaseUrl`
Utility to extract base URL for Swagger:
```javascript
const getBaseUrl = (url) => {
if (!url) return '';
return url.endsWith('/api') ? url.slice(0, -4) : url;
};
```
### Health Check Endpoint
```javascript
GET /api/health
// Response (200 - healthy):
{
"status": "ok",
"timestamp": "2026-03-30T12:00:00.000Z",
"uptime": 12345.678,
"environment": "production",
"database": "connected"
}
// Response (503 - degraded):
{
"status": "degraded",
"timestamp": "2026-03-30T12:00:00.000Z",
"uptime": 12345.678,
"environment": "production",
"database": "disconnected",
"databaseError": "Connection refused"
}
```
### Swagger/OpenAPI Configuration
```javascript
const specs = createOpenApiDocument({
serverUrl: config.server.swaggerServerUrl,
});
app.use('/api-docs', swaggerUI.serve, swaggerUI.setup(specs));
```
The canonical OpenAPI source is `backend/src/openapi/document.ts`. It defines
shared schemas, common responses and parameters, generated factory CRUD paths,
and explicit custom-route paths. Route-local JSDoc comments are not the source
of truth for Swagger UI.
### Error Handler
```javascript
app.use((err, req, res, _next) => {
if (!res.headersSent) {
const requestLog = getRequestLogger(req) ?? logger;
requestLog.error(
{ err, url: req.url, method: req.method },
'Express error middleware caught unhandled error',
);
res.status(safeStatusCode).json({
message:
safeStatusCode === 503
? 'Service temporarily unavailable'
: 'Internal server error',
});
}
});
```
Route-level `commonErrorHandler` also uses the request-scoped logger when
available and logs unexpected route failures as `Route handler failed`. Circuit
breaker rejections use status `503` instead of being collapsed to `500`.
### Server Configuration
```javascript
const PORT = config.server.port;
const server = app.listen(PORT, () => {
logger.info(
{ port: PORT, env: config.server.env },
'Server started',
);
});
server.on('error', (err) => {
logger.error(
{ err, port: PORT, env: config.server.env },
'Server failed to start',
);
exitAfterLogging();
});
```
Startup errors such as `EADDRINUSE` happen on the Node `Server`, outside the
Express request lifecycle, so the generic Express error middleware cannot catch
them. The server-level `error` listener logs the failure and exits with a
non-zero status for nodemon/PM2 supervision.
---
## 2. Configuration (`config.ts`)
### Purpose
Centralized configuration management with environment variable support and sensible defaults.
### Configuration Structure
```javascript
const config = {
// Google Cloud Storage
gcloud: {
bucket: 'fldemo-files',
hash: 'afeefb9d49f5b7977577876b99532ac7',
projectId: env.GC_PROJECT_ID,
clientEmail: env.GC_CLIENT_EMAIL,
privateKey: env.GC_PRIVATE_KEY,
},
fileStorage: {
provider: env.FILE_STORAGE_PROVIDER,
},
// AWS S3
s3: {
bucket: env.AWS_S3_BUCKET,
region: env.AWS_S3_REGION,
accessKeyId: env.AWS_ACCESS_KEY_ID,
secretAccessKey: env.AWS_SECRET_ACCESS_KEY,
prefix: env.AWS_S3_PREFIX,
connectionTimeout: env.AWS_S3_CONNECTION_TIMEOUT,
requestTimeout: env.AWS_S3_REQUEST_TIMEOUT,
maxAttempts: env.AWS_S3_MAX_ATTEMPTS,
maxSockets: env.AWS_S3_MAX_SOCKETS,
keepAlive: env.AWS_S3_KEEP_ALIVE !== 'false',
presignExpirySeconds: env.AWS_S3_PRESIGN_EXPIRY,
},
resilience: {
ffmpeg: {
reverseTimeoutMs: env.FFMPEG_REVERSE_TIMEOUT_MS,
ffprobeTimeoutMs: env.FFPROBE_TIMEOUT_MS,
breaker: {
failureThreshold: env.FFMPEG_BREAKER_FAILURE_THRESHOLD,
cooldownMs: env.FFMPEG_BREAKER_COOLDOWN_MS,
successThreshold: env.FFMPEG_BREAKER_SUCCESS_THRESHOLD,
},
},
fileStorage: {
breaker: {
failureThreshold: env.FILE_STORAGE_BREAKER_FAILURE_THRESHOLD,
cooldownMs: env.FILE_STORAGE_BREAKER_COOLDOWN_MS,
successThreshold: env.FILE_STORAGE_BREAKER_SUCCESS_THRESHOLD,
},
},
},
// Password hashing
bcrypt: {
saltRounds: 12,
},
// Default credentials
admin_pass: env.ADMIN_PASS,
user_pass: env.USER_PASS,
admin_email: env.ADMIN_EMAIL,
// Authentication providers
providers: {
LOCAL: 'local',
GOOGLE: 'google',
MICROSOFT: 'microsoft',
},
// JWT
secret_key: env.SECRET_KEY,
// Server URLs
remote: '',
port,
hostUI,
portUI,
// Swagger
swaggerUI,
swaggerPort,
// OAuth
google: {
clientId: env.GOOGLE_CLIENT_ID,
clientSecret: env.GOOGLE_CLIENT_SECRET,
},
microsoft: {
clientId: env.MS_CLIENT_ID,
clientSecret: env.MS_CLIENT_SECRET,
},
// File uploads
uploadDir: os.tmpdir(),
// Email (AWS SES)
email: {
from: 'Tour Builder Platform <app@flatlogic.app>',
host: 'email-smtp.us-east-1.amazonaws.com',
port: 587,
auth: {
user: env.EMAIL_USER,
pass: env.EMAIL_PASS,
},
tls: {
rejectUnauthorized: env.EMAIL_TLS_REJECT_UNAUTHORIZED !== 'false',
},
},
// Default roles
roles: {
admin: 'Administrator',
user: 'Analytics Viewer',
},
server: {
env: env.NODE_ENV,
port: serverPort,
swaggerServerUrl,
},
};
```
### Environment Variables Reference
| Variable | Type | Default | Description |
|----------|------|---------|-------------|
| `NODE_ENV` | string | `development` | Environment: `development`, `production`, `dev_stage`, `test` |
| `PORT` | number | `8080` | Server port |
| `SECRET_KEY` | string | UUID | JWT signing key (min 16 chars) |
| `ADMIN_EMAIL` | string | `admin@flatlogic.com` | Admin user email |
| `ADMIN_PASS` | string | Generated | Admin user password |
| `USER_PASS` | string | Generated | Default user password |
| `AWS_S3_BUCKET` | string | - | S3 bucket name |
| `AWS_S3_REGION` | string | `us-east-1` | S3 region |
| `AWS_ACCESS_KEY_ID` | string | - | AWS access key |
| `AWS_SECRET_ACCESS_KEY` | string | - | AWS secret key |
| `AWS_S3_PREFIX` | string | Hash | S3 key prefix |
| `GOOGLE_CLIENT_ID` | string | - | Google OAuth client ID |
| `GOOGLE_CLIENT_SECRET` | string | - | Google OAuth client secret |
| `MS_CLIENT_ID` | string | - | Microsoft OAuth client ID |
| `MS_CLIENT_SECRET` | string | - | Microsoft OAuth client secret |
| `EMAIL_USER` | string | - | SMTP username |
| `EMAIL_PASS` | string | - | SMTP password |
| `EMAIL_TLS_REJECT_UNAUTHORIZED` | string | `true` | TLS validation |
| `LOG_LEVEL` | string | `info` | Pino log level |
### Environment Validation
The configuration uses Joi schema validation via `utils/env-validation.js`:
```javascript
const Joi = require('joi');
const envSchema = Joi.object({
NODE_ENV: Joi.string()
.valid('development', 'test', 'production', 'dev_stage')
.default('development'),
PORT: Joi.number().default(8080),
DB_HOST: Joi.string().default('localhost'),
DB_PORT: Joi.number().default(5432),
DB_NAME: Joi.string().default('db_tour_builder_platform'),
DB_USER: Joi.string().default('postgres'),
DB_PASS: Joi.string().allow('').default(''),
SECRET_KEY: Joi.string()
.min(16)
.default('88dbeaf8-e906-405e-9e41-c3baadeda5c6'),
// ... more validations
}).unknown(true);
function validateEnv() {
const { error, value } = envSchema.validate(process.env, {
abortEarly: false,
stripUnknown: false,
});
if (error) {
const messages = error.details.map((d) => ` - ${d.message}`);
logger.error({ errors: messages }, 'Environment validation failed');
if (process.env.NODE_ENV === 'production') {
process.exit(1); // Fatal in production
} else {
logger.warn('Continuing with default values in non-production mode');
}
}
return value;
}
```
---
## 3. Helpers (`helpers.js`)
### Purpose
Utility class providing common functions used across the application.
### Class Definition
```javascript
const jwt = require('jsonwebtoken');
const config = require('./config');
module.exports = class Helpers {
/**
* Wraps async route handlers to catch errors and pass to next()
* @param {Function} fn - Async function (req, res, next) => Promise
* @returns {Function} Wrapped function
*/
static wrapAsync(fn) {
return function (req, res, next) {
fn(req, res, next).catch(next);
};
}
/**
* Common error handler middleware
* @param {Error} error - Error object with code/status property
* @param {Request} req - Express request
* @param {Response} res - Express response
* @param {Function} _next - Next middleware (unused)
*/
static commonErrorHandler(error, req, res, _next) {
const statusCode = error.code || error.status;
// Known HTTP error codes - return error message
if ([400, 401, 403, 404, 409, 422, 503].includes(statusCode)) {
return res.status(statusCode).send(error.message);
}
// Unknown errors - log and return generic message
const requestLog = getRequestLogger(req) ?? logger;
requestLog.error({ err: error }, 'Route handler failed');
return res.status(500).send('Internal server error');
}
/**
* Sign JWT token with 6-hour expiration
* @param {Object} data - Payload to sign
* @returns {string} JWT token
*/
static jwtSign(data) {
return jwt.sign(data, config.secret_key, { expiresIn: '6h' });
}
/**
* Validate UUID v4 format
* @param {string} value - String to validate
* @returns {boolean} True if valid UUID v4
*/
static isUuidV4(value) {
return /^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i.test(
value,
);
}
};
```
### Usage Examples
#### wrapAsync
```javascript
// Without wrapAsync - need try/catch
router.get('/', async (req, res, next) => {
try {
const data = await Service.findAll();
res.json(data);
} catch (error) {
next(error);
}
});
// With wrapAsync - cleaner code
const wrapAsync = require('../helpers').wrapAsync;
router.get('/', wrapAsync(async (req, res) => {
const data = await Service.findAll();
res.json(data);
}));
```
#### commonErrorHandler
```javascript
const { commonErrorHandler } = require('../helpers');
// At end of route file
router.use('/', commonErrorHandler);
// Errors with code/status are returned as-is
const error = new Error('Not found');
error.code = 404;
throw error; // → 404 "Not found"
// Unknown errors return 500
throw new Error('Database connection failed'); // → 500 "Internal server error"
```
#### jwtSign
```javascript
const { jwtSign } = require('./helpers');
// Create JWT token
const token = jwtSign({
user: {
id: user.id,
email: user.email,
},
});
// Token expires in 6 hours
// Token is signed with config.secret_key
```
#### isUuidV4
```javascript
const { isUuidV4 } = require('./helpers');
// Validate UUID format
isUuidV4('550e8400-e29b-41d4-a716-446655440000'); // true
isUuidV4('550e8400-e29b-31d4-a716-446655440000'); // false (version 3)
isUuidV4('not-a-uuid'); // false
isUuidV4(''); // false
```
---
## Module Dependencies
```
┌─────────────────────────────────────────────────────────────────┐
│ index.ts │
│ (Application Entry) │
└─────────────────────────────────────────────────────────────────┘
┌───────────────────┼───────────────────┐
▼ ▼ ▼
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ config.ts │ │ helpers.ts │ │ utils/logger │
│ (Configuration) │ │ (Utilities) │ │ (Logging) │
└─────────────────┘ └─────────────────┘ └─────────────────┘
│ │
▼ ▼
┌─────────────────┐ ┌─────────────────┐
│ env-validation │ │ jsonwebtoken │
│ (Joi) │ │ (JWT lib) │
└─────────────────┘ └─────────────────┘
External Dependencies:
• express (web framework)
• cors (CORS middleware)
• helmet (security headers)
• passport (authentication)
• body-parser (request parsing)
• swagger-ui-express (API docs)
• jsonwebtoken (JWT signing)
• joi (schema validation)
• dotenv (environment loading)
• pino (logging)
```
---
## Best Practices Implemented
### 1. Environment Validation
- Joi schema validates all environment variables at startup
- Fails fast in production, warns in development
- Provides sensible defaults for optional variables
### 2. Security
- Helmet middleware for security headers
- CORS configured for cross-origin requests
- JWT authentication with 6-hour expiration
- Trust proxy enabled for reverse proxy support
### 3. Error Handling
- `wrapAsync` wrapper for async route handlers
- Centralized error handler with status code mapping
- Generic error handler logs and returns safe message
### 4. Middleware Ordering
- File routes mounted before body parser (binary uploads)
- Rate limiters applied before routes
- Authentication applied per-route (not globally)
- Error handler at the end of middleware stack
### 5. API Documentation
- Swagger/OpenAPI 3.0 generated from `backend/src/openapi/document.ts`
- Available at `/api-docs` endpoint
- Security scheme documented (Bearer JWT)
- `backend/tests/openapi-document.test.ts` checks key paths, factory CRUD
coverage, internal `$ref` resolution, and Swagger UI handler smoke behavior
### 6. Logging
- Pino structured logging
- Request logger for all routes
- Error logging with context (URL, method)
---
## Configuration Precedence
```
1. Environment Variables (process.env.*)
↓ (fallback)
2. .env file (loaded by dotenv)
↓ (fallback)
3. Joi schema defaults
↓ (fallback)
4. Hardcoded defaults in config.ts
```
---
## Server Modes
| NODE_ENV | Port | Database | Swagger | Description |
|----------|------|----------|---------|-------------|
| `development` | 8080 | Local | localhost:8080 | Legacy local development |
| `dev_stage` | 3000 | Remote | localhost:3000 | Staging preview |
| `production` | 8080 | Remote | Disabled | Production deployment |
| `test` | 8080 | Test DB | Disabled | Automated testing |
**Standard VM note:** the VM PM2 setup runs the backend with
`NODE_ENV=dev_stage`, so the backend listens on port `3000`. The frontend runs
separately on port `3001`, and Apache proxies public traffic from port `80`.
Do not use `8080` as the VM backend health check unless the PM2 definition has
been changed. See [deployment-vm.md](../../../documentation/deployment-vm.md).