# Utilities Module ## Overview The Utilities module provides centralized helper functions, error handling, logging, environment validation, and i18n message management. Utilities are organized across several locations based on their domain. **Locations:** - `backend/src/utils/` - Core utilities (errors, logging, env validation, request context) - `backend/src/helpers.ts` - Request helpers (async wrapper, error handler, JWT) - `backend/src/db/utils.ts` - Database utilities - `backend/src/services/notifications/` - i18n messages and legacy error classes --- ## Architecture ``` backend/src/ ├── utils/ │ ├── index.ts # Re-exports │ ├── errors.ts # Error classes │ ├── logger.ts # Pino logging │ ├── request-context.ts # Request-scoped context storage │ └── env-validation.ts # Joi env schema ├── helpers.ts # Request helpers ├── db/ │ └── utils.ts # DB utilities └── services/notifications/ ├── list.ts # i18n message catalog ├── helpers.ts # Message formatting └── errors/ ├── forbidden.ts # ForbiddenError └── validation.ts # ValidationError ``` --- ## Core Utilities (`utils/`) ### Error Classes (`utils/errors.js`) Modern error hierarchy for consistent HTTP error responses. ```javascript class AppError extends Error { constructor(message, statusCode = 500, details = null) { super(message); this.statusCode = statusCode; this.details = details; this.isOperational = true; // Distinguishes from programming errors Error.captureStackTrace(this, this.constructor); } } ``` **Error Types:** | Class | Status Code | Default Message | Usage | |-------|-------------|-----------------|-------| | `AppError` | 500 | (custom) | Base class | | `NotFoundError` | 404 | `{resource} not found` | Missing resources | | `ValidationError` | 400 | (custom) | Invalid input | | `ForbiddenError` | 403 | `Access denied` | Permission denied | | `UnauthorizedError` | 401 | `Unauthorized` | Auth required | | `ConflictError` | 409 | `Resource conflict` | Duplicate resources | **Usage:** ```javascript const { NotFoundError, ValidationError, ForbiddenError } = require('./utils'); // In route handler if (!user) { throw new NotFoundError('User'); } const currentUser = getCurrentUser(req); if (!currentUser) { throw new ForbiddenError(); } if (errors.length > 0) { throw new ValidationError('Invalid data', errors); } ``` --- ### Structured Logging (`utils/logger.ts`) Pino-based logging with request correlation and environment-aware formatting. Logger initialization is a bootstrap exception where direct `process.env` access is allowed; runtime services should use `config.ts` instead. ```javascript const pino = require('pino'); const crypto = require('crypto'); const isDevelopment = process.env.NODE_ENV === 'development'; const logger = pino({ level: process.env.LOG_LEVEL || 'info', transport: isDevelopment ? { target: 'pino-pretty', options: { colorize: true } } : undefined, base: { service: 'tour-builder-api', env: process.env.NODE_ENV || 'development', }, }); ``` **Log Levels:** - `fatal` - Unrecoverable errors - `error` - Errors requiring attention - `warn` - Warning conditions (400-499 responses) - `info` - Normal operations (default) - `debug` - Detailed debugging - `trace` - Very detailed tracing **Process-Level Failure Logging:** `registerProcessErrorHandlers()` is called from `src/index.ts` during backend startup. It installs handlers for `uncaughtException` and `unhandledRejection`, logs them through Pino at `fatal`, normalizes non-`Error` rejection reasons into `Error` instances with `cause`, flushes Pino, and then exits with status `1`. These failures happen outside the Express request lifecycle, so route error middleware cannot catch them. ```typescript registerProcessErrorHandlers(); process.on('unhandledRejection', (reason) => { logger.fatal( { err: normalizeLoggedError(reason) }, 'Unhandled promise rejection, shutting down process', ); exitAfterLogging(); }); ``` **Request Logger Middleware:** ```javascript function requestLogger(req, res, next) { // Generate or use existing request ID const requestId = req.headers['x-request-id'] || crypto.randomUUID(); // Store child logger in request context setRequestLogger(req, logger.child({ requestId })); setRequestId(req, requestId); res.setHeader('X-Request-Id', requestId); const start = Date.now(); res.on('finish', () => { const duration = Date.now() - start; const logData = { method: req.method, url: req.originalUrl || req.url, status: res.statusCode, duration, userAgent: req.headers['user-agent'], }; // Log level based on status code if (res.statusCode >= 500) { getRequestLogger(req)?.error(logData, 'Request completed with server error'); } else if (res.statusCode >= 400) { getRequestLogger(req)?.warn(logData, 'Request completed with client error'); } else { getRequestLogger(req)?.info(logData, 'Request completed'); } }); next(); } ``` **Log Output Examples:** Development (pino-pretty): ``` [12:34:56.789] INFO (tour-builder-api): Request completed requestId: "abc-123" method: "GET" url: "/api/users" status: 200 duration: 45 ``` Production (JSON): ```json {"level":30,"time":1711723456789,"service":"tour-builder-api","env":"production","requestId":"abc-123","method":"GET","url":"/api/users","status":200,"duration":45,"msg":"Request completed"} ``` **Usage:** ```javascript const { logger, requestLogger } = require('./utils/logger'); // App setup app.use(requestLogger); // Manual logging logger.info({ userId: user.id }, 'User logged in'); logger.error({ err }, 'Database connection failed'); // Request-scoped logging (in routes) getRequestLogger(req)?.info({ data }, 'Processing request'); ``` --- ### Request Context (`utils/request-context.ts`) Request-scoped data is stored outside the Express `Request` object through a `WeakMap`. Do not add global `Express.Request` augmentation for project fields such as `currentUser`, `runtimeContext`, `log`, or runtime-public flags. Middleware should write with `setCurrentUser`, `setRuntimeContext`, `setRequestLogger`, and related helpers; routes/services should read through `getCurrentUser`, `getRuntimeContext`, `getRequestLogger`, and `getRouteServiceContext`. ### Environment Validation (`utils/env-validation.ts`) Joi-based validation ensuring all required environment variables are present with correct types. **Schema Definition:** ```javascript const Joi = require('joi'); const envSchema = Joi.object({ // Server NODE_ENV: Joi.string() .valid('development', 'test', 'production', 'dev_stage') .default('development'), PORT: Joi.number().default(8080), // Database DB_HOST: Joi.string().default('localhost'), DB_PORT: Joi.number().default(5432), DB_NAME: Joi.string().default('db_tour_builder_platform'), DB_USER: Joi.string().default('postgres'), DB_PASS: Joi.string().allow('').default(''), // Authentication SECRET_KEY: Joi.string() .min(16) .default('88dbeaf8-e906-405e-9e41-c3baadeda5c6'), ADMIN_PASS: Joi.string().default('88dbeaf8'), USER_PASS: Joi.string().default('c3baadeda5c6'), ADMIN_EMAIL: Joi.string().email().default('admin@flatlogic.com'), // OAuth GOOGLE_CLIENT_ID: Joi.string().allow('').default(''), GOOGLE_CLIENT_SECRET: Joi.string().allow('').default(''), MS_CLIENT_ID: Joi.string().allow('').default(''), MS_CLIENT_SECRET: Joi.string().allow('').default(''), // AWS S3 AWS_ACCESS_KEY_ID: Joi.string().allow('').default(''), AWS_SECRET_ACCESS_KEY: Joi.string().allow('').default(''), AWS_S3_BUCKET: Joi.string().allow('').default(''), AWS_S3_REGION: Joi.string().default('us-east-1'), AWS_S3_PREFIX: Joi.string().default('afeefb9d49f5b7977577876b99532ac7'), // Email EMAIL_USER: Joi.string().allow('').default(''), EMAIL_PASS: Joi.string().allow('').default(''), EMAIL_TLS_REJECT_UNAUTHORIZED: Joi.string() .valid('true', 'false') .default('true'), // External APIs PEXELS_KEY: Joi.string().allow('').default(''), // Logging LOG_LEVEL: Joi.string() .valid('fatal', 'error', 'warn', 'info', 'debug', 'trace') .default('info'), }).unknown(true); // Allow additional env vars ``` **Validation Function:** ```javascript function validateEnv() { const { error, value } = envSchema.validate(process.env, { abortEarly: false, // Report all errors stripUnknown: false, // Keep unknown vars }); if (error) { const messages = error.details.map((d) => ` - ${d.message}`); logger.error({ errors: messages }, 'Environment validation failed'); // Strict in production, lenient in development if (process.env.NODE_ENV === 'production') { process.exit(1); } else { logger.warn('Continuing with default values in non-production mode'); } } return value; // Returns validated/defaulted values } ``` **Environment Variable Categories:** | Category | Variables | Required | |----------|-----------|----------| | **Server** | `NODE_ENV`, `PORT` | Defaults | | **Database** | `DB_HOST`, `DB_PORT`, `DB_NAME`, `DB_USER`, `DB_PASS` | Defaults | | **Auth** | `SECRET_KEY`, `ADMIN_PASS`, `USER_PASS`, `ADMIN_EMAIL` | Defaults | | **OAuth** | `GOOGLE_CLIENT_ID`, `GOOGLE_CLIENT_SECRET`, `MS_CLIENT_ID`, `MS_CLIENT_SECRET` | Optional | | **AWS S3** | `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_S3_BUCKET`, `AWS_S3_REGION`, `AWS_S3_PREFIX` | Optional | | **Email** | `EMAIL_USER`, `EMAIL_PASS`, `EMAIL_TLS_REJECT_UNAUTHORIZED` | Optional | | **External APIs** | `PEXELS_KEY` | Optional | | **Logging** | `LOG_LEVEL` | Defaults | --- ### Index Re-exports (`utils/index.js`) Convenient re-export of utilities: ```javascript module.exports = { ...require('./errors'), ...require('./logger'), envValidation: require('./env-validation'), }; ``` **Exported:** - `AppError`, `NotFoundError`, `ValidationError`, `ForbiddenError`, `UnauthorizedError`, `ConflictError` - `logger`, `requestLogger`, `registerProcessErrorHandlers`, `exitAfterLogging`, `normalizeLoggedError` - `envValidation.validateEnv`, `envValidation.envSchema` --- ## Request Helpers (`helpers.js`) Core helper class for Express route handling. ```javascript const jwt = require('jsonwebtoken'); const config = require('./config'); module.exports = class Helpers { // Wrap async route handlers to catch errors static wrapAsync(fn) { return function (req, res, next) { fn(req, res, next).catch(next); }; } // Centralized error response handler static commonErrorHandler(error, req, res, _next) { const statusCode = error.code || error.status; if ([400, 401, 403, 404, 409, 422].includes(statusCode)) { return res.status(statusCode).send(error.message); } console.error(error); return res.status(500).send('Internal server error'); } // Generate JWT token static jwtSign(data) { return jwt.sign(data, config.secret_key, { expiresIn: '6h' }); } // Validate UUID v4 format static isUuidV4(value) { return /^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i.test( value, ); } }; ``` **Functions:** | Function | Purpose | Usage | |----------|---------|-------| | `wrapAsync(fn)` | Wraps async handlers to propagate errors | All async route handlers | | `commonErrorHandler(err, req, res, next)` | Standardizes error responses | Route error middleware | | `jwtSign(data)` | Creates JWT with 6h expiry | Auth service | | `isUuidV4(value)` | Validates UUID v4 format | Route parameter validation | ## Request Validation Request validation is centralized in: - `src/middlewares/validate-request.ts` - Joi middleware for `params`, `query`, and `body` - `src/validators/request-schemas.ts` - shared schemas for CRUD, auth, users, projects, tour pages, publish, and file upload endpoints New external routes must validate all incoming `params`, `query`, and `body` before calling services. Factory CRUD routes use default schemas automatically and can override them through the `validation` option. ```typescript import { validateRequest } from '../middlewares/validate-request.ts'; import { publish as publishSchemas } from '../validators/request-schemas.ts'; router.post( '/save-to-stage', validateRequest(publishSchemas.saveToStage), wrapAsync(async (req, res) => { const result = await PublishService.saveToStage( req.body.projectId, getCurrentUser(req), ); res.status(200).json(result); }), ); ``` Request validation errors return JSON: ```json { "error": "Invalid request", "details": [ { "path": "projectId", "message": "\"projectId\" must be a valid GUID", "type": "string.guid" } ] } ``` Service/domain `ValidationError` responses keep the legacy plain-text format unless they are raised by request validation middleware. **Usage Pattern:** ```javascript const { wrapAsync, commonErrorHandler, isUuidV4 } = require('../helpers'); // Async route handler router.get('/users/:id', wrapAsync(async (req, res) => { if (!isUuidV4(req.params.id)) { return res.status(400).send('Invalid ID format'); } const user = await UserService.findOne(req.params.id); res.json(user); })); // Register error handler at end of router router.use('/', commonErrorHandler); ``` --- ## Database Utilities (`db/utils.js`) Utilities specific to Sequelize database operations, including clean UUID validation functions. ```javascript const validator = require('validator'); const { v4: uuidv4 } = require('uuid'); const Sequelize = require('./models').Sequelize; module.exports = class Utils { // Check if value is a valid UUID static isValidUuid(value) { return Boolean(value && validator.isUUID(String(value))); } // Generate a new UUID v4 static generateUuid() { return uuidv4(); } // Filter array to only valid UUIDs static filterValidUuids(values) { return values.filter((v) => this.isValidUuid(v)); } // Case-insensitive LIKE query static ilike(model, column, value) { return Sequelize.where( Sequelize.fn('lower', Sequelize.col(`${model}.${column}`)), { [Sequelize.Op.like]: `%${value}%`.toLowerCase() }, ); } }; ``` **Functions:** | Function | Purpose | Returns | |----------|---------|---------| | `isValidUuid(value)` | Check if value is a valid UUID | `boolean` | | `generateUuid()` | Generate a new UUID v4 | `string` | | `filterValidUuids(values)` | Filter array to only valid UUIDs | `string[]` | | `ilike(model, column, value)` | Case-insensitive LIKE search | Sequelize where clause | **UUID Validation Behavior:** - Invalid single ID filter (`?id=xxx`) → returns `{ rows: [], count: 0 }` immediately - Invalid UUIDs in relation filters (`?project=uuid|name`) → filtered out for ID search, kept for text search - Invalid UUID field filter (`?projectId=xxx`) → returns `{ rows: [], count: 0 }` immediately **Usage in DB API:** ```javascript const Utils = require('../utils'); class GenericDBApi { async findAll(filter = {}, options = {}) { // Single ID validation if (filter.id) { if (!Utils.isValidUuid(filter.id)) { return { rows: [], count: 0 }; } where.id = filter.id; } // Relation filter with mixed UUID/text search for (const rel of this.RELATION_FILTERS) { if (filter[rel.filterKey]) { const searchTerms = filter[rel.filterKey].split('|'); const validUuids = Utils.filterValidUuids(searchTerms); // UUID search: only valid UUIDs // Text search: all terms } } } async findAllAutocomplete({ query, limit, offset }) { const orConditions = [ Utils.ilike(this.TABLE_NAME, this.AUTOCOMPLETE_FIELD, query), ]; // Only add UUID search if query is a valid UUID if (Utils.isValidUuid(query)) { orConditions.unshift({ id: query }); } // ... } } ``` --- ## Notifications Module (`services/notifications/`) i18n message management and legacy error classes. ### Message Catalog (`list.js`) Centralized message definitions with placeholders: ```javascript const errors = { app: { title: 'Tour Builder Platform', }, auth: { userDisabled: 'Your account is disabled', forbidden: 'Forbidden', unauthorized: 'Unauthorized', userNotFound: `Sorry, we don't recognize your credentials`, wrongPassword: `Sorry, we don't recognize your credentials`, weakPassword: 'This password is too weak', emailAlreadyInUse: 'Email is already in use', invalidEmail: 'Please provide a valid email', passwordReset: { invalidToken: 'Password reset link is invalid or has expired', error: `Email not recognized`, }, passwordUpdate: { samePassword: `You can't use the same password. Please create new password`, }, userNotVerified: `Sorry, your email has not been verified yet`, emailAddressVerificationEmail: { invalidToken: 'Email verification link is invalid or has expired', error: `Email not recognized`, }, }, iam: { errors: { userAlreadyExists: 'User with this email already exists', userNotFound: 'User not found', disablingHimself: `You can't disable yourself`, revokingOwnPermission: `You can't revoke your own owner permission`, deletingHimself: `You can't delete yourself`, emailRequired: 'Email is required', }, }, importer: { errors: { invalidFileEmpty: 'The file is empty', invalidFileExcel: 'Only excel (.xlsx) files are allowed', invalidFileUpload: 'Invalid file. Make sure you are using the last version of the template.', importHashRequired: 'Import hash is required', importHashExistent: 'Data has already been imported', userEmailMissing: 'Some items in the CSV do not have an email', }, }, errors: { forbidden: { message: 'Forbidden' }, validation: { message: 'An error occurred' }, searchQueryRequired: { message: 'Search query is required' }, }, emails: { invitation: { subject: `You've been invited to {0}`, body: `

Hello,

You've been invited to {0}...

`, }, emailAddressVerification: { subject: `Verify your email for {0}`, body: `

Hello,

Follow this link to verify...

`, }, passwordReset: { subject: `Reset your password for {0}`, body: `

Hello,

Follow this link to reset...

`, }, }, }; ``` **Message Categories:** | Category | Purpose | Examples | |----------|---------|----------| | `app` | Application metadata | `app.title` | | `auth` | Authentication errors | `auth.userDisabled`, `auth.wrongPassword` | | `iam` | User management errors | `iam.errors.userAlreadyExists` | | `importer` | Import/export errors | `importer.errors.invalidFileEmpty` | | `errors` | Generic errors | `errors.forbidden.message` | | `emails` | Email templates | `emails.invitation.subject` | --- ### Message Helpers (`helpers.js`) Functions for message formatting and lookup: ```javascript const _get = require('lodash/get'); const errors = require('./list'); // Format message with placeholder substitution function format(message, args) { if (!message) return null; return message.replace(/{(\d+)}/g, function (match, number) { return typeof args[number] != 'undefined' ? args[number] : match; }); } // Check if key exists in catalog const isNotification = (key) => { const message = _get(errors, key); return !!message; }; // Get formatted message by key const getNotification = (key, ...args) => { const message = _get(errors, key); if (!message) return key; return format(message, args); }; ``` **Usage:** ```javascript const { getNotification, isNotification } = require('./helpers'); // Lookup message getNotification('auth.userDisabled'); // → 'Your account is disabled' // Format with placeholders getNotification('emails.invitation.subject', 'Tour Builder'); // → "You've been invited to Tour Builder" // Check existence isNotification('auth.userDisabled'); // → true isNotification('unknown.key'); // → false ``` --- ### Legacy Error Classes (`errors/`) i18n-aware error classes (legacy pattern, prefer `utils/errors.js`): **ForbiddenError:** ```javascript const { getNotification, isNotification } = require('../helpers'); module.exports = class ForbiddenError extends Error { constructor(messageCode) { let message; if (messageCode && isNotification(messageCode)) { message = getNotification(messageCode); } message = message || getNotification('errors.forbidden.message'); super(message); this.code = 403; } }; ``` **ValidationError:** ```javascript module.exports = class ValidationError extends Error { constructor(messageCode) { let message; if (messageCode && isNotification(messageCode)) { message = getNotification(messageCode); } message = message || getNotification('errors.validation.message'); super(message); this.code = 400; } }; ``` **Usage:** ```javascript const ForbiddenError = require('./services/notifications/errors/forbidden'); const ValidationError = require('./services/notifications/errors/validation'); // With i18n key throw new ForbiddenError('auth.forbidden'); throw new ValidationError('iam.errors.emailRequired'); // With default message throw new ForbiddenError(); // → 'Forbidden' throw new ValidationError(); // → 'An error occurred' ``` --- ## Error Handling Flow ``` ┌─────────────────────────────────────────────────────────────┐ │ Route Handler │ │ │ │ router.get('/users/:id', wrapAsync(async (req, res) => { │ │ throw new NotFoundError('User'); │ │ })); │ └─────────────────────────────────────────────────────────────┘ │ wrapAsync catches │ ▼ ┌─────────────────────────────────────────────────────────────┐ │ commonErrorHandler │ │ │ │ - Checks error.code or error.status │ │ - Known codes (400-422): returns error.message │ │ - Unknown: returns 'Internal server error' │ └─────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────┐ │ HTTP Response │ │ │ │ HTTP/1.1 404 Not Found │ │ Content-Type: text/plain │ │ │ │ User not found │ └─────────────────────────────────────────────────────────────┘ ``` --- ## Usage Summary ### Recommended Imports ```javascript // Modern error classes const { NotFoundError, ValidationError, ForbiddenError } = require('./utils'); // Logging const { logger, requestLogger } = require('./utils/logger'); // Request helpers const { wrapAsync, commonErrorHandler, isUuidV4 } = require('./helpers'); // Database utilities const Utils = require('./db/utils'); // i18n messages const { getNotification } = require('./services/notifications/helpers'); ``` ### Error Class Selection | Scenario | Recommended Class | |----------|-------------------| | Resource not found | `NotFoundError` from `utils/errors.js` | | Invalid input | `ValidationError` from `utils/errors.js` | | Permission denied | `ForbiddenError` from `utils/errors.js` | | Auth required | `UnauthorizedError` from `utils/errors.js` | | Duplicate resource | `ConflictError` from `utils/errors.js` | | i18n error message | Legacy classes from `notifications/errors/` | --- ## Environment Variables Reference | Variable | Type | Default | Description | |----------|------|---------|-------------| | `NODE_ENV` | string | `development` | `development`, `test`, `production`, `dev_stage` | | `PORT` | number | `8080` | Server port | | `DB_HOST` | string | `localhost` | PostgreSQL host | | `DB_PORT` | number | `5432` | PostgreSQL port | | `DB_NAME` | string | `db_tour_builder_platform` | Database name | | `DB_USER` | string | `postgres` | Database user | | `DB_PASS` | string | `` | Database password | | `SECRET_KEY` | string | UUID | JWT signing key (min 16 chars) | | `ADMIN_EMAIL` | email | `admin@flatlogic.com` | Admin account email | | `ADMIN_PASS` | string | `88dbeaf8` | Admin account password | | `USER_PASS` | string | `c3baadeda5c6` | Default user password | | `GOOGLE_CLIENT_ID` | string | `` | Google OAuth client ID | | `GOOGLE_CLIENT_SECRET` | string | `` | Google OAuth client secret | | `MS_CLIENT_ID` | string | `` | Microsoft OAuth client ID | | `MS_CLIENT_SECRET` | string | `` | Microsoft OAuth client secret | | `AWS_ACCESS_KEY_ID` | string | `` | AWS access key | | `AWS_SECRET_ACCESS_KEY` | string | `` | AWS secret key | | `AWS_S3_BUCKET` | string | `` | S3 bucket name | | `AWS_S3_REGION` | string | `us-east-1` | S3 region | | `AWS_S3_PREFIX` | string | UUID | S3 key prefix | | `EMAIL_USER` | string | `` | SMTP username | | `EMAIL_PASS` | string | `` | SMTP password | | `EMAIL_TLS_REJECT_UNAUTHORIZED` | string | `true` | TLS cert validation | | `PEXELS_KEY` | string | `` | Pexels API key | | `LOG_LEVEL` | string | `info` | Pino log level | --- ## Related Documentation - [Backend Architecture](../backend-architecture.md) - Overall backend structure - [Auth Module](./auth.md) - Authentication using JWT helpers - [Middleware Module](./middleware.md) - Request middleware - [Routes Module](./routes.md) - Route handlers using wrapAsync - [Services Module](./services.md) - Business logic services - [DB Config Module](./db-config.md) - Database configuration