39948-vm/backend/docs/modules/utilities.md
2026-07-03 16:11:24 +02:00

27 KiB

Utilities Module

Overview

The Utilities module provides centralized helper functions, error handling, logging, environment validation, and i18n message management. Utilities are organized across several locations based on their domain.

Locations:

  • backend/src/utils/ - Core utilities (errors, logging, env validation, request context)
  • backend/src/helpers.ts - Request helpers (async wrapper, error handler, JWT)
  • backend/src/db/utils.ts - Database utilities
  • backend/src/services/notifications/ - i18n messages and legacy error classes

Architecture

backend/src/
├── utils/
│   ├── index.ts              # Re-exports
│   ├── errors.ts             # Error classes
│   ├── logger.ts             # Pino logging
│   ├── request-context.ts    # Request-scoped context storage
│   └── env-validation.ts     # Joi env schema
├── helpers.ts                # Request helpers
├── db/
│   └── utils.ts              # DB utilities
└── services/notifications/
    ├── list.ts               # i18n message catalog
    ├── helpers.ts            # Message formatting
    └── errors/
        ├── forbidden.ts      # ForbiddenError
        └── validation.ts     # ValidationError

Core Utilities (utils/)

Error Classes (utils/errors.js)

Modern error hierarchy for consistent HTTP error responses.

class AppError extends Error {
  constructor(message, statusCode = 500, details = null) {
    super(message);
    this.statusCode = statusCode;
    this.details = details;
    this.isOperational = true;  // Distinguishes from programming errors
    Error.captureStackTrace(this, this.constructor);
  }
}

Error Types:

Class Status Code Default Message Usage
AppError 500 (custom) Base class
NotFoundError 404 {resource} not found Missing resources
ValidationError 400 (custom) Invalid input
ForbiddenError 403 Access denied Permission denied
UnauthorizedError 401 Unauthorized Auth required
ConflictError 409 Resource conflict Duplicate resources

Usage:

const { NotFoundError, ValidationError, ForbiddenError } = require('./utils');

// In route handler
if (!user) {
  throw new NotFoundError('User');
}

const currentUser = getCurrentUser(req);
if (!currentUser) {
  throw new ForbiddenError();
}

if (errors.length > 0) {
  throw new ValidationError('Invalid data', errors);
}

Structured Logging (utils/logger.ts)

Pino-based logging with request correlation and environment-aware formatting. Logger initialization is a bootstrap exception where direct process.env access is allowed; runtime services should use config.ts instead.

const pino = require('pino');
const crypto = require('crypto');

const isDevelopment = process.env.NODE_ENV === 'development';

const logger = pino({
  level: process.env.LOG_LEVEL || 'info',
  transport: isDevelopment
    ? { target: 'pino-pretty', options: { colorize: true } }
    : undefined,
  base: {
    service: 'tour-builder-api',
    env: process.env.NODE_ENV || 'development',
  },
});

Log Levels:

  • fatal - Unrecoverable errors
  • error - Errors requiring attention
  • warn - Warning conditions (400-499 responses)
  • info - Normal operations (default)
  • debug - Detailed debugging
  • trace - Very detailed tracing

Process-Level Failure Logging:

registerProcessErrorHandlers() is called from src/index.ts during backend startup. It installs handlers for uncaughtException and unhandledRejection, logs them through Pino at fatal, normalizes non-Error rejection reasons into Error instances with cause, flushes Pino, and then exits with status 1. These failures happen outside the Express request lifecycle, so route error middleware cannot catch them.

registerProcessErrorHandlers();

process.on('unhandledRejection', (reason) => {
  logger.fatal(
    { err: normalizeLoggedError(reason) },
    'Unhandled promise rejection, shutting down process',
  );
  exitAfterLogging();
});

Request Logger Middleware:

function requestLogger(req, res, next) {
  // Generate or use existing request ID
  const requestId = req.headers['x-request-id'] || crypto.randomUUID();

  // Store child logger in request context
  setRequestLogger(req, logger.child({ requestId }));
  setRequestId(req, requestId);
  res.setHeader('X-Request-Id', requestId);

  const start = Date.now();
  res.on('finish', () => {
    const duration = Date.now() - start;
    const logData = {
      method: req.method,
      url: req.originalUrl || req.url,
      status: res.statusCode,
      duration,
      userAgent: req.headers['user-agent'],
    };

    // Log level based on status code
    if (res.statusCode >= 500) {
      getRequestLogger(req)?.error(logData, 'Request completed with server error');
    } else if (res.statusCode >= 400) {
      getRequestLogger(req)?.warn(logData, 'Request completed with client error');
    } else {
      getRequestLogger(req)?.info(logData, 'Request completed');
    }
  });

  next();
}

Log Output Examples:

Development (pino-pretty):

[12:34:56.789] INFO (tour-builder-api): Request completed
    requestId: "abc-123"
    method: "GET"
    url: "/api/users"
    status: 200
    duration: 45

Production (JSON):

{"level":30,"time":1711723456789,"service":"tour-builder-api","env":"production","requestId":"abc-123","method":"GET","url":"/api/users","status":200,"duration":45,"msg":"Request completed"}

Usage:

const { logger, requestLogger } = require('./utils/logger');

// App setup
app.use(requestLogger);

// Manual logging
logger.info({ userId: user.id }, 'User logged in');
logger.error({ err }, 'Database connection failed');

// Request-scoped logging (in routes)
getRequestLogger(req)?.info({ data }, 'Processing request');

Request Context (utils/request-context.ts)

Request-scoped data is stored outside the Express Request object through a WeakMap<Request, AppRequestContext>. Do not add global Express.Request augmentation for project fields such as currentUser, runtimeContext, log, or runtime-public flags. Middleware should write with setCurrentUser, setRuntimeContext, setRequestLogger, and related helpers; routes/services should read through getCurrentUser, getRuntimeContext, getRequestLogger, and getRouteServiceContext.

Environment Validation (utils/env-validation.ts)

Joi-based validation ensuring all required environment variables are present with correct types.

Schema Definition:

const Joi = require('joi');

const envSchema = Joi.object({
  // Server
  NODE_ENV: Joi.string()
    .valid('development', 'test', 'production', 'dev_stage')
    .default('development'),
  PORT: Joi.number().default(8080),

  // Database
  DB_HOST: Joi.string().default('localhost'),
  DB_PORT: Joi.number().default(5432),
  DB_NAME: Joi.string().default('db_tour_builder_platform'),
  DB_USER: Joi.string().default('postgres'),
  DB_PASS: Joi.string().allow('').default(''),

  // Authentication
  SECRET_KEY: Joi.string()
    .min(16)
    .default('88dbeaf8-e906-405e-9e41-c3baadeda5c6'),
  ADMIN_PASS: Joi.string().default('88dbeaf8'),
  USER_PASS: Joi.string().default('c3baadeda5c6'),
  ADMIN_EMAIL: Joi.string().email().default('admin@flatlogic.com'),

  // OAuth
  GOOGLE_CLIENT_ID: Joi.string().allow('').default(''),
  GOOGLE_CLIENT_SECRET: Joi.string().allow('').default(''),
  MS_CLIENT_ID: Joi.string().allow('').default(''),
  MS_CLIENT_SECRET: Joi.string().allow('').default(''),

  // AWS S3
  AWS_ACCESS_KEY_ID: Joi.string().allow('').default(''),
  AWS_SECRET_ACCESS_KEY: Joi.string().allow('').default(''),
  AWS_S3_BUCKET: Joi.string().allow('').default(''),
  AWS_S3_REGION: Joi.string().default('us-east-1'),
  AWS_S3_PREFIX: Joi.string().default('afeefb9d49f5b7977577876b99532ac7'),

  // Email
  EMAIL_USER: Joi.string().allow('').default(''),
  EMAIL_PASS: Joi.string().allow('').default(''),
  EMAIL_TLS_REJECT_UNAUTHORIZED: Joi.string()
    .valid('true', 'false')
    .default('true'),

  // External APIs
  PEXELS_KEY: Joi.string().allow('').default(''),

  // Logging
  LOG_LEVEL: Joi.string()
    .valid('fatal', 'error', 'warn', 'info', 'debug', 'trace')
    .default('info'),
}).unknown(true);  // Allow additional env vars

Validation Function:

function validateEnv() {
  const { error, value } = envSchema.validate(process.env, {
    abortEarly: false,     // Report all errors
    stripUnknown: false,   // Keep unknown vars
  });

  if (error) {
    const messages = error.details.map((d) => `  - ${d.message}`);
    logger.error({ errors: messages }, 'Environment validation failed');

    // Strict in production, lenient in development
    if (process.env.NODE_ENV === 'production') {
      process.exit(1);
    } else {
      logger.warn('Continuing with default values in non-production mode');
    }
  }

  return value;  // Returns validated/defaulted values
}

Environment Variable Categories:

Category Variables Required
Server NODE_ENV, PORT Defaults
Database DB_HOST, DB_PORT, DB_NAME, DB_USER, DB_PASS Defaults
Auth SECRET_KEY, ADMIN_PASS, USER_PASS, ADMIN_EMAIL Defaults
OAuth GOOGLE_CLIENT_ID, GOOGLE_CLIENT_SECRET, MS_CLIENT_ID, MS_CLIENT_SECRET Optional
AWS S3 AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_S3_BUCKET, AWS_S3_REGION, AWS_S3_PREFIX Optional
Email EMAIL_USER, EMAIL_PASS, EMAIL_TLS_REJECT_UNAUTHORIZED Optional
External APIs PEXELS_KEY Optional
Logging LOG_LEVEL Defaults

Index Re-exports (utils/index.js)

Convenient re-export of utilities:

module.exports = {
  ...require('./errors'),
  ...require('./logger'),
  envValidation: require('./env-validation'),
};

Exported:

  • AppError, NotFoundError, ValidationError, ForbiddenError, UnauthorizedError, ConflictError
  • logger, requestLogger, registerProcessErrorHandlers, exitAfterLogging, normalizeLoggedError
  • envValidation.validateEnv, envValidation.envSchema

Request Helpers (helpers.js)

Core helper class for Express route handling.

const jwt = require('jsonwebtoken');
const config = require('./config');

module.exports = class Helpers {
  // Wrap async route handlers to catch errors
  static wrapAsync(fn) {
    return function (req, res, next) {
      fn(req, res, next).catch(next);
    };
  }

  // Centralized error response handler
  static commonErrorHandler(error, req, res, _next) {
    const statusCode = error.code || error.status;

    if ([400, 401, 403, 404, 409, 422].includes(statusCode)) {
      return res.status(statusCode).send(error.message);
    }

    console.error(error);
    return res.status(500).send('Internal server error');
  }

  // Generate JWT token
  static jwtSign(data) {
    return jwt.sign(data, config.secret_key, { expiresIn: '6h' });
  }

  // Validate UUID v4 format
  static isUuidV4(value) {
    return /^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i.test(
      value,
    );
  }
};

Functions:

Function Purpose Usage
wrapAsync(fn) Wraps async handlers to propagate errors All async route handlers
commonErrorHandler(err, req, res, next) Standardizes error responses Route error middleware
jwtSign(data) Creates JWT with 6h expiry Auth service
isUuidV4(value) Validates UUID v4 format Route parameter validation

Request Validation

Request validation is centralized in:

  • src/middlewares/validate-request.ts - Joi middleware for params, query, and body
  • src/validators/request-schemas.ts - shared schemas for CRUD, auth, users, projects, tour pages, publish, and file upload endpoints

New external routes must validate all incoming params, query, and body before calling services. Factory CRUD routes use default schemas automatically and can override them through the validation option.

import { validateRequest } from '../middlewares/validate-request.ts';
import { publish as publishSchemas } from '../validators/request-schemas.ts';

router.post(
  '/save-to-stage',
  validateRequest(publishSchemas.saveToStage),
  wrapAsync(async (req, res) => {
    const result = await PublishService.saveToStage(
      req.body.projectId,
      getCurrentUser(req),
    );
    res.status(200).json(result);
  }),
);

Request validation errors return JSON:

{
  "error": "Invalid request",
  "details": [
    {
      "path": "projectId",
      "message": "\"projectId\" must be a valid GUID",
      "type": "string.guid"
    }
  ]
}

Service/domain ValidationError responses keep the legacy plain-text format unless they are raised by request validation middleware.

Usage Pattern:

const { wrapAsync, commonErrorHandler, isUuidV4 } = require('../helpers');

// Async route handler
router.get('/users/:id', wrapAsync(async (req, res) => {
  if (!isUuidV4(req.params.id)) {
    return res.status(400).send('Invalid ID format');
  }

  const user = await UserService.findOne(req.params.id);
  res.json(user);
}));

// Register error handler at end of router
router.use('/', commonErrorHandler);

Database Utilities (db/utils.js)

Utilities specific to Sequelize database operations, including clean UUID validation functions.

const validator = require('validator');
const { v4: uuidv4 } = require('uuid');
const Sequelize = require('./models').Sequelize;

module.exports = class Utils {
  // Check if value is a valid UUID
  static isValidUuid(value) {
    return Boolean(value && validator.isUUID(String(value)));
  }

  // Generate a new UUID v4
  static generateUuid() {
    return uuidv4();
  }

  // Filter array to only valid UUIDs
  static filterValidUuids(values) {
    return values.filter((v) => this.isValidUuid(v));
  }

  // Case-insensitive LIKE query
  static ilike(model, column, value) {
    return Sequelize.where(
      Sequelize.fn('lower', Sequelize.col(`${model}.${column}`)),
      { [Sequelize.Op.like]: `%${value}%`.toLowerCase() },
    );
  }
};

Functions:

Function Purpose Returns
isValidUuid(value) Check if value is a valid UUID boolean
generateUuid() Generate a new UUID v4 string
filterValidUuids(values) Filter array to only valid UUIDs string[]
ilike(model, column, value) Case-insensitive LIKE search Sequelize where clause

UUID Validation Behavior:

  • Invalid single ID filter (?id=xxx) → returns { rows: [], count: 0 } immediately
  • Invalid UUIDs in relation filters (?project=uuid|name) → filtered out for ID search, kept for text search
  • Invalid UUID field filter (?projectId=xxx) → returns { rows: [], count: 0 } immediately

Usage in DB API:

const Utils = require('../utils');

class GenericDBApi {
  async findAll(filter = {}, options = {}) {
    // Single ID validation
    if (filter.id) {
      if (!Utils.isValidUuid(filter.id)) {
        return { rows: [], count: 0 };
      }
      where.id = filter.id;
    }

    // Relation filter with mixed UUID/text search
    for (const rel of this.RELATION_FILTERS) {
      if (filter[rel.filterKey]) {
        const searchTerms = filter[rel.filterKey].split('|');
        const validUuids = Utils.filterValidUuids(searchTerms);

        // UUID search: only valid UUIDs
        // Text search: all terms
      }
    }
  }

  async findAllAutocomplete({ query, limit, offset }) {
    const orConditions = [
      Utils.ilike(this.TABLE_NAME, this.AUTOCOMPLETE_FIELD, query),
    ];

    // Only add UUID search if query is a valid UUID
    if (Utils.isValidUuid(query)) {
      orConditions.unshift({ id: query });
    }

    // ...
  }
}

Notifications Module (services/notifications/)

i18n message management and legacy error classes.

Message Catalog (list.js)

Centralized message definitions with placeholders:

const errors = {
  app: {
    title: 'Tour Builder Platform',
  },

  auth: {
    userDisabled: 'Your account is disabled',
    forbidden: 'Forbidden',
    unauthorized: 'Unauthorized',
    userNotFound: `Sorry, we don't recognize your credentials`,
    wrongPassword: `Sorry, we don't recognize your credentials`,
    weakPassword: 'This password is too weak',
    emailAlreadyInUse: 'Email is already in use',
    invalidEmail: 'Please provide a valid email',
    passwordReset: {
      invalidToken: 'Password reset link is invalid or has expired',
      error: `Email not recognized`,
    },
    passwordUpdate: {
      samePassword: `You can't use the same password. Please create new password`,
    },
    userNotVerified: `Sorry, your email has not been verified yet`,
    emailAddressVerificationEmail: {
      invalidToken: 'Email verification link is invalid or has expired',
      error: `Email not recognized`,
    },
  },

  iam: {
    errors: {
      userAlreadyExists: 'User with this email already exists',
      userNotFound: 'User not found',
      disablingHimself: `You can't disable yourself`,
      revokingOwnPermission: `You can't revoke your own owner permission`,
      deletingHimself: `You can't delete yourself`,
      emailRequired: 'Email is required',
    },
  },

  importer: {
    errors: {
      invalidFileEmpty: 'The file is empty',
      invalidFileExcel: 'Only excel (.xlsx) files are allowed',
      invalidFileUpload: 'Invalid file. Make sure you are using the last version of the template.',
      importHashRequired: 'Import hash is required',
      importHashExistent: 'Data has already been imported',
      userEmailMissing: 'Some items in the CSV do not have an email',
    },
  },

  errors: {
    forbidden: { message: 'Forbidden' },
    validation: { message: 'An error occurred' },
    searchQueryRequired: { message: 'Search query is required' },
  },

  emails: {
    invitation: {
      subject: `You've been invited to {0}`,
      body: `<p>Hello,</p><p>You've been invited to {0}...</p>`,
    },
    emailAddressVerification: {
      subject: `Verify your email for {0}`,
      body: `<p>Hello,</p><p>Follow this link to verify...</p>`,
    },
    passwordReset: {
      subject: `Reset your password for {0}`,
      body: `<p>Hello,</p><p>Follow this link to reset...</p>`,
    },
  },
};

Message Categories:

Category Purpose Examples
app Application metadata app.title
auth Authentication errors auth.userDisabled, auth.wrongPassword
iam User management errors iam.errors.userAlreadyExists
importer Import/export errors importer.errors.invalidFileEmpty
errors Generic errors errors.forbidden.message
emails Email templates emails.invitation.subject

Message Helpers (helpers.js)

Functions for message formatting and lookup:

const _get = require('lodash/get');
const errors = require('./list');

// Format message with placeholder substitution
function format(message, args) {
  if (!message) return null;
  return message.replace(/{(\d+)}/g, function (match, number) {
    return typeof args[number] != 'undefined' ? args[number] : match;
  });
}

// Check if key exists in catalog
const isNotification = (key) => {
  const message = _get(errors, key);
  return !!message;
};

// Get formatted message by key
const getNotification = (key, ...args) => {
  const message = _get(errors, key);
  if (!message) return key;
  return format(message, args);
};

Usage:

const { getNotification, isNotification } = require('./helpers');

// Lookup message
getNotification('auth.userDisabled');
// → 'Your account is disabled'

// Format with placeholders
getNotification('emails.invitation.subject', 'Tour Builder');
// → "You've been invited to Tour Builder"

// Check existence
isNotification('auth.userDisabled');  // → true
isNotification('unknown.key');        // → false

Legacy Error Classes (errors/)

i18n-aware error classes (legacy pattern, prefer utils/errors.js):

ForbiddenError:

const { getNotification, isNotification } = require('../helpers');

module.exports = class ForbiddenError extends Error {
  constructor(messageCode) {
    let message;

    if (messageCode && isNotification(messageCode)) {
      message = getNotification(messageCode);
    }

    message = message || getNotification('errors.forbidden.message');

    super(message);
    this.code = 403;
  }
};

ValidationError:

module.exports = class ValidationError extends Error {
  constructor(messageCode) {
    let message;

    if (messageCode && isNotification(messageCode)) {
      message = getNotification(messageCode);
    }

    message = message || getNotification('errors.validation.message');

    super(message);
    this.code = 400;
  }
};

Usage:

const ForbiddenError = require('./services/notifications/errors/forbidden');
const ValidationError = require('./services/notifications/errors/validation');

// With i18n key
throw new ForbiddenError('auth.forbidden');
throw new ValidationError('iam.errors.emailRequired');

// With default message
throw new ForbiddenError();  // → 'Forbidden'
throw new ValidationError(); // → 'An error occurred'

Error Handling Flow

┌─────────────────────────────────────────────────────────────┐
│                     Route Handler                           │
│                                                             │
│   router.get('/users/:id', wrapAsync(async (req, res) => {  │
│     throw new NotFoundError('User');                        │
│   }));                                                      │
└─────────────────────────────────────────────────────────────┘
                              │
                    wrapAsync catches
                              │
                              ▼
┌─────────────────────────────────────────────────────────────┐
│                  commonErrorHandler                         │
│                                                             │
│   - Checks error.code or error.status                       │
│   - Known codes (400-422): returns error.message            │
│   - Unknown: returns 'Internal server error'                │
└─────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────┐
│                    HTTP Response                            │
│                                                             │
│   HTTP/1.1 404 Not Found                                    │
│   Content-Type: text/plain                                  │
│                                                             │
│   User not found                                            │
└─────────────────────────────────────────────────────────────┘

Usage Summary

// Modern error classes
const { NotFoundError, ValidationError, ForbiddenError } = require('./utils');

// Logging
const { logger, requestLogger } = require('./utils/logger');

// Request helpers
const { wrapAsync, commonErrorHandler, isUuidV4 } = require('./helpers');

// Database utilities
const Utils = require('./db/utils');

// i18n messages
const { getNotification } = require('./services/notifications/helpers');

Error Class Selection

Scenario Recommended Class
Resource not found NotFoundError from utils/errors.js
Invalid input ValidationError from utils/errors.js
Permission denied ForbiddenError from utils/errors.js
Auth required UnauthorizedError from utils/errors.js
Duplicate resource ConflictError from utils/errors.js
i18n error message Legacy classes from notifications/errors/

Environment Variables Reference

Variable Type Default Description
NODE_ENV string development development, test, production, dev_stage
PORT number 8080 Server port
DB_HOST string localhost PostgreSQL host
DB_PORT number 5432 PostgreSQL port
DB_NAME string db_tour_builder_platform Database name
DB_USER string postgres Database user
DB_PASS string `` Database password
SECRET_KEY string UUID JWT signing key (min 16 chars)
ADMIN_EMAIL email admin@flatlogic.com Admin account email
ADMIN_PASS string 88dbeaf8 Admin account password
USER_PASS string c3baadeda5c6 Default user password
GOOGLE_CLIENT_ID string `` Google OAuth client ID
GOOGLE_CLIENT_SECRET string `` Google OAuth client secret
MS_CLIENT_ID string `` Microsoft OAuth client ID
MS_CLIENT_SECRET string `` Microsoft OAuth client secret
AWS_ACCESS_KEY_ID string `` AWS access key
AWS_SECRET_ACCESS_KEY string `` AWS secret key
AWS_S3_BUCKET string `` S3 bucket name
AWS_S3_REGION string us-east-1 S3 region
AWS_S3_PREFIX string UUID S3 key prefix
EMAIL_USER string `` SMTP username
EMAIL_PASS string `` SMTP password
EMAIL_TLS_REJECT_UNAUTHORIZED string true TLS cert validation
PEXELS_KEY string `` Pexels API key
LOG_LEVEL string info Pino log level