Dmitri 9f111d6226 added documentation

2026-07-03 16:11:24 +02:00

14 KiB

Raw Blame History

DB Config Module

Overview

The DB Config module manages database connection settings, environment validation, and database utilities. It provides environment-aware configuration for PostgreSQL connections via Sequelize ORM.

Location: backend/src/db/

Key Files:

db-config.ts - Typed ESM database connection settings per environment
umzug.ts - Typed Umzug runner for migrations, seeders, create/drop
utils.ts - Database utility functions
sync.ts - Development database sync script
reset.ts - Database reset script

Related Files:

backend/src/config.ts - Application configuration
backend/src/utils/env-validation.ts - Environment variable validation

Architecture

backend/
├── src/
│   ├── config.ts                     # Application config
│   ├── utils/
│   │   └── env-validation.ts         # Joi schema validation
│   └── db/
│       ├── db-config.ts              # Typed database connection config
│       ├── umzug.ts                  # Typed migration/seed runner
│       ├── utils.ts                  # DB utilities
│       ├── sync.ts                   # Dev sync script (21 LOC)
│       ├── reset.ts                  # Reset script (18 LOC)
│       ├── models/
│       │   ├── loader.ts             # Typed model loader + Sequelize init
│       │   └── index.ts              # ESM model registry entrypoint
│       ├── migrations/               # Schema migrations
│       └── seeders/                  # Data seeders

Database Configuration

db-config.ts

db-config.ts exports environment-specific database settings for Sequelize. It uses the official Sequelize Options type plus project-specific DatabaseConfigMap/DatabaseEnvironmentConfig reusable contracts from backend/src/types/db-config.ts.

Database scripts use backend/src/db/umzug.ts directly.

DB_PORT is parsed to a number for the Sequelize Options contract. If the env var is absent or invalid, the port property is omitted.

Environment Comparison

Setting	Production	Development	Dev Stage
Dialect	postgres	postgres	postgres
Credentials	Env vars	Hardcoded	Env vars
Logging	Disabled	Pino debug	Pino debug
Host	Env var	localhost	Env var
Migration Storage	SequelizeMeta	SequelizeMeta	SequelizeMeta
Seeder Storage	SequelizeData	SequelizeData	SequelizeData

Environment Variables

Database Variables

Variable	Required	Default	Description
`NODE_ENV`	No	development	Environment selection
`DB_HOST`	Prod/Stage	localhost	Database host
`DB_PORT`	Prod/Stage	5432	Database port
`DB_NAME`	Prod/Stage	db_tour_builder_platform	Database name
`DB_USER`	Prod/Stage	postgres	Database username
`DB_PASS`	Prod/Stage	(empty)	Database password

Environment Validation

The env-validation.ts file uses Joi schema to validate all environment variables:

const Joi = require('joi');

const envSchema = Joi.object({
  NODE_ENV: Joi.string()
    .valid('development', 'test', 'production', 'dev_stage')
    .default('development'),

  PORT: Joi.number().default(8080),

  DB_HOST: Joi.string().default('localhost'),
  DB_PORT: Joi.number().default(5432),
  DB_NAME: Joi.string().default('db_tour_builder_platform'),
  DB_USER: Joi.string().default('postgres'),
  DB_PASS: Joi.string().allow('').default(''),

  SECRET_KEY: Joi.string()
    .min(16)
    .default('88dbeaf8-e906-405e-9e41-c3baadeda5c6'),

  // ... more variables
}).unknown(true);

Complete Environment Variable Schema

Category	Variable	Validation	Default
Server	NODE_ENV	enum: development, test, production, dev_stage	development
	PORT	number	8080
Database	DB_HOST	string	localhost
	DB_PORT	number	5432
	DB_NAME	string	db_tour_builder_platform
	DB_USER	string	postgres
	DB_PASS	string (allow empty)	(empty)
Auth	SECRET_KEY	string, min 16 chars	(default UUID)
	ADMIN_PASS	string	88dbeaf8
	USER_PASS	string	c3baadeda5c6
	ADMIN_EMAIL	email	admin@flatlogic.com
OAuth	GOOGLE_CLIENT_ID	string (allow empty)	(empty)
	GOOGLE_CLIENT_SECRET	string (allow empty)	(empty)
	MS_CLIENT_ID	string (allow empty)	(empty)
	MS_CLIENT_SECRET	string (allow empty)	(empty)
AWS S3	AWS_ACCESS_KEY_ID	string (allow empty)	(empty)
	AWS_SECRET_ACCESS_KEY	string (allow empty)	(empty)
	AWS_S3_BUCKET	string (allow empty)	(empty)
	AWS_S3_REGION	string	us-east-1
	AWS_S3_PREFIX	string	(default hash)
	AWS_S3_CONNECTION_TIMEOUT	number (ms)	5000
	AWS_S3_REQUEST_TIMEOUT	number (ms)	30000
	AWS_S3_MAX_ATTEMPTS	number	3
	AWS_S3_MAX_SOCKETS	number	50
	AWS_S3_KEEP_ALIVE	boolean string	true
	AWS_S3_PRESIGN_EXPIRY	number (seconds)	3600
Email	EMAIL_USER	string (allow empty)	(empty)
	EMAIL_PASS	string (allow empty)	(empty)
	EMAIL_TLS_REJECT_UNAUTHORIZED	enum: true, false	true
External APIs	PEXELS_KEY	string (allow empty)	(empty)
Logging	LOG_LEVEL	enum: fatal, error, warn, info, debug, trace	info

Validation Behavior

function validateEnv() {
  const { error, value } = envSchema.validate(process.env, {
    abortEarly: false,      // Report all errors, not just first
    stripUnknown: false,    // Keep unknown env vars
  });

  if (error) {
    const messages = error.details.map((d) => `  - ${d.message}`);
    logger.error({ errors: messages }, 'Environment validation failed');

    if (process.env.NODE_ENV === 'production') {
      process.exit(1);  // Fatal in production
    } else {
      logger.warn('Continuing with default values in non-production mode');
    }
  }

  return value;
}

Umzug Configuration

The database command entrypoint is backend/src/db/umzug.ts.

Runtime Paths

Setting	Path
Config	`src/db/db-config.ts`
Runner	`src/db/umzug.ts`
Models	`src/db/models/`
Seeders	`src/db/seeders/`
Migrations	`src/db/migrations/`

Database Initialization

Model Loader

src/db/models/loader.ts explicitly imports model factories and builds the Sequelize registry. src/db/models/index.ts is the ESM entrypoint. The typed service-specific overload facade lives in src/types/db-models.ts.

Initialization Flow

Server Start
    │
    ▼
config.ts loads
    │
    ├─ dotenv.config()  ─── Load .env file
    │
    ├─ validateEnv()    ─── Validate with Joi schema
    │                        │
    │                        ├─ Production: Exit on error
    │                        └─ Development: Warn, use defaults
    │
    └─ Export config object
    │
    ▼
db/models/index.ts loads
    │
    ├─ Read NODE_ENV
    │
    ├─ Load db-config.ts[env]
    │
    ├─ Create Sequelize instance
    │
    ├─ Initialize explicitly imported model factories
    │
    ├─ Call model.associate() for relationships
    │
    └─ Export db object { sequelize, Sequelize, models... }

Database Utilities

utils.ts

Database utility helpers are TypeScript/ESM source. Runtime DB access goes through src/db/models/index.ts, whose typed facade is provided by src/types/db-models.ts.

Usage Examples

import Utils from '../db/utils.ts';

// UUID validation
Utils.isValidUuid('550e8400-e29b-41d4-a716-446655440000');  // true
Utils.isValidUuid('not-a-uuid');                             // false

// Generate new UUID
const id = Utils.generateUuid();  // Returns new UUID v4

// Filter array to valid UUIDs only
const validIds = Utils.filterValidUuids(['uuid1', 'invalid', 'uuid2']);

// Case-insensitive search
const where = {
  [Op.or]: [
    Utils.ilike('users', 'firstName', searchTerm),
    Utils.ilike('users', 'lastName', searchTerm),
    Utils.ilike('users', 'email', searchTerm),
  ]
};

Database Scripts

sync.ts (Development Only)

Synchronizes models to database schema using Sequelize's alter mode.

async function syncDatabase() {
  // Safety check - never run in production
  if (process.env.NODE_ENV === 'production') {
    console.error('ERROR: sync.ts should not be run in production. Use migrations instead.');
    process.exit(1);
  }

  try {
    console.log('Syncing database...');
    await db.sequelize.sync({ alter: true });
    console.log('Database synced successfully!');
    process.exit(0);
  } catch (error) {
    console.error('Error syncing database:', error);
    process.exit(1);
  }
}

Usage:

node src/db/sync.ts

Sync Modes:

Mode	Description	Use Case
`{ force: true }`	Drop and recreate all tables	Fresh start
`{ alter: true }`	Modify tables to match models	Development
(none)	Create only missing tables	Safe default

reset.ts

Forcefully resets database and runs seeders.

db.sequelize
  .sync({ force: true })
  .then(() => {
    execSync('sequelize db:seed:all');
    console.log('OK');
    process.exit();
  })
  .catch((error) => {
    console.error(error);
    process.exit(1);
  });

Usage:

node src/db/reset.ts

Warning: This drops ALL tables and recreates them. Data loss!

Application Configuration

config.ts

Centralized application configuration with environment-aware settings.

const env = validateEnv();

const config = {
  gcloud: {
    bucket: 'fldemo-files',
    hash: 'afeefb9d49f5b7977577876b99532ac7',
    projectId: env.GC_PROJECT_ID,
    clientEmail: env.GC_CLIENT_EMAIL,
    privateKey: env.GC_PRIVATE_KEY,
  },
  fileStorage: {
    provider: env.FILE_STORAGE_PROVIDER,
  },
  s3: {
    bucket: env.AWS_S3_BUCKET,
    region: env.AWS_S3_REGION,
    accessKeyId: env.AWS_ACCESS_KEY_ID,
    secretAccessKey: env.AWS_SECRET_ACCESS_KEY,
    prefix: env.AWS_S3_PREFIX,
    connectionTimeout: env.AWS_S3_CONNECTION_TIMEOUT,
    requestTimeout: env.AWS_S3_REQUEST_TIMEOUT,
    maxAttempts: env.AWS_S3_MAX_ATTEMPTS,
    maxSockets: env.AWS_S3_MAX_SOCKETS,
    keepAlive: env.AWS_S3_KEEP_ALIVE !== 'false',
    presignExpirySeconds: env.AWS_S3_PRESIGN_EXPIRY,
  },
  resilience: {
    ffmpeg: { reverseTimeoutMs, ffprobeTimeoutMs, breaker },
    fileStorage: { breaker },
  },
  server: {
    env: env.NODE_ENV,
    port: serverPort,
    swaggerServerUrl,
  },
  secret_key: env.SECRET_KEY,
  admin_pass: env.ADMIN_PASS,
  user_pass: env.USER_PASS,
  admin_email: env.ADMIN_EMAIL,

  // Roles
  roles: {
    admin: 'Administrator',
    user: 'Analytics Viewer',
  },
  apiUrl: `${host}${port ? `:${port}` : ''}/api`,
  swaggerUrl: `${swaggerUI}${swaggerPort}`,
  uiUrl: `${hostUI}${portUI ? `:${portUI}` : ''}/#`,
  backUrl: `${hostUI}${portUI ? `:${portUI}` : ''}`,
};

Configuration Categories

Storage Configuration

Provider	Variables	Purpose
AWS S3	AWS_S3_BUCKET, AWS_S3_REGION, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY	File storage
GCloud	(hardcoded bucket)	Legacy support
Local	uploadDir (os.tmpdir())	Development fallback

S3 Performance Tuning

Variable	Default	Description
AWS_S3_CONNECTION_TIMEOUT	5000ms	TCP connection timeout
AWS_S3_REQUEST_TIMEOUT	30000ms	Total request timeout
AWS_S3_MAX_ATTEMPTS	3	Retry attempts on failure
AWS_S3_MAX_SOCKETS	50	Connection pool size
AWS_S3_KEEP_ALIVE	true	Reuse TCP connections
AWS_S3_PRESIGN_EXPIRY	3600s	Presigned URL validity (1 hour)

Security Configuration

Setting	Value	Purpose
bcrypt.saltRounds	12	Password hashing strength
SECRET_KEY	16+ char string	JWT signing key
EMAIL_TLS_REJECT_UNAUTHORIZED	true/false	TLS certificate validation

URL Configuration

URL	Development	Production
apiUrl	http://localhost:3000/api	(remote)/api
swaggerUrl	http://localhost:3000	(remote)
uiUrl	http://localhost:3001/#	(remote)/#
backUrl	http://localhost:3001	(remote)

Running Commands

Development

cd backend
npm run start-dev

start-dev runs the standard start pipeline (db:migrate, db:seed, watch) with LOG_PRETTY=true. src/load-env.ts loads backend/.env before DB config selection; when NODE_ENV is absent it defaults to dev_stage, which matches the standard VM backend flow and listens on port 3000.

VM / Dev Stage

cd backend
npm run start

In the standard VM PM2 setup, NODE_ENV=dev_stage makes the backend listen on port 3000; the frontend runs separately on 3001 and Apache proxies public traffic from port 80. See deployment-vm.md.

Do not add NODE_ENV=production only to make local startup work. The current flow loads .env through src/load-env.ts and defaults missing NODE_ENV to dev_stage, matching the VM backend process.

Best Practices

1. Never Commit Secrets

# .env file should be in .gitignore
# Use environment variables in deployment

2. Use Migrations in Production

// Never use sync.ts or reset.ts in production
if (process.env.NODE_ENV === 'production') {
  process.exit(1);
}

3. Validate Environment Early

// config.ts loads validation at import time
import { validateEnv } from './utils/env-validation.ts';
validateEnv();  // Called before app starts

4. Environment-Specific Logging

// Production: logging disabled (performance)
// Development/dev_stage: SQL logs use structured Pino debug entries
logging: process.env.NODE_ENV === 'production'
  ? false
  : (sql) => logger.debug({ sql }, 'Sequelize query'),

DB Models - Sequelize model definitions
DB API - Database access layer
DB Migrations - Schema evolution
DB Seeders - Initial data population
Core Module - Application entry point

14 KiB Raw Blame History