39948-vm/backend/docs/modules/db-config.md

# DB Config Module

## Overview

The DB Config module manages database connection settings, environment validation, and database utilities. It provides environment-aware configuration for PostgreSQL connections via Sequelize ORM.

**Location:** `backend/src/db/`

**Key Files:**
- `db-config.ts` - Typed ESM database connection settings per environment
- `umzug.ts` - Typed Umzug runner for migrations, seeders, create/drop
- `utils.ts` - Database utility functions
- `sync.ts` - Development database sync script
- `reset.ts` - Database reset script

**Related Files:**
- `backend/src/config.ts` - Application configuration
- `backend/src/utils/env-validation.ts` - Environment variable validation

---

## Architecture

```
backend/
├── src/
│   ├── config.ts                     # Application config
│   ├── utils/
│   │   └── env-validation.ts         # Joi schema validation
│   └── db/
│       ├── db-config.ts              # Typed database connection config
│       ├── umzug.ts                  # Typed migration/seed runner
│       ├── utils.ts                  # DB utilities
│       ├── sync.ts                   # Dev sync script (21 LOC)
│       ├── reset.ts                  # Reset script (18 LOC)
│       ├── models/
│       │   ├── loader.ts             # Typed model loader + Sequelize init
│       │   └── index.ts              # ESM model registry entrypoint
│       ├── migrations/               # Schema migrations
│       └── seeders/                  # Data seeders
```

---

## Database Configuration

### db-config.ts

`db-config.ts` exports environment-specific database settings for Sequelize.
It uses the official Sequelize `Options` type plus project-specific
`DatabaseConfigMap`/`DatabaseEnvironmentConfig` reusable contracts from
`backend/src/types/db-config.ts`.

Database scripts use `backend/src/db/umzug.ts` directly.

`DB_PORT` is parsed to a number for the Sequelize `Options` contract. If the
env var is absent or invalid, the `port` property is omitted.

### Environment Comparison

| Setting | Production | Development | Dev Stage |
|---------|------------|-------------|-----------|
| **Dialect** | postgres | postgres | postgres |
| **Credentials** | Env vars | Hardcoded | Env vars |
| **Logging** | Disabled | Pino debug | Pino debug |
| **Host** | Env var | localhost | Env var |
| **Migration Storage** | SequelizeMeta | SequelizeMeta | SequelizeMeta |
| **Seeder Storage** | SequelizeData | SequelizeData | SequelizeData |

---

## Environment Variables

### Database Variables

| Variable | Required | Default | Description |
|----------|----------|---------|-------------|
| `NODE_ENV` | No | development | Environment selection |
| `DB_HOST` | Prod/Stage | localhost | Database host |
| `DB_PORT` | Prod/Stage | 5432 | Database port |
| `DB_NAME` | Prod/Stage | db_tour_builder_platform | Database name |
| `DB_USER` | Prod/Stage | postgres | Database username |
| `DB_PASS` | Prod/Stage | (empty) | Database password |

### Environment Validation

The `env-validation.ts` file uses Joi schema to validate all environment variables:

```javascript
const Joi = require('joi');

const envSchema = Joi.object({
  NODE_ENV: Joi.string()
    .valid('development', 'test', 'production', 'dev_stage')
    .default('development'),

  PORT: Joi.number().default(8080),

  DB_HOST: Joi.string().default('localhost'),
  DB_PORT: Joi.number().default(5432),
  DB_NAME: Joi.string().default('db_tour_builder_platform'),
  DB_USER: Joi.string().default('postgres'),
  DB_PASS: Joi.string().allow('').default(''),

  SECRET_KEY: Joi.string()
    .min(16)
    .default('88dbeaf8-e906-405e-9e41-c3baadeda5c6'),

  // ... more variables
}).unknown(true);
```

### Complete Environment Variable Schema

| Category | Variable | Validation | Default |
|----------|----------|------------|---------|
| **Server** | NODE_ENV | enum: development, test, production, dev_stage | development |
| | PORT | number | 8080 |
| **Database** | DB_HOST | string | localhost |
| | DB_PORT | number | 5432 |
| | DB_NAME | string | db_tour_builder_platform |
| | DB_USER | string | postgres |
| | DB_PASS | string (allow empty) | (empty) |
| **Auth** | SECRET_KEY | string, min 16 chars | (default UUID) |
| | ADMIN_PASS | string | 88dbeaf8 |
| | USER_PASS | string | c3baadeda5c6 |
| | ADMIN_EMAIL | email | admin@flatlogic.com |
| **OAuth** | GOOGLE_CLIENT_ID | string (allow empty) | (empty) |
| | GOOGLE_CLIENT_SECRET | string (allow empty) | (empty) |
| | MS_CLIENT_ID | string (allow empty) | (empty) |
| | MS_CLIENT_SECRET | string (allow empty) | (empty) |
| **AWS S3** | AWS_ACCESS_KEY_ID | string (allow empty) | (empty) |
| | AWS_SECRET_ACCESS_KEY | string (allow empty) | (empty) |
| | AWS_S3_BUCKET | string (allow empty) | (empty) |
| | AWS_S3_REGION | string | us-east-1 |
| | AWS_S3_PREFIX | string | (default hash) |
| | AWS_S3_CONNECTION_TIMEOUT | number (ms) | 5000 |
| | AWS_S3_REQUEST_TIMEOUT | number (ms) | 30000 |
| | AWS_S3_MAX_ATTEMPTS | number | 3 |
| | AWS_S3_MAX_SOCKETS | number | 50 |
| | AWS_S3_KEEP_ALIVE | boolean string | true |
| | AWS_S3_PRESIGN_EXPIRY | number (seconds) | 3600 |
| **Email** | EMAIL_USER | string (allow empty) | (empty) |
| | EMAIL_PASS | string (allow empty) | (empty) |
| | EMAIL_TLS_REJECT_UNAUTHORIZED | enum: true, false | true |
| **External APIs** | PEXELS_KEY | string (allow empty) | (empty) |
| **Logging** | LOG_LEVEL | enum: fatal, error, warn, info, debug, trace | info |

### Validation Behavior

```javascript
function validateEnv() {
  const { error, value } = envSchema.validate(process.env, {
    abortEarly: false,      // Report all errors, not just first
    stripUnknown: false,    // Keep unknown env vars
  });

  if (error) {
    const messages = error.details.map((d) => `  - ${d.message}`);
    logger.error({ errors: messages }, 'Environment validation failed');

    if (process.env.NODE_ENV === 'production') {
      process.exit(1);  // Fatal in production
    } else {
      logger.warn('Continuing with default values in non-production mode');
    }
  }

  return value;
}
```

---

## Umzug Configuration

The database command entrypoint is `backend/src/db/umzug.ts`.

### Runtime Paths

| Setting | Path |
|---------|------|
| Config | `src/db/db-config.ts` |
| Runner | `src/db/umzug.ts` |
| Models | `src/db/models/` |
| Seeders | `src/db/seeders/` |
| Migrations | `src/db/migrations/` |

---

## Database Initialization

### Model Loader

`src/db/models/loader.ts` explicitly imports model factories and builds the
Sequelize registry. `src/db/models/index.ts` is the ESM entrypoint. The typed
service-specific overload facade lives in `src/types/db-models.ts`.

### Initialization Flow

```
Server Start
    │
    ▼
config.ts loads
    │
    ├─ dotenv.config()  ─── Load .env file
    │
    ├─ validateEnv()    ─── Validate with Joi schema
    │                        │
    │                        ├─ Production: Exit on error
    │                        └─ Development: Warn, use defaults
    │
    └─ Export config object
    │
    ▼
db/models/index.ts loads
    │
    ├─ Read NODE_ENV
    │
    ├─ Load db-config.ts[env]
    │
    ├─ Create Sequelize instance
    │
    ├─ Initialize explicitly imported model factories
    │
    ├─ Call model.associate() for relationships
    │
    └─ Export db object { sequelize, Sequelize, models... }
```

---

## Database Utilities

### utils.ts

Database utility helpers are TypeScript/ESM source. Runtime DB access goes
through `src/db/models/index.ts`, whose typed facade is provided by
`src/types/db-models.ts`.

### Usage Examples

```typescript
import Utils from '../db/utils.ts';

// UUID validation
Utils.isValidUuid('550e8400-e29b-41d4-a716-446655440000');  // true
Utils.isValidUuid('not-a-uuid');                             // false

// Generate new UUID
const id = Utils.generateUuid();  // Returns new UUID v4

// Filter array to valid UUIDs only
const validIds = Utils.filterValidUuids(['uuid1', 'invalid', 'uuid2']);

// Case-insensitive search
const where = {
  [Op.or]: [
    Utils.ilike('users', 'firstName', searchTerm),
    Utils.ilike('users', 'lastName', searchTerm),
    Utils.ilike('users', 'email', searchTerm),
  ]
};
```

---

## Database Scripts

### sync.ts (Development Only)

Synchronizes models to database schema using Sequelize's `alter` mode.

```javascript
async function syncDatabase() {
  // Safety check - never run in production
  if (process.env.NODE_ENV === 'production') {
    console.error('ERROR: sync.ts should not be run in production. Use migrations instead.');
    process.exit(1);
  }

  try {
    console.log('Syncing database...');
    await db.sequelize.sync({ alter: true });
    console.log('Database synced successfully!');
    process.exit(0);
  } catch (error) {
    console.error('Error syncing database:', error);
    process.exit(1);
  }
}
```

**Usage:**
```bash
node src/db/sync.ts
```

**Sync Modes:**

| Mode | Description | Use Case |
|------|-------------|----------|
| `{ force: true }` | Drop and recreate all tables | Fresh start |
| `{ alter: true }` | Modify tables to match models | Development |
| (none) | Create only missing tables | Safe default |

### reset.ts

Forcefully resets database and runs seeders.

```javascript
db.sequelize
  .sync({ force: true })
  .then(() => {
    execSync('sequelize db:seed:all');
    console.log('OK');
    process.exit();
  })
  .catch((error) => {
    console.error(error);
    process.exit(1);
  });
```

**Usage:**
```bash
node src/db/reset.ts
```

**Warning:** This drops ALL tables and recreates them. Data loss!

---

## Application Configuration

### config.ts

Centralized application configuration with environment-aware settings.

```javascript
const env = validateEnv();

const config = {
  gcloud: {
    bucket: 'fldemo-files',
    hash: 'afeefb9d49f5b7977577876b99532ac7',
    projectId: env.GC_PROJECT_ID,
    clientEmail: env.GC_CLIENT_EMAIL,
    privateKey: env.GC_PRIVATE_KEY,
  },
  fileStorage: {
    provider: env.FILE_STORAGE_PROVIDER,
  },
  s3: {
    bucket: env.AWS_S3_BUCKET,
    region: env.AWS_S3_REGION,
    accessKeyId: env.AWS_ACCESS_KEY_ID,
    secretAccessKey: env.AWS_SECRET_ACCESS_KEY,
    prefix: env.AWS_S3_PREFIX,
    connectionTimeout: env.AWS_S3_CONNECTION_TIMEOUT,
    requestTimeout: env.AWS_S3_REQUEST_TIMEOUT,
    maxAttempts: env.AWS_S3_MAX_ATTEMPTS,
    maxSockets: env.AWS_S3_MAX_SOCKETS,
    keepAlive: env.AWS_S3_KEEP_ALIVE !== 'false',
    presignExpirySeconds: env.AWS_S3_PRESIGN_EXPIRY,
  },
  resilience: {
    ffmpeg: { reverseTimeoutMs, ffprobeTimeoutMs, breaker },
    fileStorage: { breaker },
  },
  server: {
    env: env.NODE_ENV,
    port: serverPort,
    swaggerServerUrl,
  },
  secret_key: env.SECRET_KEY,
  admin_pass: env.ADMIN_PASS,
  user_pass: env.USER_PASS,
  admin_email: env.ADMIN_EMAIL,

  // Roles
  roles: {
    admin: 'Administrator',
    user: 'Analytics Viewer',
  },
  apiUrl: `${host}${port ? `:${port}` : ''}/api`,
  swaggerUrl: `${swaggerUI}${swaggerPort}`,
  uiUrl: `${hostUI}${portUI ? `:${portUI}` : ''}/#`,
  backUrl: `${hostUI}${portUI ? `:${portUI}` : ''}`,
};
```

---

## Configuration Categories

### Storage Configuration

| Provider | Variables | Purpose |
|----------|-----------|---------|
| **AWS S3** | AWS_S3_BUCKET, AWS_S3_REGION, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY | File storage |
| **GCloud** | (hardcoded bucket) | Legacy support |
| **Local** | uploadDir (os.tmpdir()) | Development fallback |

### S3 Performance Tuning

| Variable | Default | Description |
|----------|---------|-------------|
| AWS_S3_CONNECTION_TIMEOUT | 5000ms | TCP connection timeout |
| AWS_S3_REQUEST_TIMEOUT | 30000ms | Total request timeout |
| AWS_S3_MAX_ATTEMPTS | 3 | Retry attempts on failure |
| AWS_S3_MAX_SOCKETS | 50 | Connection pool size |
| AWS_S3_KEEP_ALIVE | true | Reuse TCP connections |
| AWS_S3_PRESIGN_EXPIRY | 3600s | Presigned URL validity (1 hour) |

### Security Configuration

| Setting | Value | Purpose |
|---------|-------|---------|
| bcrypt.saltRounds | 12 | Password hashing strength |
| SECRET_KEY | 16+ char string | JWT signing key |
| EMAIL_TLS_REJECT_UNAUTHORIZED | true/false | TLS certificate validation |

### URL Configuration

| URL | Development | Production |
|-----|-------------|------------|
| apiUrl | http://localhost:3000/api | (remote)/api |
| swaggerUrl | http://localhost:3000 | (remote) |
| uiUrl | http://localhost:3001/# | (remote)/# |
| backUrl | http://localhost:3001 | (remote) |

---

## Running Commands

### Development
```bash
cd backend
npm run start-dev
```

`start-dev` runs the standard `start` pipeline (`db:migrate`, `db:seed`,
`watch`) with `LOG_PRETTY=true`. `src/load-env.ts` loads `backend/.env` before
DB config selection; when `NODE_ENV` is absent it defaults to `dev_stage`, which
matches the standard VM backend flow and listens on port `3000`.

### VM / Dev Stage
```bash
cd backend
npm run start
```

In the standard VM PM2 setup, `NODE_ENV=dev_stage` makes the backend listen on
port `3000`; the frontend runs separately on `3001` and Apache proxies public
traffic from port `80`. See [`deployment-vm.md`](../../../documentation/deployment-vm.md).

Do not add `NODE_ENV=production` only to make local startup work. The current
flow loads `.env` through `src/load-env.ts` and defaults missing `NODE_ENV` to
`dev_stage`, matching the VM backend process.

---

## Best Practices

### 1. Never Commit Secrets
```bash
# .env file should be in .gitignore
# Use environment variables in deployment
```

### 2. Use Migrations in Production
```javascript
// Never use sync.ts or reset.ts in production
if (process.env.NODE_ENV === 'production') {
  process.exit(1);
}
```

### 3. Validate Environment Early
```javascript
// config.ts loads validation at import time
import { validateEnv } from './utils/env-validation.ts';
validateEnv();  // Called before app starts
```

### 4. Environment-Specific Logging
```javascript
// Production: logging disabled (performance)
// Development/dev_stage: SQL logs use structured Pino debug entries
logging: process.env.NODE_ENV === 'production'
  ? false
  : (sql) => logger.debug({ sql }, 'Sequelize query'),
```

---

## Related Documentation

- [DB Models](./db-models.md) - Sequelize model definitions
- [DB API](./db-api.md) - Database access layer
- [DB Migrations](./db-migrations.md) - Schema evolution
- [DB Seeders](./db-seeders.md) - Initial data population
- [Core Module](./core.md) - Application entry point