Dmitri 9f111d6226 added documentation

2026-07-03 16:11:24 +02:00

46 KiB

Raw Blame History

Backend Services Module

Overview

The Services module implements the business logic layer of the backend application. Services sit between routes (controllers) and the database API layer, encapsulating complex operations, transaction management, and cross-cutting concerns.

Location: backend/src/services/

Total Files: 37 (24 root services + 13 subdirectory files)

Architecture Diagram

┌─────────────────────────────────────────────────────────────────────┐
│                           Routes Layer                              │
│                    (HTTP Request Handling)                          │
└─────────────────────────────────────────────────────────────────────┘
                                 │
                                 ▼
┌─────────────────────────────────────────────────────────────────────┐
│                         Services Layer                              │
│  ┌───────────────────────────────────────────────────────────────┐  │
│  │                    Service Categories                         │  │
│  │  ┌─────────────┐ ┌─────────────┐ ┌─────────────┐              │  │
│  │  │   Factory   │ │   Custom    │ │ Specialized │              │  │
│  │  │  Services   │ │  Services   │ │  Modules    │              │  │
│  │  │ (9 files)   │ │ (12 files)  │ │ (3 dirs)    │              │  │
│  │  └─────────────┘ └─────────────┘ └─────────────┘              │  │
│  └───────────────────────────────────────────────────────────────┘  │
└─────────────────────────────────────────────────────────────────────┘
                                 │
                                 ▼
┌─────────────────────────────────────────────────────────────────────┐
│                       Database API Layer                            │
│                 (Sequelize ORM Operations)                          │
└─────────────────────────────────────────────────────────────────────┘

Service Categories

1. Factory-Generated Services (Simple CRUD)

Generated using createEntityService() from factories/service.factory.ts. These provide standardized CRUD operations with transaction handling.

Service	File	Entity	LOC
tour_pages	`tour_pages.ts`	Tour Pages (includes reverse video generation)	~1,300
permissions	`permissions.ts`	Permissions	6
asset_variants	`asset_variants.ts`	Asset Variants	6
presigned_url_requests	`presigned_url_requests.ts`	Presigned URL Requests	6
publish_events	`publish_events.ts`	Publish Events	6
pwa_caches	`pwa_caches.ts`	PWA Caches	6
access_logs	`access_logs.ts`	Access Logs	6
element_type_defaults	`element_type_defaults.ts`	Element Type Defaults	6
project_memberships	`project_memberships.ts`	Project Memberships	6
global_transition_defaults	`global_transition_defaults.ts`	Global transition defaults	6

Example - Factory Service:

// permissions.ts
import PermissionsDBApi from '../db/api/permissions.ts';
import { createEntityService } from '../factories/service.factory.ts';

export default createEntityService(PermissionsDBApi, {
  entityName: 'permissions',
});

2. Custom Services (Business Logic)

Services with domain-specific business logic beyond simple CRUD.

Service	File	Purpose	LOC
assets	`assets.ts`	Asset management, MIME validation, embed URL validation, stored media metadata probing	~300
auth	`auth.ts`	Authentication, password reset, email verification	~210
users	`users.ts`	User management, invitation emails, Public viewer grants	~350
projects	`projects.ts`	Project cloning, slug generation, slug uniqueness validation	~680
roles	`roles.ts`	Role management, permission assignment, CSV import, Public-role hardening	~170
file	`file.ts`	Multi-provider file storage, downloadToBuffer, uploadBuffer, S3/GCloud circuit breaker for processing paths	~1,600
publish	`publish.ts`	Dev→Stage→Production publishing	~400
search	`search.ts`	Global full-text search	178
pwa_manifest	`pwa_manifest.js`	PWA offline manifest generation	315
project_audio_tracks	`project_audio_tracks.ts`	Audio track management	117
project_transition_settings	`project_transition_settings.ts`	Environment-aware transition settings	209
project_element_defaults	`project_element_defaults.ts`	Element defaults with reset/diff	34
global_ui_control_defaults	`global_ui_control_defaults.ts`	Global defaults CRUD service for system controls	6
project_ui_control_settings	`project_ui_control_settings.ts`	Transactional find/upsert/delete for project UI-control overrides	51
videoProcessing	`videoProcessing.ts`	FFmpeg video reversal for transition videos with single-worker queue, `-threads 1`, hard timeout, metadata logs, and circuit breaker	~240

3. Specialized Module Directories

services/
├── file/                    # Storage providers (Strategy Pattern)
│   ├── BaseStorageProvider.ts     # Abstract interface
│   ├── S3StorageProvider.ts       # AWS S3 implementation
│   ├── LocalStorageProvider.ts    # Local filesystem
│   ├── UploadSessionManager.ts    # Chunked uploads
│   └── index.js                   # Provider factory (27 LOC)
│
├── email/                   # Email sending
│   ├── index.js                   # EmailSender class (44 LOC)
│   └── list/
│       ├── passwordReset.ts       # Password reset email
│       ├── addressVerification.ts # Email verification
│       └── invitation.ts          # User invitation email
│
└── notifications/           # Error handling & i18n
    ├── helpers.js                 # getNotification helper (30 LOC)
    ├── list.js                    # Error message catalog (100 LOC)
    └── errors/
        ├── validation.js          # ValidationError class (16 LOC)
        └── forbidden.js           # ForbiddenError class (16 LOC)

Service Factory Pattern

`factories/service.factory.ts`

Generates standardized service classes with transaction handling.

function createEntityService(DBApi, options = {}) {
  const entityName = options.entityName || 'Entity';

  return class GenericService {
    // Create with transaction
    static async create({ data, currentUser, transaction, runtimeContext }) {
      const transaction = await db.sequelize.transaction();
      try {
        const record = await DBApi.create({ data, currentUser, transaction, runtimeContext });
        await transaction.commit();
        return record;
      } catch (error) {
        await transaction.rollback();
        throw error;
      }
    }

    // Bulk import from CSV
    static async bulkImport(req, res) { ... }

    // Update with existence check
    static async update({ id, data, currentUser, transaction, runtimeContext }) { ... }

    // Delete multiple by IDs
    static async deleteByIds({ ids, currentUser, transaction, runtimeContext }) { ... }

    // Soft delete single
    static async remove({ id, currentUser, transaction, runtimeContext }) { ... }
  };
}

Generated Methods:

Method	Signature	Description
`create`	`(data, currentUser) → record`	Create with transaction
`bulkImport`	`(req, res) → void`	CSV import with validation
`update`	`(data, id, currentUser) → record`	Update with existence check
`deleteByIds`	`(ids, currentUser) → void`	Bulk soft delete
`remove`	`(id, currentUser) → void`	Single soft delete

Core Services Detail

Tour Pages Service (`tour_pages.ts`)

Extends the factory-generated CRUD service with page-specific operations.

Reorder operation:

TourPagesService.reorder(data, currentUser) accepts projectId, environment, and orderedPageIds.
Reordering is allowed only for environment='dev'; this preserves the publishing model where stage and production are derived environments.
The service loads all pages for the project/dev environment inside a transaction, verifies that the ordered list contains every page exactly once, and then updates sort_order sequentially.
The operation updates only sort_order; it does not change page content, slugs, backgrounds, ui_schema_json, navigation, transitions, or media.
Stage receives the new order after Save to Stage; production receives it after Publish.

Auth Service (`auth.ts`)

Handles authentication, password management, and email verification.

class Auth {
  // User login with bcrypt verification
  static async signin(email, password) → JWT

  // Send email verification link
  static async sendEmailAddressVerificationEmail(email, host)

  // Send password reset or invitation email
  static async sendPasswordResetEmail(email, type, host)

  // Verify email from token
  static async verifyEmail(token, options) → boolean

  // Update password (requires current password)
  static async passwordUpdate(currentPassword, newPassword, options)

  // Reset password from token
  static async passwordReset(token, password, options)

  // Update user profile
  static async updateProfile(data, currentUser)
}

Security Features:

bcrypt password hashing (config.bcrypt.saltRounds)
JWT token generation via helpers.jwtSign()
Email verification required for login
Password reset tokens with expiration

Assets Service (`assets.ts`)

Extended factory service with strict reusable TypeScript contracts, MIME type validation for asset uploads, embed URL validation, and stored audio/video metadata probing.

class AssetsService extends BaseService {
  // Create asset with MIME type validation
  static async create({ data, currentUser, transaction, runtimeContext })

  // Update asset with MIME type validation
  static async update({ id, data, currentUser, transaction, runtimeContext })
}

MIME Type Validation:

Asset Type	Valid MIME Prefixes	Description
`image`	`image/`	JPEG, PNG, GIF, WebP, SVG, etc.
`video`	`video/`	MP4, WebM, MOV, etc.
`audio`	`audio/`	MP3, WAV, OGG, etc.
`embed`	n/a	MIME validation skipped; HTTPS embed URL domain is validated

Validation Rules:

asset_type and mime_type must be consistent
If asset_type is image, mime_type must start with image/
If asset_type is video, mime_type must start with video/
If asset_type is audio, mime_type must start with audio/
Asset types not in the validation list (e.g., file) skip validation
Missing mime_type is allowed (browser may not always send it)

Error Response:

throw new ValidationError(
  `Invalid file type for ${assetType}. Expected ${patterns.description}, got "${mimeType}"`
);

File Service (`file.js`)

Unified file storage using Strategy Pattern for multiple backends. Features comprehensive error handling, AbortController support for client disconnect handling, path validation for security, and structured Pino logging.

// Provider auto-detection
const getFileStorageProvider = () → 's3' | 'gcloud' | 'local'

// Core operations (with AbortController support for S3)
const uploadFile = async (folder, req, res) → { url }
const downloadFile = async (req, res) → stream  // Aborts on client disconnect
const deleteFile = async (privateUrl, { throwOnError }) → { success, error? }

// Server-side file copy (S3 uses CopyObjectCommand, Local uses fs.copyFile)
const copyFile = async (sourceKey, destKey, options) → { url } | { key }
const copyFilesParallel = async (copies, options) → { succeeded, failed }

// Chunked upload session management
const initUploadSession = async (req, res) → { sessionId, totalChunks }
const getUploadSession = async (req, res) → { status, uploadedChunks }
const uploadChunk = async (req, res) → { chunkIndex, uploadedChunks }
const finalizeUploadSession = async (req, res) → { url, privateUrl }

// Presigned URL generation (S3 only, with path validation)
const generatePresignedUrls = async (urls) → { [url]: presignedUrl }

// Utilities (exported for route layer)
const isValidPath = (urlPath) → boolean  // Path traversal protection
const createErrorResponse = (message, code, details) → { message, code?, details? }
const getS3ErrorStatusCode = (error) → number  // HTTP status code mapping

Server-Side File Copy (S3 Native):

The copyFile() function uses provider-native copy operations for optimal performance:

Provider	Implementation	Performance
S3	`CopyObjectCommand` (server-side)	15x faster, zero memory
Local	`fs.promises.copyFile`	Kernel-level copy
GCloud	Download + Upload (fallback)	Legacy behavior

// Single file copy
const copyFile = async (sourceKey, destKey, { contentType }) → { url }

// Parallel batch copy with concurrency control
const copyFilesParallel = async (copies, { concurrency = 10, continueOnError = true })
  → { succeeded: [{ sourceKey, destKey }], failed: [{ sourceKey, error }] }

Benefits over download-then-upload:

15x faster: Server-side copy, no data through backend
Zero memory: No file buffering in Node.js
No timeouts: Works for large files (>100MB)
Reduced bandwidth: No double network transfer

Error Response Format:

// Standardized across all file endpoints
{
  message: 'Human-readable error message',
  code: 'ERROR_CODE',           // For programmatic handling
  details: { ... }               // Optional additional context
}

Request Cancellation: Downloads automatically abort S3 requests when client disconnects, preventing wasted bandwidth and server resources.

Provider Selection:

Priority	Provider	Detection
1	`config.fileStorage.provider`	Validated `FILE_STORAGE_PROVIDER` override
2	S3	`S3_BUCKET` + `S3_REGION` + credentials
3	GCloud	Validated `GC_PROJECT_ID` + `GC_CLIENT_EMAIL` + `GC_PRIVATE_KEY`
4	Local	Default fallback

External S3/GCloud operations used by processing paths are protected by the shared file-storage circuit breaker. For S3, only retryable SDK/network errors count toward opening the breaker; expected 4xx errors do not.

Chunked Upload Flow:

1. POST /api/file/upload-sessions/init
   ← { sessionId, totalChunks }

2. PUT /api/file/upload-sessions/:sessionId/chunks/:chunkIndex
   ← { uploadedChunks: N }

3. POST /api/file/upload-sessions/:sessionId/finalize
   ← { url, privateUrl }

S3 Local Cache (Atomic Writes):

When S3 is the storage provider, downloads are cached locally to reduce S3 requests and improve latency. The cache uses atomic writes to prevent race conditions:

┌─────────────────────────────────────────────────────────────────┐
│  Request A                    Request B (concurrent)            │
├─────────────────────────────────────────────────────────────────┤
│  1. Check cache → miss        1. Check cache → miss             │
│  2. Create .downloading       2. See .downloading → skip cache  │
│  3. Stream to .tmp file       3. Stream directly from S3        │
│  4. Verify size matches       4. Complete                       │
│  5. Rename .tmp → final       ↓                                 │
│  6. Delete .downloading       Request C (later)                 │
│     ↓                         1. Check cache → hit              │
│                               2. Serve from cache               │
└─────────────────────────────────────────────────────────────────┘

Cache validation:

File exists AND age < S3_CACHE_MAX_AGE (default: 1 hour)
No .downloading marker file (indicates download in progress)
Size matches Content-Length header (verified before rename)

Why atomic writes: Without atomic writes, concurrent requests could serve truncated cache files:

Request A starts downloading 12MB file, writes to cache
After 2MB written, Request B checks cache - file exists, age valid
Request B serves 2MB truncated file → corrupted video/image

The atomic write pattern (write to .tmp, verify size, rename) prevents this.

Publish Service (`publish.ts`)

Three-tier content environment publishing workflow.

module.exports = class PublishService {
  // Acquire project lock and run callback
  static async withProjectPublishLock(projectId, callback)

  // Copy stage content to production (blocking)
  static async publishToProduction(projectId, currentUser, title, description)

  // Copy dev content to stage (non-blocking, returns immediately)
  static async saveToStage(projectId, currentUser)

  // Generic environment copy
  static async copyEnvironment(projectId, fromEnv, toEnv, currentUser, transaction)
}

Non-Blocking vs Blocking:

saveToStage() - Returns immediately, copy runs in background via setImmediate()
publishToProduction() - Waits for entire copy operation before returning

Publishing Flow:

┌─────────┐       ┌─────────┐       ┌────────────┐
│   DEV   │──────▶│  STAGE  │──────▶│ PRODUCTION │
│(editing)│       │(preview)│       │  (public)  │
└─────────┘       └─────────┘       └────────────┘
    │                  │                    │
saveToStage()    publishToProduction()      │
    │                  │                    │
    └──────────────────┴────────────────────┘
         Creates publish_event record

Event Status Lifecycle:

queued - Event created
running - Processing started
success / failed - Completed

Search Service (`search.ts`)

Global full-text search with permission filtering.

export default class SearchService {
  // Search across all permitted entities
  static async search(searchQuery: string, currentUser: CurrentUser | undefined): Promise<SearchResult>
}

Searchable Tables:

Table	Text Fields	Numeric Fields
users	firstName, lastName, phoneNumber, email	-
projects	name, slug, description, logo_url, favicon_url, og_image_url	-
assets	name, cdn_url, storage_key, mime_type, checksum	size_mb, width_px, height_px, duration_sec
asset_variants	cdn_url	width_px, height_px, size_mb
presigned_url_requests	requested_key, mime_type, status	requested_size_mb
tour_pages	source_key, name, slug, background_image_url, background_video_url, background_audio_url, ui_schema_json	sort_order
project_audio_tracks	source_key, name, slug, url	volume, sort_order
publish_events	error_message	pages_copied, transitions_copied, audios_copied
pwa_caches	cache_version, manifest_json, asset_list_json	-
access_logs	path, ip_address, user_agent	-

Permission Check:

// Only search tables user has READ permission for
if (!hasPermission(permissionSet, `READ_${tableName.toUpperCase()}`)) {
  return [];
}

Projects Service (`projects.ts`)

Project management with cloning capabilities and slug uniqueness validation.

export default class ProjectsService {
  // Normalize slug for URL safety
  static normalizeSlug(value) → slug

  // Generate unique slug with -copy suffix
  static async generateUniqueSlug(baseSlug, transaction) → uniqueSlug

  // Validate slug uniqueness before create/update (throws ValidationError if duplicate)
  static async validateSlugUniqueness(slug, excludeId, transaction) → normalizedSlug

  // Create new project (validates slug uniqueness)
  static async create({ data, currentUser, transaction, runtimeContext }) → project

  // Clone project with all assets and variants
  static async cloneFromProject(sourceProjectId, currentUser) → clonedProject

  // Update project (validates slug uniqueness if changed)
  static async update({ id, data, currentUser, transaction, runtimeContext }) → project
}

Slug Validation:

validateSlugUniqueness() normalizes the slug and checks for duplicates
Uses excludeId parameter to skip the current project during updates
Checks soft-deleted projects (paranoid: false) to prevent conflicts
Throws ValidationError('iam.errors.slugAlreadyExists') if duplicate found

Clone Process (Optimized with S3 Native Copy):

The clone process uses S3's native CopyObjectCommand for server-side file copying (15x faster than download-then-upload). Files are copied in parallel with configurable concurrency.

Phase A: Create cloned project record
    ↓
Phase B: Collect all copy operations (assets + non-reversed variants)
    ↓
Phase C: Execute parallel S3 copy (10 concurrent, continueOnError=true)
    ↓
Phase D: Build assetPathMap from copy results (failed → use original path)
    ↓
Phase E: Create asset/variant records, build assetIdMap (old → new asset IDs)
    ↓
Phase F: Copy reversed videos using asset ID mapping
    └── Reversed videos use pattern: assets/{assetId}/reversed.mp4
    └── Copy from old asset ID path to new asset ID path
    ↓
Phase G: Clone tour_pages, audio_tracks, element_defaults
    └── Transform ui_schema_json asset paths using assetPathMap

Key Implementation Details:

Phase	Operation	Notes
B-C	Parallel file copy	Uses `FileService.copyFilesParallel()` with S3 `CopyObjectCommand`
E	Asset ID mapping	Tracks `oldAssetId → newAssetId` for reversed video copying
F	Reversed video copy	Separate phase because reversed videos use asset-ID-based paths, not project-ID-based
G	Path transformation	`transformUiSchemaAssetPaths()` updates all asset URLs in `ui_schema_json`

Reversed Video Storage Pattern:

Primary assets: assets/{projectId}/{uuid}.ext
Reversed videos: assets/{assetId}/reversed.mp4 (uses asset ID, not project ID)

Error Handling:

Failed file copies fall back to original storage path (cloned project still functional, shares assets with source)
Most assets won't have reversed videos - this is expected (only navigation elements with transitions generate them)
Transaction rollback on DB errors; orphaned S3 files acceptable (can be cleaned later)

Roles Service (`roles.ts`)

Role management service for standard CRUD, CSV bulk import, and permission assignment. The service wraps DB writes in transactions when a caller does not provide one.

export default class RolesService {
  static assertPublicRoleHasNoPermissions(data, existingRole)
  static async create(options)
  static async bulkImport(req, res)
  static async update(options)
  static async deleteByIds(options)
  static async remove(options)
}

Public Role Hardening:

Creating or updating a role named Public rejects non-empty permissions.
Existing role lookup is done before update so renaming a role to Public cannot retain assigned permissions through the service boundary.
This keeps customer viewer access separate from admin RBAC permissions.

~~Generates offline manifests for PWA asset downloads.~~

Project Element Defaults Service (`project_element_defaults.ts`)

Extended factory service with custom methods for managing project-level element defaults.

class Project_element_defaultsService extends BaseService {
  // Reset project default to current global default
  static async resetToGlobal(id, options) → updated record

  // Get diff between project default and global default
  static async getDiffFromGlobal(id) → { hasChanges, diff }

  // Snapshot all global defaults to a project (called on project creation)
  static async snapshotGlobalDefaults(projectId, options) → created records
}

Methods:

Method	Description
`resetToGlobal(id, options)`	Resets a project element default to match the current global element type default
`getDiffFromGlobal(id)`	Compares project element default with global default, returns differences
`snapshotGlobalDefaults(projectId, options)`	Creates project element defaults by copying all global element type defaults

Use Cases:

Project Creation: snapshotGlobalDefaults is called to copy global defaults to new project
Reset to Global: User can reset customized project defaults back to global values
Diff View: UI can show which settings differ from global defaults

File Storage Module (`services/file/`)

Strategy Pattern Implementation

┌─────────────────────────────────────────────────────────────────────┐
│                     BaseStorageProvider                             │
│                    (Abstract Interface)                             │
│  ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌───────────────┐  │
│  │   upload    │ │  download   │ │   delete    │ │ getSignedUrl  │  │
│  │ (abstract)  │ │ (abstract)  │ │ (abstract)  │ │  (optional)   │  │
│  └─────────────┘ └─────────────┘ └─────────────┘ └───────────────┘  │
└─────────────────────────────────────────────────────────────────────┘
           △                    △                    △
           │                    │                    │
┌──────────┴─────────┐ ┌───────┴────────┐ ┌────────┴───────┐
│ S3StorageProvider  │ │LocalStorage    │ │(GCloud inline) │
│                    │ │Provider        │ │                │
│ • AWS SDK v3       │ │ • fs module    │ │ • @google-cloud│
│ • Presigned URLs   │ │ • MIME types   │ │   /storage     │
│ • Batch delete     │ │ • Recursive    │ │ • Resumable    │
└────────────────────┘ └────────────────┘ └────────────────┘

BaseStorageProvider Interface

export default class BaseStorageProvider {
  static get providerName(): string
  upload(key, data, options): Promise<StorageUploadResult>
  download(key): Promise<StorageDownloadResult>
  delete(key): Promise<void>
  deleteMany(keys): Promise<void>
  exists(key): Promise<boolean>
  list(prefix): Promise<string[]>
  getSignedUrl(key, expiresIn): Promise<string | null>
}

S3StorageProvider

AWS S3 implementation using SDK v3 with robust timeout, retry, and error handling.

S3StorageProvider.ts uses official AWS SDK v3 types for client config, command inputs, signed URL generation, streaming download bodies, and S3ServiceException metadata. Project-specific storage contracts live in backend/src/types/file.ts.

class S3StorageProvider extends BaseStorageProvider {
  constructor({
    bucket, region, accessKeyId, secretAccessKey, prefix,
    connectionTimeout,   // Default: 5000ms
    requestTimeout,      // Default: 30000ms
    maxAttempts,         // Default: 3 (adaptive retry)
    maxSockets,          // Default: 50 (connection pool)
    keepAlive            // Default: true
  })

  // Static methods for error handling
  static getErrorStatusCode(error)  // Map S3 errors to HTTP status codes
  static isRetryableError(error)    // Check if error should be retried

  // Instance methods (all support AbortSignal for cancellation)
  buildKey(key)                            // Add prefix to key
  async upload(key, data, { signal })      // PutObjectCommand
  async download(key, { signal })          // GetObjectCommand
  async copy(sourceKey, destKey, { signal, contentType })  // CopyObjectCommand (server-side)
  async delete(key, { signal })            // DeleteObjectCommand
  async deleteMany(keys, { signal })       // DeleteObjectsCommand (batched 1000)
  async exists(key, { signal })            // HeadObjectCommand
  async list(prefix, { signal })           // ListObjectsV2Command (paginated)
  async getSignedUrl(key, expiry)          // @aws-sdk/s3-request-presigner
  getConfig()                              // Get provider config for debugging
  destroy()                                // Cleanup connection pool
}

S3 Error to HTTP Status Code Mapping:

S3 Error	HTTP Status
NoSuchKey, NotFound, NoSuchBucket	404
AccessDenied, InvalidAccessKeyId	403
ExpiredToken	401
TimeoutError, RequestTimeout	504
NetworkingError, ServiceUnavailable	503
ThrottlingException	429
InternalError	500

LocalStorageProvider

Local filesystem implementation.

LocalStorageProvider.ts shares the same typed storage contracts as S3 while using Node filesystem and stream APIs.

class LocalStorageProvider extends BaseStorageProvider {
  constructor({ basePath = './uploads' })

  buildPath(key)                   // path.join(basePath, key)
  async upload(key, data)          // fs.writeFileSync / stream.pipeline
  async download(key)              // fs.createReadStream
  async copy(sourceKey, destKey)   // fs.promises.copyFile (kernel-level)
  async delete(key)                // fs.unlinkSync
  async exists(key)                // fs.existsSync
  async list(prefix)               // fs.readdirSync (recursive)
  getContentType(ext)              // MIME type mapping
}

UploadSessionManager

Chunked upload session management for large files.

UploadSessionManager.ts uses reusable upload-session metadata contracts from backend/src/types/file.ts and validates meta.json after parsing instead of relying on type assertions.

class UploadSessionManager {
  constructor({ sessionDir, ttlMs = 24h })

  createSession(options)           // → sessionId (UUID)
  readMeta(sessionId)              // → session metadata
  writeMeta(sessionId, payload)    // Save session state
  saveChunk(sessionId, index, data) // Save chunk file
  chunkExists(sessionId, index)    // Check chunk exists
  isComplete(sessionId)            // All chunks uploaded?
  assembleChunks(sessionId, path)  // Combine chunks
  removeSession(sessionId)         // Cleanup session
  cleanupExpiredSessions()         // Remove stale sessions
}

Session Directory Structure:

upload_sessions/
└── {sessionId}/
    ├── meta.json          # Session metadata
    └── chunks/
        ├── 0.part
        ├── 1.part
        └── N.part

Email Module (`services/email/`)

EmailSender Class

Core email sending using Nodemailer.

export default class EmailSender {
  constructor(email: EmailTemplate)
  async send(): Promise<EmailSendResult>
  static get isConfigured(): boolean
  get transportConfig(): SMTPTransport.Options
  get from(): string
}

Configuration (config.email):

{
  host: 'email-smtp.us-east-1.amazonaws.com',
  port: 587,
  auth: {
    user: process.env.EMAIL_USER || '',
    pass: process.env.EMAIL_PASS || ''
  },
  from: 'Tour Builder Platform <app@flatlogic.app>'
}

Email Templates

Template	Class	Fields
Password Reset	`PasswordResetEmail`	to, link
Email Verification	`EmailAddressVerificationEmail`	to, link
User Invitation	`InvitationEmail`	to, host

Template Pattern:

export default class PasswordResetEmail implements EmailTemplate {
  constructor({ to, link }: LinkEmailTemplateOptions) {}

  get subject(): string {
    return getNotification('emails.passwordReset.subject', getNotification('app.title'));
  }

  async html(): Promise<string> {
    const template = await fs.readFile(templatePath, 'utf8');
    return template
      .replace(/{appTitle}/g, appTitle)
      .replace(/{resetUrl}/g, this.link)
      .replace(/{accountName}/g, this.to);
  }
};

Notifications Module (`services/notifications/`)

Error Classes

ValidationError (400 Bad Request):

class ValidationError extends Error {
  constructor(messageCode) {
    const message = isNotification(messageCode)
      ? getNotification(messageCode)
      : getNotification('errors.validation.message');
    super(message);
    this.code = 400;
  }
}

ForbiddenError (403 Forbidden):

class ForbiddenError extends Error {
  constructor(messageCode) {
    const message = isNotification(messageCode)
      ? getNotification(messageCode)
      : getNotification('errors.forbidden.message');
    super(message);
    this.code = 403;
  }
}

Notification Catalog (`list.js`)

Centralized error messages and i18n strings.

const errors = {
  app: {
    title: 'Tour Builder Platform',
  },
  auth: {
    userDisabled: 'Your account is disabled',
    forbidden: 'Forbidden',
    unauthorized: 'Unauthorized',
    userNotFound: "Sorry, we don't recognize your credentials",
    wrongPassword: "Sorry, we don't recognize your credentials",
    // ...
  },
  iam: {
    errors: {
      userAlreadyExists: 'User with this email already exists',
      userNotFound: 'User not found',
      // ...
    }
  },
  emails: {
    invitation: {
      subject: "You've been invited to {0}",
      body: "..."
    },
    // ...
  }
};

Helper Functions

// Get notification with parameter substitution
getNotification('emails.invitation.subject', 'Tour Builder')
// → "You've been invited to Tour Builder"

// Check if key exists in catalog
isNotification('auth.userNotFound') // → true
isNotification('custom.message')    // → false

Transaction Patterns

Standard Transaction Pattern

All services use consistent transaction handling:

static async create({ data, currentUser, transaction: externalTransaction, runtimeContext }) {
  const transaction = externalTransaction || await db.sequelize.transaction();
  const ownsTransaction = !externalTransaction;
  try {
    const record = await DBApi.create({ data, currentUser, transaction, runtimeContext });
    if (ownsTransaction) await transaction.commit();
    return record;
  } catch (error) {
    if (ownsTransaction) await transaction.rollback();
    throw error;
  }
}

Lock Pattern (Publish Service)

static async withProjectPublishLock(projectId, callback) {
  return db.sequelize.transaction(async (transaction) => {
    // Acquire row lock
    const project = await db.projects.findByPk(projectId, {
      transaction,
      lock: transaction.LOCK.UPDATE,
    });

    // Check for concurrent operations
    const runningEvent = await db.publish_events.findOne({
      where: { projectId, status: EVENT_STATUS.RUNNING },
      transaction,
      lock: transaction.LOCK.UPDATE,
    });

    if (runningEvent) {
      throw new Error('Publish is already running for this project');
    }

    return callback(transaction);
  });
}

Dependencies

External Packages

Package	Usage
`bcrypt`	Password hashing
`nodemailer`	Email sending
`csv-parser`	CSV import parsing
`axios`	External API calls (widgets)
`uuid`	Upload session IDs
`@aws-sdk/client-s3`	S3 operations
`@aws-sdk/s3-request-presigner`	Presigned URLs
`@google-cloud/storage`	GCloud storage
`lodash/get`	Deep object access

Internal Dependencies

Module	Services Using
`db/models`	All services (transaction)
`db/api/*`	All entity services
`factories/service.factory`	9 factory services
`config`	auth, file, users, roles
`helpers`	auth (jwtSign)

Service Relationships

┌─────────────────────────────────────────────────────────────────────┐
│                         Service Graph                               │
│                                                                     │
│  ┌──────────┐                                                       │
│  │   auth   │◄────────────────────┐                                 │
│  └────┬─────┘                     │                                 │
│       │ uses                      │ sends invitations               │
│       ▼                           │                                 │
│  ┌──────────┐     ┌───────────┐   │                                 │
│  │  users   │────▶│  email    │───┘                                 │
│  │ (DBApi)  │     └───────────┘                                     │
│  └──────────┘           │                                           │
│                         │ uses templates                            │
│                         ▼                                           │
│                  ┌─────────────────┐                                │
│                  │  notifications  │                                │
│                  │ (error catalog) │                                │
│                  └─────────────────┘                                │
│                         ▲                                           │
│                         │ throws errors                             │
│                         │                                           │
│  ┌──────────┐     ┌─────┴─────┐     ┌──────────┐                    │
│  │ projects │     │  publish  │     │  search  │                    │
│  │ (clone)  │     │ (env copy)│     │(fulltext)│                    │
│  └──────────┘     └───────────┘     └──────────┘                    │
│                                                                     │
│  ┌──────────────────────────────────────────────────────────────┐   │
│  │                      File Service                            │   │
│  │  ┌──────────────┐ ┌──────────────┐ ┌──────────────────────┐  │   │
│  │  │ S3Provider   │ │LocalProvider │ │UploadSessionManager  │  │   │
│  │  └──────────────┘ └──────────────┘ └──────────────────────┘  │   │
│  └──────────────────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────────────┘

Configuration

Environment Variables

Variable	Service	Description
`FILE_STORAGE_PROVIDER`	file	Force provider ('s3', 'gcloud', 'local')
`AWS_S3_BUCKET`	file	S3 bucket name
`AWS_S3_REGION`	file	AWS region (default: us-east-1)
`AWS_ACCESS_KEY_ID`	file	AWS access key
`AWS_SECRET_ACCESS_KEY`	file	AWS secret key
`AWS_S3_PREFIX`	file	Key prefix
`AWS_S3_CONNECTION_TIMEOUT`	file	S3 connection timeout in ms (default: 5000)
`AWS_S3_REQUEST_TIMEOUT`	file	S3 request timeout in ms (default: 30000)
`AWS_S3_MAX_ATTEMPTS`	file	S3 retry attempts (default: 3)
`AWS_S3_MAX_SOCKETS`	file	S3 connection pool size (default: 50)
`AWS_S3_KEEP_ALIVE`	file	Enable HTTP keep-alive (default: true)
`AWS_S3_PRESIGN_EXPIRY`	file	Presigned URL expiry in seconds (default: 3600)
`GC_PROJECT_ID`	file	GCloud project
`GC_CLIENT_EMAIL`	file	GCloud service account
`GC_PRIVATE_KEY`	file	GCloud private key
`FFMPEG_REVERSE_TIMEOUT_MS`	videoProcessing	Reverse-video hard timeout in ms (default: 600000)
`FFPROBE_TIMEOUT_MS`	videoProcessing	Metadata probe timeout in ms (default: 30000)
`FFMPEG_BREAKER_FAILURE_THRESHOLD`	videoProcessing	Failures before FFmpeg breaker opens (default: 3)
`FFMPEG_BREAKER_COOLDOWN_MS`	videoProcessing	FFmpeg breaker cooldown in ms (default: 120000)
`FFMPEG_BREAKER_SUCCESS_THRESHOLD`	videoProcessing	Half-open successes required to close FFmpeg breaker (default: 1)
`FILE_STORAGE_BREAKER_FAILURE_THRESHOLD`	file	Failures before storage breaker opens (default: 5)
`FILE_STORAGE_BREAKER_COOLDOWN_MS`	file	Storage breaker cooldown in ms (default: 30000)
`FILE_STORAGE_BREAKER_SUCCESS_THRESHOLD`	file	Half-open successes required to close storage breaker (default: 2)
`EMAIL_USER`	email	SMTP username
`EMAIL_PASS`	email	SMTP password

Config References

// config.ts
module.exports = {
  bcrypt: { saltRounds: 12 },
  email: { host, port, auth, from },
  s3: {
    bucket,
    region,
    accessKeyId,
    secretAccessKey,
    prefix,
    // Timeout/retry configuration
    connectionTimeout,     // 5000ms default
    requestTimeout,        // 30000ms default
    maxAttempts,           // 3 retries with adaptive backoff
    // Connection pool
    maxSockets,            // 50 concurrent connections
    keepAlive,             // HTTP keep-alive enabled
    // Presigned URLs
    presignExpirySeconds,  // 3600 (1 hour) default
  },
  gcloud: { bucket, hash },
  uploadDir: './uploads',
  flHost: 'https://flatlogic.host',  // Widget service
  project_uuid: '...',
};

Testing Guidelines

Unit Testing Services

// Mock transaction
jest.mock('../db/models', () => ({
  sequelize: {
    transaction: jest.fn(() => ({
      commit: jest.fn(),
      rollback: jest.fn(),
    })),
  },
}));

// Mock DB API
jest.mock('../db/api/assets');

describe('AssetsService', () => {
  it('should create asset with transaction', async () => {
    const result = await AssetsService.create({
      data: mockData,
      currentUser: mockUser,
    });
    expect(transaction.commit).toHaveBeenCalled();
  });
});

Integration Testing

describe('PublishService', () => {
  it('should copy dev to stage', async () => {
    // Create test project with pages
    const project = await createTestProject();
    await createTestPages(project.id, 'dev');

    // Publish to stage
    const result = await PublishService.saveToStage(project.id, mockUser);

    // Verify stage pages created
    const stagePages = await findPages(project.id, 'stage');
    expect(stagePages.length).toBe(result.summary.pages_copied);
  });
});

Best Practices

1. Transaction Handling

Always wrap multi-step operations in transactions
Use try/catch/rollback pattern consistently
Pass transaction to all DB API calls

2. Error Handling

Use ValidationError for client errors (400)
Use ForbiddenError for authorization failures (403)
Use notification catalog for consistent messages

3. Service Design

Keep services focused on business logic
Delegate DB operations to DB API layer
Use factories for simple CRUD services

4. File Operations

Use Strategy Pattern for multi-provider support
Implement chunked uploads for large files
Clean up sessions on completion/failure

Backend Architecture - Overall backend design
Database Schema - Data models
API Endpoints - REST API reference
Auth Module - Authentication details
Routes Module - Route implementations

46 KiB Raw Blame History