30 KiB
30 KiB
Backend Architecture Documentation
This document provides a comprehensive analysis of the Tour Builder Platform backend architecture, including design patterns, layers, and implementation details.
Overview
- Runtime: Node.js 24 LTS
- Framework: Express.js 4.x
- ORM: Sequelize 6.x with PostgreSQL
- Authentication: Passport.js (JWT, Google OAuth, Microsoft OAuth)
- Documentation: Swagger/OpenAPI 3.0
- Logging: Pino (structured JSON logging)
- TypeScript/ESM: the backend package is
"type": "module"and active backend source is strict TypeScript ESM. - File Storage: AWS S3 / Local filesystem (Strategy Pattern, provider-based)
Architecture Diagram
┌──────────────────────────────────────────────────────────────────────────┐
│ Express Application │
│ (src/index.ts) │
└──────────────────────────────────────────────────────────────────────────┘
│
┌─────────────────┼─────────────────┐
▼ ▼ ▼
┌────────────────────────┐ ┌────────────────┐ ┌────────────────────────┐
│ Middleware │ │ Rate Limiter │ │ Request Logger │
│ • runtimeContext │ │ • auth (10/15m)│ │ • Pino structured │
│ • runtimePublic │ │ • api (100/1m) │ │ • Request ID tracking │
│ • checkPermissions │ │ • upload (10/m)│ │ • Duration metrics │
│ • passport JWT │ │ • download(200)│ └────────────────────────┘
└────────────────────────┘ │ • search (30/m)│
│ • AI (20/1m) │
└────────────────┘
│
┌──────────────────────────────────────────────────────────────────────────┐
│ Routes Layer │
│ (src/routes/*.ts) │
│ ┌────────────────────────────────────────────────────────────────────┐ │
│ │ Factory-generated routes: router.factory.js │ │
│ │ • Standard CRUD: POST /, PUT /:id, DELETE /:id, GET /, GET /:id │ │
│ │ • Extra: GET /count, GET /autocomplete, POST /deleteByIds │ │
│ │ • CSV export: GET /?filetype=csv │ │
│ └────────────────────────────────────────────────────────────────────┘ │
│ ┌────────────────────────────────────────────────────────────────────┐ │
│ │ Custom routes: auth, file, publish, search, sql, runtime │ │
│ └────────────────────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────────────────┘
│
▼
┌──────────────────────────────────────────────────────────────────────────┐
│ Service Layer │
│ (src/services/*.js) │
│ ┌────────────────────────────────────────────────────────────────────┐ │
│ │ Factory-generated services: service.factory.js │ │
│ │ • Transaction-wrapped CRUD operations │ │
│ │ • Delegates to DB API layer │ │
│ └────────────────────────────────────────────────────────────────────┘ │
│ ┌────────────────────────────────────────────────────────────────────┐ │
│ │ Custom services: │ │
│ │ • auth.js - Authentication logic (signin, signup, password) │ │
│ │ • publish.ts - Publishing workflow service (dev→stage→production)│ │
│ │ • file.ts - File storage operations │ │
│ │ • email/index.js - Email sending via Nodemailer/SES │ │
│ │ • search.ts - Full-text search route │ │
│ │ • pwa_manifest.js - PWA offline manifest generation │ │
│ └────────────────────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────────────────┘
│
▼
┌──────────────────────────────────────────────────────────────────────────┐
│ Database API Layer │
│ (src/db/api/*.js) │
│ ┌────────────────────────────────────────────────────────────────────┐ │
│ │ GenericDBApi (base.api.js) - Template Method Pattern │ │
│ │ • Configurable: MODEL, SEARCHABLE_FIELDS, RANGE_FIELDS, etc. │ │
│ │ • CRUD: create, update, remove, deleteByIds │ │
│ │ • Query: findBy, findAll, findAllAutocomplete │ │
│ │ • Data Transform: getFieldMapping(), JSON_FIELDS, FIELD_DEFAULTS │ │
│ └────────────────────────────────────────────────────────────────────┘ │
│ ┌────────────────────────────────────────────────────────────────────┐ │
│ │ Entity APIs extend GenericDBApi: │ │
│ │ UsersDBApi, ProjectsDBApi, AssetsDBApi, TourPagesDBApi, etc. │ │
│ └────────────────────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────────────────┘
│
▼
┌──────────────────────────────────────────────────────────────────────────┐
│ Sequelize Models │
│ (src/db/models/*.ts) │
│ Entity models + file model, loaded dynamically via loader.ts │
└──────────────────────────────────────────────────────────────────────────┘
│
▼
┌──────────────────────────────────────────────────────────────────────────┐
│ PostgreSQL │
└──────────────────────────────────────────────────────────────────────────┘
Directory Structure
backend/src/
├── index.ts # Application entry point
├── config.ts # Environment configuration
├── helpers.ts # Utility functions (wrapAsync, JWT, UUID validation)
├── types/ # Reusable strict TypeScript contracts for migrated code
│
├── auth/
│ └── auth.ts # Passport strategies (JWT, Google, Microsoft)
│
├── middlewares/
│ ├── check-permissions.ts # RBAC permission checking
│ ├── runtime-context.ts # Runtime environment context (dev/stage/production)
│ ├── runtime-public.ts # Public runtime access control & field sanitization
│ ├── rateLimiter.ts # Rate limiting (auth, API, upload, download)
│ └── upload.ts # File upload handling (multer)
│
├── routes/
│ ├── auth.ts # Authentication routes (custom)
│ ├── file.ts # File upload/download routes (custom)
│ ├── publish.ts # Publishing workflow routes (custom)
│ ├── search.ts # Full-text search routes (custom)
│ ├── runtime-context.ts # Runtime context detection (custom)
│ └── [entity].ts # Entity CRUD routes
│
├── services/
│ ├── auth.ts # Authentication service
│ ├── publish.ts # Publishing workflow (dev→stage→production)
│ ├── file.ts # Unified file storage service
│ ├── search.ts # Full-text search service
│ ├── file/ # File storage providers (S3, Local)
│ │ ├── index.ts # Module exports & provider factory
│ │ ├── BaseStorageProvider.ts # Abstract base class
│ │ ├── S3StorageProvider.ts # AWS S3 implementation
│ │ ├── LocalStorageProvider.ts # Local filesystem implementation
│ │ └── UploadSessionManager.ts # Chunked upload session management
│ ├── email/
│ │ ├── index.ts # Email sender (Nodemailer/SES)
│ │ └── list/ # Email templates
│ ├── notifications/
│ │ ├── helpers.ts # Notification helpers
│ │ └── errors/ # Error classes (ValidationError, ForbiddenError)
│ └── [entity].ts # Entity services
│
├── factories/
│ ├── router.factory.ts # Route generator (createEntityRouter)
│ └── service.factory.ts # Service generator (createEntityService)
│
├── db/
│ ├── db-config.ts # Database configuration (env-based)
│ ├── umzug.ts # Migration and seeder runner
│ ├── models/
│ │ ├── index.ts # Model registry entrypoint
│ │ ├── loader.ts # Model loader
│ │ └── [entity].ts # Sequelize models
│ ├── api/
│ │ ├── base.api.ts # GenericDBApi base class
│ │ ├── runtime-context.ts # Runtime filtering helpers
│ │ └── [entity].ts # Entity DB APIs
│ ├── migrations/ # Applied migration history
│ └── seeders/ # Typed seed data files
│
└── utils/
├── index.ts # Utils barrel export
├── errors.ts # Error classes (AppError, NotFoundError, etc.)
├── logger.ts # Pino logger configuration
└── env-validation.ts # Environment variable validation
Design Patterns
1. Factory Pattern
Router Factory (factories/router.factory.js)
Generates standardized CRUD routes for entities:
const { createEntityRouter } = require('../factories/router.factory');
// Creates routes: POST /, PUT /:id, DELETE /:id, GET /, GET /:id, GET /count, GET /autocomplete
module.exports = createEntityRouter('assets', AssetsService, AssetsDBApi, {
permissionEntity: 'assets',
csvFields: ['id', 'name', 'asset_type', 'createdAt'],
});
Service Factory (factories/service.factory.js)
Generates transaction-wrapped service classes:
const { createEntityService } = require('../factories/service.factory');
// Creates: create(), update(), remove(), deleteByIds(), bulkImport()
module.exports = createEntityService(AssetsDBApi, { entityName: 'Asset' });
2. Template Method Pattern
GenericDBApi (db/api/base.api.js)
Base class with configurable hooks for entity-specific behavior:
class AssetsDBApi extends GenericDBApi {
// Required: Define the Sequelize model
static get MODEL() { return db.assets; }
// Configurable behavior via static getters
static get SEARCHABLE_FIELDS() { return ['name', 'cdn_url']; }
static get RANGE_FIELDS() { return ['size_mb', 'width_px']; }
static get ENUM_FIELDS() { return ['asset_type', 'is_public']; }
static get JSON_FIELDS() { return ['settings_json']; }
static get FIELD_DEFAULTS() { return { type: { default: 'general' } }; }
static get ASSOCIATIONS() { return [{ field: 'project', setter: 'setProject' }]; }
static get FIND_BY_INCLUDES() { return [{ association: 'project' }]; }
static get FIND_ALL_INCLUDES() { return [{ model: db.projects, as: 'project' }]; }
// Custom field transformation
static getFieldMapping(data) {
return {
name: data.name || null,
asset_type: data.asset_type || null,
type: data.type || 'general',
// ...
};
}
}
3. Strategy Pattern
File Storage Providers (services/file/)
Two concrete implementations with pluggable architecture:
BaseStorageProvider (abstract)
│
├── S3StorageProvider # AWS S3 implementation (with timeout/retry)
└── LocalStorageProvider # Local filesystem implementation
The storage provider base, S3 provider, and local provider are migrated TS/ESM modules. The S3 implementation uses official AWS SDK v3 types; shared provider-domain contracts are in src/types/file.ts.
Interface:
upload(key, data, options)→{ key, url }download(key)→{ body, contentType }delete(key)→voiddeleteMany(keys)→voidexists(key)→booleanlist(prefix)→string[]getSignedUrl(key, expiresIn)→string
4. Middleware Chain Pattern
Request flow through middleware stack:
Request → requestLogger → runtimeContextMiddleware → rateLimiter
→ passport.authenticate('jwt') → checkCrudPermissions
→ Route Handler → Service → DB API → Database
→ Response
Layers in Detail
Entry Point (src/index.ts)
Application bootstrap:
- Security: Helmet, CORS configuration
- Logging: Request logger middleware (Pino)
- Authentication: Passport JWT initialization
- Rate Limiting: Per-route rate limiters
- Body Parsing: JSON (1mb limit), applied after file routes
- Runtime Context: Environment detection middleware
- Route Mounting: Entity routes with auth/permissions
- Error Handling: Generic error handler
- Static Files: Public directory serving
// Key route mounting patterns
app.use('/api/auth', authRoutes); // No JWT required
app.use('/api/users', jwtAuth, usersRoutes); // JWT required
// Runtime public routes (production content accessible without auth)
const mountRuntimeEntityRoute = (path, entityName, router) => {
app.use(path,
requireRuntimeReadOrAuth, // JWT or public production
blockNonPublicRuntimeListEndpoints, // Block non-list endpoints
sanitizePublicRuntimeListResponse(entityName), // Filter sensitive fields
router
);
};
mountRuntimeEntityRoute('/api/projects', 'projects', projectsRoutes);
mountRuntimeEntityRoute('/api/tour_pages', 'tour_pages', tour_pagesRoutes);
Routes Layer
Factory-Generated Routes provide standard CRUD:
| Method | Path | Description |
|---|---|---|
| POST | / |
Create record |
| POST | /bulk-import |
Bulk import from CSV |
| PUT | /:id |
Update record |
| DELETE | /:id |
Delete record |
| POST | /deleteByIds |
Bulk delete |
| GET | / |
List with pagination & filters |
| GET | /count |
Count only |
| GET | /autocomplete |
Autocomplete search |
| GET | /:id |
Get single record |
Custom Routes (auth, file, publish, search, runtime-context):
| Route | Endpoints |
|---|---|
/api/auth |
signin, signup, me, password-reset, verify-email, Google/Microsoft OAuth |
/api/file |
upload, download, presign, upload-sessions (chunked) |
/api/publish |
publish (stage→production), save-to-stage (dev→stage) |
/api/search |
Global full-text search |
/api/runtime-context |
Runtime environment detection |
Service Layer
Transaction Management: Services wrap operations in transactions:
static async create({ data, currentUser, transaction: externalTransaction, runtimeContext }) {
const transaction = externalTransaction || await db.sequelize.transaction();
const ownsTransaction = !externalTransaction;
try {
const record = await DBApi.create({ data, currentUser, transaction, runtimeContext });
if (ownsTransaction) await transaction.commit();
return record;
} catch (error) {
if (ownsTransaction) await transaction.rollback();
throw error;
}
}
Publish Service (services/publish.ts):
Implements the dev→stage→production workflow with:
- Transaction locking to prevent concurrent publishes
- Source key tracking for content lineage
- Bulk copy operations for pages and audio tracks
Database API Layer
Query Building in findAll():
| Filter Type | Example | SQL |
|---|---|---|
| Text search | ?name=foo |
name ILIKE '%foo%' |
| Range | ?size_mbRange=[0,100] |
size_mb >= 0 AND size_mb <= 100 |
| Enum | ?asset_type=image |
asset_type = 'image' |
| Relation | ?project=uuid |
JOIN with projects table |
| Sort | ?field=name&sort=asc |
ORDER BY name ASC |
| Pagination | ?page=1&limit=10 |
OFFSET 0 LIMIT 10 |
Authentication & Authorization
Authentication Flow
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ POST /signin │ │ Passport JWT │ │ Protected │
│ → JWT Token │────▶│ Middleware │────▶│ Route │
└─────────────────┘ └─────────────────┘ └─────────────────┘
│
▼
req.currentUser = {
id, email, app_role,
custom_permissions
}
Authorization (RBAC)
Permission Check Flow (middlewares/check-permissions.ts):
- Self-access bypass (user accessing own resource)
- Check custom permissions (user-specific)
- Check role permissions (from app_role)
- Fallback to Public role for unauthenticated
Permission Naming Convention:
CREATE_<ENTITY>- Create recordsREAD_<ENTITY>- Read recordsUPDATE_<ENTITY>- Modify recordsDELETE_<ENTITY>- Delete records
// Auto-generated from HTTP method
const permissionName = `${METHOD_MAP[req.method]}_${name.toUpperCase()}`;
// POST /api/assets → CREATE_ASSETS
// GET /api/assets → READ_ASSETS
Runtime Public Access
For production content accessible without authentication:
const requireRuntimeReadOrAuth = (req, res, next) => {
const isPublicEnvironment = req.runtimeContext?.headerEnvironment === 'production';
const isReadOnlyRequest = ['GET', 'OPTIONS'].includes(req.method);
if (isPublicEnvironment && isReadOnlyRequest && !hasAuthHeader) {
req.isRuntimePublicRequest = true;
return next(); // Allow without JWT
}
return jwtAuth(req, res, next); // Require JWT
};
Rate Limiting
Pre-configured limiters (middlewares/rateLimiter.ts):
| Limiter | Window | Max Requests | Use Case |
|---|---|---|---|
authLimiter |
15 min | 10 | Authentication endpoints |
passwordResetLimiter |
1 hour | 5 | Password reset |
apiLimiter |
1 min | 100 | General API |
uploadLimiter |
1 min | 10 | File uploads |
downloadLimiter |
1 min | 200 | File downloads |
searchLimiter |
1 min | 30 | Search queries |
Headers returned:
X-RateLimit-Limit: Maximum requestsX-RateLimit-Remaining: Remaining requestsX-RateLimit-Reset: Reset time (ISO timestamp)Retry-After: Seconds until reset (when limited)
File Storage
Storage Provider Selection:
const provider = config.fileStorage.provider ||
(hasS3Credentials ? 's3' : hasGCloudCredentials ? 'gcloud' : 'local');
S3 Operations:
| Operation | Method | Description |
|---|---|---|
| Upload | upload(key, data, options) |
Put object with metadata |
| Download | download(key) |
Get object stream |
| Presign | getSignedUrl(key, expiresIn) |
Generate presigned URL |
| Delete | delete(key) / deleteMany(keys) |
Remove objects |
| Check | exists(key) |
Head object |
| List | list(prefix) |
List objects with prefix |
Chunked Uploads (UploadSessionManager):
For large files, supports multipart upload sessions:
POST /upload-sessions/init- Create sessionPOST /upload-sessions/:id/chunk- Upload chunkPOST /upload-sessions/:id/finalize- Complete upload
Error Handling
Error Classes (utils/errors.js):
class AppError extends Error {
constructor(message, statusCode = 500, details = null) {
super(message);
this.statusCode = statusCode;
this.details = details;
this.isOperational = true;
}
}
class NotFoundError extends AppError { statusCode = 404 }
class ValidationError extends AppError { statusCode = 400 }
class ForbiddenError extends AppError { statusCode = 403 }
class UnauthorizedError extends AppError { statusCode = 401 }
class ConflictError extends AppError { statusCode = 409 }
Async Handler (helpers.ts):
// Wraps async route handlers to catch errors
static wrapAsync(fn) {
return function(req, res, next) {
fn(req, res, next).catch(next);
};
}
Common Error Handler (helpers.ts):
static commonErrorHandler(error, req, res, _next) {
const statusCode = error.code || error.status;
if ([400, 401, 403, 404, 409, 422].includes(statusCode)) {
return res.status(statusCode).send(error.message);
}
console.error(error);
return res.status(500).send('Internal server error');
}
Logging
Pino Logger (utils/logger.js):
Logger initialization is a bootstrap exception: it may read process.env
directly because importing config.ts would create an initialization cycle.
const logger = pino({
level: process.env.LOG_LEVEL || 'info',
transport: isDevelopment
? { target: 'pino-pretty', options: { colorize: true } }
: undefined,
base: {
service: 'tour-builder-api',
env: process.env.NODE_ENV || 'development',
},
});
Request Logging:
function requestLogger(req, res, next) {
const requestId = req.headers['x-request-id'] || crypto.randomUUID();
req.log = logger.child({ requestId });
req.requestId = requestId;
res.setHeader('X-Request-Id', requestId);
res.on('finish', () => {
req.log.info({
method: req.method,
url: req.originalUrl,
status: res.statusCode,
duration: Date.now() - start,
}, 'Request completed');
});
}
Configuration
Environment Variables (config.ts):
| Variable | Description | Default |
|---|---|---|
SECRET_KEY |
JWT signing key | UUID-based default |
ADMIN_EMAIL |
Admin user email | admin@flatlogic.com |
ADMIN_PASS |
Admin user password | Generated |
AWS_S3_BUCKET |
S3 bucket name | - |
AWS_S3_REGION |
S3 region | us-east-1 |
AWS_ACCESS_KEY_ID |
AWS access key | - |
AWS_SECRET_ACCESS_KEY |
AWS secret key | - |
GOOGLE_CLIENT_ID |
Google OAuth client ID | - |
GOOGLE_CLIENT_SECRET |
Google OAuth client secret | - |
MS_CLIENT_ID |
Microsoft OAuth client ID | - |
MS_CLIENT_SECRET |
Microsoft OAuth client secret | - |
EMAIL_USER |
SMTP username | - |
EMAIL_PASS |
SMTP password | - |
LOG_LEVEL |
Logging level | info |
Database Configuration (db/db-config.ts):
| Environment | Database | Logging |
|---|---|---|
production |
DB_* env vars |
Disabled |
development |
db_tour_builder_platform |
Console |
dev_stage |
DB_* env vars |
Console |
API Documentation
Swagger/OpenAPI documentation is available at /api-docs.
The served document is centralized in backend/src/openapi/document.ts. The
OpenAPI module defines shared schemas, common parameters, reusable responses,
and generated standard CRUD paths for every createEntityRouter resource. This
keeps the documented factory contract aligned with the route factory endpoints:
create, bulk import, update, delete, bulk delete, list, count, autocomplete,
and get by ID.
const specs = createOpenApiDocument({
serverUrl: config.server.swaggerServerUrl,
});
When adding a new route, update backend/src/openapi/document.ts in the same
change. For new factory-backed entities, add the entity schema and CrudResource
entry; for custom routes, add an explicit path item.
Health Check
GET /api/health
{
"status": "ok", // or "degraded"
"timestamp": "2026-03-29T...",
"uptime": 12345.678,
"environment": "production",
"database": "connected" // or "disconnected"
}
Key Implementation Files
| File | Purpose |
|---|---|
src/index.ts |
Application entry, middleware setup, route mounting |
src/config.ts |
Environment configuration |
src/helpers.ts |
wrapAsync, commonErrorHandler, jwtSign, isUuidV4 |
src/auth/auth.ts |
Passport strategies (JWT, Google, Microsoft) |
src/factories/router.factory.ts |
Route generator for entities |
src/factories/service.factory.ts |
Service generator for entities |
src/db/api/base.api.ts |
GenericDBApi base class |
src/middlewares/check-permissions.ts |
RBAC permission checking |
src/middlewares/rateLimiter.ts |
Rate limiting configuration |
src/middlewares/runtime-context.ts |
Runtime environment detection |
src/middlewares/runtime-public.ts |
Public runtime access control & field sanitization |
src/services/publish.ts |
Publishing workflow service |
src/services/file/S3StorageProvider.ts |
S3 storage implementation using official AWS SDK v3 types |
src/utils/logger.ts |
Pino logger configuration |
src/utils/errors.ts |
Error class definitions |
Best Practices Implemented
- Factory Patterns - Reduce boilerplate for CRUD operations
- Template Method - Configurable base class for DB operations
- Strategy Pattern - Pluggable storage providers
- Middleware Chain - Composable request processing
- Transaction Management - Consistent rollback on errors
- Rate Limiting - Protection against abuse
- Structured Logging - JSON logs with request IDs
- Environment-Based Config - Secure credential handling
- Soft Deletes - Paranoid models for data recovery
- RBAC - Fine-grained permission control