Dmitri 9f111d6226 added documentation

2026-07-03 16:11:24 +02:00

29 KiB

Raw Blame History

Backend Middleware Module Documentation

Overview

The Middleware module provides cross-cutting concerns for the Express application including rate limiting, permission checking, runtime context management, file uploads, and public access control.

Files:

File	Lines	Purpose
`src/middlewares/rateLimiter.js`	268	Configurable rate limiting with in-memory store
`src/middlewares/check-permissions.ts`	RBAC permission checking through AccessPolicy
`src/middlewares/runtime-context.ts`	34	Runtime environment context from headers
`src/middlewares/runtime-public.ts`	200	Public runtime access control and response sanitization
`src/middlewares/upload.ts`	34	Multer-based file upload handling

Architecture Diagram

┌──────────────────────────────────────────────────────────────────────┐
│                         Incoming Request                             │
└──────────────────────────────────┬───────────────────────────────────┘
                                   │
                                   ▼
┌──────────────────────────────────────────────────────────────────────┐
│                    Express Middleware Stack                          │
│                                                                      │
│  ┌────────────────────────────────────────────────────────────────┐  │
│  │ 1. helmet() - Security headers                                 │  │
│  └────────────────────────────────────────────────────────────────┘  │
│                                   │                                  │
│                                   ▼                                  │
│  ┌────────────────────────────────────────────────────────────────┐  │
│  │ 2. cors() - Cross-origin resource sharing                     │  │
│  └────────────────────────────────────────────────────────────────┘  │
│                                   │                                  │
│                                   ▼                                  │
│  ┌────────────────────────────────────────────────────────────────┐  │
│  │ 3. requestLogger - Pino HTTP logging                          │  │
│  └────────────────────────────────────────────────────────────────┘  │
│                                   │                                  │
│                                   ▼                                  │
│  ┌────────────────────────────────────────────────────────────────┐  │
│  │ 4. Rate Limiters (route-specific)                             │  │
│  │    • downloadLimiter → /api/file/download, /api/file/presign  │  │
│  │    • uploadLimiter → /api/file/upload                         │  │
│  │    • searchLimiter → /api/search                              │  │
│  │    • authLimiter → /api/auth/signin                           │  │
│  └────────────────────────────────────────────────────────────────┘  │
│                                   │                                  │
│                                   ▼                                  │
│  ┌────────────────────────────────────────────────────────────────┐  │
│  │ 5. bodyParser.json() - JSON body parsing (after file routes)  │  │
│  └────────────────────────────────────────────────────────────────┘  │
│                                   │                                  │
│                                   ▼                                  │
│  ┌────────────────────────────────────────────────────────────────┐  │
│  │ 6. runtimeContextMiddleware - Environment context             │  │
│  │    Reads: X-Runtime-Environment, X-Runtime-Project-Slug       │  │
│  └────────────────────────────────────────────────────────────────┘  │
│                                   │                                  │
│                                   ▼                                  │
│  ┌────────────────────────────────────────────────────────────────┐  │
│  │ 7. JWT Authentication (route-specific)                        │  │
│  │    passport.authenticate('jwt', { session: false })           │  │
│  └────────────────────────────────────────────────────────────────┘  │
│                                   │                                  │
│                                   ▼                                  │
│  ┌────────────────────────────────────────────────────────────────┐  │
│  │ 8. Runtime Public Middleware (route-specific)                 │  │
│  │    • blockNonPublicRuntimeListEndpoints                       │  │
│  │    • sanitizePublicRuntimeListResponse                        │  │
│  └────────────────────────────────────────────────────────────────┘  │
│                                   │                                  │
│                                   ▼                                  │
│  ┌────────────────────────────────────────────────────────────────┐  │
│  │ 9. checkCrudPermissions / checkPermissions (route-specific)   │  │
│  └────────────────────────────────────────────────────────────────┘  │
│                                   │                                  │
│                                   ▼                                  │
│  ┌────────────────────────────────────────────────────────────────┐  │
│  │ 10. Route Handler                                             │  │
│  └────────────────────────────────────────────────────────────────┘  │
│                                   │                                  │
│                                   ▼                                  │
│  ┌────────────────────────────────────────────────────────────────┐  │
│  │ 11. Error Handler (commonErrorHandler)                        │  │
│  └────────────────────────────────────────────────────────────────┘  │
└──────────────────────────────────────────────────────────────────────┘

File Details

1. rateLimiter.js (268 lines)

In-memory rate limiting middleware with configurable windows and limits.

Storage Architecture

// In-memory store (Map)
const rateLimitStore = new Map();

// Entry structure
{
  count: number,      // Request count in window
  expiresAt: number,  // Window expiration timestamp
  resetTime: string   // ISO timestamp for headers
}

// Automatic cleanup every 5 minutes
setInterval(() => {
  for (const [key, entry] of rateLimitStore.entries()) {
    if (entry.expiresAt <= now) {
      rateLimitStore.delete(key);
    }
  }
}, 5 * 60 * 1000);

Factory: createRateLimiter(options)

Creates a configurable rate limiter middleware.

Parameters:

Parameter	Type	Default	Description
`keyPrefix`	string	`'rate-limit'`	Prefix for rate limit keys
`windowMs`	number	`900000` (15min)	Time window in milliseconds
`max`	number	`100`	Maximum requests per window
`message`	string	`'Too many requests...'`	Error message on limit
`skipFailedRequests`	boolean	`false`	Don't count 4xx/5xx responses
`keyGenerator`	function	`null`	Custom key generator `(req) => string`
`skip`	function	`null`	Skip rate limiting `(req) => boolean`

Returns: Express middleware function

Response Headers:

X-RateLimit-Limit: 100
X-RateLimit-Remaining: 99
X-RateLimit-Reset: 2024-01-01T00:15:00.000Z
Retry-After: 300  (only when limit exceeded)

Rate Limit Exceeded Response (429):

{
  "error": "Too Many Requests",
  "message": "Too many requests. Please try again later.",
  "retryAfter": 300
}

Factory: createAuthenticatedRateLimiter(options)

Creates rate limiter that uses IP + User ID as key.

const createAuthenticatedRateLimiter = (options = {}) => {
  return createRateLimiter({
    ...options,
    keyGenerator: (req) => {
      const userId = req.currentUser?.id || 'anonymous';
      const ip = req.ip || 'unknown';
      return `${ip}:${userId}`;
    },
  });
};

Pre-configured Limiters

Limiter	Key Prefix	Window	Max	Skip Failed	Use Case
`authLimiter`	`auth`	15 min	10	No	Login attempts
`passwordResetLimiter`	`password-reset`	1 hour	5	No	Password reset
`apiLimiter`	`api`	1 min	100	Yes	General API
`uploadLimiter`	`upload`	1 min	10	No	File uploads
`downloadLimiter`	`download`	1 min	200	Yes	File downloads
`searchLimiter`	`search`	1 min	30	No	Search queries

Route Mapping

// index.js
app.use('/api/file/download', downloadLimiter);
app.use('/api/file/presign', downloadLimiter);
app.use('/api/file/upload', uploadLimiter);
app.use('/api/file/upload-sessions', uploadLimiter);
app.use('/api/search', jwtAuth, searchLimiter, searchRoutes);

// routes/auth.js
router.post('/signin/local', signinLimiter, handler);
router.post('/send-password-reset-email', passwordResetLimiter, handler);

Development Mode

Rate limiting is skipped for localhost in development:

if (
  config.server.env === 'development' &&
  (req.ip === '127.0.0.1' || req.ip === '::1')
) {
  return next(); // Skip rate limiting
}

2. check-permissions.ts (194 lines)

Role-based access control (RBAC) middleware. Permission decisions are delegated to src/services/access-policy.ts so role/custom permission resolution and Public-user hardening stay centralized.

Public Role Caching

let publicRoleCache = null;

// Fetched on module load (startup)
async function fetchAndCachePublicRole() {
  publicRoleCache = await RolesDBApi.findBy({ name: 'Public' });
}

// Called immediately when module is imported
fetchAndCachePublicRole();

Function: checkPermissions(permission)

Creates middleware that checks if user has specific permission.

Permission Check Flow:

1. AccessPolicy.hasPermission(user, permission)
   ├── Public users are always denied admin API permissions
   └── Internal users use app_role.permissions + custom_permissions

2. Public role fallback
   └── Unauthenticated/no-role requests use cached Public role, but Public role permissions are ignored

3. Role lacks permission → 403 Forbidden

Self-access bypass is not part of checkPermissions. It is explicitly limited to GET, PUT, and PATCH on the authenticated user's own /api/users/:id route in checkCrudPermissions.

Usage:

const { checkPermissions } = require('./middlewares/check-permissions');

// Check specific permission
router.get('/admin', checkPermissions('ADMIN_ACCESS'), handler);

// Check entity permission
router.get('/users', checkPermissions('READ_USERS'), handler);

Error Response (403):

{
  "message": "Forbidden"
}

Function: checkCrudPermissions(name)

Creates middleware that maps HTTP method to CRUD permission.

Method Mapping:

HTTP Method	Permission Prefix
`POST`	`CREATE_`
`GET`	`READ_`
`PUT`	`UPDATE_`
`PATCH`	`UPDATE_`
`DELETE`	`DELETE_`

Permission Name Format: {METHOD}_{ENTITY}

Examples:

GET /api/users → READ_USERS
POST /api/projects → CREATE_PROJECTS
DELETE /api/assets/123 → DELETE_ASSETS

Routes can set req.permissionNameOverride before checkCrudPermissions when the HTTP verb does not describe the domain operation. The middleware uses the override as the exact permission name and otherwise falls back to {METHOD}_{ENTITY}. For example, environment-level resets for project runtime settings use DELETE to remove an override row, but the user-facing operation is "use inherited defaults", so those routes require UPDATE_PAGE_ELEMENTS rather than DELETE_PAGE_ELEMENTS.

Usage:

const { checkCrudPermissions } = require('./middlewares/check-permissions');

// In router factory
router.get('/', checkCrudPermissions('users'), listHandler);
router.post('/', checkCrudPermissions('users'), createHandler);
router.delete('/:id', checkCrudPermissions('users'), deleteHandler);

// For reset/update semantics implemented as DELETE
router.use((req, _res, next) => {
  if (req.method === 'DELETE' && req.path.startsWith('/project/')) {
    req.permissionNameOverride = 'UPDATE_PAGE_ELEMENTS';
  }
  next();
});
router.use(checkCrudPermissions('page_elements'));

Runtime Public Read Bypass

Certain entities allow public read access in production runtime:

const RUNTIME_PUBLIC_READ_ENTITIES = new Set([
  'PROJECTS',
  'TOUR_PAGES',
  'PAGE_ELEMENTS',
  'PAGE_LINKS',
  'TRANSITIONS',
  'PROJECT_AUDIO_TRACKS',
  'GLOBAL_TRANSITION_DEFAULTS',
  'PROJECT_TRANSITION_SETTINGS',
]);

// Bypass permission check for public runtime reads
const isRuntimePublicRead =
  req.isRuntimePublicRequest === true &&
  req.method === 'GET' &&
  RUNTIME_PUBLIC_READ_ENTITIES.has(name.toUpperCase());

if (isRuntimePublicRead) {
  return next(); // Skip permission check
}

⚠️ Middleware Ordering Requirement:

For public read bypass to work, the middleware that sets req.isRuntimePublicRequest = true MUST run before checkCrudPermissions. Common mistake:

// ❌ WRONG - allowPublicRead runs AFTER checkCrudPermissions
router.use(checkCrudPermissions('entity'));
router.get('/', allowPublicRead, handler);  // Too late!

// ✅ CORRECT - allowPublicRead runs BEFORE checkCrudPermissions
router.use(allowPublicRead);
router.use(checkCrudPermissions('entity'));
router.get('/', handler);

When using router.use(), middleware is applied to ALL routes before route-specific middleware runs.

3. runtime-context.ts (34 lines)

Middleware that extracts runtime environment context from request headers.

Function: runtimeContextMiddleware

Reads environment and project slug from headers for route-based access.

Headers:

Header	Values	Description
`X-Runtime-Environment`	`production`, `stage`, `dev`	Content environment
`X-Runtime-Project-Slug`	string	Project identifier

Context Object:

req.runtimeContext = {
  mode: 'admin',              // Default mode
  projectSlug: null,          // Extracted from path or header
  headerEnvironment: 'production',  // From X-Runtime-Environment
  headerProjectSlug: 'my-tour'      // From X-Runtime-Project-Slug
};

Usage in Routes:

// index.js
app.use(runtimeContextMiddleware);

// Access in handlers
const env = req.runtimeContext?.headerEnvironment;
if (env === 'production') {
  // Filter for production content
}

Route-Based Environment Access:

Route	Environment	Access
`/p/[slug]`	`production`	Public (no auth)
`/p/[slug]/stage`	`stage`	Authenticated only
`/constructor?projectId=`	`dev`	Authenticated only

4. runtime-public.ts (200 lines)

Middleware for controlling public runtime access and sanitizing responses.

Allowed Fields (Whitelist)

Only these fields are returned for public runtime requests:

const PUBLIC_RUNTIME_ENTITY_FIELDS = {
  projects: [
    'id', 'name', 'slug', 'description', 'logo_url', 'favicon_url', 'og_image_url',
  ],
  tour_pages: [
    'id', 'projectId', 'environment', 'source_key', 'name', 'slug',
    'sort_order', 'background_image_url', 'background_video_url',
    'background_audio_url', 'background_loop', 'requires_auth', 'ui_schema_json',
  ],
  project_audio_tracks: [
    'id', 'projectId', 'environment', 'source_key', 'name', 'slug',
    'url', 'loop', 'volume', 'sort_order', 'is_enabled',
  ],
};

Function: blockNonPublicRuntimeListEndpoints

Restricts public runtime requests to list endpoints only.

const blockNonPublicRuntimeListEndpoints = (req, res, next) => {
  if (!isPublicRuntimeReadRequest(req)) {
    return next(); // Not a public request, continue
  }

  // Only allow root path (list endpoint)
  if (req.path !== '/') {
    return res.status(404).send({ message: 'Not found' });
  }

  // Block CSV exports
  if (req.query.filetype === 'csv') {
    return res.status(404).send({ message: 'Not found' });
  }

  return next();
};

Blocked:

Individual record access: GET /api/projects/123 → 404
CSV exports: GET /api/projects?filetype=csv → 404

Allowed:

List endpoints: GET /api/projects/ → Continue

Function: sanitizePublicRuntimeListResponse(entityName)

Filters response data to only include whitelisted fields.

const sanitizePublicRuntimeListResponse = (entityName) => {
  const fields = PUBLIC_RUNTIME_ENTITY_FIELDS[entityName] || [];

  return (req, res, next) => {
    // Intercept res.send()
    const originalSend = res.send.bind(res);

    res.send = (body) => {
      if (Array.isArray(body.rows)) {
        const sanitizedRows = body.rows.map((row) => pickFields(row, fields));
        return originalSend({ ...body, rows: sanitizedRows });
      }
      return originalSend(body);
    };

    return next();
  };
};

Before Sanitization:

{
  "rows": [{
    "id": "123",
    "name": "My Tour",
    "slug": "my-tour",
    "createdAt": "2024-01-01",
    "createdById": "user-456",
    "internalNotes": "sensitive data"
  }]
}

After Sanitization:

{
  "rows": [{
    "id": "123",
    "name": "My Tour",
    "slug": "my-tour"
  }]
}

Usage in index.js

const mountRuntimeEntityRoute = (path, entityName, router) => {
  app.use(
    path,
    requireRuntimeReadOrAuth,           // JWT or public access
    blockNonPublicRuntimeListEndpoints, // Block non-list endpoints
    sanitizePublicRuntimeListResponse(entityName), // Filter fields
    router,
  );
};

mountRuntimeEntityRoute('/api/projects', 'projects', projectsRoutes);
mountRuntimeEntityRoute('/api/tour_pages', 'tour_pages', tour_pagesRoutes);
mountRuntimeEntityRoute('/api/project_audio_tracks', 'project_audio_tracks', ...);

5. upload.ts (34 lines)

Simple Multer-based file upload middleware.

const util = require('util');
const Multer = require('multer');

let processFile = Multer({
  storage: Multer.memoryStorage(),
}).single('file');

let processFileMiddleware = util.promisify(processFile);
module.exports = processFileMiddleware;

Configuration:

Setting	Value
Storage	Memory (Buffer)
Field Name	`file`
Max Files	1 (single)

Usage:

const upload = require('./middlewares/upload');

router.post('/upload', async (req, res) => {
  await upload(req, res);
  // req.file contains:
  // - buffer: File data
  // - originalname: Original filename
  // - mimetype: MIME type
  // - size: File size in bytes
});

Note: This middleware is primarily used for legacy uploads. The main file upload system uses chunked uploads without this middleware.

Middleware Composition Patterns

Pattern 1: Route-Level Rate Limiting

// Apply limiter before route handler
app.use('/api/file/upload', uploadLimiter);
app.use('/api/file', fileRoutes);

Pattern 2: Inline Middleware Chain

// Multiple middlewares in route definition
router.post(
  '/signin/local',
  signinLimiter,           // Rate limit
  wrapAsync(async (req, res) => { ... }),
);

Pattern 3: JWT + Feature Middleware

// JWT auth + rate limit + routes
app.use('/api/search', jwtAuth, searchLimiter, searchRoutes);

Pattern 4: Conditional Auth (Runtime)

const requireRuntimeReadOrAuth = (req, res, next) => {
  const headerEnvironment = req.runtimeContext?.headerEnvironment;
  const isReadOnlyRequest = ['GET', 'OPTIONS'].includes(req.method);
  const hasAuthHeader = Boolean(req.headers.authorization);
  const isPublicEnvironment = headerEnvironment === 'production';

  if (isPublicEnvironment && isReadOnlyRequest && !hasAuthHeader) {
    req.isRuntimePublicRequest = true;
    return next(); // Allow without auth
  }

  req.isRuntimePublicRequest = false;
  return jwtAuth(req, res, next); // Require auth
};

Pattern 5: Response Interception

// Intercept and modify response before sending
const sanitizeResponse = (req, res, next) => {
  const originalSend = res.send.bind(res);

  res.send = (body) => {
    const modified = transformBody(body);
    return originalSend(modified);
  };

  return next();
};

Request Flow Examples

Example 1: Authenticated API Request

GET /api/users
Authorization: Bearer <jwt>

1. helmet() → Security headers
2. cors() → CORS headers
3. requestLogger → Log request
4. bodyParser.json() → Parse body
5. runtimeContextMiddleware → Set req.runtimeContext
6. jwtAuth → Validate JWT, set req.currentUser
7. checkCrudPermissions('users') → Check READ_USERS permission
8. Route handler → Return users

Example 2: Public Runtime Request

GET /api/projects
X-Runtime-Environment: production

1. helmet() → Security headers
2. cors() → CORS headers
3. requestLogger → Log request
4. bodyParser.json() → Parse body
5. runtimeContextMiddleware → Set req.runtimeContext.headerEnvironment = 'production'
6. requireRuntimeReadOrAuth → Set req.isRuntimePublicRequest = true, skip JWT
7. blockNonPublicRuntimeListEndpoints → Allow (path is '/')
8. sanitizePublicRuntimeListResponse('projects') → Filter response fields
9. checkCrudPermissions('projects') → Skip (isRuntimePublicRequest)
10. Route handler → Return sanitized projects

Example 3: Rate Limited Upload

POST /api/file/upload
Authorization: Bearer <jwt>
Content-Type: multipart/form-data

1. uploadLimiter → Check rate limit (10/min)
   ├── Under limit → Continue
   └── Over limit → 429 Too Many Requests
2. fileRoutes handles request (own body parsing)

Error Handling

Rate Limit Errors

// 429 Too Many Requests
{
  "error": "Too Many Requests",
  "message": "Too many requests. Please try again later.",
  "retryAfter": 300
}

Permission Errors

// 403 Forbidden (via ValidationError)
{
  "message": "Role 'User' denied access to 'DELETE_USERS'."
}

Public Access Errors

// 404 Not Found (blocked endpoint)
{
  "message": "Not found"
}

Configuration

Environment Variables

Variable	Affects	Description
`NODE_ENV`	Rate limiting	Skip localhost in development

Constants

// rateLimiter.js
const CLEANUP_INTERVAL_MS = 5 * 60 * 1000; // 5 minutes

// check-permissions.ts
const METHOD_MAP = {
  POST: 'CREATE',
  GET: 'READ',
  PUT: 'UPDATE',
  PATCH: 'UPDATE',
  DELETE: 'DELETE',
};

const RUNTIME_PUBLIC_READ_ENTITIES = new Set([
  'PROJECTS', 'TOUR_PAGES', 'PAGE_ELEMENTS',
  'PAGE_LINKS', 'TRANSITIONS', 'PROJECT_AUDIO_TRACKS',
]);

// runtime-public.ts
const PUBLIC_RUNTIME_ALLOWED_PATH = '/';

Dependencies

Package	Version	Purpose
`multer`	^1.4.5	Multipart form data parsing
`util`	built-in	Promisify multer

Internal Dependencies:

../utils/logger - Pino logger for rate limit logging
../services/notifications/errors/validation - ValidationError class
../db/api/roles - RolesDBApi for Public role

Security Considerations

Rate Limiting: Prevents brute force and DoS attacks
Permission Checking: RBAC with role hierarchy
Public Role Fallback: Unauthenticated users get minimal permissions
Response Sanitization: Prevents data leakage in public runtime
Self-Access Bypass: Users can always access their own resources
Memory Store: Not suitable for horizontal scaling (use Redis)

Testing

Test Rate Limiting

# Should succeed (under limit)
for i in {1..10}; do
  curl -X POST http://localhost:3000/api/auth/signin/local \
    -H "Content-Type: application/json" \
    -d '{"email": "test@test.com", "password": "wrong"}'
done

# Should return 429 (over limit)
curl -X POST http://localhost:3000/api/auth/signin/local \
  -H "Content-Type: application/json" \
  -d '{"email": "test@test.com", "password": "wrong"}'

Test Public Runtime Access

# Should return sanitized projects
curl http://localhost:3000/api/projects \
  -H "X-Runtime-Environment: production"

# Should return 404 (individual record blocked)
curl http://localhost:3000/api/projects/123 \
  -H "X-Runtime-Environment: production"

Test Permission Check

# Should return 403 if user lacks permission
curl http://localhost:3000/api/users \
  -H "Authorization: Bearer <limited_user_jwt>"

Summary

The Middleware module provides:

rateLimiter.js - 8 pre-configured rate limiters with in-memory store
check-permissions.ts - RBAC through AccessPolicy with user-route-only self access
runtime-context.ts - Runtime environment context from headers
runtime-public.ts - Public access control and response sanitization
upload.ts - Simple Multer-based file upload

Key Features:

Configurable rate limiting per endpoint type
Role-based permission checking with method-to-CRUD mapping
Public runtime access for production presentations
Response field filtering for public access
Memory-based storage (scales vertically, needs Redis for horizontal)

29 KiB Raw Blame History

Backend Middleware Module Documentation

Overview

Architecture Diagram

File Details

1. rateLimiter.js (268 lines)

Storage Architecture

Factory: createRateLimiter(options)

Factory: createAuthenticatedRateLimiter(options)

Pre-configured Limiters

Route Mapping

Development Mode

2. check-permissions.ts (194 lines)

Public Role Caching

Function: checkPermissions(permission)

Function: checkCrudPermissions(name)

Runtime Public Read Bypass

3. runtime-context.ts (34 lines)

Function: runtimeContextMiddleware

4. runtime-public.ts (200 lines)

Allowed Fields (Whitelist)

Function: blockNonPublicRuntimeListEndpoints

Function: sanitizePublicRuntimeListResponse(entityName)

Usage in index.js

5. upload.ts (34 lines)

Middleware Composition Patterns

Pattern 1: Route-Level Rate Limiting

Pattern 2: Inline Middleware Chain

Pattern 3: JWT + Feature Middleware

Pattern 4: Conditional Auth (Runtime)

Pattern 5: Response Interception

Request Flow Examples

Example 1: Authenticated API Request

Example 2: Public Runtime Request

Example 3: Rate Limited Upload

Error Handling

Rate Limit Errors

Permission Errors

Public Access Errors

Configuration

Environment Variables

Constants

Dependencies

Security Considerations

Testing

Test Rate Limiting

Test Public Runtime Access

Test Permission Check

Summary

29 KiB

Raw Blame History