# Backend Middleware Module Documentation

## Overview

The Middleware module provides cross-cutting concerns for the Express application including rate limiting, permission checking, runtime context management, file uploads, and public access control.

**Files:**
| File | Lines | Purpose |
|------|-------|---------|
| `src/middlewares/rateLimiter.js` | 268 | Configurable rate limiting with in-memory store |
| `src/middlewares/check-permissions.ts` | RBAC permission checking through AccessPolicy |
| `src/middlewares/runtime-context.ts` | 34 | Runtime environment context from headers |
| `src/middlewares/runtime-public.ts` | 200 | Public runtime access control and response sanitization |
| `src/middlewares/upload.ts` | 34 | Multer-based file upload handling |

---

## Architecture Diagram

```
┌──────────────────────────────────────────────────────────────────────┐
│                         Incoming Request                             │
└──────────────────────────────────┬───────────────────────────────────┘
                                   │
                                   ▼
┌──────────────────────────────────────────────────────────────────────┐
│                    Express Middleware Stack                          │
│                                                                      │
│  ┌────────────────────────────────────────────────────────────────┐  │
│  │ 1. helmet() - Security headers                                 │  │
│  └────────────────────────────────────────────────────────────────┘  │
│                                   │                                  │
│                                   ▼                                  │
│  ┌────────────────────────────────────────────────────────────────┐  │
│  │ 2. cors() - Cross-origin resource sharing                     │  │
│  └────────────────────────────────────────────────────────────────┘  │
│                                   │                                  │
│                                   ▼                                  │
│  ┌────────────────────────────────────────────────────────────────┐  │
│  │ 3. requestLogger - Pino HTTP logging                          │  │
│  └────────────────────────────────────────────────────────────────┘  │
│                                   │                                  │
│                                   ▼                                  │
│  ┌────────────────────────────────────────────────────────────────┐  │
│  │ 4. Rate Limiters (route-specific)                             │  │
│  │    • downloadLimiter → /api/file/download, /api/file/presign  │  │
│  │    • uploadLimiter → /api/file/upload                         │  │
│  │    • searchLimiter → /api/search                              │  │
│  │    • authLimiter → /api/auth/signin                           │  │
│  └────────────────────────────────────────────────────────────────┘  │
│                                   │                                  │
│                                   ▼                                  │
│  ┌────────────────────────────────────────────────────────────────┐  │
│  │ 5. bodyParser.json() - JSON body parsing (after file routes)  │  │
│  └────────────────────────────────────────────────────────────────┘  │
│                                   │                                  │
│                                   ▼                                  │
│  ┌────────────────────────────────────────────────────────────────┐  │
│  │ 6. runtimeContextMiddleware - Environment context             │  │
│  │    Reads: X-Runtime-Environment, X-Runtime-Project-Slug       │  │
│  └────────────────────────────────────────────────────────────────┘  │
│                                   │                                  │
│                                   ▼                                  │
│  ┌────────────────────────────────────────────────────────────────┐  │
│  │ 7. JWT Authentication (route-specific)                        │  │
│  │    passport.authenticate('jwt', { session: false })           │  │
│  └────────────────────────────────────────────────────────────────┘  │
│                                   │                                  │
│                                   ▼                                  │
│  ┌────────────────────────────────────────────────────────────────┐  │
│  │ 8. Runtime Public Middleware (route-specific)                 │  │
│  │    • blockNonPublicRuntimeListEndpoints                       │  │
│  │    • sanitizePublicRuntimeListResponse                        │  │
│  └────────────────────────────────────────────────────────────────┘  │
│                                   │                                  │
│                                   ▼                                  │
│  ┌────────────────────────────────────────────────────────────────┐  │
│  │ 9. checkCrudPermissions / checkPermissions (route-specific)   │  │
│  └────────────────────────────────────────────────────────────────┘  │
│                                   │                                  │
│                                   ▼                                  │
│  ┌────────────────────────────────────────────────────────────────┐  │
│  │ 10. Route Handler                                             │  │
│  └────────────────────────────────────────────────────────────────┘  │
│                                   │                                  │
│                                   ▼                                  │
│  ┌────────────────────────────────────────────────────────────────┐  │
│  │ 11. Error Handler (commonErrorHandler)                        │  │
│  └────────────────────────────────────────────────────────────────┘  │
└──────────────────────────────────────────────────────────────────────┘
```

---

## File Details

### 1. rateLimiter.js (268 lines)

In-memory rate limiting middleware with configurable windows and limits.

#### Storage Architecture

```javascript
// In-memory store (Map)
const rateLimitStore = new Map();

// Entry structure
{
  count: number,      // Request count in window
  expiresAt: number,  // Window expiration timestamp
  resetTime: string   // ISO timestamp for headers
}

// Automatic cleanup every 5 minutes
setInterval(() => {
  for (const [key, entry] of rateLimitStore.entries()) {
    if (entry.expiresAt <= now) {
      rateLimitStore.delete(key);
    }
  }
}, 5 * 60 * 1000);
```

#### Factory: createRateLimiter(options)

Creates a configurable rate limiter middleware.

**Parameters:**
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `keyPrefix` | string | `'rate-limit'` | Prefix for rate limit keys |
| `windowMs` | number | `900000` (15min) | Time window in milliseconds |
| `max` | number | `100` | Maximum requests per window |
| `message` | string | `'Too many requests...'` | Error message on limit |
| `skipFailedRequests` | boolean | `false` | Don't count 4xx/5xx responses |
| `keyGenerator` | function | `null` | Custom key generator `(req) => string` |
| `skip` | function | `null` | Skip rate limiting `(req) => boolean` |

**Returns:** Express middleware function

**Response Headers:**
```
X-RateLimit-Limit: 100
X-RateLimit-Remaining: 99
X-RateLimit-Reset: 2024-01-01T00:15:00.000Z
Retry-After: 300  (only when limit exceeded)
```

**Rate Limit Exceeded Response (429):**
```json
{
  "error": "Too Many Requests",
  "message": "Too many requests. Please try again later.",
  "retryAfter": 300
}
```

#### Factory: createAuthenticatedRateLimiter(options)

Creates rate limiter that uses IP + User ID as key.

```javascript
const createAuthenticatedRateLimiter = (options = {}) => {
  return createRateLimiter({
    ...options,
    keyGenerator: (req) => {
      const userId = req.currentUser?.id || 'anonymous';
      const ip = req.ip || 'unknown';
      return `${ip}:${userId}`;
    },
  });
};
```

#### Pre-configured Limiters

| Limiter | Key Prefix | Window | Max | Skip Failed | Use Case |
|---------|------------|--------|-----|-------------|----------|
| `authLimiter` | `auth` | 15 min | 10 | No | Login attempts |
| `passwordResetLimiter` | `password-reset` | 1 hour | 5 | No | Password reset |
| `apiLimiter` | `api` | 1 min | 100 | Yes | General API |
| `uploadLimiter` | `upload` | 1 min | 10 | No | File uploads |
| `downloadLimiter` | `download` | 1 min | 200 | Yes | File downloads |
| `searchLimiter` | `search` | 1 min | 30 | No | Search queries |

#### Route Mapping

```javascript
// index.js
app.use('/api/file/download', downloadLimiter);
app.use('/api/file/presign', downloadLimiter);
app.use('/api/file/upload', uploadLimiter);
app.use('/api/file/upload-sessions', uploadLimiter);
app.use('/api/search', jwtAuth, searchLimiter, searchRoutes);

// routes/auth.js
router.post('/signin/local', signinLimiter, handler);
router.post('/send-password-reset-email', passwordResetLimiter, handler);
```

#### Development Mode

Rate limiting is skipped for localhost in development:

```javascript
if (
  config.server.env === 'development' &&
  (req.ip === '127.0.0.1' || req.ip === '::1')
) {
  return next(); // Skip rate limiting
}
```

---

### 2. check-permissions.ts (194 lines)

Role-based access control (RBAC) middleware. Permission decisions are delegated
to `src/services/access-policy.ts` so role/custom permission resolution and
Public-user hardening stay centralized.

#### Public Role Caching

```javascript
let publicRoleCache = null;

// Fetched on module load (startup)
async function fetchAndCachePublicRole() {
  publicRoleCache = await RolesDBApi.findBy({ name: 'Public' });
}

// Called immediately when module is imported
fetchAndCachePublicRole();
```

#### Function: checkPermissions(permission)

Creates middleware that checks if user has specific permission.

**Permission Check Flow:**
```
1. AccessPolicy.hasPermission(user, permission)
   ├── Public users are always denied admin API permissions
   └── Internal users use app_role.permissions + custom_permissions

2. Public role fallback
   └── Unauthenticated/no-role requests use cached Public role, but Public role permissions are ignored

3. Role lacks permission → 403 Forbidden
```

Self-access bypass is not part of `checkPermissions`. It is explicitly limited
to `GET`, `PUT`, and `PATCH` on the authenticated user's own `/api/users/:id`
route in `checkCrudPermissions`.

**Usage:**
```javascript
const { checkPermissions } = require('./middlewares/check-permissions');

// Check specific permission
router.get('/admin', checkPermissions('ADMIN_ACCESS'), handler);

// Check entity permission
router.get('/users', checkPermissions('READ_USERS'), handler);
```

**Error Response (403):**
```json
{
  "message": "Forbidden"
}
```

#### Function: checkCrudPermissions(name)

Creates middleware that maps HTTP method to CRUD permission.

**Method Mapping:**
| HTTP Method | Permission Prefix |
|-------------|-------------------|
| `POST` | `CREATE_` |
| `GET` | `READ_` |
| `PUT` | `UPDATE_` |
| `PATCH` | `UPDATE_` |
| `DELETE` | `DELETE_` |

**Permission Name Format:** `{METHOD}_{ENTITY}`

Examples:
- `GET /api/users` → `READ_USERS`
- `POST /api/projects` → `CREATE_PROJECTS`
- `DELETE /api/assets/123` → `DELETE_ASSETS`

Routes can set `req.permissionNameOverride` before `checkCrudPermissions` when
the HTTP verb does not describe the domain operation. The middleware uses the
override as the exact permission name and otherwise falls back to
`{METHOD}_{ENTITY}`. For example, environment-level resets for project runtime
settings use `DELETE` to remove an override row, but the user-facing operation
is "use inherited defaults", so those routes require `UPDATE_PAGE_ELEMENTS`
rather than `DELETE_PAGE_ELEMENTS`.

**Usage:**
```javascript
const { checkCrudPermissions } = require('./middlewares/check-permissions');

// In router factory
router.get('/', checkCrudPermissions('users'), listHandler);
router.post('/', checkCrudPermissions('users'), createHandler);
router.delete('/:id', checkCrudPermissions('users'), deleteHandler);

// For reset/update semantics implemented as DELETE
router.use((req, _res, next) => {
  if (req.method === 'DELETE' && req.path.startsWith('/project/')) {
    req.permissionNameOverride = 'UPDATE_PAGE_ELEMENTS';
  }
  next();
});
router.use(checkCrudPermissions('page_elements'));
```

#### Runtime Public Read Bypass

Certain entities allow public read access in production runtime:

```javascript
const RUNTIME_PUBLIC_READ_ENTITIES = new Set([
  'PROJECTS',
  'TOUR_PAGES',
  'PAGE_ELEMENTS',
  'PAGE_LINKS',
  'TRANSITIONS',
  'PROJECT_AUDIO_TRACKS',
  'GLOBAL_TRANSITION_DEFAULTS',
  'PROJECT_TRANSITION_SETTINGS',
]);

// Bypass permission check for public runtime reads
const isRuntimePublicRead =
  req.isRuntimePublicRequest === true &&
  req.method === 'GET' &&
  RUNTIME_PUBLIC_READ_ENTITIES.has(name.toUpperCase());

if (isRuntimePublicRead) {
  return next(); // Skip permission check
}
```

**⚠️ Middleware Ordering Requirement:**

For public read bypass to work, the middleware that sets `req.isRuntimePublicRequest = true` **MUST run before** `checkCrudPermissions`. Common mistake:

```javascript
// ❌ WRONG - allowPublicRead runs AFTER checkCrudPermissions
router.use(checkCrudPermissions('entity'));
router.get('/', allowPublicRead, handler);  // Too late!

// ✅ CORRECT - allowPublicRead runs BEFORE checkCrudPermissions
router.use(allowPublicRead);
router.use(checkCrudPermissions('entity'));
router.get('/', handler);
```

When using `router.use()`, middleware is applied to ALL routes before route-specific middleware runs.

---

### 3. runtime-context.ts (34 lines)

Middleware that extracts runtime environment context from request headers.

#### Function: runtimeContextMiddleware

Reads environment and project slug from headers for route-based access.

**Headers:**
| Header | Values | Description |
|--------|--------|-------------|
| `X-Runtime-Environment` | `production`, `stage`, `dev` | Content environment |
| `X-Runtime-Project-Slug` | string | Project identifier |

**Context Object:**
```javascript
req.runtimeContext = {
  mode: 'admin',              // Default mode
  projectSlug: null,          // Extracted from path or header
  headerEnvironment: 'production',  // From X-Runtime-Environment
  headerProjectSlug: 'my-tour'      // From X-Runtime-Project-Slug
};
```

**Usage in Routes:**
```javascript
// index.js
app.use(runtimeContextMiddleware);

// Access in handlers
const env = req.runtimeContext?.headerEnvironment;
if (env === 'production') {
  // Filter for production content
}
```

**Route-Based Environment Access:**
| Route | Environment | Access |
|-------|-------------|--------|
| `/p/[slug]` | `production` | Public (no auth) |
| `/p/[slug]/stage` | `stage` | Authenticated only |
| `/constructor?projectId=` | `dev` | Authenticated only |

---

### 4. runtime-public.ts (200 lines)

Middleware for controlling public runtime access and sanitizing responses.

#### Allowed Fields (Whitelist)

Only these fields are returned for public runtime requests:

```javascript
const PUBLIC_RUNTIME_ENTITY_FIELDS = {
  projects: [
    'id', 'name', 'slug', 'description', 'logo_url', 'favicon_url', 'og_image_url',
  ],
  tour_pages: [
    'id', 'projectId', 'environment', 'source_key', 'name', 'slug',
    'sort_order', 'background_image_url', 'background_video_url',
    'background_audio_url', 'background_loop', 'requires_auth', 'ui_schema_json',
  ],
  project_audio_tracks: [
    'id', 'projectId', 'environment', 'source_key', 'name', 'slug',
    'url', 'loop', 'volume', 'sort_order', 'is_enabled',
  ],
};
```

#### Function: blockNonPublicRuntimeListEndpoints

Restricts public runtime requests to list endpoints only.

```javascript
const blockNonPublicRuntimeListEndpoints = (req, res, next) => {
  if (!isPublicRuntimeReadRequest(req)) {
    return next(); // Not a public request, continue
  }

  // Only allow root path (list endpoint)
  if (req.path !== '/') {
    return res.status(404).send({ message: 'Not found' });
  }

  // Block CSV exports
  if (req.query.filetype === 'csv') {
    return res.status(404).send({ message: 'Not found' });
  }

  return next();
};
```

**Blocked:**
- Individual record access: `GET /api/projects/123` → 404
- CSV exports: `GET /api/projects?filetype=csv` → 404

**Allowed:**
- List endpoints: `GET /api/projects/` → Continue

#### Function: sanitizePublicRuntimeListResponse(entityName)

Filters response data to only include whitelisted fields.

```javascript
const sanitizePublicRuntimeListResponse = (entityName) => {
  const fields = PUBLIC_RUNTIME_ENTITY_FIELDS[entityName] || [];

  return (req, res, next) => {
    // Intercept res.send()
    const originalSend = res.send.bind(res);

    res.send = (body) => {
      if (Array.isArray(body.rows)) {
        const sanitizedRows = body.rows.map((row) => pickFields(row, fields));
        return originalSend({ ...body, rows: sanitizedRows });
      }
      return originalSend(body);
    };

    return next();
  };
};
```

**Before Sanitization:**
```json
{
  "rows": [{
    "id": "123",
    "name": "My Tour",
    "slug": "my-tour",
    "createdAt": "2024-01-01",
    "createdById": "user-456",
    "internalNotes": "sensitive data"
  }]
}
```

**After Sanitization:**
```json
{
  "rows": [{
    "id": "123",
    "name": "My Tour",
    "slug": "my-tour"
  }]
}
```

#### Usage in index.js

```javascript
const mountRuntimeEntityRoute = (path, entityName, router) => {
  app.use(
    path,
    requireRuntimeReadOrAuth,           // JWT or public access
    blockNonPublicRuntimeListEndpoints, // Block non-list endpoints
    sanitizePublicRuntimeListResponse(entityName), // Filter fields
    router,
  );
};

mountRuntimeEntityRoute('/api/projects', 'projects', projectsRoutes);
mountRuntimeEntityRoute('/api/tour_pages', 'tour_pages', tour_pagesRoutes);
mountRuntimeEntityRoute('/api/project_audio_tracks', 'project_audio_tracks', ...);
```

---

### 5. upload.ts (34 lines)

Simple Multer-based file upload middleware.

```javascript
const util = require('util');
const Multer = require('multer');

let processFile = Multer({
  storage: Multer.memoryStorage(),
}).single('file');

let processFileMiddleware = util.promisify(processFile);
module.exports = processFileMiddleware;
```

**Configuration:**
| Setting | Value |
|---------|-------|
| Storage | Memory (Buffer) |
| Field Name | `file` |
| Max Files | 1 (single) |

**Usage:**
```javascript
const upload = require('./middlewares/upload');

router.post('/upload', async (req, res) => {
  await upload(req, res);
  // req.file contains:
  // - buffer: File data
  // - originalname: Original filename
  // - mimetype: MIME type
  // - size: File size in bytes
});
```

**Note:** This middleware is primarily used for legacy uploads. The main file upload system uses chunked uploads without this middleware.

---

## Middleware Composition Patterns

### Pattern 1: Route-Level Rate Limiting

```javascript
// Apply limiter before route handler
app.use('/api/file/upload', uploadLimiter);
app.use('/api/file', fileRoutes);
```

### Pattern 2: Inline Middleware Chain

```javascript
// Multiple middlewares in route definition
router.post(
  '/signin/local',
  signinLimiter,           // Rate limit
  wrapAsync(async (req, res) => { ... }),
);
```

### Pattern 3: JWT + Feature Middleware

```javascript
// JWT auth + rate limit + routes
app.use('/api/search', jwtAuth, searchLimiter, searchRoutes);
```

### Pattern 4: Conditional Auth (Runtime)

```javascript
const requireRuntimeReadOrAuth = (req, res, next) => {
  const headerEnvironment = req.runtimeContext?.headerEnvironment;
  const isReadOnlyRequest = ['GET', 'OPTIONS'].includes(req.method);
  const hasAuthHeader = Boolean(req.headers.authorization);
  const isPublicEnvironment = headerEnvironment === 'production';

  if (isPublicEnvironment && isReadOnlyRequest && !hasAuthHeader) {
    req.isRuntimePublicRequest = true;
    return next(); // Allow without auth
  }

  req.isRuntimePublicRequest = false;
  return jwtAuth(req, res, next); // Require auth
};
```

### Pattern 5: Response Interception

```javascript
// Intercept and modify response before sending
const sanitizeResponse = (req, res, next) => {
  const originalSend = res.send.bind(res);

  res.send = (body) => {
    const modified = transformBody(body);
    return originalSend(modified);
  };

  return next();
};
```

---

## Request Flow Examples

### Example 1: Authenticated API Request

```
GET /api/users
Authorization: Bearer <jwt>

1. helmet() → Security headers
2. cors() → CORS headers
3. requestLogger → Log request
4. bodyParser.json() → Parse body
5. runtimeContextMiddleware → Set req.runtimeContext
6. jwtAuth → Validate JWT, set req.currentUser
7. checkCrudPermissions('users') → Check READ_USERS permission
8. Route handler → Return users
```

### Example 2: Public Runtime Request

```
GET /api/projects
X-Runtime-Environment: production

1. helmet() → Security headers
2. cors() → CORS headers
3. requestLogger → Log request
4. bodyParser.json() → Parse body
5. runtimeContextMiddleware → Set req.runtimeContext.headerEnvironment = 'production'
6. requireRuntimeReadOrAuth → Set req.isRuntimePublicRequest = true, skip JWT
7. blockNonPublicRuntimeListEndpoints → Allow (path is '/')
8. sanitizePublicRuntimeListResponse('projects') → Filter response fields
9. checkCrudPermissions('projects') → Skip (isRuntimePublicRequest)
10. Route handler → Return sanitized projects
```

### Example 3: Rate Limited Upload

```
POST /api/file/upload
Authorization: Bearer <jwt>
Content-Type: multipart/form-data

1. uploadLimiter → Check rate limit (10/min)
   ├── Under limit → Continue
   └── Over limit → 429 Too Many Requests
2. fileRoutes handles request (own body parsing)
```

---

## Error Handling

### Rate Limit Errors

```javascript
// 429 Too Many Requests
{
  "error": "Too Many Requests",
  "message": "Too many requests. Please try again later.",
  "retryAfter": 300
}
```

### Permission Errors

```javascript
// 403 Forbidden (via ValidationError)
{
  "message": "Role 'User' denied access to 'DELETE_USERS'."
}
```

### Public Access Errors

```javascript
// 404 Not Found (blocked endpoint)
{
  "message": "Not found"
}
```

---

## Configuration

### Environment Variables

| Variable | Affects | Description |
|----------|---------|-------------|
| `NODE_ENV` | Rate limiting | Skip localhost in development |

### Constants

```javascript
// rateLimiter.js
const CLEANUP_INTERVAL_MS = 5 * 60 * 1000; // 5 minutes

// check-permissions.ts
const METHOD_MAP = {
  POST: 'CREATE',
  GET: 'READ',
  PUT: 'UPDATE',
  PATCH: 'UPDATE',
  DELETE: 'DELETE',
};

const RUNTIME_PUBLIC_READ_ENTITIES = new Set([
  'PROJECTS', 'TOUR_PAGES', 'PAGE_ELEMENTS',
  'PAGE_LINKS', 'TRANSITIONS', 'PROJECT_AUDIO_TRACKS',
]);

// runtime-public.ts
const PUBLIC_RUNTIME_ALLOWED_PATH = '/';
```

---

## Dependencies

| Package | Version | Purpose |
|---------|---------|---------|
| `multer` | ^1.4.5 | Multipart form data parsing |
| `util` | built-in | Promisify multer |

**Internal Dependencies:**
- `../utils/logger` - Pino logger for rate limit logging
- `../services/notifications/errors/validation` - ValidationError class
- `../db/api/roles` - RolesDBApi for Public role

---

## Security Considerations

1. **Rate Limiting:** Prevents brute force and DoS attacks
2. **Permission Checking:** RBAC with role hierarchy
3. **Public Role Fallback:** Unauthenticated users get minimal permissions
4. **Response Sanitization:** Prevents data leakage in public runtime
5. **Self-Access Bypass:** Users can always access their own resources
6. **Memory Store:** Not suitable for horizontal scaling (use Redis)

---

## Testing

### Test Rate Limiting

```bash
# Should succeed (under limit)
for i in {1..10}; do
  curl -X POST http://localhost:3000/api/auth/signin/local \
    -H "Content-Type: application/json" \
    -d '{"email": "test@test.com", "password": "wrong"}'
done

# Should return 429 (over limit)
curl -X POST http://localhost:3000/api/auth/signin/local \
  -H "Content-Type: application/json" \
  -d '{"email": "test@test.com", "password": "wrong"}'
```

### Test Public Runtime Access

```bash
# Should return sanitized projects
curl http://localhost:3000/api/projects \
  -H "X-Runtime-Environment: production"

# Should return 404 (individual record blocked)
curl http://localhost:3000/api/projects/123 \
  -H "X-Runtime-Environment: production"
```

### Test Permission Check

```bash
# Should return 403 if user lacks permission
curl http://localhost:3000/api/users \
  -H "Authorization: Bearer <limited_user_jwt>"
```

---

## Summary

The Middleware module provides:

1. **rateLimiter.js** - 8 pre-configured rate limiters with in-memory store
2. **check-permissions.ts** - RBAC through AccessPolicy with user-route-only self access
3. **runtime-context.ts** - Runtime environment context from headers
4. **runtime-public.ts** - Public access control and response sanitization
5. **upload.ts** - Simple Multer-based file upload

**Key Features:**
- Configurable rate limiting per endpoint type
- Role-based permission checking with method-to-CRUD mapping
- Public runtime access for production presentations
- Response field filtering for public access
- Memory-based storage (scales vertically, needs Redis for horizontal)