admin/39948-vm
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
39948-vm/documentation/access-logs-audit-trail.md
Dmitri 9f111d6226 added documentation
2026-07-03 16:11:24 +02:00

24 KiB
Raw Blame History

Access Logs & Audit Trail

Complete documentation for the Tour Builder Platform's Access Logs and Audit Trail system including activity tracking, security auditing, and data preservation.

Overview

The platform implements a comprehensive audit system with two complementary mechanisms:

  • Access Logs - Dedicated table recording API requests with user, path, and client info
  • Audit Trail - CreatedBy/UpdatedBy fields on all entities tracking who modified what
┌─────────────────────────────────────────────────────────────────────────────┐
│                    Access Logs & Audit Trail Architecture                    │
│                                                                              │
│  ┌────────────────────────────────────────────────────────────────────────┐ │
│  │                        Request Flow                                     │ │
│  │                                                                         │ │
│  │   Client Request                                                        │ │
│  │        │                                                                │ │
│  │        ▼                                                                │ │
│  │   ┌─────────────────────────────────────────────────────────────────┐  │ │
│  │   │                    Request Logger Middleware                     │  │ │
│  │   │                                                                  │  │ │
│  │   │   • Generate/Extract X-Request-Id                               │  │ │
│  │   │   • Capture: method, path, user-agent, duration                 │  │ │
│  │   │   • Log to Pino (structured logging)                            │  │ │
│  │   └──────────────────────────┬──────────────────────────────────────┘  │ │
│  │                              │                                          │ │
│  │                              ▼                                          │ │
│  │   ┌─────────────────────────────────────────────────────────────────┐  │ │
│  │   │                    JWT Authentication                            │  │ │
│  │   │                                                                  │  │ │
│  │   │   • Validate token                                              │  │ │
│  │   │   • Attach currentUser to request                               │  │ │
│  │   └──────────────────────────┬──────────────────────────────────────┘  │ │
│  │                              │                                          │ │
│  │                              ▼                                          │ │
│  │   ┌─────────────────────────────────────────────────────────────────┐  │ │
│  │   │                    Permission Check                              │  │ │
│  │   │                                                                  │  │ │
│  │   │   • Check READ/CREATE/UPDATE/DELETE permissions                 │  │ │
│  │   │   • Role-based + custom permissions                             │  │ │
│  │   └──────────────────────────┬──────────────────────────────────────┘  │ │
│  │                              │                                          │ │
│  │                              ▼                                          │ │
│  │   ┌─────────────────────────────────────────────────────────────────┐  │ │
│  │   │                    Database Operation                            │  │ │
│  │   │                                                                  │  │ │
│  │   │   CREATE: set createdById, updatedById                          │  │ │
│  │   │   UPDATE: set updatedById                                       │  │ │
│  │   │   DELETE: set deletedBy, mark deletedAt (soft delete)           │  │ │
│  │   └─────────────────────────────────────────────────────────────────┘  │ │
│  └────────────────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────────┘

Data Model

Access Logs Schema

Table: access_logs

Source: backend/src/db/models/access_logs.js

Field Type Required Default Description
id UUID Yes UUIDv4 Primary key
projectId UUID (FK) No null Related project (CASCADE delete)
userId UUID (FK) No null User who accessed (SET NULL delete)
environment ENUM Yes - 'admin', 'stage', 'production'
path TEXT No null API path accessed (max 2048 chars)
ip_address TEXT No null Client IP address (max 45 chars for IPv6)
user_agent TEXT No null HTTP User-Agent header (max 1024 chars)
accessed_at TIMESTAMP Yes NOW When access occurred
importHash VARCHAR(255) No null Unique hash for CSV imports
createdAt TIMESTAMP Yes Auto Record creation time
updatedAt TIMESTAMP Yes Auto Record update time
deletedAt TIMESTAMP No null Soft delete timestamp

Indexes

indexes: [
  { fields: ['projectId'] },     // Query logs by project
  { fields: ['environment'] },   // Filter by environment
  { fields: ['userId'] },        // Query logs by user
  { fields: ['accessed_at'] },   // Time-range queries and sorting
]

Relationships

access_logs
├── belongsTo projects (as project)
│   ├── foreignKey: projectId
│   ├── onDelete: CASCADE        ← Logs deleted when project deleted
│   └── onUpdate: CASCADE
│
├── belongsTo users (as user)
│   ├── foreignKey: userId
│   ├── onDelete: CASCADE        ← Logs deleted when user deleted
│   └── onUpdate: CASCADE
│
├── belongsTo users (as createdBy) ─── Audit: who created this log
└── belongsTo users (as updatedBy) ─── Audit: who last updated this log

Audit Trail Pattern

CreatedBy/UpdatedBy Implementation

All entities in the system implement the audit trail pattern:

Source: backend/src/db/api/base.api.js

// On CREATE
static async create({ data, currentUser = { id: null }, transaction }) {
  const mappedData = this.getFieldMapping(data);

  const record = await this.MODEL.create({
    ...mappedData,
    createdById: currentUser.id,  // Track creator
    updatedById: currentUser.id,  // Initial updater = creator
  }, { transaction });
}

// On UPDATE
static async update({ id, data, currentUser = { id: null }, transaction }) {

  await record.update({
    ...updatePayload,
    updatedById: currentUser.id,  // Track who modified
  }, { transaction });
}

// On DELETE (soft delete)
static async remove({ id, currentUser = { id: null }, transaction }) {

  await record.update({ deletedBy: currentUser.id }, { transaction });
  await record.destroy({ transaction });  // Sets deletedAt
}

Entities with Audit Trail

All models include audit relationships:

// Applied to ALL models:
db.[entity].belongsTo(db.users, { as: 'createdBy' });
db.[entity].belongsTo(db.users, { as: 'updatedBy' });

Audited Entities:

  • users, roles, permissions
  • projects, project_memberships
  • tour_pages, project_audio_tracks
  • assets, asset_variants
  • element_type_defaults, project_element_defaults
  • publish_events, pwa_caches
  • presigned_url_requests, access_logs

Note: Page elements, navigation links, and transitions are stored in tour_pages.ui_schema_json and audited as part of the tour_pages record.

Timestamp Tracking

Sequelize provides automatic timestamps:

{
  timestamps: true,   // Enables createdAt, updatedAt
  paranoid: true,     // Enables deletedAt (soft delete)
}
Field Set When Never Changes
createdAt Record created ✓ Immutable
updatedAt Any modification Updates each save
deletedAt Soft delete called -

Request Logging Middleware

Logger Configuration

Source: backend/src/utils/logger.js

const logger = pino({
  level: process.env.LOG_LEVEL || 'info',
  transport: isDevelopment
    ? { target: 'pino-pretty', options: { colorize: true } }
    : undefined,
  base: {
    service: 'tour-builder-api',
    env: process.env.NODE_ENV || 'development',
  },
});

Request Logger Middleware

function requestLogger(req, res, next) {
  // Generate or extract request ID for tracing
  const requestId = req.headers['x-request-id'] || crypto.randomUUID();
  req.log = logger.child({ requestId });
  req.requestId = requestId;
  res.setHeader('X-Request-Id', requestId);

  const start = Date.now();

  res.on('finish', () => {
    const duration = Date.now() - start;
    const logData = {
      method: req.method,
      url: req.originalUrl || req.url,
      status: res.statusCode,
      duration,
      userAgent: req.headers['user-agent'],
    };

    // Log level based on response status
    if (res.statusCode >= 500) {
      req.log.error(logData, 'Request completed with server error');
    } else if (res.statusCode >= 400) {
      req.log.warn(logData, 'Request completed with client error');
    } else {
      req.log.info(logData, 'Request completed');
    }
  });

  next();
}

Applied in Application

Source: backend/src/index.ts

app.enable('trust proxy');  // Extract real IP behind proxies
app.use(requestLogger);     // Apply to all routes

Log Output Format

Development (pino-pretty):

[10:30:45.123] INFO: Request completed
    method: "GET"
    url: "/api/projects"
    status: 200
    duration: 45
    userAgent: "Mozilla/5.0..."
    requestId: "abc123-def456"

Production (JSON):

{
  "level": 30,
  "time": 1711234567890,
  "service": "tour-builder-api",
  "env": "production",
  "requestId": "abc123-def456",
  "method": "GET",
  "url": "/api/projects",
  "status": 200,
  "duration": 45,
  "userAgent": "Mozilla/5.0..."
}

API Reference

Endpoints

Source: backend/src/routes/access_logs.ts

Method Endpoint Permission Description
GET /api/access_logs READ_ACCESS_LOGS List with filtering
GET /api/access_logs/:id READ_ACCESS_LOGS Get single log
POST /api/access_logs CREATE_ACCESS_LOGS Create log entry
PUT /api/access_logs/:id UPDATE_ACCESS_LOGS Update log
DELETE /api/access_logs/:id DELETE_ACCESS_LOGS Soft delete log

Query Parameters

Source: backend/src/db/api/access_logs.ts

Parameter Type Description
limit number Results per page (default: 50)
page number Page number (0-based)
field string Sort field
sort 'asc' | 'desc' Sort direction
path string Search path (ILIKE)
ip_address string Search IP address (ILIKE)
user_agent string Search user agent (ILIKE)
accessed_atRange [start, end] Date range filter
environment enum Filter by environment
project UUID | string Filter by project
user UUID | string Filter by user
filetype 'csv' Export as CSV

Request/Response Examples

List Logs with Filters:

GET /api/access_logs?limit=50&page=1&environment=production&accessed_atRange=["2024-01-01","2024-12-31"]&sort=DESC&field=accessed_at
Authorization: Bearer <jwt-token>

Response:

{
  "rows": [
    {
      "id": "log-uuid-1",
      "projectId": "project-uuid",
      "userId": "user-uuid",
      "environment": "production",
      "path": "/api/tour_pages",
      "ip_address": "192.168.1.100",
      "user_agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64)...",
      "accessed_at": "2024-03-15T10:30:00Z",
      "project": {
        "id": "project-uuid",
        "name": "My Tour"
      },
      "user": {
        "id": "user-uuid",
        "firstName": "John",
        "lastName": "Doe"
      },
      "createdAt": "2024-03-15T10:30:00Z",
      "updatedAt": "2024-03-15T10:30:00Z"
    }
  ],
  "count": 1542
}

Create Log Entry:

POST /api/access_logs
Content-Type: application/json
Authorization: Bearer <jwt-token>

{
  "data": {
    "project": "project-uuid",
    "user": "user-uuid",
    "environment": "production",
    "path": "/api/tour_pages",
    "ip_address": "192.168.1.100",
    "user_agent": "Mozilla/5.0...",
    "accessed_at": "2024-03-15T10:30:00Z"
  }
}

CSV Export

Fields Exported:

  • id
  • environment
  • path
  • ip_address
  • user_agent
  • accessed_at
  • createdAt
GET /api/access_logs?filetype=csv
Authorization: Bearer <jwt-token>

Frontend Implementation

Admin Pages

Location: frontend/src/pages/access_logs/

Page Path Description
List /access_logs/access_logs-list Filterable table view
Table /access_logs/access_logs-table Data grid variant
New /access_logs/access_logs-new Create log entry
Edit /access_logs/access_logs-edit?id=<uuid> Edit log
View /access_logs/access_logs-view?id=<uuid> Read-only details
Dynamic /access_logs/[access_logsId] Dynamic route (ID in path)

Note: Uses useEditPageSync hook for edit page data synchronization.

List Page Filters

Source: frontend/src/pages/access_logs/access_logs-list.tsx

Filter Type Description
Path Text Search by API path
IP Address Text Search by client IP
User Agent Text Search by browser/client
Accessed At Date Range Filter by time range
Project Dropdown Filter by project
User Dropdown Filter by user
Environment Enum admin / stage / production

Table Columns

Source: frontend/src/components/Access_logs/configureAccess_logsCols.tsx

Uses createColumnLoader factory from configBuilderFactory.tsx:

Column Editable Type
project Yes singleSelectRelation (projects)
environment Yes text
user Yes singleSelectRelation (users)
path Yes text
ip_address Yes text
user_agent Yes text
accessed_at Yes datetime
actions - actions (Edit / View / Delete)

TypeScript Interface

Source: frontend/src/types/entities.ts

export interface AccessLog extends BaseEntity {
  user?: User | string | null;
  project?: Project | string | null;
  environment?: 'admin' | 'stage' | 'production';
  path?: string;
  action?: string;
  ip_address?: string;
  user_agent?: string;
  accessed_at?: string | Date;
  metadata?: Record<string, unknown>;
}

Redux State

Source: frontend/src/stores/access_logs/access_logsSlice.ts

Uses createEntitySlice factory for standardized CRUD operations:

const { slice, actions, reducer } = createEntitySlice<AccessLog>({
  name: 'access_logs',
  endpoint: 'access_logs',
  singularName: 'Access Log',
});

export const {
  fetch,           // Load logs with query
  create,          // Create new log
  update,          // Update existing log
  deleteItem,      // Delete single log
  deleteItemsByIds,// Batch delete
  uploadCsv,       // Import from CSV
  setRefetch,      // Trigger refresh
} = actions;

// Usage
dispatch(fetch({ query: '?environment=production&limit=100' }));

Security & Permissions

Role-Permission Matrix

Source: backend/src/db/seeders/20200430130760-user-roles.js

Role CREATE READ UPDATE DELETE
PlatformOwner ✓ ✓ ✓ ✓
Administrator ✓ ✓ ✓ ✓
AccountManager ✗ ✓ ✗ ✗
TourDesigner ✗ ✓ ✗ ✗
ContentReviewer ✗ ✓ ✗ ✗
AnalyticsViewer ✗ ✓ ✗ ✗

Permission Check Flow

Request → JWT Auth → checkCrudPermissions('access_logs') → Route Handler
                              │
                              ▼
              ┌───────────────────────────────┐
              │  1. AccessPolicy              │
              │  2. Custom user permissions   │
              │  3. Role-based permissions    │
              │  4. Public hardening          │
              └───────────────────────────────┘

Cascade Delete Behavior

Both user and project deletions cascade to access logs:

// access_logs -> users
onDelete: 'CASCADE'   // Logs deleted with user

// access_logs -> projects
onDelete: 'CASCADE'   // Logs deleted with project

Note: Access logs are NOT preserved when users or projects are deleted. If audit history retention is required, consider archiving logs before deletion or implementing SET NULL behavior.

Data Capture Scope

What IS Captured

Data Point Location Purpose
User ID access_logs.userId Who made the request
Project ID access_logs.projectId Which project context
API Path access_logs.path What endpoint accessed
IP Address access_logs.ip_address Client location/identity
User Agent access_logs.user_agent Browser/client identification
Timestamp access_logs.accessed_at When it happened
Environment access_logs.environment admin/stage/production

What IS NOT Captured

Data Point Reason
Request body Privacy protection
Response content Privacy/performance
Query parameters Only path stored
Authentication credentials Security
Personal user data Beyond ID reference

Audit Trail vs Access Logs

Aspect Audit Trail Access Logs
Scope All entities Dedicated table
Granularity Per-record changes Per-request
Data Who modified when Request details
Retention With entity Separate lifecycle
Purpose Change tracking Security monitoring

Soft Delete & Retention

Paranoid Mode

{
  timestamps: true,
  paranoid: true,  // Enables soft deletes
}

Deletion Behavior

  1. Soft Delete: Sets deletedAt timestamp
  2. Query Filtering: Normal queries exclude deleted records
  3. Restoration: Possible by clearing deletedAt
  4. Hard Delete: Requires explicit permanent removal

Retention Considerations

Current Implementation:

  • No automatic log pruning
  • Logs retained indefinitely
  • Manual cleanup via DELETE endpoints

Recommended Practices:

  • Implement TTL-based cleanup for old logs
  • Archive logs to cold storage after 90 days
  • Consider GDPR right-to-erasure requirements

File Structure

Backend

File Purpose
backend/src/db/models/access_logs.js Sequelize model definition
backend/src/db/api/access_logs.ts Database API (CRUD)
backend/src/db/api/base.api.js Audit trail implementation
backend/src/routes/access_logs.ts REST endpoints
backend/src/services/access_logs.ts Business logic
backend/src/utils/logger.js Request logging middleware
backend/src/middlewares/check-permissions.ts Permission enforcement

Frontend

File Purpose
frontend/src/pages/access_logs/access_logs-list.tsx List page
frontend/src/pages/access_logs/access_logs-table.tsx Table page
frontend/src/pages/access_logs/access_logs-new.tsx Create page
frontend/src/pages/access_logs/access_logs-edit.tsx Edit page
frontend/src/pages/access_logs/access_logs-view.tsx View page
frontend/src/pages/access_logs/[access_logsId].tsx Dynamic route
frontend/src/stores/access_logs/access_logsSlice.ts Redux state (createEntitySlice)
frontend/src/components/Access_logs/TableAccess_logs.tsx Data grid component
frontend/src/components/Access_logs/ListAccess_logs.tsx List view component
frontend/src/components/Access_logs/CardAccess_logs.tsx Card view component
frontend/src/components/Access_logs/configureAccess_logsCols.tsx Column config (createColumnLoader)
frontend/src/types/entities.ts TypeScript interfaces

Known Considerations

1. Manual Log Creation

Issue: Access logs are typically created manually via API, not automatically by middleware.

Impact: Requires explicit logging calls or additional middleware to auto-populate.

2. User Deletion Cascades

Issue: CASCADE on userId means logs are deleted when user is deleted.

Impact: Audit history lost with user deletion. Consider archiving logs before user deletion or changing to SET NULL for preservation.

3. Project Deletion Cascades

Issue: All access logs deleted when project is deleted.

Impact: Audit history lost with project. Consider archiving before deletion.

4. No Query Parameter Logging

Issue: Only path is stored, not query strings.

Impact: Cannot reconstruct full request URL. May miss filter/search context.

5. IPv6 Support

Issue: ip_address limited to 45 characters.

Impact: Supports full IPv6 addresses (39 chars max). Adequate for current needs.

6. User Agent Truncation

Issue: user_agent limited to 1024 characters.

Impact: Very long user agents may be truncated. Rare but possible.

7. No Automatic Retention

Issue: Logs grow indefinitely without cleanup.

Impact: Database size increases over time. Recommend implementing retention policy.

8. TypeScript Interface Extras

Issue: Frontend AccessLog interface includes action and metadata fields not in database model.

Impact: These fields exist only in TypeScript for potential future use. Currently not persisted.

Integration Examples

Log Access in Custom Middleware

import Access_logsDBApi from '../db/api/access_logs.ts';

async function logAccess(req, res, next) {
  // Log after response completes
  res.on('finish', async () => {
    try {
      await Access_logsDBApi.create({
        project: req.project?.id,
        user: req.currentUser?.id,
        environment: determineEnvironment(req),
        path: req.path,
        ip_address: req.ip,
        user_agent: req.headers['user-agent'],
        accessed_at: new Date(),
      }, { currentUser: req.currentUser });
    } catch (err) {
      // Don't fail request on logging error
      console.error('Failed to log access:', err);
    }
  });
  next();
}

Query Logs by Time Range

// Frontend
dispatch(fetch({
  query: '?accessed_atRange=["2024-01-01T00:00:00Z","2024-01-31T23:59:59Z"]&environment=production&sort=DESC&field=accessed_at'
}));

Export Logs for Compliance

curl -H "Authorization: Bearer $TOKEN" \
  "https://api.example.com/api/access_logs?filetype=csv&accessed_atRange=[\"2024-01-01\",\"2024-03-31\"]" \
  > access_logs_q1_2024.csv