# Access Logs & Audit Trail

Complete documentation for the Tour Builder Platform's Access Logs and Audit Trail system including activity tracking, security auditing, and data preservation.

## Overview

The platform implements a comprehensive audit system with two complementary mechanisms:
- **Access Logs** - Dedicated table recording API requests with user, path, and client info
- **Audit Trail** - CreatedBy/UpdatedBy fields on all entities tracking who modified what

```
┌─────────────────────────────────────────────────────────────────────────────┐
│                    Access Logs & Audit Trail Architecture                    │
│                                                                              │
│  ┌────────────────────────────────────────────────────────────────────────┐ │
│  │                        Request Flow                                     │ │
│  │                                                                         │ │
│  │   Client Request                                                        │ │
│  │        │                                                                │ │
│  │        ▼                                                                │ │
│  │   ┌─────────────────────────────────────────────────────────────────┐  │ │
│  │   │                    Request Logger Middleware                     │  │ │
│  │   │                                                                  │  │ │
│  │   │   • Generate/Extract X-Request-Id                               │  │ │
│  │   │   • Capture: method, path, user-agent, duration                 │  │ │
│  │   │   • Log to Pino (structured logging)                            │  │ │
│  │   └──────────────────────────┬──────────────────────────────────────┘  │ │
│  │                              │                                          │ │
│  │                              ▼                                          │ │
│  │   ┌─────────────────────────────────────────────────────────────────┐  │ │
│  │   │                    JWT Authentication                            │  │ │
│  │   │                                                                  │  │ │
│  │   │   • Validate token                                              │  │ │
│  │   │   • Attach currentUser to request                               │  │ │
│  │   └──────────────────────────┬──────────────────────────────────────┘  │ │
│  │                              │                                          │ │
│  │                              ▼                                          │ │
│  │   ┌─────────────────────────────────────────────────────────────────┐  │ │
│  │   │                    Permission Check                              │  │ │
│  │   │                                                                  │  │ │
│  │   │   • Check READ/CREATE/UPDATE/DELETE permissions                 │  │ │
│  │   │   • Role-based + custom permissions                             │  │ │
│  │   └──────────────────────────┬──────────────────────────────────────┘  │ │
│  │                              │                                          │ │
│  │                              ▼                                          │ │
│  │   ┌─────────────────────────────────────────────────────────────────┐  │ │
│  │   │                    Database Operation                            │  │ │
│  │   │                                                                  │  │ │
│  │   │   CREATE: set createdById, updatedById                          │  │ │
│  │   │   UPDATE: set updatedById                                       │  │ │
│  │   │   DELETE: set deletedBy, mark deletedAt (soft delete)           │  │ │
│  │   └─────────────────────────────────────────────────────────────────┘  │ │
│  └────────────────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────────┘
```

## Data Model

### Access Logs Schema

**Table:** `access_logs`

**Source:** `backend/src/db/models/access_logs.js`

| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| id | UUID | Yes | UUIDv4 | Primary key |
| projectId | UUID (FK) | No | null | Related project (CASCADE delete) |
| userId | UUID (FK) | No | null | User who accessed (SET NULL delete) |
| environment | ENUM | Yes | - | 'admin', 'stage', 'production' |
| path | TEXT | No | null | API path accessed (max 2048 chars) |
| ip_address | TEXT | No | null | Client IP address (max 45 chars for IPv6) |
| user_agent | TEXT | No | null | HTTP User-Agent header (max 1024 chars) |
| accessed_at | TIMESTAMP | Yes | NOW | When access occurred |
| importHash | VARCHAR(255) | No | null | Unique hash for CSV imports |
| createdAt | TIMESTAMP | Yes | Auto | Record creation time |
| updatedAt | TIMESTAMP | Yes | Auto | Record update time |
| deletedAt | TIMESTAMP | No | null | Soft delete timestamp |

### Indexes

```javascript
indexes: [
  { fields: ['projectId'] },     // Query logs by project
  { fields: ['environment'] },   // Filter by environment
  { fields: ['userId'] },        // Query logs by user
  { fields: ['accessed_at'] },   // Time-range queries and sorting
]
```

### Relationships

```
access_logs
├── belongsTo projects (as project)
│   ├── foreignKey: projectId
│   ├── onDelete: CASCADE        ← Logs deleted when project deleted
│   └── onUpdate: CASCADE
│
├── belongsTo users (as user)
│   ├── foreignKey: userId
│   ├── onDelete: CASCADE        ← Logs deleted when user deleted
│   └── onUpdate: CASCADE
│
├── belongsTo users (as createdBy) ─── Audit: who created this log
└── belongsTo users (as updatedBy) ─── Audit: who last updated this log
```

## Audit Trail Pattern

### CreatedBy/UpdatedBy Implementation

All entities in the system implement the audit trail pattern:

**Source:** `backend/src/db/api/base.api.js`

```javascript
// On CREATE
static async create({ data, currentUser = { id: null }, transaction }) {
  const mappedData = this.getFieldMapping(data);

  const record = await this.MODEL.create({
    ...mappedData,
    createdById: currentUser.id,  // Track creator
    updatedById: currentUser.id,  // Initial updater = creator
  }, { transaction });
}

// On UPDATE
static async update({ id, data, currentUser = { id: null }, transaction }) {

  await record.update({
    ...updatePayload,
    updatedById: currentUser.id,  // Track who modified
  }, { transaction });
}

// On DELETE (soft delete)
static async remove({ id, currentUser = { id: null }, transaction }) {

  await record.update({ deletedBy: currentUser.id }, { transaction });
  await record.destroy({ transaction });  // Sets deletedAt
}
```

### Entities with Audit Trail

All models include audit relationships:

```javascript
// Applied to ALL models:
db.[entity].belongsTo(db.users, { as: 'createdBy' });
db.[entity].belongsTo(db.users, { as: 'updatedBy' });
```

**Audited Entities:**
- users, roles, permissions
- projects, project_memberships
- tour_pages, project_audio_tracks
- assets, asset_variants
- element_type_defaults, project_element_defaults
- publish_events, pwa_caches
- presigned_url_requests, access_logs

**Note:** Page elements, navigation links, and transitions are stored in `tour_pages.ui_schema_json` and audited as part of the tour_pages record.

### Timestamp Tracking

Sequelize provides automatic timestamps:

```javascript
{
  timestamps: true,   // Enables createdAt, updatedAt
  paranoid: true,     // Enables deletedAt (soft delete)
}
```

| Field | Set When | Never Changes |
|-------|----------|---------------|
| `createdAt` | Record created | ✓ Immutable |
| `updatedAt` | Any modification | Updates each save |
| `deletedAt` | Soft delete called | - |

## Request Logging Middleware

### Logger Configuration

**Source:** `backend/src/utils/logger.js`

```javascript
const logger = pino({
  level: process.env.LOG_LEVEL || 'info',
  transport: isDevelopment
    ? { target: 'pino-pretty', options: { colorize: true } }
    : undefined,
  base: {
    service: 'tour-builder-api',
    env: process.env.NODE_ENV || 'development',
  },
});
```

### Request Logger Middleware

```javascript
function requestLogger(req, res, next) {
  // Generate or extract request ID for tracing
  const requestId = req.headers['x-request-id'] || crypto.randomUUID();
  req.log = logger.child({ requestId });
  req.requestId = requestId;
  res.setHeader('X-Request-Id', requestId);

  const start = Date.now();

  res.on('finish', () => {
    const duration = Date.now() - start;
    const logData = {
      method: req.method,
      url: req.originalUrl || req.url,
      status: res.statusCode,
      duration,
      userAgent: req.headers['user-agent'],
    };

    // Log level based on response status
    if (res.statusCode >= 500) {
      req.log.error(logData, 'Request completed with server error');
    } else if (res.statusCode >= 400) {
      req.log.warn(logData, 'Request completed with client error');
    } else {
      req.log.info(logData, 'Request completed');
    }
  });

  next();
}
```

### Applied in Application

**Source:** `backend/src/index.ts`

```javascript
app.enable('trust proxy');  // Extract real IP behind proxies
app.use(requestLogger);     // Apply to all routes
```

### Log Output Format

**Development (pino-pretty):**
```
[10:30:45.123] INFO: Request completed
    method: "GET"
    url: "/api/projects"
    status: 200
    duration: 45
    userAgent: "Mozilla/5.0..."
    requestId: "abc123-def456"
```

**Production (JSON):**
```json
{
  "level": 30,
  "time": 1711234567890,
  "service": "tour-builder-api",
  "env": "production",
  "requestId": "abc123-def456",
  "method": "GET",
  "url": "/api/projects",
  "status": 200,
  "duration": 45,
  "userAgent": "Mozilla/5.0..."
}
```

## API Reference

### Endpoints

**Source:** `backend/src/routes/access_logs.ts`

| Method | Endpoint | Permission | Description |
|--------|----------|------------|-------------|
| GET | `/api/access_logs` | READ_ACCESS_LOGS | List with filtering |
| GET | `/api/access_logs/:id` | READ_ACCESS_LOGS | Get single log |
| POST | `/api/access_logs` | CREATE_ACCESS_LOGS | Create log entry |
| PUT | `/api/access_logs/:id` | UPDATE_ACCESS_LOGS | Update log |
| DELETE | `/api/access_logs/:id` | DELETE_ACCESS_LOGS | Soft delete log |

### Query Parameters

**Source:** `backend/src/db/api/access_logs.ts`

| Parameter | Type | Description |
|-----------|------|-------------|
| `limit` | number | Results per page (default: 50) |
| `page` | number | Page number (0-based) |
| `field` | string | Sort field |
| `sort` | 'asc' \| 'desc' | Sort direction |
| `path` | string | Search path (ILIKE) |
| `ip_address` | string | Search IP address (ILIKE) |
| `user_agent` | string | Search user agent (ILIKE) |
| `accessed_atRange` | [start, end] | Date range filter |
| `environment` | enum | Filter by environment |
| `project` | UUID \| string | Filter by project |
| `user` | UUID \| string | Filter by user |
| `filetype` | 'csv' | Export as CSV |

### Request/Response Examples

**List Logs with Filters:**
```http
GET /api/access_logs?limit=50&page=1&environment=production&accessed_atRange=["2024-01-01","2024-12-31"]&sort=DESC&field=accessed_at
Authorization: Bearer <jwt-token>
```

**Response:**
```json
{
  "rows": [
    {
      "id": "log-uuid-1",
      "projectId": "project-uuid",
      "userId": "user-uuid",
      "environment": "production",
      "path": "/api/tour_pages",
      "ip_address": "192.168.1.100",
      "user_agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64)...",
      "accessed_at": "2024-03-15T10:30:00Z",
      "project": {
        "id": "project-uuid",
        "name": "My Tour"
      },
      "user": {
        "id": "user-uuid",
        "firstName": "John",
        "lastName": "Doe"
      },
      "createdAt": "2024-03-15T10:30:00Z",
      "updatedAt": "2024-03-15T10:30:00Z"
    }
  ],
  "count": 1542
}
```

**Create Log Entry:**
```http
POST /api/access_logs
Content-Type: application/json
Authorization: Bearer <jwt-token>

{
  "data": {
    "project": "project-uuid",
    "user": "user-uuid",
    "environment": "production",
    "path": "/api/tour_pages",
    "ip_address": "192.168.1.100",
    "user_agent": "Mozilla/5.0...",
    "accessed_at": "2024-03-15T10:30:00Z"
  }
}
```

### CSV Export

**Fields Exported:**
- id
- environment
- path
- ip_address
- user_agent
- accessed_at
- createdAt

```http
GET /api/access_logs?filetype=csv
Authorization: Bearer <jwt-token>
```

## Frontend Implementation

### Admin Pages

**Location:** `frontend/src/pages/access_logs/`

| Page | Path | Description |
|------|------|-------------|
| List | `/access_logs/access_logs-list` | Filterable table view |
| Table | `/access_logs/access_logs-table` | Data grid variant |
| New | `/access_logs/access_logs-new` | Create log entry |
| Edit | `/access_logs/access_logs-edit?id=<uuid>` | Edit log |
| View | `/access_logs/access_logs-view?id=<uuid>` | Read-only details |
| Dynamic | `/access_logs/[access_logsId]` | Dynamic route (ID in path) |

**Note:** Uses `useEditPageSync` hook for edit page data synchronization.

### List Page Filters

**Source:** `frontend/src/pages/access_logs/access_logs-list.tsx`

| Filter | Type | Description |
|--------|------|-------------|
| Path | Text | Search by API path |
| IP Address | Text | Search by client IP |
| User Agent | Text | Search by browser/client |
| Accessed At | Date Range | Filter by time range |
| Project | Dropdown | Filter by project |
| User | Dropdown | Filter by user |
| Environment | Enum | admin / stage / production |

### Table Columns

**Source:** `frontend/src/components/Access_logs/configureAccess_logsCols.tsx`

Uses `createColumnLoader` factory from `configBuilderFactory.tsx`:

| Column | Editable | Type |
|--------|----------|------|
| project | Yes | singleSelectRelation (projects) |
| environment | Yes | text |
| user | Yes | singleSelectRelation (users) |
| path | Yes | text |
| ip_address | Yes | text |
| user_agent | Yes | text |
| accessed_at | Yes | datetime |
| actions | - | actions (Edit / View / Delete) |

### TypeScript Interface

**Source:** `frontend/src/types/entities.ts`

```typescript
export interface AccessLog extends BaseEntity {
  user?: User | string | null;
  project?: Project | string | null;
  environment?: 'admin' | 'stage' | 'production';
  path?: string;
  action?: string;
  ip_address?: string;
  user_agent?: string;
  accessed_at?: string | Date;
  metadata?: Record<string, unknown>;
}
```

### Redux State

**Source:** `frontend/src/stores/access_logs/access_logsSlice.ts`

Uses `createEntitySlice` factory for standardized CRUD operations:

```typescript
const { slice, actions, reducer } = createEntitySlice<AccessLog>({
  name: 'access_logs',
  endpoint: 'access_logs',
  singularName: 'Access Log',
});

export const {
  fetch,           // Load logs with query
  create,          // Create new log
  update,          // Update existing log
  deleteItem,      // Delete single log
  deleteItemsByIds,// Batch delete
  uploadCsv,       // Import from CSV
  setRefetch,      // Trigger refresh
} = actions;

// Usage
dispatch(fetch({ query: '?environment=production&limit=100' }));
```

## Security & Permissions

### Role-Permission Matrix

**Source:** `backend/src/db/seeders/20200430130760-user-roles.js`

| Role | CREATE | READ | UPDATE | DELETE |
|------|--------|------|--------|--------|
| PlatformOwner | ✓ | ✓ | ✓ | ✓ |
| Administrator | ✓ | ✓ | ✓ | ✓ |
| AccountManager | ✗ | ✓ | ✗ | ✗ |
| TourDesigner | ✗ | ✓ | ✗ | ✗ |
| ContentReviewer | ✗ | ✓ | ✗ | ✗ |
| AnalyticsViewer | ✗ | ✓ | ✗ | ✗ |

### Permission Check Flow

```
Request → JWT Auth → checkCrudPermissions('access_logs') → Route Handler
                              │
                              ▼
              ┌───────────────────────────────┐
              │  1. AccessPolicy              │
              │  2. Custom user permissions   │
              │  3. Role-based permissions    │
              │  4. Public hardening          │
              └───────────────────────────────┘
```

### Cascade Delete Behavior

Both user and project deletions cascade to access logs:

```javascript
// access_logs -> users
onDelete: 'CASCADE'   // Logs deleted with user

// access_logs -> projects
onDelete: 'CASCADE'   // Logs deleted with project
```

> **Note:** Access logs are NOT preserved when users or projects are deleted. If audit history retention is required, consider archiving logs before deletion or implementing SET NULL behavior.

## Data Capture Scope

### What IS Captured

| Data Point | Location | Purpose |
|------------|----------|---------|
| User ID | access_logs.userId | Who made the request |
| Project ID | access_logs.projectId | Which project context |
| API Path | access_logs.path | What endpoint accessed |
| IP Address | access_logs.ip_address | Client location/identity |
| User Agent | access_logs.user_agent | Browser/client identification |
| Timestamp | access_logs.accessed_at | When it happened |
| Environment | access_logs.environment | admin/stage/production |

### What IS NOT Captured

| Data Point | Reason |
|------------|--------|
| Request body | Privacy protection |
| Response content | Privacy/performance |
| Query parameters | Only path stored |
| Authentication credentials | Security |
| Personal user data | Beyond ID reference |

### Audit Trail vs Access Logs

| Aspect | Audit Trail | Access Logs |
|--------|-------------|-------------|
| Scope | All entities | Dedicated table |
| Granularity | Per-record changes | Per-request |
| Data | Who modified when | Request details |
| Retention | With entity | Separate lifecycle |
| Purpose | Change tracking | Security monitoring |

## Soft Delete & Retention

### Paranoid Mode

```javascript
{
  timestamps: true,
  paranoid: true,  // Enables soft deletes
}
```

### Deletion Behavior

1. **Soft Delete**: Sets `deletedAt` timestamp
2. **Query Filtering**: Normal queries exclude deleted records
3. **Restoration**: Possible by clearing `deletedAt`
4. **Hard Delete**: Requires explicit permanent removal

### Retention Considerations

**Current Implementation:**
- No automatic log pruning
- Logs retained indefinitely
- Manual cleanup via DELETE endpoints

**Recommended Practices:**
- Implement TTL-based cleanup for old logs
- Archive logs to cold storage after 90 days
- Consider GDPR right-to-erasure requirements

## File Structure

### Backend

| File | Purpose |
|------|---------|
| `backend/src/db/models/access_logs.js` | Sequelize model definition |
| `backend/src/db/api/access_logs.ts` | Database API (CRUD) |
| `backend/src/db/api/base.api.js` | Audit trail implementation |
| `backend/src/routes/access_logs.ts` | REST endpoints |
| `backend/src/services/access_logs.ts` | Business logic |
| `backend/src/utils/logger.js` | Request logging middleware |
| `backend/src/middlewares/check-permissions.ts` | Permission enforcement |

### Frontend

| File | Purpose |
|------|---------|
| `frontend/src/pages/access_logs/access_logs-list.tsx` | List page |
| `frontend/src/pages/access_logs/access_logs-table.tsx` | Table page |
| `frontend/src/pages/access_logs/access_logs-new.tsx` | Create page |
| `frontend/src/pages/access_logs/access_logs-edit.tsx` | Edit page |
| `frontend/src/pages/access_logs/access_logs-view.tsx` | View page |
| `frontend/src/pages/access_logs/[access_logsId].tsx` | Dynamic route |
| `frontend/src/stores/access_logs/access_logsSlice.ts` | Redux state (createEntitySlice) |
| `frontend/src/components/Access_logs/TableAccess_logs.tsx` | Data grid component |
| `frontend/src/components/Access_logs/ListAccess_logs.tsx` | List view component |
| `frontend/src/components/Access_logs/CardAccess_logs.tsx` | Card view component |
| `frontend/src/components/Access_logs/configureAccess_logsCols.tsx` | Column config (createColumnLoader) |
| `frontend/src/types/entities.ts` | TypeScript interfaces |

## Known Considerations

### 1. Manual Log Creation

**Issue:** Access logs are typically created manually via API, not automatically by middleware.

**Impact:** Requires explicit logging calls or additional middleware to auto-populate.

### 2. User Deletion Cascades

**Issue:** `CASCADE` on userId means logs are deleted when user is deleted.

**Impact:** Audit history lost with user deletion. Consider archiving logs before user deletion or changing to `SET NULL` for preservation.

### 3. Project Deletion Cascades

**Issue:** All access logs deleted when project is deleted.

**Impact:** Audit history lost with project. Consider archiving before deletion.

### 4. No Query Parameter Logging

**Issue:** Only `path` is stored, not query strings.

**Impact:** Cannot reconstruct full request URL. May miss filter/search context.

### 5. IPv6 Support

**Issue:** `ip_address` limited to 45 characters.

**Impact:** Supports full IPv6 addresses (39 chars max). Adequate for current needs.

### 6. User Agent Truncation

**Issue:** `user_agent` limited to 1024 characters.

**Impact:** Very long user agents may be truncated. Rare but possible.

### 7. No Automatic Retention

**Issue:** Logs grow indefinitely without cleanup.

**Impact:** Database size increases over time. Recommend implementing retention policy.

### 8. TypeScript Interface Extras

**Issue:** Frontend `AccessLog` interface includes `action` and `metadata` fields not in database model.

**Impact:** These fields exist only in TypeScript for potential future use. Currently not persisted.

## Integration Examples

### Log Access in Custom Middleware

```javascript
import Access_logsDBApi from '../db/api/access_logs.ts';

async function logAccess(req, res, next) {
  // Log after response completes
  res.on('finish', async () => {
    try {
      await Access_logsDBApi.create({
        project: req.project?.id,
        user: req.currentUser?.id,
        environment: determineEnvironment(req),
        path: req.path,
        ip_address: req.ip,
        user_agent: req.headers['user-agent'],
        accessed_at: new Date(),
      }, { currentUser: req.currentUser });
    } catch (err) {
      // Don't fail request on logging error
      console.error('Failed to log access:', err);
    }
  });
  next();
}
```

### Query Logs by Time Range

```typescript
// Frontend
dispatch(fetch({
  query: '?accessed_atRange=["2024-01-01T00:00:00Z","2024-01-31T23:59:59Z"]&environment=production&sort=DESC&field=accessed_at'
}));
```

### Export Logs for Compliance

```bash
curl -H "Authorization: Bearer $TOKEN" \
  "https://api.example.com/api/access_logs?filetype=csv&accessed_atRange=[\"2024-01-01\",\"2024-03-31\"]" \
  > access_logs_q1_2024.csv
```