Dmitri 9f111d6226 added documentation

2026-07-03 16:11:24 +02:00

41 KiB

Raw Blame History

Project Architecture - E2E Documentation

Overview

Tour Builder Platform - A sophisticated web application for building and managing virtual tours with interactive pages, assets, transitions, and offline capabilities.

Technology Stack:

Frontend: Next.js 15 with React 19, TypeScript, Redux Toolkit, Tailwind CSS, Serwist PWA
Backend: Node.js/Express with Sequelize ORM
Database: PostgreSQL with migrations and seeders
File Storage: AWS S3 (configurable via FILE_STORAGE_PROVIDER)

High-Level Architecture

┌─────────────────────────────────────────────────────────────────┐
│                         Frontend                                 │
│  Next.js 15 │ React 19 │ Redux Toolkit │ Tailwind │ Serwist    │
└─────────────────────────────────────────────────────────────────┘
                              │
                         HTTP/REST
                              │
┌─────────────────────────────────────────────────────────────────┐
│                         Backend                                  │
│  Express.js │ Passport.js │ Sequelize ORM │ Factory Patterns    │
└─────────────────────────────────────────────────────────────────┘
                              │
                         Sequelize
                              │
┌─────────────────────────────────────────────────────────────────┐
│                        Database                                  │
│  PostgreSQL │ Migrations │ Seeders                              │
└─────────────────────────────────────────────────────────────────┘
                              │
┌─────────────────────────────────────────────────────────────────┐
│                      File Storage                                │
│  AWS S3 │ Google Cloud Storage │ Local Filesystem               │
└─────────────────────────────────────────────────────────────────┘

Standard VM Deployment

The standard Flatlogic VM adds Apache, PM2, and executor services around the application:

Cloudflare
  -> Apache :80
    -> Next.js frontend PM2 app `frontend-dev` :3001 (production server)
    -> Express backend PM2 app `backend-dev` :3000 (NODE_ENV=dev_stage)

PM2 systemd unit: pm2-ubuntu.service
Auxiliary PM2 apps: fl-executor, fl-telemetry

Local development now mirrors this port split: frontend 3001, backend 3000. A protected backend endpoint returning 401 Unauthorized is a healthy response when no JWT is supplied.

For the detailed VM runbook, OOM recovery, process checks, and executor notes, see deployment-vm.md.

Frontend Architecture

Directory Structure

frontend/src/
├── pages/              # Next.js pages (dynamic routing)
├── components/         # React components (PascalCase naming)
├── stores/             # Redux Toolkit slices (entity-based)
├── layouts/            # Page layouts (Authenticated, Guest)
├── hooks/              # Custom React hooks (~32 hooks)
├── lib/                # Utility libraries (offline, logging)
├── types/              # TypeScript interfaces and types
├── helpers/            # Utility functions
├── css/                # Global CSS (Tailwind)
├── config/             # App configuration
├── context/            # React context providers
├── schemas/            # Zod validation schemas
└── factories/          # Page and component factories

Pages Structure

Page Type	Pattern	Description
List	`{entity}-list.tsx`	Data grid with CRUD, filtering, sorting
Edit	`{entity}-edit.tsx`	Form-based entity modification
View	`{entity}-view.tsx`	Read-only entity display
New	`{entity}-new.tsx`	Create new entity form

Special Pages:

constructor.tsx - Visual tour builder (complex)
p/[projectSlug]/index.tsx - Production tour presentation
p/[projectSlug]/stage.tsx - Stage tour preview
login.tsx - Authentication page
dashboard.tsx - Overview and analytics

State Management (Redux Toolkit)

Store Configuration (stores/store.ts):

// Entity slices (13 total)
users, roles, permissions, projects, assets, tour_pages,
asset_variants, project_audio_tracks, publish_events, pwa_caches,
access_logs, presigned_url_requests, project_memberships

// Global slices
authSlice      // Authentication state, JWT token
mainSlice      // UI state, notifications
styleSlice     // Theme, styling preferences

Redux Pattern:

// Entity slice factory
import { createEntitySlice } from './createEntitySlice';

const assetsSlice = createEntitySlice('assets', {
  // Custom reducers if needed
});

// Usage in components
import { fetch, create, update, deleteItem } from '@/stores/assets/assetsSlice';
import { useAppDispatch, useAppSelector } from '@/stores/hooks';

const dispatch = useAppDispatch();
const { rows, loading } = useAppSelector((state) => state.assets);

dispatch(fetch({ query: '?limit=100&page=1' }));

Component Organization

components/
├── Assets/
│   ├── AssetCard.tsx
│   ├── AssetGrid.tsx
│   ├── useAssetUploader.ts      # Custom hook
│   └── ProjectSelector.tsx
├── Offline/
│   ├── OfflineToggle.tsx
│   ├── OfflineStatusIndicator.tsx
│   └── DownloadProgressPanel.tsx
├── Layout/
│   ├── Header.tsx
│   ├── Sidebar.tsx
│   └── Footer.tsx
└── [Entity]/
    ├── [Entity]Card.tsx
    ├── [Entity]Form.tsx
    └── [Entity]List.tsx

Custom Hooks

Hook	Location	Purpose
`useAssetUploader`	`components/Assets/`	Chunked file upload management
`usePreloadOrchestrator`	`hooks/`	Asset preloading with S3 presigned URLs and ready blob URLs
`usePageSwitch`	`hooks/`	Smooth page navigation with preloaded blob URLs
`useTransitionPlayback`	`hooks/`	Transition video playback coordination
`useNeighborGraph`	`hooks/`	Navigation graph for preloading (maxDepth: 1)
`useNetworkAware`	`hooks/`	Network condition detection
`useReversePlayback`	`hooks/`	Video reverse playback
`useOfflineMode`	`hooks/`	Offline download management
`useStorageQuota`	`hooks/`	Storage quota monitoring

TypeScript Types

types/
├── api.ts          # PaginatedResponse, FetchParams, ApiError
├── entities.ts     # BaseEntity, User, Role, Project, Asset, etc.
├── redux.ts        # Entity slice configuration
├── constructor.ts  # Tour builder specific types
├── runtime.ts      # Runtime environment types
├── offline.ts      # PWA and offline types
├── preload.ts      # Preloading system types
├── permissions.ts  # Permission enum definitions
├── presentation.ts # Presentation runtime types
├── filters.ts      # Filter system types
├── forms.ts        # Form field types
└── index.ts        # Central exports

Frontend Configuration

// config/config.ts
const config = {
  apiBaseUrl: process.env.NEXT_PUBLIC_BACK_API || 'http://localhost:3000/api',
  appTitle: 'Shimahara Visual',
  storageKeys: {
    darkMode: 'tour-builder-dark-mode',
    stylePrefs: 'tour-builder-style',
  },
};

Build Configuration

// next.config.mjs
{
  // PWA with Serwist
  swSrc: 'src/sw.ts',
  swDest: 'public/sw.js',
  disable: process.env.NODE_ENV === 'development',

  // Next.js settings
  trailingSlash: true,
  distDir: process.env.NODE_ENV === 'development' ? '.next' : 'build',
  output: process.env.NEXT_OUTPUT || undefined,
  images: { unoptimized: true },

  // Development
  experimental: { turbo: true }
}

Backend Architecture

Directory Structure

backend/src/
├── routes/             # Express route handlers (22 routes)
├── db/
│   ├── models/         # Sequelize model definitions (16 models)
│   ├── api/            # Database access layer (GenericDBApi)
│   ├── migrations/     # Database schema migrations
│   ├── seeders/        # Seed data scripts
│   ├── db-config.ts    # Database configuration
│   └── umzug.ts        # Migration and seeder runner
├── services/           # Business logic services
├── auth/               # Passport.js strategies
├── middlewares/        # Express middlewares
├── factories/          # Factory patterns
└── utils/              # Logger, error handling

Three-Tier Architecture

┌─────────────────────────────────────────────────────────────────┐
│                        Routes Layer                              │
│  Express handlers │ Request validation │ Response formatting    │
└─────────────────────────────────────────────────────────────────┘
                              │
┌─────────────────────────────────────────────────────────────────┐
│                       Services Layer                             │
│  Business logic │ Transactions │ External integrations          │
└─────────────────────────────────────────────────────────────────┘
                              │
┌─────────────────────────────────────────────────────────────────┐
│                        DB API Layer                              │
│  Sequelize queries │ Filtering │ Associations │ Transformations │
└─────────────────────────────────────────────────────────────────┘

Factory Patterns

Router Factory (factories/router.factory.js):

// Generates standard CRUD routes automatically
module.exports = createEntityRouter('assets', AssetsService, AssetsDBApi, {
  permissionEntity: 'assets',
  csvFields: ['name', 'url', 'type'],
  customRoutes: (router) => {
    router.get('/custom', customHandler);
  }
});

// Generated routes:
// GET    /api/assets          - List with pagination/filtering
// GET    /api/assets/:id      - Get single entity
// POST   /api/assets          - Create entity
// PUT    /api/assets/:id      - Update entity
// DELETE /api/assets/:id      - Delete entity
// POST   /api/assets/bulk-import   - CSV import
// DELETE /api/assets/deleteByIds   - Bulk delete

Service Factory (factories/service.factory.js):

// Creates service class with transaction handling
module.exports = createEntityService('assets', AssetsDBApi);

// Generated methods:
// create({ data, currentUser, transaction, runtimeContext }) - Create with transaction
// update({ id, data, currentUser, transaction, runtimeContext }) - Update with transaction
// remove({ id, currentUser, transaction, runtimeContext }) - Delete with transaction
// bulkImport(data, options) - Batch import
// deleteByIds({ ids, currentUser, transaction, runtimeContext }) - Batch delete

GenericDBApi (db/api/base.api.js):

class GenericDBApi {
  // Override in entity APIs
  MODEL = null;
  SEARCHABLE_FIELDS = [];      // Text search fields
  RANGE_FIELDS = [];           // Numeric/date range filters
  ENUM_FIELDS = [];            // Exact match filters
  ASSOCIATIONS = [];           // Related entity loading

  // Transform input before save
  getFieldMapping(data) { return data; }

  // Standard operations
  findAll(query, options) { }
  findOne(id, options) { }
  create(data, options) { }
  update({ id, data, currentUser, transaction, runtimeContext }) { }
  remove({ id, currentUser, transaction, runtimeContext }) { }
}

Middleware Stack

// Applied in order
app.use(helmet());                    // Security headers
app.use(cors({ origin: true }));      // CORS
app.use(requestLogger);               // Request logging
app.use(runtimeContext);              // Admin/stage/production detection
app.use(bodyParser.json());           // JSON parsing
app.use(passport.initialize());       // Auth initialization

// Per-route middleware
router.use(passport.authenticate('jwt'));  // JWT validation
router.use(checkPermissions);              // RBAC check

Authentication System

Passport Strategies:

// 1. JWT Strategy (primary)
passport.use(new JwtStrategy({
  jwtFromRequest: ExtractJwt.fromAuthHeaderAsBearerToken(),
  secretOrKey: process.env.SECRET_KEY,
}, async (payload, done) => {
  const user = await findUserByEmail(payload.email);
  if (user && !user.disabled) {
    return done(null, user);
  }
  return done(null, false);
}));

// 2. Google OAuth 2.0
passport.use(new GoogleStrategy({
  clientID: process.env.GOOGLE_CLIENT_ID,
  clientSecret: process.env.GOOGLE_CLIENT_SECRET,
  callbackURL: '/api/auth/signin/google/callback',
}, findOrCreateUser));

// 3. Microsoft OAuth
passport.use(new MicrosoftStrategy({...}, findOrCreateUser));

JWT Configuration:

Secret: process.env.SECRET_KEY
Expiration: 6 hours
Payload: { id, email, name }

Runtime Context System

The platform uses route-based environment access (not subdomains):

Route	Environment	Purpose
`/p/[slug]`	production	Public presentation
`/p/[slug]/stage`	stage	Preview presentation
`/constructor?projectId=`	dev	Editor (always dev)

Middleware (middleware/runtime-context.ts):

// Detects context from hostname AND headers
req.runtimeContext = {
  mode: detectFromHostname(hostname),     // 'admin' | 'stage' | 'production'
  projectSlug: fromHostname || null,
  headerEnvironment: req.headers['x-runtime-environment'],  // Route-based
  headerProjectSlug: req.headers['x-runtime-project-slug'], // Route-based
};

Environment Resolution (db/api/runtime-context.ts):

// Priority: hostname-based → header-based
// SECURITY: Only 'production' and 'stage' allowed from headers
// 'dev' from header is ignored to prevent unauthorized access

Frontend Filtering (RuntimePresentation.tsx):

// STRICT: Only exact environment match
.filter((p) => p.environment === environment)

See Publishing Workflow for complete environment isolation details.

Permissions System

// middlewares/check-permissions.ts
const checkPermissions = async (req, res, next) => {
  // 1. Self-access bypass
  if (req.params.id === req.currentUser.id) {
    return next();
  }

  // 2. Check custom permissions
  const customPermission = await findCustomPermission(
    req.currentUser.id,
    req.permissionEntity,
    req.method
  );
  if (customPermission) return next();

  // 3. Check role-based permissions
  const rolePermission = await findRolePermission(
    req.currentUser.app_role,
    req.permissionEntity,
    req.method
  );
  if (rolePermission) return next();

  return res.status(403).json({ error: 'Forbidden' });
};

File Storage System

// Configurable via FILE_STORAGE_PROVIDER env var
const providers = {
  s3: new S3StorageProvider({
    bucket: process.env.AWS_S3_BUCKET,
    region: process.env.AWS_S3_REGION,
  }),
  gcloud: new GCloudStorageProvider({...}),
  local: new LocalStorageProvider({
    basePath: './uploads',
  }),
};

// File service with chunked upload support
class FileService {
  async initUpload(filename, totalChunks) { }
  async uploadChunk(sessionId, chunkIndex, data) { }
  async finalizeUpload(sessionId) { }
  async getDownloadUrl(fileKey) { }
}

S3 Presigned URLs for Direct Download:

The platform supports direct asset downloads from S3 for better performance:

// POST /api/file/presign - Get presigned URLs for batch download
// Request: { urls: ["assets/image1.jpg", "assets/video.mp4", ...] }
// Response: { presignedUrls: { "assets/image1.jpg": "https://s3...", ... } }
// - Max 50 URLs per request
// - 1-hour expiry
// - Public endpoint (no auth required for runtime)

Frontend preloading uses presigned URLs:

Request presigned URLs via POST /api/file/presign
Download directly from S3 (bypasses backend)
Store in Cache API (dual key: download URL + storage key)
Create blob URL for instant playback
Store in readyBlobUrlsRef (dual key: download URL + storage key) for O(1) lookup

Storage Key Mapping: Assets are cached under both download URL and canonical storage key (e.g., assets/project/video.mp4). This ensures cache hits even when presigned URL signatures change.

Database Schema

Core Domain Models

┌─────────────────────────────────────────────────────────────────┐
│                    Element Defaults Hierarchy                    │
│                                                                  │
│  ┌─────────────────────┐                                        │
│  │ element_type_defaults│ ─── Global platform-wide defaults     │
│  └──────────┬──────────┘                                        │
│             │ (auto-snapshot on project creation)               │
│             ▼                                                   │
│  ┌─────────────────────┐                                        │
│  │project_element_defaults│ ─── Project-specific overrides      │
│  └──────────┬──────────┘                                        │
│             │ (applies defaults to new elements)                │
│             ▼                                                   │
│  ┌─────────────────────┐                                        │
│  │   Page Elements     │ ─── Instance-specific on pages         │
│  └─────────────────────┘                                        │
└─────────────────────────────────────────────────────────────────┘

┌─────────────────┐     ┌─────────────────┐
│    Projects     │────<│   Tour Pages    │
└─────────────────┘     └─────────────────┘
        │                       │
        │                       ▼
        │               ┌─────────────────────────────────────┐
        │               │       ui_schema_json (inline)       │
        │               │  • Page Elements                    │
        │               │  • Navigation Links (targetPageSlug)│
        │               │  • Transition Videos                │
        │               └─────────────────────────────────────┘
        ▼
┌─────────────────┐
│     Assets      │
└─────────────────┘
        │
        ▼
┌─────────────────┐
│ Asset Variants  │
└─────────────────┘

Note: Page elements, navigation links, and transition configurations
are stored inline in tour_pages.ui_schema_json (not separate tables).
Navigation uses targetPageSlug for cross-environment consistency.

Entity Relationships

Entity	Relationships
Projects	hasMany: tour_pages, assets, project_audio_tracks, publish_events, project_element_defaults, project_memberships, presigned_url_requests, pwa_caches, access_logs
Tour Pages	belongsTo: project; contains ui_schema_json with elements, navigation (targetPageSlug), and transitions (transitionVideoUrl)
Assets	belongsTo: project; hasMany: asset_variants
Users	belongsTo: role; hasMany: custom_permissions
Element Type Defaults	hasMany: project_element_defaults (as source_element)
Project Element Defaults	belongsTo: project, element_type_defaults (as source_element)

Note: Page elements, navigation links, and transitions are stored inline in tour_pages.ui_schema_json rather than as separate tables. Navigation uses targetPageSlug (not UUIDs) for cross-environment consistency during publishing.

Common Fields Pattern

All entities include:

{
  id: UUID,                 // Primary key
  createdAt: TIMESTAMP,     // Creation timestamp
  updatedAt: TIMESTAMP,     // Last update timestamp
  deletedAt: TIMESTAMP,     // Soft delete (paranoid)
  createdBy: UUID,          // User who created
  updatedBy: UUID,          // User who last updated
}

Key Models

Projects:

{
  id, name, slug, description,
  logo_url, favicon_url, og_image_url,
}

Tour Pages:

{
  id, name, slug, sort_order,
  projectId, environment,
  background_image_url,
  background_video_url,
  background_audio_url,
  background_loop,
  requires_auth,
  ui_schema_json,            // Page layout/elements
}

Page Elements:

{
  id, element_type, name, sort_order,  // element_type is TEXT (not ENUM)
  pageId,
  is_visible,
  x_percent, y_percent,      // Position as percentage (0-100)
  width_percent, height_percent,
  rotation_deg,
  content_json,              // Element-specific data
  style_json,                // CSS properties
}

Element Type Defaults (Global):

{
  id, element_type, name, sort_order,  // Unique element_type per platform
  default_settings_json,      // JSON-serialized default settings
  // 11 predefined types: navigation_next, navigation_prev, spot, tooltip,
  // description, gallery, carousel, logo, video_player, audio_player, popup
}

Project Element Defaults:

{
  id, element_type, name, sort_order,
  projectId,
  settings_json,              // Project-specific settings override
  source_element_id,          // FK to element_type_defaults (for tracking)
  snapshot_version,           // Version for "check for updates" feature
  // Auto-snapshotted from global defaults on project creation
}

Assets:

{
  id, name, projectId,
  asset_type,                // 'image' | 'video' | 'audio' | 'file'
  type,                      // 'general' | 'icon' | 'background_image' | 'audio' | 'video' | 'transition' | 'logo' | 'favicon' | 'document'
  cdn_url, storage_key, mime_type,
  size_mb, width_px, height_px, duration_sec,
  checksum, is_public,
}

Navigation Elements (stored in ui_schema_json):

// Navigation elements with transitions are stored inline in tour_pages.ui_schema_json
{
  id, element_type,  // 'navigation_next', 'navigation_prev', 'spot', etc.
  targetPageSlug,    // Target page for navigation (not UUID)
  transitionVideoUrl,          // Forward transition video
  transitionDurationSec,       // Duration in seconds
  transitionReverseMode,       // 'auto_reverse' | 'separate_video' (replaces supportsReverse)
  reverseVideoUrl,             // Separate reverse video (when mode is 'separate_video')
  // ...other element properties
}

Note: There is no separate transitions table. Transition data is stored directly on navigation elements within each page's ui_schema_json.

Data Flow

Authentication Flow

┌─────────────────┐     ┌─────────────────┐     ┌─────────────────┐
│   Login Page    │────>│  POST /signin   │────>│  Validate Creds │
└─────────────────┘     └─────────────────┘     └─────────────────┘
                                                        │
                                                        ▼
┌─────────────────┐     ┌─────────────────┐     ┌─────────────────┐
│  Store Token    │<────│  Return JWT     │<────│  Generate JWT   │
│  (sessionStorage)│     │                 │     │  (6h expiry)    │
└─────────────────┘     └─────────────────┘     └─────────────────┘
        │
        ▼
┌─────────────────────────────────────────────────────────────────┐
│  Axios Interceptor attaches token to all subsequent requests    │
│  Authorization: Bearer <token>                                  │
└─────────────────────────────────────────────────────────────────┘

CRUD Operation Flow

┌─────────────────┐
│ Redux dispatch  │
│ fetch({query})  │
└────────┬────────┘
         │
         ▼
┌─────────────────┐     ┌─────────────────┐
│   Axios GET     │────>│  JWT Middleware │
│  /api/entity    │     │  (validate)     │
└─────────────────┘     └────────┬────────┘
                                 │
                                 ▼
┌─────────────────┐     ┌─────────────────┐
│  Permissions    │<────│  Route Handler  │
│  Middleware     │     │                 │
└────────┬────────┘     └─────────────────┘
         │
         ▼
┌─────────────────┐     ┌─────────────────┐
│    Service      │────>│    DB API       │
│  (transactions) │     │  (Sequelize)    │
└─────────────────┘     └────────┬────────┘
                                 │
                                 ▼
┌─────────────────┐     ┌─────────────────┐
│   PostgreSQL    │────>│  JSON Response  │
│    Query        │     │  {rows, count}  │
└─────────────────┘     └────────┬────────┘
                                 │
                                 ▼
┌─────────────────────────────────────────┐
│  Redux slice updates state              │
│  Components re-render with new data     │
└─────────────────────────────────────────┘

Asset Upload Flow

┌─────────────────┐     ┌─────────────────┐
│ useAssetUploader│────>│  Split into     │
│  hook           │     │  chunks (5MB)   │
└─────────────────┘     └────────┬────────┘
                                 │
                                 ▼
┌─────────────────┐     ┌─────────────────┐
│  POST /init     │────>│  Create upload  │
│  (get sessionId)│     │  session        │
└─────────────────┘     └────────┬────────┘
                                 │
                                 ▼
┌─────────────────┐     ┌─────────────────┐
│  POST /chunk    │────>│  Store chunk    │
│  (each chunk)   │     │  (S3/local)     │
└─────────────────┘     └────────┬────────┘
                                 │ (repeat)
                                 ▼
┌─────────────────┐     ┌─────────────────┐
│ POST /finalize  │────>│  Combine chunks │
│                 │     │  Create Asset   │
└─────────────────┘     └────────┬────────┘
                                 │
                                 ▼
┌─────────────────────────────────────────┐
│  Return asset metadata with CDN URL     │
└─────────────────────────────────────────┘

Asset Preloading Flow

┌─────────────────────────────────────────────────────────────────┐
│  1. Page Load - Initialize preloading                            │
│     └── usePreloadOrchestrator initializes                       │
│     └── extractPageLinksAndElements(pages) extracts assets       │
│     └── useNeighborGraph builds navigation graph (maxDepth: 1)   │
└─────────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────────┐
│  2. Batch presigned URL request                                  │
│     └── POST /api/file/presign { urls: [...] }                   │
│     └── Returns S3 presigned URLs (1-hour expiry, max 50)        │
└─────────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────────┐
│  3. Priority queue downloads (3 concurrent)                      │
│     └── Transition: +150 │ Image: +100 │ Audio: +50 │ Video: +30│
│     └── Current page: 1000 base │ Neighbor: 500 base            │
└─────────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────────┐
│  4. Download directly from S3                                    │
│     └── fetch(presignedUrl)                                      │
│     └── Store in Cache API (<5MB) or IndexedDB (≥5MB)           │
└─────────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────────┐
│  5. Create ready blob URL (dual key storage)                     │
│     └── URL.createObjectURL(blob)                                │
│     └── Decode image (if applicable)                             │
│     └── Store in readyBlobUrlsRef Map:                          │
│         • key: downloadUrl → blobUrl                             │
│         • key: storageKey → blobUrl (canonical, never changes)  │
└─────────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────────┐
│  6. Navigation - Instant asset display                           │
│     └── usePageSwitch.switchToPage(targetPage)                   │
│     └── getReadyBlobUrl(storageKey) → O(1) instant lookup        │
│     └── Smooth page transition without black flash               │
└─────────────────────────────────────────────────────────────────┘

API Patterns

Standard Endpoints

Every entity follows this pattern:

Method	Endpoint	Description
GET	`/api/{entity}`	List with pagination
GET	`/api/{entity}/:id`	Get single entity
POST	`/api/{entity}`	Create entity
PUT	`/api/{entity}/:id`	Update entity
DELETE	`/api/{entity}/:id`	Delete entity
POST	`/api/{entity}/bulk-import`	CSV import
DELETE	`/api/{entity}/deleteByIds`	Bulk delete

Query Parameters

# Pagination
?limit=100&page=1&offset=0

# Sorting
?sortField=createdAt&sortOrder=DESC

# Text search (SEARCHABLE_FIELDS)
?search=keyword

# Range filters (RANGE_FIELDS)
?createdAtRange=2024-01-01,2024-12-31
?sizeRange=0,1000000

# Enum filters (ENUM_FIELDS)
?type=image&status=active

# Export
?filetype=csv

Response Format

// List response
{
  rows: [...],           // Array of entities
  count: 100,            // Total count (for pagination)
}

// Single entity
{
  id: "uuid",
  ...entityFields,
  createdAt: "2024-01-01T00:00:00Z",
  updatedAt: "2024-01-01T00:00:00Z",
}

// Error response
{
  error: "Error message",
  details: {...},        // Optional
}

Environment Configuration

Backend (.env)

# Database
DB_NAME=app_39215
DB_USER=app_39215
DB_PASS=password
DB_HOST=localhost
DB_PORT=5432

# Authentication
SECRET_KEY=jwt-secret-key
GOOGLE_CLIENT_ID=...
GOOGLE_CLIENT_SECRET=...
MS_CLIENT_ID=...
MS_CLIENT_SECRET=...

# File Storage (S3)
FILE_STORAGE_PROVIDER=s3
AWS_ACCESS_KEY_ID=...
AWS_SECRET_ACCESS_KEY=...
AWS_S3_BUCKET=tour-builder-assets
AWS_S3_REGION=us-east-1
AWS_S3_PREFIX=uploads/

# Email (AWS SES)
EMAIL_USER=...
EMAIL_PASS=...

PEXELS_KEY=...

# Runtime
# Optional. If omitted by backend scripts, src/load-env.ts defaults to dev_stage.
NODE_ENV=dev_stage

Frontend (.env.local)

NEXT_PUBLIC_BACK_API=http://localhost:3000/api

Database Configuration

Database configuration lives in backend/src/db/db-config.ts. Database commands are executed by the typed Umzug runner in backend/src/db/umzug.ts. backend/src/load-env.ts loads backend/.env for app and DB entrypoints; if NODE_ENV is absent it defaults to dev_stage, matching the standard VM backend process.

Key Files Reference

Frontend

File	Purpose
`pages/_app.tsx`	App wrapper, providers, axios interceptor
`stores/store.ts`	Redux store configuration
`stores/createEntitySlice.ts`	Entity slice factory
`config/config.ts`	App configuration
`config/preload.config.ts`	Preload priorities and settings
`next.config.mjs`	Next.js + Serwist config
`src/sw.ts`	Service worker
`hooks/usePreloadOrchestrator.ts`	Asset preloading with S3 presigned URLs
`hooks/usePageSwitch.ts`	Page navigation with preloaded blob URLs
`lib/extractPageLinks.ts`	Extract navigation links from ui_schema_json
`lib/offline/StorageManager.ts`	Cache API and IndexedDB storage
`components/RuntimePresentation.tsx`	Runtime with environment filtering

Backend

File	Purpose
`index.ts`	Express app setup, middleware stack
`factories/router.factory.ts`	CRUD route generator
`factories/service.factory.ts`	Service class generator
`db/api/base.api.ts`	GenericDBApi base class
`db/api/element_type_defaults.ts`	Global element defaults with auto-seed
`db/api/project_element_defaults.ts`	Project element defaults with snapshot
`services/publish.ts`	Dev → Stage → Production publishing
`auth/auth.ts`	Passport strategies
`middlewares/check-permissions.ts`	RBAC middleware

Common Commands

Backend

cd backend

# Install dependencies
npm install

# Start server (port 3000 in dev_stage)
npm run start-dev

# Database operations
npm run db:create    # Create database
npm run db:migrate   # Run migrations
npm run db:seed      # Seed data
npm run db:drop      # Drop database

# Linting
npm run lint

Frontend

cd frontend

# Install dependencies
npm install

# Development server (port 3001)
npm run dev

# Production build
npm run build

# Linting and formatting
npm run lint
npm run format

Docker

cd docker

# First time setup
chmod +x start-backend.sh wait-for-it.sh

# Start all services
docker-compose up

# Fresh database
rm -rf data && docker-compose up

Security Considerations

JWT Authentication - 6-hour expiration, secure secret key
RBAC - Role-based access control with custom overrides
Helmet - Security headers (XSS, CSRF protection)
CORS - Configured for specific origins
SQL Injection - Prevented via Sequelize parameterized queries
Soft Delete - Paranoid mode preserves audit trail
File Validation - MIME type and size checks on upload

Performance Optimizations

Chunked Uploads - Prevents memory overload for large files
Pagination - All list endpoints support limit/offset
Lazy Loading - Redux thunks load data on demand
Asset Variants - Multiple quality levels (thumbnail, preview, etc.)
PWA Caching - Service worker caches static assets
S3 Direct Download - Assets downloaded directly from S3 via presigned URLs (bypasses backend)
Ready Blob URLs - Pre-created blob URLs stored in memory for O(1) instant lookup (getReadyBlobUrl)
Storage Key Mapping - Assets cached by canonical storage key (e.g., assets/vid.mp4) enabling cache hits regardless of presigned URL signature changes
Neighbor Preloading - BFS traversal preloads immediate neighbors (maxDepth: 1)
Priority Queue - Transitions (+150), images (+100), audio (+50), video (+30)
IndexedDB - Large files (≥5MB) stored locally for offline access
Cache API - Small files (<5MB) stored in browser Cache API
Blob URL Rendering - Native <img> tags for blob URLs to prevent Next.js Image re-fetch on re-renders

41 KiB Raw Blame History

Project Architecture - E2E Documentation

Overview

High-Level Architecture

Standard VM Deployment

Frontend Architecture

Directory Structure

Pages Structure

State Management (Redux Toolkit)

Component Organization

Custom Hooks

TypeScript Types

Frontend Configuration

Build Configuration

Backend Architecture

Directory Structure

Three-Tier Architecture

Factory Patterns

Middleware Stack

Authentication System

Runtime Context System

Permissions System

File Storage System

Database Schema

Core Domain Models

Entity Relationships

Common Fields Pattern

Key Models

Data Flow

Authentication Flow

CRUD Operation Flow

Asset Upload Flow

Asset Preloading Flow

API Patterns

Standard Endpoints

Query Parameters

Response Format

Environment Configuration

Backend (.env)

Frontend (.env.local)

Database Configuration

Key Files Reference

Frontend

Backend

Common Commands

Backend

Frontend

Docker

Security Considerations

Performance Optimizations

41 KiB

Raw Blame History