1053 lines
41 KiB
Markdown
1053 lines
41 KiB
Markdown
# Project Architecture - E2E Documentation
|
|
|
|
## Overview
|
|
|
|
**Tour Builder Platform** - A sophisticated web application for building and managing virtual tours with interactive pages, assets, transitions, and offline capabilities.
|
|
|
|
**Technology Stack:**
|
|
- **Frontend**: Next.js 15 with React 19, TypeScript, Redux Toolkit, Tailwind CSS, Serwist PWA
|
|
- **Backend**: Node.js/Express with Sequelize ORM
|
|
- **Database**: PostgreSQL with migrations and seeders
|
|
- **File Storage**: AWS S3 (configurable via `FILE_STORAGE_PROVIDER`)
|
|
|
|
---
|
|
|
|
## High-Level Architecture
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ Frontend │
|
|
│ Next.js 15 │ React 19 │ Redux Toolkit │ Tailwind │ Serwist │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
│
|
|
HTTP/REST
|
|
│
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ Backend │
|
|
│ Express.js │ Passport.js │ Sequelize ORM │ Factory Patterns │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
│
|
|
Sequelize
|
|
│
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ Database │
|
|
│ PostgreSQL │ Migrations │ Seeders │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
│
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ File Storage │
|
|
│ AWS S3 │ Google Cloud Storage │ Local Filesystem │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
---
|
|
|
|
## Standard VM Deployment
|
|
|
|
The standard Flatlogic VM adds Apache, PM2, and executor services around the
|
|
application:
|
|
|
|
```
|
|
Cloudflare
|
|
-> Apache :80
|
|
-> Next.js frontend PM2 app `frontend-dev` :3001 (production server)
|
|
-> Express backend PM2 app `backend-dev` :3000 (NODE_ENV=dev_stage)
|
|
|
|
PM2 systemd unit: pm2-ubuntu.service
|
|
Auxiliary PM2 apps: fl-executor, fl-telemetry
|
|
```
|
|
|
|
Local development now mirrors this port split: frontend `3001`, backend
|
|
`3000`. A protected backend endpoint returning `401 Unauthorized` is a healthy
|
|
response when no JWT is supplied.
|
|
|
|
For the detailed VM runbook, OOM recovery, process checks, and executor notes,
|
|
see [deployment-vm.md](deployment-vm.md).
|
|
|
|
---
|
|
|
|
## Frontend Architecture
|
|
|
|
### Directory Structure
|
|
|
|
```
|
|
frontend/src/
|
|
├── pages/ # Next.js pages (dynamic routing)
|
|
├── components/ # React components (PascalCase naming)
|
|
├── stores/ # Redux Toolkit slices (entity-based)
|
|
├── layouts/ # Page layouts (Authenticated, Guest)
|
|
├── hooks/ # Custom React hooks (~32 hooks)
|
|
├── lib/ # Utility libraries (offline, logging)
|
|
├── types/ # TypeScript interfaces and types
|
|
├── helpers/ # Utility functions
|
|
├── css/ # Global CSS (Tailwind)
|
|
├── config/ # App configuration
|
|
├── context/ # React context providers
|
|
├── schemas/ # Zod validation schemas
|
|
└── factories/ # Page and component factories
|
|
```
|
|
|
|
### Pages Structure
|
|
|
|
| Page Type | Pattern | Description |
|
|
|-----------|---------|-------------|
|
|
| List | `{entity}-list.tsx` | Data grid with CRUD, filtering, sorting |
|
|
| Edit | `{entity}-edit.tsx` | Form-based entity modification |
|
|
| View | `{entity}-view.tsx` | Read-only entity display |
|
|
| New | `{entity}-new.tsx` | Create new entity form |
|
|
|
|
**Special Pages:**
|
|
- `constructor.tsx` - Visual tour builder (complex)
|
|
- `p/[projectSlug]/index.tsx` - Production tour presentation
|
|
- `p/[projectSlug]/stage.tsx` - Stage tour preview
|
|
- `login.tsx` - Authentication page
|
|
- `dashboard.tsx` - Overview and analytics
|
|
|
|
### State Management (Redux Toolkit)
|
|
|
|
**Store Configuration** (`stores/store.ts`):
|
|
|
|
```typescript
|
|
// Entity slices (13 total)
|
|
users, roles, permissions, projects, assets, tour_pages,
|
|
asset_variants, project_audio_tracks, publish_events, pwa_caches,
|
|
access_logs, presigned_url_requests, project_memberships
|
|
|
|
// Global slices
|
|
authSlice // Authentication state, JWT token
|
|
mainSlice // UI state, notifications
|
|
styleSlice // Theme, styling preferences
|
|
```
|
|
|
|
**Redux Pattern:**
|
|
|
|
```typescript
|
|
// Entity slice factory
|
|
import { createEntitySlice } from './createEntitySlice';
|
|
|
|
const assetsSlice = createEntitySlice('assets', {
|
|
// Custom reducers if needed
|
|
});
|
|
|
|
// Usage in components
|
|
import { fetch, create, update, deleteItem } from '@/stores/assets/assetsSlice';
|
|
import { useAppDispatch, useAppSelector } from '@/stores/hooks';
|
|
|
|
const dispatch = useAppDispatch();
|
|
const { rows, loading } = useAppSelector((state) => state.assets);
|
|
|
|
dispatch(fetch({ query: '?limit=100&page=1' }));
|
|
```
|
|
|
|
### Component Organization
|
|
|
|
```
|
|
components/
|
|
├── Assets/
|
|
│ ├── AssetCard.tsx
|
|
│ ├── AssetGrid.tsx
|
|
│ ├── useAssetUploader.ts # Custom hook
|
|
│ └── ProjectSelector.tsx
|
|
├── Offline/
|
|
│ ├── OfflineToggle.tsx
|
|
│ ├── OfflineStatusIndicator.tsx
|
|
│ └── DownloadProgressPanel.tsx
|
|
├── Layout/
|
|
│ ├── Header.tsx
|
|
│ ├── Sidebar.tsx
|
|
│ └── Footer.tsx
|
|
└── [Entity]/
|
|
├── [Entity]Card.tsx
|
|
├── [Entity]Form.tsx
|
|
└── [Entity]List.tsx
|
|
```
|
|
|
|
### Custom Hooks
|
|
|
|
| Hook | Location | Purpose |
|
|
|------|----------|---------|
|
|
| `useAssetUploader` | `components/Assets/` | Chunked file upload management |
|
|
| `usePreloadOrchestrator` | `hooks/` | Asset preloading with S3 presigned URLs and ready blob URLs |
|
|
| `usePageSwitch` | `hooks/` | Smooth page navigation with preloaded blob URLs |
|
|
| `useTransitionPlayback` | `hooks/` | Transition video playback coordination |
|
|
| `useNeighborGraph` | `hooks/` | Navigation graph for preloading (maxDepth: 1) |
|
|
| `useNetworkAware` | `hooks/` | Network condition detection |
|
|
| `useReversePlayback` | `hooks/` | Video reverse playback |
|
|
| `useOfflineMode` | `hooks/` | Offline download management |
|
|
| `useStorageQuota` | `hooks/` | Storage quota monitoring |
|
|
|
|
### TypeScript Types
|
|
|
|
```
|
|
types/
|
|
├── api.ts # PaginatedResponse, FetchParams, ApiError
|
|
├── entities.ts # BaseEntity, User, Role, Project, Asset, etc.
|
|
├── redux.ts # Entity slice configuration
|
|
├── constructor.ts # Tour builder specific types
|
|
├── runtime.ts # Runtime environment types
|
|
├── offline.ts # PWA and offline types
|
|
├── preload.ts # Preloading system types
|
|
├── permissions.ts # Permission enum definitions
|
|
├── presentation.ts # Presentation runtime types
|
|
├── filters.ts # Filter system types
|
|
├── forms.ts # Form field types
|
|
└── index.ts # Central exports
|
|
```
|
|
|
|
### Frontend Configuration
|
|
|
|
```typescript
|
|
// config/config.ts
|
|
const config = {
|
|
apiBaseUrl: process.env.NEXT_PUBLIC_BACK_API || 'http://localhost:3000/api',
|
|
appTitle: 'Shimahara Visual',
|
|
storageKeys: {
|
|
darkMode: 'tour-builder-dark-mode',
|
|
stylePrefs: 'tour-builder-style',
|
|
},
|
|
};
|
|
```
|
|
|
|
### Build Configuration
|
|
|
|
```javascript
|
|
// next.config.mjs
|
|
{
|
|
// PWA with Serwist
|
|
swSrc: 'src/sw.ts',
|
|
swDest: 'public/sw.js',
|
|
disable: process.env.NODE_ENV === 'development',
|
|
|
|
// Next.js settings
|
|
trailingSlash: true,
|
|
distDir: process.env.NODE_ENV === 'development' ? '.next' : 'build',
|
|
output: process.env.NEXT_OUTPUT || undefined,
|
|
images: { unoptimized: true },
|
|
|
|
// Development
|
|
experimental: { turbo: true }
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Backend Architecture
|
|
|
|
### Directory Structure
|
|
|
|
```
|
|
backend/src/
|
|
├── routes/ # Express route handlers (22 routes)
|
|
├── db/
|
|
│ ├── models/ # Sequelize model definitions (16 models)
|
|
│ ├── api/ # Database access layer (GenericDBApi)
|
|
│ ├── migrations/ # Database schema migrations
|
|
│ ├── seeders/ # Seed data scripts
|
|
│ ├── db-config.ts # Database configuration
|
|
│ └── umzug.ts # Migration and seeder runner
|
|
├── services/ # Business logic services
|
|
├── auth/ # Passport.js strategies
|
|
├── middlewares/ # Express middlewares
|
|
├── factories/ # Factory patterns
|
|
└── utils/ # Logger, error handling
|
|
```
|
|
|
|
### Three-Tier Architecture
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ Routes Layer │
|
|
│ Express handlers │ Request validation │ Response formatting │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
│
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ Services Layer │
|
|
│ Business logic │ Transactions │ External integrations │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
│
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ DB API Layer │
|
|
│ Sequelize queries │ Filtering │ Associations │ Transformations │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
### Factory Patterns
|
|
|
|
**Router Factory** (`factories/router.factory.js`):
|
|
|
|
```javascript
|
|
// Generates standard CRUD routes automatically
|
|
module.exports = createEntityRouter('assets', AssetsService, AssetsDBApi, {
|
|
permissionEntity: 'assets',
|
|
csvFields: ['name', 'url', 'type'],
|
|
customRoutes: (router) => {
|
|
router.get('/custom', customHandler);
|
|
}
|
|
});
|
|
|
|
// Generated routes:
|
|
// GET /api/assets - List with pagination/filtering
|
|
// GET /api/assets/:id - Get single entity
|
|
// POST /api/assets - Create entity
|
|
// PUT /api/assets/:id - Update entity
|
|
// DELETE /api/assets/:id - Delete entity
|
|
// POST /api/assets/bulk-import - CSV import
|
|
// DELETE /api/assets/deleteByIds - Bulk delete
|
|
```
|
|
|
|
**Service Factory** (`factories/service.factory.js`):
|
|
|
|
```javascript
|
|
// Creates service class with transaction handling
|
|
module.exports = createEntityService('assets', AssetsDBApi);
|
|
|
|
// Generated methods:
|
|
// create({ data, currentUser, transaction, runtimeContext }) - Create with transaction
|
|
// update({ id, data, currentUser, transaction, runtimeContext }) - Update with transaction
|
|
// remove({ id, currentUser, transaction, runtimeContext }) - Delete with transaction
|
|
// bulkImport(data, options) - Batch import
|
|
// deleteByIds({ ids, currentUser, transaction, runtimeContext }) - Batch delete
|
|
```
|
|
|
|
**GenericDBApi** (`db/api/base.api.js`):
|
|
|
|
```javascript
|
|
class GenericDBApi {
|
|
// Override in entity APIs
|
|
MODEL = null;
|
|
SEARCHABLE_FIELDS = []; // Text search fields
|
|
RANGE_FIELDS = []; // Numeric/date range filters
|
|
ENUM_FIELDS = []; // Exact match filters
|
|
ASSOCIATIONS = []; // Related entity loading
|
|
|
|
// Transform input before save
|
|
getFieldMapping(data) { return data; }
|
|
|
|
// Standard operations
|
|
findAll(query, options) { }
|
|
findOne(id, options) { }
|
|
create(data, options) { }
|
|
update({ id, data, currentUser, transaction, runtimeContext }) { }
|
|
remove({ id, currentUser, transaction, runtimeContext }) { }
|
|
}
|
|
```
|
|
|
|
### Middleware Stack
|
|
|
|
```javascript
|
|
// Applied in order
|
|
app.use(helmet()); // Security headers
|
|
app.use(cors({ origin: true })); // CORS
|
|
app.use(requestLogger); // Request logging
|
|
app.use(runtimeContext); // Admin/stage/production detection
|
|
app.use(bodyParser.json()); // JSON parsing
|
|
app.use(passport.initialize()); // Auth initialization
|
|
|
|
// Per-route middleware
|
|
router.use(passport.authenticate('jwt')); // JWT validation
|
|
router.use(checkPermissions); // RBAC check
|
|
```
|
|
|
|
### Authentication System
|
|
|
|
**Passport Strategies:**
|
|
|
|
```javascript
|
|
// 1. JWT Strategy (primary)
|
|
passport.use(new JwtStrategy({
|
|
jwtFromRequest: ExtractJwt.fromAuthHeaderAsBearerToken(),
|
|
secretOrKey: process.env.SECRET_KEY,
|
|
}, async (payload, done) => {
|
|
const user = await findUserByEmail(payload.email);
|
|
if (user && !user.disabled) {
|
|
return done(null, user);
|
|
}
|
|
return done(null, false);
|
|
}));
|
|
|
|
// 2. Google OAuth 2.0
|
|
passport.use(new GoogleStrategy({
|
|
clientID: process.env.GOOGLE_CLIENT_ID,
|
|
clientSecret: process.env.GOOGLE_CLIENT_SECRET,
|
|
callbackURL: '/api/auth/signin/google/callback',
|
|
}, findOrCreateUser));
|
|
|
|
// 3. Microsoft OAuth
|
|
passport.use(new MicrosoftStrategy({...}, findOrCreateUser));
|
|
```
|
|
|
|
**JWT Configuration:**
|
|
- Secret: `process.env.SECRET_KEY`
|
|
- Expiration: 6 hours
|
|
- Payload: `{ id, email, name }`
|
|
|
|
### Runtime Context System
|
|
|
|
The platform uses **route-based environment access** (not subdomains):
|
|
|
|
| Route | Environment | Purpose |
|
|
|-------|-------------|---------|
|
|
| `/p/[slug]` | production | Public presentation |
|
|
| `/p/[slug]/stage` | stage | Preview presentation |
|
|
| `/constructor?projectId=` | dev | Editor (always dev) |
|
|
|
|
**Middleware** (`middleware/runtime-context.ts`):
|
|
|
|
```javascript
|
|
// Detects context from hostname AND headers
|
|
req.runtimeContext = {
|
|
mode: detectFromHostname(hostname), // 'admin' | 'stage' | 'production'
|
|
projectSlug: fromHostname || null,
|
|
headerEnvironment: req.headers['x-runtime-environment'], // Route-based
|
|
headerProjectSlug: req.headers['x-runtime-project-slug'], // Route-based
|
|
};
|
|
```
|
|
|
|
**Environment Resolution** (`db/api/runtime-context.ts`):
|
|
|
|
```javascript
|
|
// Priority: hostname-based → header-based
|
|
// SECURITY: Only 'production' and 'stage' allowed from headers
|
|
// 'dev' from header is ignored to prevent unauthorized access
|
|
```
|
|
|
|
**Frontend Filtering** (`RuntimePresentation.tsx`):
|
|
|
|
```javascript
|
|
// STRICT: Only exact environment match
|
|
.filter((p) => p.environment === environment)
|
|
```
|
|
|
|
See [Publishing Workflow](./publishing-workflow.md) for complete environment isolation details.
|
|
|
|
### Permissions System
|
|
|
|
```javascript
|
|
// middlewares/check-permissions.ts
|
|
const checkPermissions = async (req, res, next) => {
|
|
// 1. Self-access bypass
|
|
if (req.params.id === req.currentUser.id) {
|
|
return next();
|
|
}
|
|
|
|
// 2. Check custom permissions
|
|
const customPermission = await findCustomPermission(
|
|
req.currentUser.id,
|
|
req.permissionEntity,
|
|
req.method
|
|
);
|
|
if (customPermission) return next();
|
|
|
|
// 3. Check role-based permissions
|
|
const rolePermission = await findRolePermission(
|
|
req.currentUser.app_role,
|
|
req.permissionEntity,
|
|
req.method
|
|
);
|
|
if (rolePermission) return next();
|
|
|
|
return res.status(403).json({ error: 'Forbidden' });
|
|
};
|
|
```
|
|
|
|
### File Storage System
|
|
|
|
```javascript
|
|
// Configurable via FILE_STORAGE_PROVIDER env var
|
|
const providers = {
|
|
s3: new S3StorageProvider({
|
|
bucket: process.env.AWS_S3_BUCKET,
|
|
region: process.env.AWS_S3_REGION,
|
|
}),
|
|
gcloud: new GCloudStorageProvider({...}),
|
|
local: new LocalStorageProvider({
|
|
basePath: './uploads',
|
|
}),
|
|
};
|
|
|
|
// File service with chunked upload support
|
|
class FileService {
|
|
async initUpload(filename, totalChunks) { }
|
|
async uploadChunk(sessionId, chunkIndex, data) { }
|
|
async finalizeUpload(sessionId) { }
|
|
async getDownloadUrl(fileKey) { }
|
|
}
|
|
```
|
|
|
|
**S3 Presigned URLs for Direct Download:**
|
|
|
|
The platform supports direct asset downloads from S3 for better performance:
|
|
|
|
```javascript
|
|
// POST /api/file/presign - Get presigned URLs for batch download
|
|
// Request: { urls: ["assets/image1.jpg", "assets/video.mp4", ...] }
|
|
// Response: { presignedUrls: { "assets/image1.jpg": "https://s3...", ... } }
|
|
// - Max 50 URLs per request
|
|
// - 1-hour expiry
|
|
// - Public endpoint (no auth required for runtime)
|
|
```
|
|
|
|
Frontend preloading uses presigned URLs:
|
|
1. Request presigned URLs via `POST /api/file/presign`
|
|
2. Download directly from S3 (bypasses backend)
|
|
3. Store in Cache API (dual key: download URL + storage key)
|
|
4. Create blob URL for instant playback
|
|
5. Store in `readyBlobUrlsRef` (dual key: download URL + storage key) for O(1) lookup
|
|
|
|
**Storage Key Mapping:** Assets are cached under both download URL and canonical storage key (e.g., `assets/project/video.mp4`). This ensures cache hits even when presigned URL signatures change.
|
|
|
|
---
|
|
|
|
## Database Schema
|
|
|
|
### Core Domain Models
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ Element Defaults Hierarchy │
|
|
│ │
|
|
│ ┌─────────────────────┐ │
|
|
│ │ element_type_defaults│ ─── Global platform-wide defaults │
|
|
│ └──────────┬──────────┘ │
|
|
│ │ (auto-snapshot on project creation) │
|
|
│ ▼ │
|
|
│ ┌─────────────────────┐ │
|
|
│ │project_element_defaults│ ─── Project-specific overrides │
|
|
│ └──────────┬──────────┘ │
|
|
│ │ (applies defaults to new elements) │
|
|
│ ▼ │
|
|
│ ┌─────────────────────┐ │
|
|
│ │ Page Elements │ ─── Instance-specific on pages │
|
|
│ └─────────────────────┘ │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
|
|
┌─────────────────┐ ┌─────────────────┐
|
|
│ Projects │────<│ Tour Pages │
|
|
└─────────────────┘ └─────────────────┘
|
|
│ │
|
|
│ ▼
|
|
│ ┌─────────────────────────────────────┐
|
|
│ │ ui_schema_json (inline) │
|
|
│ │ • Page Elements │
|
|
│ │ • Navigation Links (targetPageSlug)│
|
|
│ │ • Transition Videos │
|
|
│ └─────────────────────────────────────┘
|
|
▼
|
|
┌─────────────────┐
|
|
│ Assets │
|
|
└─────────────────┘
|
|
│
|
|
▼
|
|
┌─────────────────┐
|
|
│ Asset Variants │
|
|
└─────────────────┘
|
|
|
|
Note: Page elements, navigation links, and transition configurations
|
|
are stored inline in tour_pages.ui_schema_json (not separate tables).
|
|
Navigation uses targetPageSlug for cross-environment consistency.
|
|
```
|
|
|
|
### Entity Relationships
|
|
|
|
| Entity | Relationships |
|
|
|--------|--------------|
|
|
| **Projects** | hasMany: tour_pages, assets, project_audio_tracks, publish_events, project_element_defaults, project_memberships, presigned_url_requests, pwa_caches, access_logs |
|
|
| **Tour Pages** | belongsTo: project; contains ui_schema_json with elements, navigation (targetPageSlug), and transitions (transitionVideoUrl) |
|
|
| **Assets** | belongsTo: project; hasMany: asset_variants |
|
|
| **Users** | belongsTo: role; hasMany: custom_permissions |
|
|
| **Element Type Defaults** | hasMany: project_element_defaults (as source_element) |
|
|
| **Project Element Defaults** | belongsTo: project, element_type_defaults (as source_element) |
|
|
|
|
**Note:** Page elements, navigation links, and transitions are stored inline in `tour_pages.ui_schema_json` rather than as separate tables. Navigation uses `targetPageSlug` (not UUIDs) for cross-environment consistency during publishing.
|
|
|
|
### Common Fields Pattern
|
|
|
|
All entities include:
|
|
|
|
```javascript
|
|
{
|
|
id: UUID, // Primary key
|
|
createdAt: TIMESTAMP, // Creation timestamp
|
|
updatedAt: TIMESTAMP, // Last update timestamp
|
|
deletedAt: TIMESTAMP, // Soft delete (paranoid)
|
|
createdBy: UUID, // User who created
|
|
updatedBy: UUID, // User who last updated
|
|
}
|
|
```
|
|
|
|
### Key Models
|
|
|
|
**Projects:**
|
|
```javascript
|
|
{
|
|
id, name, slug, description,
|
|
logo_url, favicon_url, og_image_url,
|
|
}
|
|
```
|
|
|
|
**Tour Pages:**
|
|
```javascript
|
|
{
|
|
id, name, slug, sort_order,
|
|
projectId, environment,
|
|
background_image_url,
|
|
background_video_url,
|
|
background_audio_url,
|
|
background_loop,
|
|
requires_auth,
|
|
ui_schema_json, // Page layout/elements
|
|
}
|
|
```
|
|
|
|
**Page Elements:**
|
|
```javascript
|
|
{
|
|
id, element_type, name, sort_order, // element_type is TEXT (not ENUM)
|
|
pageId,
|
|
is_visible,
|
|
x_percent, y_percent, // Position as percentage (0-100)
|
|
width_percent, height_percent,
|
|
rotation_deg,
|
|
content_json, // Element-specific data
|
|
style_json, // CSS properties
|
|
}
|
|
```
|
|
|
|
**Element Type Defaults (Global):**
|
|
```javascript
|
|
{
|
|
id, element_type, name, sort_order, // Unique element_type per platform
|
|
default_settings_json, // JSON-serialized default settings
|
|
// 11 predefined types: navigation_next, navigation_prev, spot, tooltip,
|
|
// description, gallery, carousel, logo, video_player, audio_player, popup
|
|
}
|
|
```
|
|
|
|
**Project Element Defaults:**
|
|
```javascript
|
|
{
|
|
id, element_type, name, sort_order,
|
|
projectId,
|
|
settings_json, // Project-specific settings override
|
|
source_element_id, // FK to element_type_defaults (for tracking)
|
|
snapshot_version, // Version for "check for updates" feature
|
|
// Auto-snapshotted from global defaults on project creation
|
|
}
|
|
```
|
|
|
|
**Assets:**
|
|
```javascript
|
|
{
|
|
id, name, projectId,
|
|
asset_type, // 'image' | 'video' | 'audio' | 'file'
|
|
type, // 'general' | 'icon' | 'background_image' | 'audio' | 'video' | 'transition' | 'logo' | 'favicon' | 'document'
|
|
cdn_url, storage_key, mime_type,
|
|
size_mb, width_px, height_px, duration_sec,
|
|
checksum, is_public,
|
|
}
|
|
```
|
|
|
|
**Navigation Elements (stored in ui_schema_json):**
|
|
```javascript
|
|
// Navigation elements with transitions are stored inline in tour_pages.ui_schema_json
|
|
{
|
|
id, element_type, // 'navigation_next', 'navigation_prev', 'spot', etc.
|
|
targetPageSlug, // Target page for navigation (not UUID)
|
|
transitionVideoUrl, // Forward transition video
|
|
transitionDurationSec, // Duration in seconds
|
|
transitionReverseMode, // 'auto_reverse' | 'separate_video' (replaces supportsReverse)
|
|
reverseVideoUrl, // Separate reverse video (when mode is 'separate_video')
|
|
// ...other element properties
|
|
}
|
|
```
|
|
|
|
**Note:** There is no separate `transitions` table. Transition data is stored directly on navigation elements within each page's `ui_schema_json`.
|
|
|
|
---
|
|
|
|
## Data Flow
|
|
|
|
### Authentication Flow
|
|
|
|
```
|
|
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
|
|
│ Login Page │────>│ POST /signin │────>│ Validate Creds │
|
|
└─────────────────┘ └─────────────────┘ └─────────────────┘
|
|
│
|
|
▼
|
|
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
|
|
│ Store Token │<────│ Return JWT │<────│ Generate JWT │
|
|
│ (sessionStorage)│ │ │ │ (6h expiry) │
|
|
└─────────────────┘ └─────────────────┘ └─────────────────┘
|
|
│
|
|
▼
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ Axios Interceptor attaches token to all subsequent requests │
|
|
│ Authorization: Bearer <token> │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
### CRUD Operation Flow
|
|
|
|
```
|
|
┌─────────────────┐
|
|
│ Redux dispatch │
|
|
│ fetch({query}) │
|
|
└────────┬────────┘
|
|
│
|
|
▼
|
|
┌─────────────────┐ ┌─────────────────┐
|
|
│ Axios GET │────>│ JWT Middleware │
|
|
│ /api/entity │ │ (validate) │
|
|
└─────────────────┘ └────────┬────────┘
|
|
│
|
|
▼
|
|
┌─────────────────┐ ┌─────────────────┐
|
|
│ Permissions │<────│ Route Handler │
|
|
│ Middleware │ │ │
|
|
└────────┬────────┘ └─────────────────┘
|
|
│
|
|
▼
|
|
┌─────────────────┐ ┌─────────────────┐
|
|
│ Service │────>│ DB API │
|
|
│ (transactions) │ │ (Sequelize) │
|
|
└─────────────────┘ └────────┬────────┘
|
|
│
|
|
▼
|
|
┌─────────────────┐ ┌─────────────────┐
|
|
│ PostgreSQL │────>│ JSON Response │
|
|
│ Query │ │ {rows, count} │
|
|
└─────────────────┘ └────────┬────────┘
|
|
│
|
|
▼
|
|
┌─────────────────────────────────────────┐
|
|
│ Redux slice updates state │
|
|
│ Components re-render with new data │
|
|
└─────────────────────────────────────────┘
|
|
```
|
|
|
|
### Asset Upload Flow
|
|
|
|
```
|
|
┌─────────────────┐ ┌─────────────────┐
|
|
│ useAssetUploader│────>│ Split into │
|
|
│ hook │ │ chunks (5MB) │
|
|
└─────────────────┘ └────────┬────────┘
|
|
│
|
|
▼
|
|
┌─────────────────┐ ┌─────────────────┐
|
|
│ POST /init │────>│ Create upload │
|
|
│ (get sessionId)│ │ session │
|
|
└─────────────────┘ └────────┬────────┘
|
|
│
|
|
▼
|
|
┌─────────────────┐ ┌─────────────────┐
|
|
│ POST /chunk │────>│ Store chunk │
|
|
│ (each chunk) │ │ (S3/local) │
|
|
└─────────────────┘ └────────┬────────┘
|
|
│ (repeat)
|
|
▼
|
|
┌─────────────────┐ ┌─────────────────┐
|
|
│ POST /finalize │────>│ Combine chunks │
|
|
│ │ │ Create Asset │
|
|
└─────────────────┘ └────────┬────────┘
|
|
│
|
|
▼
|
|
┌─────────────────────────────────────────┐
|
|
│ Return asset metadata with CDN URL │
|
|
└─────────────────────────────────────────┘
|
|
```
|
|
|
|
### Asset Preloading Flow
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ 1. Page Load - Initialize preloading │
|
|
│ └── usePreloadOrchestrator initializes │
|
|
│ └── extractPageLinksAndElements(pages) extracts assets │
|
|
│ └── useNeighborGraph builds navigation graph (maxDepth: 1) │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
│
|
|
▼
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ 2. Batch presigned URL request │
|
|
│ └── POST /api/file/presign { urls: [...] } │
|
|
│ └── Returns S3 presigned URLs (1-hour expiry, max 50) │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
│
|
|
▼
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ 3. Priority queue downloads (3 concurrent) │
|
|
│ └── Transition: +150 │ Image: +100 │ Audio: +50 │ Video: +30│
|
|
│ └── Current page: 1000 base │ Neighbor: 500 base │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
│
|
|
▼
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ 4. Download directly from S3 │
|
|
│ └── fetch(presignedUrl) │
|
|
│ └── Store in Cache API (<5MB) or IndexedDB (≥5MB) │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
│
|
|
▼
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ 5. Create ready blob URL (dual key storage) │
|
|
│ └── URL.createObjectURL(blob) │
|
|
│ └── Decode image (if applicable) │
|
|
│ └── Store in readyBlobUrlsRef Map: │
|
|
│ • key: downloadUrl → blobUrl │
|
|
│ • key: storageKey → blobUrl (canonical, never changes) │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
│
|
|
▼
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ 6. Navigation - Instant asset display │
|
|
│ └── usePageSwitch.switchToPage(targetPage) │
|
|
│ └── getReadyBlobUrl(storageKey) → O(1) instant lookup │
|
|
│ └── Smooth page transition without black flash │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
---
|
|
|
|
## API Patterns
|
|
|
|
### Standard Endpoints
|
|
|
|
Every entity follows this pattern:
|
|
|
|
| Method | Endpoint | Description |
|
|
|--------|----------|-------------|
|
|
| GET | `/api/{entity}` | List with pagination |
|
|
| GET | `/api/{entity}/:id` | Get single entity |
|
|
| POST | `/api/{entity}` | Create entity |
|
|
| PUT | `/api/{entity}/:id` | Update entity |
|
|
| DELETE | `/api/{entity}/:id` | Delete entity |
|
|
| POST | `/api/{entity}/bulk-import` | CSV import |
|
|
| DELETE | `/api/{entity}/deleteByIds` | Bulk delete |
|
|
|
|
### Query Parameters
|
|
|
|
```
|
|
# Pagination
|
|
?limit=100&page=1&offset=0
|
|
|
|
# Sorting
|
|
?sortField=createdAt&sortOrder=DESC
|
|
|
|
# Text search (SEARCHABLE_FIELDS)
|
|
?search=keyword
|
|
|
|
# Range filters (RANGE_FIELDS)
|
|
?createdAtRange=2024-01-01,2024-12-31
|
|
?sizeRange=0,1000000
|
|
|
|
# Enum filters (ENUM_FIELDS)
|
|
?type=image&status=active
|
|
|
|
# Export
|
|
?filetype=csv
|
|
```
|
|
|
|
### Response Format
|
|
|
|
```javascript
|
|
// List response
|
|
{
|
|
rows: [...], // Array of entities
|
|
count: 100, // Total count (for pagination)
|
|
}
|
|
|
|
// Single entity
|
|
{
|
|
id: "uuid",
|
|
...entityFields,
|
|
createdAt: "2024-01-01T00:00:00Z",
|
|
updatedAt: "2024-01-01T00:00:00Z",
|
|
}
|
|
|
|
// Error response
|
|
{
|
|
error: "Error message",
|
|
details: {...}, // Optional
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Environment Configuration
|
|
|
|
### Backend (.env)
|
|
|
|
```bash
|
|
# Database
|
|
DB_NAME=app_39215
|
|
DB_USER=app_39215
|
|
DB_PASS=password
|
|
DB_HOST=localhost
|
|
DB_PORT=5432
|
|
|
|
# Authentication
|
|
SECRET_KEY=jwt-secret-key
|
|
GOOGLE_CLIENT_ID=...
|
|
GOOGLE_CLIENT_SECRET=...
|
|
MS_CLIENT_ID=...
|
|
MS_CLIENT_SECRET=...
|
|
|
|
# File Storage (S3)
|
|
FILE_STORAGE_PROVIDER=s3
|
|
AWS_ACCESS_KEY_ID=...
|
|
AWS_SECRET_ACCESS_KEY=...
|
|
AWS_S3_BUCKET=tour-builder-assets
|
|
AWS_S3_REGION=us-east-1
|
|
AWS_S3_PREFIX=uploads/
|
|
|
|
# Email (AWS SES)
|
|
EMAIL_USER=...
|
|
EMAIL_PASS=...
|
|
|
|
PEXELS_KEY=...
|
|
|
|
# Runtime
|
|
# Optional. If omitted by backend scripts, src/load-env.ts defaults to dev_stage.
|
|
NODE_ENV=dev_stage
|
|
```
|
|
|
|
### Frontend (.env.local)
|
|
|
|
```bash
|
|
NEXT_PUBLIC_BACK_API=http://localhost:3000/api
|
|
```
|
|
|
|
### Database Configuration
|
|
|
|
Database configuration lives in `backend/src/db/db-config.ts`. Database
|
|
commands are executed by the typed Umzug runner in `backend/src/db/umzug.ts`.
|
|
`backend/src/load-env.ts` loads `backend/.env` for app and DB entrypoints; if
|
|
`NODE_ENV` is absent it defaults to `dev_stage`, matching the standard VM
|
|
backend process.
|
|
|
|
---
|
|
|
|
## Key Files Reference
|
|
|
|
### Frontend
|
|
|
|
| File | Purpose |
|
|
|------|---------|
|
|
| `pages/_app.tsx` | App wrapper, providers, axios interceptor |
|
|
| `stores/store.ts` | Redux store configuration |
|
|
| `stores/createEntitySlice.ts` | Entity slice factory |
|
|
| `config/config.ts` | App configuration |
|
|
| `config/preload.config.ts` | Preload priorities and settings |
|
|
| `next.config.mjs` | Next.js + Serwist config |
|
|
| `src/sw.ts` | Service worker |
|
|
| `hooks/usePreloadOrchestrator.ts` | Asset preloading with S3 presigned URLs |
|
|
| `hooks/usePageSwitch.ts` | Page navigation with preloaded blob URLs |
|
|
| `lib/extractPageLinks.ts` | Extract navigation links from ui_schema_json |
|
|
| `lib/offline/StorageManager.ts` | Cache API and IndexedDB storage |
|
|
| `components/RuntimePresentation.tsx` | Runtime with environment filtering |
|
|
|
|
### Backend
|
|
|
|
| File | Purpose |
|
|
|------|---------|
|
|
| `index.ts` | Express app setup, middleware stack |
|
|
| `factories/router.factory.ts` | CRUD route generator |
|
|
| `factories/service.factory.ts` | Service class generator |
|
|
| `db/api/base.api.ts` | GenericDBApi base class |
|
|
| `db/api/element_type_defaults.ts` | Global element defaults with auto-seed |
|
|
| `db/api/project_element_defaults.ts` | Project element defaults with snapshot |
|
|
| `services/publish.ts` | Dev → Stage → Production publishing |
|
|
| `auth/auth.ts` | Passport strategies |
|
|
| `middlewares/check-permissions.ts` | RBAC middleware |
|
|
|
|
---
|
|
|
|
## Common Commands
|
|
|
|
### Backend
|
|
|
|
```bash
|
|
cd backend
|
|
|
|
# Install dependencies
|
|
npm install
|
|
|
|
# Start server (port 3000 in dev_stage)
|
|
npm run start-dev
|
|
|
|
# Database operations
|
|
npm run db:create # Create database
|
|
npm run db:migrate # Run migrations
|
|
npm run db:seed # Seed data
|
|
npm run db:drop # Drop database
|
|
|
|
# Linting
|
|
npm run lint
|
|
```
|
|
|
|
### Frontend
|
|
|
|
```bash
|
|
cd frontend
|
|
|
|
# Install dependencies
|
|
npm install
|
|
|
|
# Development server (port 3001)
|
|
npm run dev
|
|
|
|
# Production build
|
|
npm run build
|
|
|
|
# Linting and formatting
|
|
npm run lint
|
|
npm run format
|
|
```
|
|
|
|
### Docker
|
|
|
|
```bash
|
|
cd docker
|
|
|
|
# First time setup
|
|
chmod +x start-backend.sh wait-for-it.sh
|
|
|
|
# Start all services
|
|
docker-compose up
|
|
|
|
# Fresh database
|
|
rm -rf data && docker-compose up
|
|
```
|
|
|
|
---
|
|
|
|
## Security Considerations
|
|
|
|
1. **JWT Authentication** - 6-hour expiration, secure secret key
|
|
2. **RBAC** - Role-based access control with custom overrides
|
|
3. **Helmet** - Security headers (XSS, CSRF protection)
|
|
4. **CORS** - Configured for specific origins
|
|
5. **SQL Injection** - Prevented via Sequelize parameterized queries
|
|
6. **Soft Delete** - Paranoid mode preserves audit trail
|
|
7. **File Validation** - MIME type and size checks on upload
|
|
|
|
---
|
|
|
|
## Performance Optimizations
|
|
|
|
1. **Chunked Uploads** - Prevents memory overload for large files
|
|
2. **Pagination** - All list endpoints support limit/offset
|
|
3. **Lazy Loading** - Redux thunks load data on demand
|
|
4. **Asset Variants** - Multiple quality levels (thumbnail, preview, etc.)
|
|
5. **PWA Caching** - Service worker caches static assets
|
|
6. **S3 Direct Download** - Assets downloaded directly from S3 via presigned URLs (bypasses backend)
|
|
7. **Ready Blob URLs** - Pre-created blob URLs stored in memory for O(1) instant lookup (`getReadyBlobUrl`)
|
|
8. **Storage Key Mapping** - Assets cached by canonical storage key (e.g., `assets/vid.mp4`) enabling cache hits regardless of presigned URL signature changes
|
|
9. **Neighbor Preloading** - BFS traversal preloads immediate neighbors (maxDepth: 1)
|
|
10. **Priority Queue** - Transitions (+150), images (+100), audio (+50), video (+30)
|
|
11. **IndexedDB** - Large files (≥5MB) stored locally for offline access
|
|
12. **Cache API** - Small files (<5MB) stored in browser Cache API
|
|
13. **Blob URL Rendering** - Native `<img>` tags for blob URLs to prevent Next.js Image re-fetch on re-renders
|