# Publishing Workflow Complete documentation for the Tour Builder Platform's publishing system including the three-tier environment model (dev → stage → production), publish events, and transaction locking. ## Overview The platform implements a **three-tier environment publishing system**: - **Dev** - Active editing environment (constructor always edits here) - **Stage** - Preview/testing environment for stakeholder review - **Production** - Live environment for public access The publishing workflow has two steps: 1. **Save to Stage**: Copy dev content to stage for preview 2. **Publish to Production**: Copy stage content to production Projects maintain environment-specific data (pages, audio tracks) that can be independently edited and published. The system uses database transactions with project-level locks to prevent concurrent publishing conflicts. Page elements and transitions are stored directly in `tour_pages.ui_schema_json`. ## Architecture ``` ┌─────────────────────────────────────────────────────────────────────────┐ │ Three-Tier Publishing Flow │ │ │ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ │ │ Dev │ │ Stage │ │ Production │ │ │ │ Environment │───▶│ Environment │───▶│ Environment │ │ │ └─────────────┘ └─────────────┘ └─────────────┘ │ │ │ │ │ │ │ │ Edit in │ Preview │ Public Access │ │ │ Constructor │ │ │ │ ▼ ▼ ▼ │ │ /constructor /p/[slug]/stage /p/[slug] │ │ │ │ [Save to Stage]──────────▶ │ │ [Publish to Production]──────────▶ │ └─────────────────────────────────────────────────────────────────────────┘ ┌─────────────────────────────────────────────────────────────────────────┐ │ Database Structure │ │ │ │ Projects ─────────┬─────────────────┬─────────────────┐ │ │ │ │ │ │ │ │ ▼ ▼ ▼ ▼ │ │ tour_pages audio_tracks transition_settings publish_events │ │ (env field) (env field) (env field) (history) │ │ │ │ environment: 'dev' | 'stage' | 'production' │ │ │ │ Note: Elements, links, and transitions are stored in │ │ tour_pages.ui_schema_json (no separate tables) │ └─────────────────────────────────────────────────────────────────────────┘ ``` ## Stage vs Production Environments ### Environment-Specific Entities Each of these entities maintains an `environment` field: | Entity | Environment Field | Description | |--------|------------------|-------------| | `tour_pages` | Yes | Pages with backgrounds, content, and `ui_schema_json` | | `project_audio_tracks` | Yes | Project audio files | | `project_transition_settings` | Yes | CSS transition settings (type, duration, easing, overlay color) | **Note:** Page elements, navigation links, and video transitions are stored directly in `tour_pages.ui_schema_json` and are copied with the page when publishing. Project-level CSS transition settings are stored separately in `project_transition_settings` and also copied during publish. ### URL Patterns **Stage Environment:** ``` /p/[projectSlug]/stage ``` - Route: `frontend/src/pages/p/[projectSlug]/stage.tsx` - Loads pages with `environment='stage'` only **Production Environment:** ``` /p/[projectSlug] ``` - Route: `frontend/src/pages/p/[projectSlug]/index.tsx` - Loads pages with `environment='production'` only ## Environment Isolation (Security) **CRITICAL:** Strict environment filtering prevents data leaks between environments. ### Defense in Depth The system uses **two layers** of environment protection: | Layer | File | Protection | |-------|------|------------| | **Frontend** | `RuntimePresentation.tsx` | Strict filter: `p.environment === environment` | | **Backend** | `db/api/runtime-context.ts` | Filters by `X-Runtime-Environment` header | ### Frontend Filtering Environment filtering happens in the `usePageDataLoader` hook, which is used by `RuntimePresentation.tsx`: ```typescript // hooks/usePageDataLoader.ts - STRICT environment match only // For runtime mode, filter by environment client-side if (projectSlug) { pageRows = pageRows.filter((p) => p.environment === environment); } // Sort by sort_order pageRows.sort((a, b) => (a.sort_order ?? 0) - (b.sort_order ?? 0)); ``` - Production mode (`/p/cardiff`): Only shows `environment='production'` pages - Stage mode (`/p/cardiff/stage`): Only shows `environment='stage'` pages - Dev mode (Constructor): Only shows `environment='dev'` pages - **No fallbacks** - missing environment data shows empty, never leaks from other environments ### Backend Filtering The backend also filters based on the `X-Runtime-Environment` header: ```javascript // db/api/runtime-context.ts // Only 'production' and 'stage' allowed from header // 'dev' is BLOCKED to prevent unauthorized access if (runtimeContext.headerEnvironment === 'production') return 'production'; if (runtimeContext.headerEnvironment === 'stage') return 'stage'; // 'dev' header returns null → no backend filter (constructor handles separately) ``` ### Database Constraints Environment columns are enforced at the database level: ```sql -- tour_pages.environment: NOT NULL, default 'dev' -- project_audio_tracks.environment: NOT NULL, default 'dev' -- project_transition_settings.environment: NOT NULL ``` This prevents NULL values from bypassing environment filters. **Constructor (Dev Environment):** ``` /constructor?projectId=[id] ``` - Route: `frontend/src/pages/constructor.tsx` - Always loads and edits pages with `environment='dev'` - "Save to Stage" button copies dev → stage ### Runtime Mode Detection **Primary Method: Route-Based Access** The platform uses **route-based environment access**, not subdomains: | Route | Environment | Component | |-------|-------------|-----------| | `/p/[slug]` | production | `pages/p/[projectSlug]/index.tsx` | | `/p/[slug]/stage` | stage | `pages/p/[projectSlug]/stage.tsx` | | `/constructor?projectId=` | dev | `pages/constructor.tsx` | **Frontend Headers for API Calls:** ```javascript // RuntimePresentation sends these headers headers: { 'X-Runtime-Project-Slug': projectSlug, 'X-Runtime-Environment': environment // 'production' | 'stage' | 'dev' } ``` **Backend Middleware** (`backend/src/middlewares/runtime-context.ts`): The middleware reads environment and project slug from headers for route-based access: ```javascript // req.runtimeContext structure { mode: 'admin', // default mode projectSlug: null, // not used in current implementation headerEnvironment: string | null, // from X-Runtime-Environment header ('production', 'stage', 'dev') headerProjectSlug: string | null, // from X-Runtime-Project-Slug header } ``` **Environment Resolution** (`backend/src/db/api/runtime-context.ts`): The `getRuntimeEnvironment()` function resolves environment in order: 1. Hostname-based detection (stage/production subdomains) 2. Header-based fallback (`X-Runtime-Environment` header) ```javascript // Only 'production' and 'stage' are allowed from headers // 'dev' is blocked to prevent unauthorized access to dev data if (runtimeContext.headerEnvironment === 'production') return 'production'; if (runtimeContext.headerEnvironment === 'stage') return 'stage'; ``` ### Project Slug The `slug` field on projects determines public URLs: | Property | Value | |----------|-------| | Pattern | `/^[a-z0-9_-]+$/i` | | Length | 1-255 characters | | Uniqueness | Must be unique across all projects | | Mutability | Should not change (breaks public URLs) | **Example:** ``` slug: 'cardiff' Stage URL: /p/cardiff/stage Production URL: /p/cardiff ``` ## Publish Events ### Database Schema **File:** `backend/src/db/models/publish_events.js` | Field | Type | Required | Description | |-------|------|----------|-------------| | id | UUID | Yes | Primary key | | title | STRING(255) | No* | User-provided event name (*DB allows null, service validates, max 255 chars) | | description | TEXT | No* | User-provided details (*DB allows null, service validates, max 5000 chars) | | from_environment | ENUM | Yes | Source: 'dev', 'stage', 'production' | | to_environment | ENUM | Yes | Target: 'dev', 'stage', 'production' | | status | ENUM | Yes | 'queued', 'running', 'success', 'failed' | | started_at | DATETIME | No | When publish began | | finished_at | DATETIME | No | When publish completed | | error_message | TEXT | No | Failure reason if applicable | | pages_copied | INTEGER | No | Number of pages published | | transitions_copied | INTEGER | No | Number of transitions published (legacy field, not currently populated) | | audios_copied | INTEGER | No | Number of audio tracks published | | projectId | UUID | Yes | FK to projects (CASCADE) | | userId | UUID | Yes | FK to users who initiated | ### Status Lifecycle ``` queued → running → success ↘ failed ``` | Status | Meaning | Fields Set | |--------|---------|------------| | `queued` | Event created, waiting to process | title, description, environments | | `running` | Publish in progress | started_at | | `success` | Completed successfully | finished_at, *_copied counts | | `failed` | Encountered error | finished_at, error_message | ### API Endpoints **Save to Stage (Dev → Stage):** ```http POST /api/publish/save-to-stage Content-Type: application/json Authorization: Bearer {token} { "projectId": "uuid" } ``` **Response (Success):** ```json { "success": true, "publishEventId": "uuid" } ``` **Note:** Save to Stage is **non-blocking** - the API returns immediately after creating the publish event, and the actual copy operation continues in the background. Check the `publish_events` table for final status (`success` or `failed`). **Publish to Production (Stage → Production):** *Note: Both `/api/publish` and `/api/publish/publish` route to the same handler.* ```http POST /api/publish Content-Type: application/json Authorization: Bearer {token} { "projectId": "uuid", "title": "Release 1.0.3", "description": "Added new tour pages and fixed navigation" } ``` **Response (Success):** ```json { "success": true, "publishEventId": "uuid", "summary": { "pages_copied": 5, "audios_copied": 2 } } ``` **Publish Events CRUD:** ``` GET /api/publish_events - List with pagination GET /api/publish_events/:id - Get details PUT /api/publish_events/:id - Update DELETE /api/publish_events/:id - Delete GET /api/publish_events?project=id - Filter by project ``` ## Publishing Process ### Blocking vs Non-Blocking Operations | Operation | Blocking | Behavior | |-----------|----------|----------| | **Save to Stage** | No | Returns immediately, copy runs in background via `setImmediate()` | | **Publish to Production** | Yes | Waits for entire copy operation before returning | **Save to Stage** uses background processing because it's a frequent operation during development and shouldn't block the UI. **Publish to Production** remains blocking because it's a deliberate action that users expect to complete before seeing results. ### Complete Flow (Publish to Production) ``` ┌─────────────────────────────────────────────────────────────────────────┐ │ Phase 1: Validation & Event Creation │ ├─────────────────────────────────────────────────────────────────────────┤ │ POST /api/publish │ │ ↓ │ │ PublishService.publishToProduction() │ │ ├─ Validate projectId exists │ │ ├─ Validate title is non-empty │ │ ├─ Validate description is non-empty │ │ └─ CREATE publish_events record with status='queued' │ └─────────────────────────────────────────────────────────────────────────┘ ↓ ┌─────────────────────────────────────────────────────────────────────────┐ │ Phase 2: Acquire Lock & Begin Transaction │ ├─────────────────────────────────────────────────────────────────────────┤ │ PublishService.withProjectPublishLock(projectId, callback) │ │ ├─ BEGIN TRANSACTION │ │ ├─ SELECT * FROM projects WHERE id={id} FOR UPDATE │ │ ├─ SELECT * FROM publish_events WHERE status='running' FOR UPDATE │ │ └─ If running publish exists → ERROR 400 │ └─────────────────────────────────────────────────────────────────────────┘ ↓ ┌─────────────────────────────────────────────────────────────────────────┐ │ Phase 3: Copy Stage → Production │ ├─────────────────────────────────────────────────────────────────────────┤ │ 1. FETCH all stage data: │ │ ├─ tour_pages WHERE environment='stage' │ │ ├─ project_audio_tracks WHERE environment='stage' │ │ └─ project_transition_settings WHERE environment='stage' │ │ │ │ 2. PURGE existing production data: │ │ ├─ DELETE tour_pages WHERE environment='production' │ │ ├─ DELETE project_audio_tracks WHERE environment='production' │ │ └─ DELETE project_transition_settings WHERE environment='production'│ │ │ │ 3. CREATE production records via bulkCreate: │ │ ├─ tour_pages (with environment='production', source_key tracking) │ │ ├─ project_audio_tracks │ │ └─ project_transition_settings (CSS transition defaults) │ │ │ │ Note: Video transitions are stored in tour_pages.ui_schema_json and │ │ copied with the page. CSS transition settings (fade, duration, │ │ easing) are stored in project_transition_settings per environment.│ │ Slugs are used for navigation (not IDs), so no remapping needed. │ │ │ │ Element defaults (element_type_defaults → project_element_defaults) │ │ are NOT environment-specific. Settings are embedded in elements when │ │ created, so ui_schema_json contains complete elements with settings. │ └─────────────────────────────────────────────────────────────────────────┘ ↓ ┌─────────────────────────────────────────────────────────────────────────┐ │ Phase 4: Mark Success or Failure │ ├─────────────────────────────────────────────────────────────────────────┤ │ Success: │ │ UPDATE publish_events SET status='success', finished_at=NOW(), │ │ pages_copied=N, audios_copied=N │ │ │ │ Failure: │ │ UPDATE publish_events SET status='failed', finished_at=NOW(), │ │ error_message=error.message │ │ ROLLBACK TRANSACTION │ └─────────────────────────────────────────────────────────────────────────┘ ``` ### Data Copied During Publish **Entities Copied:** | Entity | Copied | Notes | |--------|--------|-------| | Tour Pages | ✅ | All stage pages → production (includes `sort_order`, `ui_schema_json` with elements, navigation, video transitions, and page media fields) | | Project Audio | ✅ | All stage audio → production | | Project Transition Settings | ✅ | CSS transition settings (type, duration, easing, overlay color) | **Entities NOT Copied:** | Entity | Reason | |--------|--------| | Assets | Shared across environments | | Project metadata | Name, slug, description unchanged | | User permissions | Independent of environment | | PWA caches | Generated separately | | Element type defaults | Global platform-wide (not project-specific) | | Project element defaults | Project-wide (not environment-specific) - settings are already embedded in elements within `ui_schema_json` when elements are created | ### Page Order Propagation Page order is stored on `tour_pages.sort_order`. Runtime loaders sort pages by this field and use the first sorted page as the presentation entry page. The constructor can change the dev page set and order directly: - Reorder pages via `POST /api/tour_pages/reorder`. This updates only `tour_pages.sort_order`. - Duplicate the active dev page via `POST /api/tour_pages/:id/duplicate`. This creates a new independent dev page at the end of the order, copies page settings and `ui_schema_json`, and regenerates inline element IDs. - Delete the active dev page via `DELETE /api/tour_pages/:id` after constructor confirmation. Stage and production are intentionally read-only for direct constructor page writes: 1. Reorder, duplicate, or delete pages in the constructor (`environment='dev'`). 2. Click Save to Stage to copy dev pages, including `sort_order`, to `environment='stage'`. 3. Publish to Production to copy stage pages, including `sort_order`, to `environment='production'`. This means stage preview keeps the previous page set/order until Save to Stage finishes, and the public production presentation keeps the previous page set/order until Publish finishes. ### Data Sanitization Before copying, records are sanitized: ```javascript sanitizeRecordForClone(modelInstance) { const data = modelInstance.toJSON(); // Delete auto-generated fields delete data.id; // Gets new UUID delete data.createdAt; // New timestamp delete data.updatedAt; // New timestamp delete data.deletedAt; // Not paranoid for production delete data.deletedBy; // Clear soft-delete actor delete data.importHash; // Unique field // Ensure JSON fields are objects, not strings (avoid double-encoding) if (data.ui_schema_json && typeof data.ui_schema_json === 'string') { try { data.ui_schema_json = JSON.parse(data.ui_schema_json); } catch { // Keep as-is if parsing fails } } return data; } ``` ## Transaction Locking ### Concurrency Control The system prevents concurrent publishing using row-level locks: ```javascript // Acquire exclusive lock on project row const project = await db.projects.findByPk(projectId, { transaction, lock: transaction.LOCK.UPDATE // SELECT FOR UPDATE }); // Check for running publish (also locked) const runningEvent = await db.publish_events.findOne({ where: { projectId, status: 'running' }, transaction, lock: transaction.LOCK.UPDATE }); if (runningEvent) { throw Error('Publish already running for this project'); } ``` ### Lock Properties | Property | Value | |----------|-------| | **Isolation Level** | READ COMMITTED (Postgres default) | | **Lock Duration** | Transaction start → commit/rollback | | **Lock Scope** | Project row + publish_events rows | | **Deadlock Handling** | Auto-rollback on deadlock | | **Lock Timeout** | 5-30 seconds (DB config) | ### Lock Workflow ``` 1. Transaction Starts └─ BEGIN TRANSACTION 2. Acquire Project Lock └─ SELECT * FROM projects WHERE id=? FOR UPDATE 3. Check Running Publishes └─ SELECT * FROM publish_events WHERE status='running' FOR UPDATE └─ If exists → ROLLBACK + ERROR 400 4. Execute Publish Operations └─ All operations protected by transaction └─ Concurrent requests block until lock released 5. Commit or Rollback └─ Success → COMMIT (locks released) └─ Error → ROLLBACK (locks released) ``` ### Error Handling **Pre-Lock Validation:** ```javascript if (!projectId || !title || !description) { throw new ValidationError('Missing required fields'); // No transaction started } ``` **Lock Conflict:** ```javascript if (runningEvent) { throw new ValidationError('Publish is already running for this project'); // HTTP 400 returned } ``` **Data Errors:** ```javascript try { await copyStageToProduction(projectId, userId, transaction); } catch (error) { // Transaction automatically rolls back // Error captured in publish_events.error_message throw error; } ``` ## Frontend Integration ### Save to Stage Button (Constructor) **Files:** - `frontend/src/pages/constructor.tsx` - Uses the hook - `frontend/src/hooks/useConstructorPageActions.ts` - Contains the `saveToStage` implementation The Constructor uses the `useConstructorPageActions` hook which provides the `saveToStage` function: ```typescript // hooks/useConstructorPageActions.ts const saveToStage = useCallback(async () => { if (!projectId) { onError?.('Project ID is required to save to stage.'); return; } // First save current state, then copy to stage await saveConstructor(); try { setIsSavingToStage(true); // Note: axios baseURL adds '/api' prefix automatically // Non-blocking: returns immediately, copy runs in background await axios.post('/publish/save-to-stage', { projectId }); onSuccess?.('Saved to stage.'); } catch (error: any) { onError?.(error?.response?.data?.message || 'Failed to save to stage'); } finally { setIsSavingToStage(false); } }, [projectId, saveConstructor, onError, onSuccess]); ``` **Note:** The Save to Stage operation is non-blocking - the button returns to normal immediately while the actual copy operation continues in the background. The user sees a brief "Saved to stage" confirmation. // constructor.tsx - Hook usage const { isSavingToStage, saveToStage, // ... other actions } = useConstructorPageActions({ projectId, pages, // ... other options }); // UI passes saveToStage to ConstructorMenu component ``` ### Publish Status Visibility The platform displays timestamps showing when content was last saved or published, providing users with visual feedback on content freshness. #### usePublishStatus Hook **File:** `frontend/src/hooks/usePublishStatus.ts` Fetches the last successful publish events for a project: ```typescript interface UsePublishStatusResult { lastSavedToStage: string | null; // Last dev → stage timestamp lastPublishedToProduction: string | null; // Last stage → production timestamp isLoading: boolean; refresh: () => Promise; // Refresh after new publish } const { lastPublishedToProduction, refresh } = usePublishStatus({ projectId }); ``` #### Timestamp Display Timestamps are displayed inside buttons using the `subtitle` prop of `BaseButton`: **Project Dashboard (Publish to Production):** ```typescript setIsPublishModalActive(true)} /> // Shows: "Publish to Production" with "Last: 5 min ago" below ``` **Constructor Menu (Save / Save to Stage):** ```typescript // Project-level save timestamp: most recent updatedAt across all pages const lastProjectSaveAt = useMemo(() => { if (!pages.length) return null; return pages.reduce((latest, page) => { if (!page.updatedAt) return latest; if (!latest) return page.updatedAt; return new Date(page.updatedAt) > new Date(latest) ? page.updatedAt : latest; }, null as string | null); }, [pages]); // Shows: "Save" with project-level timestamp (same on all pages) ``` #### Relative Timestamp Format The `dataFormatter.relativeTimestamp()` method formats dates as human-readable relative times: | Time Difference | Display Format | |-----------------|----------------| | < 2 minutes | "Just now" | | < 60 minutes | "5 min ago" | | < 24 hours | "2 hours ago" | | Same day | "Today at 14:30" | | Yesterday | "Yesterday at 09:15" | | Older | "Apr 28 at 16:45" | ### Publish Button (Project Dashboard) **File:** `frontend/src/pages/projects/[projectsId].tsx` ```typescript const [isPublishing, setIsPublishing] = useState(false); const [isPublishModalActive, setIsPublishModalActive] = useState(false); const [publishTitle, setPublishTitle] = useState(''); const [publishDescription, setPublishDescription] = useState(''); // Publish status for timestamp display const { lastPublishedToProduction, refresh: refreshPublishStatus } = usePublishStatus({ projectId, }); setIsPublishModalActive(true)} disabled={isPublishing || !projectId} /> ``` ### Publish Modal ```typescript { if (!isPublishing) setIsPublishModalActive(false); }} > setPublishTitle(e.target.value)} />