# Assets Preloading Feature - E2E Documentation ## Overview The Tour Builder Platform implements a sophisticated dual-mode asset preloading system that optimizes user experience in both online and offline scenarios. The system intelligently prefetches assets based on navigation patterns, network conditions, and storage constraints. --- ## Architecture ``` ┌─────────────────────────────────────────────────────────────────┐ │ Configuration Layer │ │ preload.config.ts │ offline.config.ts │ └─────────────────────────────────────────────────────────────────┘ │ ┌─────────────────────────────────────────────────────────────────┐ │ Hook Layer │ │ usePreloadOrchestrator │ usePageNavigationState │ useNetworkAware│ │ usePreloadProgress │ usePWAPreload │ useOfflineMode │ │ useStorageQuota │ useIconPreload │ └─────────────────────────────────────────────────────────────────┘ │ ┌─────────────────────────────────────────────────────────────────┐ │ Library Layer │ │ DownloadEventBus │ DownloadManager │ StorageManager │ │ OfflineDbManager │ extractPageLinks │ assetCache │ └─────────────────────────────────────────────────────────────────┘ │ ┌─────────────────────────────────────────────────────────────────┐ │ Service Worker (Serwist) │ │ sw.ts - CacheFirst, NetworkFirst, Range Request Support │ └─────────────────────────────────────────────────────────────────┘ ``` **Recent Refactor:** The preloading system was simplified to use a "stream-first" approach: - **Online mode:** Preloads current page + outgoing transition videos only (no neighbor preloading) - **Offline mode:** Full project assets downloaded via `useOfflineMode.startDownload()` - **Video playback:** Streams on-demand, then caches for replay (no bandwidth competition) - `useNeighborGraph` was removed (neighbor preloading caused excessive network requests) - `usePageSwitch`, `useBackgroundTransition`, and other hooks were consolidated into `usePageNavigationState` --- ## Online Mode (Stream-First) ### How It Works Online mode uses a **stream-first** approach orchestrated by `usePreloadOrchestrator.ts`: - **Current page assets** are preloaded (backgrounds, element images) - **Outgoing transition videos** are preloaded for instant playback - **Other videos** stream on-demand using presigned URLs (browser handles buffering) - **No neighbor page preloading** - eliminated to reduce network contention ### Asset Discovery Flow 1. **Page Change Detection** - Monitors `currentPageId` changes 2. **Current Page Asset Extraction** - Extracts URLs from: - **Page backgrounds** - `background_image_url`, `background_video_url`, `background_audio_url` - **Element content** - URLs from `ui_schema_json` elements 3. **Transition Video Extraction** - Gets `transitionVideoUrl` from outgoing page links 4. **Priority Assignment** - Assigns download priority based on asset type ### Asset URL Extraction Fields The system extracts URLs from these `content_json` fields (configured in `PRELOAD_CONFIG.assetFields`): **Direct Fields:** `iconUrl`, `imageUrl`, `mediaUrl`, `videoUrl`, `audioUrl`, `transitionVideoUrl`, `backgroundImageUrl`, `reverseVideoUrl`, `carouselPrevIconUrl`, `carouselNextIconUrl`, `src`, `url`, `poster`, `thumbnail` **Nested Arrays:** - `galleryCards[].imageUrl`, `galleryCards[].videoUrl` - `carouselSlides[].imageUrl`, `carouselSlides[].videoUrl` **Page Links:** - `pageLink.transition.video_url` (eagerly loaded from backend API) ### Priority Calculation ``` Priority = basePriority + assetTypeBonus + variantBonus Current Page Assets: priority = 1000 + bonuses Transition Videos: priority = 1000 + 150 = 1150 (highest) ``` **Asset Type Multipliers:** | Type | Weight | Notes | |------|--------|-------| | Transition | 150 | Highest - needed immediately on navigation click | | Image | 100 | Background and UI elements | | Audio | 50 | Background audio tracks | | Video | 30 | Lower priority since they stream | **Note:** Neighbor page preloading was removed to simplify the system and reduce network contention. Videos stream on-demand using presigned URLs with browser-managed buffering. **Variant Multipliers:** | Variant | Weight | |---------|--------| | Thumbnail | 50 | | Preview | 40 | | WebP | 35 | | MP4 Low | 20 | | MP4 High | 10 | | Original | 5 | ### Network-Aware Concurrency The system adapts download concurrency based on network conditions via `useNetworkAware.ts`: | Connection | Concurrent Downloads | Strategy | |------------|---------------------|----------| | 4g / Good | 3 | Aggressive | | 3g | 2 | Normal | | 2g / Slow | 1 | Conservative | | Save-Data | 1 | Minimal | **Network Thresholds:** - Aggressive: 4g OR downlink ≥ 5Mbps - Suggest offline: slow-2g OR 2g OR RTT > 500ms OR downlink < 0.5Mbps - Low quality: save-data flag OR slow-2g OR downlink < 1Mbps ### Download Process 1. Fetch asset with progress tracking via ReadableStream 2. Emit progress events via `DownloadEventBus` 3. Store in Cache API (`tour-builder-assets-v1`) under download URL 4. **Store in Cache API under storage key** (enables post-refresh lookups) 5. **Create Blob URL**: `URL.createObjectURL(blob)` for local reference 6. **Image Decoding**: Call `image.decode()` on the blob URL to eliminate white flash on navigation 7. **Store Ready URLs**: Map both download URL AND storage key to blob URL in `readyBlobUrlsRef` 8. Track cached assets in memory Set for quick lookups **Storage Key Mapping (Key Feature):** The system maps assets by **canonical storage key** (e.g., `assets/project-123/video.mp4`) in addition to download URLs. This solves the presigned URL mismatch problem where different requests generate different signatures. ``` ┌─────────────────────────────────────────────────────────────────────────────────────────┐ │ STORAGE KEY MAPPING │ │ │ │ Storage Key: "assets/project-123/video.mp4" ← CANONICAL IDENTIFIER (never changes) │ │ │ │ │ ┌──────────────┴──────────────┐ │ │ ▼ ▼ │ │ Download URL #1 Download URL #2 │ │ "https://s3...?Sig=ABC" "https://s3...?Sig=XYZ" │ │ │ │ │ │ ▼ ▼ │ │ ┌─────────────────────────────────────────────────────────────────────┐ │ │ │ readyBlobUrlsRef Map │ │ │ │ key: storageKey ────────────► blob://... │ │ │ │ key: downloadUrl ────────────► blob://... (same blob URL) │ │ │ └─────────────────────────────────────────────────────────────────────┘ │ │ │ │ Lookup: getReadyBlobUrl(storageKey) → O(1) instant → blob://... │ └─────────────────────────────────────────────────────────────────────────────────────────┘ ``` **Why storage key mapping matters:** | Scenario | Download URL | Storage Key | Lookup Result | |----------|-------------|-------------|---------------| | Same session | `https://s3...?Sig=ABC` | `assets/vid.mp4` | ✅ Instant via storage key | | New presigned URL | `https://s3...?Sig=XYZ` | `assets/vid.mp4` | ✅ Instant via storage key | | Page refresh | N/A (in-memory cleared) | `assets/vid.mp4` | ✅ From Cache API by storage key | **Ready Blob URL Flow:** ``` Download → Cache API (URL key) → Cache API (storage key) → Blob URL → Image Decode ↓ ┌── readyBlobUrlsRef.set(downloadUrl, blobUrl) └── readyBlobUrlsRef.set(storageKey, blobUrl) └── readyUrlsVersion++ (triggers re-render) ↓ Navigation Click → getReadyBlobUrl(storageKey) → O(1) lookup → Instant display ``` **Re-render Trigger Mechanism:** Since `readyBlobUrlsRef` is a ref (not state), updates don't trigger React re-renders. The `readyUrlsVersion` counter solves this: 1. When a blob URL becomes ready, `setReadyUrlsVersion(v => v + 1)` is called 2. Components using `readyUrlsVersion` in dependency arrays re-render 3. `resolveUrlWithBlob` callbacks are recreated with fresh lookups 4. UI switches from direct URLs to cached blob URLs ```typescript // In constructor.tsx const resolveUrlWithBlob = useCallback( (url) => { const blobUrl = preloadOrchestrator.getReadyBlobUrl(url); if (blobUrl) return blobUrl; return resolveAssetPlaybackUrl(url); }, [preloadOrchestrator, preloadOrchestrator.readyUrlsVersion], // Re-render when ready ); ``` ### Presigned URL Integration For S3 storage, the preloader fetches presigned URLs before downloading assets for direct S3 access: ```typescript // In usePreloadOrchestrator.ts // 1. Collect storage paths that need presigning const storagePaths = assets .map(a => a.url) .filter(isRelativeStoragePath); // 2. Batch fetch presigned URLs (auto-batches and caches) queuePresignedUrls(storagePaths) .then(() => { // 3. Add assets to queue with resolved URLs assets.forEach(asset => { const presignedUrl = getPresignedUrl(asset.url); const resolvedUrl = presignedUrl || resolveAssetPlaybackUrl(asset.url); addToQueue({ url: resolvedUrl, storageKey: asset.url, ... }); }); }) .catch(() => { // Fallback to proxy URLs addAssetsToQueue(); }); // 4. On successful download, mark presigned URLs as verified // (enables resolveAssetPlaybackUrl to use presigned URLs) if (isPresignedUrl(item.url)) { markPresignedUrlsVerified(); } ``` **Key Functions (lib/assetUrl.ts):** | Function | Purpose | |----------|---------| | `queuePresignedUrl()` | Queue single URL for presigning | | `queuePresignedUrls()` | Batch queue URLs for presigning | | `flushPresignedUrlQueue()` | Fetch queued presigned URLs | | `getPresignedUrl()` | Get cached presigned URL (sync) | | `resolveAssetPlaybackUrl()` | Resolve URL with presigned cache lookup | | `markPresignedUrlFailed()` | Mark presigned URL as failed for specific key | | `markPresignedUrlsVerified()` | Mark presigned URLs as verified (enables playback URL resolution) | | `isRelativeStoragePath()` | Check if URL is a relative storage path (needs presigning) | | `extractStoragePath()` | Extract storage path from full URL (handles S3, CDN, proxy URLs) | | `setupPresignedUrlInterceptor()` | Setup Axios interceptor for CORS failure detection | | `disablePresignedUrls()` | Globally disable presigned URLs (fallback to proxy) | | `arePresignedUrlsDisabled()` | Check if presigned URLs are disabled | | `clearPresignedUrlCache()` | Clear all cached presigned URLs | **Key Functions (usePreloadOrchestrator):** | Function | Purpose | |----------|---------| | `getReadyBlobUrl(key)` | O(1) instant lookup by storage key or resolved URL | | `getCachedBlobUrl(key)` | Async lookup from Cache API by storage key or resolved URL | | `isUrlPreloaded(key)` | Check if asset is ready for instant display | **Lookup Priority (in usePageSwitch and useTransitionPlayback):** ``` 1. getReadyBlobUrl(storageKey) → instant O(1) (same session) 2. getCachedBlobUrl(storageKey) → Cache API (~5ms, post-refresh) 3. getReadyBlobUrl(resolvedUrl) → fallback 4. getCachedBlobUrl(resolvedUrl) → fallback 5. Network fetch → last resort ``` **Fallback Behavior:** - If presigned URL fails (CORS not configured), DownloadManager automatically retries with proxy URL - This fallback is built into DownloadManager as single source of truth (no duplicate retry logic) - Retry happens immediately with reset retry count - Logger tracks presigned URL status changes: `[DownloadManager] Presigned URL failed, retrying with proxy` ### Key Files | File | Location | Purpose | |------|----------|---------| | `usePreloadOrchestrator.ts` | `frontend/src/hooks/` | Main orchestrator with ready blob URL management | | `usePageSwitch.ts` | `frontend/src/hooks/` | Page navigation using preloaded blob URLs | | `useNeighborGraph.ts` | `frontend/src/hooks/` | BFS navigation graph, extracts page backgrounds + element assets | | `useNetworkAware.ts` | `frontend/src/hooks/` | Network condition monitoring | | `useIconPreload.ts` | `frontend/src/hooks/` | Constructor icon preloading to prevent flash | | `extractPageLinks.ts` | `frontend/src/lib/` | Extract navigation targets and preload elements from pages | | `preload.config.ts` | `frontend/src/config/` | Queue and priority settings | | `assetUrl.ts` | `frontend/src/lib/` | URL resolution with presigned URL cache | ### Key Methods | Method | Hook | Purpose | |--------|------|---------| | `getReadyBlobUrl(key)` | usePreloadOrchestrator | O(1) instant lookup by storage key or resolved URL | | `getCachedBlobUrl(key)` | usePreloadOrchestrator | Async lookup from Cache API by storage key or URL | | `isUrlPreloaded(key)` | usePreloadOrchestrator | Check if asset is ready for instant display | | `preloadAsset(url)` | usePreloadOrchestrator | Manually trigger preload for a specific URL | | `clearQueue()` | usePreloadOrchestrator | Clear the preload queue | | `switchToPage(page, onSwitched?)` | usePageSwitch | Navigate with preloaded assets (resolves by storage key first) | | `setBackgroundsDirectly(img, vid, aud)` | usePageSwitch | Set backgrounds without transition overlay (initial load) | | `markBackgroundReady()` | usePageSwitch | Mark new background as ready (call from Image onLoad) | | `clearPreviousBackground()` | usePageSwitch | Clear overlay after transition completes | --- ## Offline Mode ### Storage Strategy The system uses a hybrid storage approach for optimal performance: ``` Assets < 5MB → Cache API (tour-builder-assets-v1) Assets ≥ 5MB → IndexedDB (TourBuilderOffline) ``` **Why Hybrid:** - Cache API has better browser support but size limitations - IndexedDB provides reliable large file storage - `StorageManager` abstracts both for transparent access ### IndexedDB Schema (Dexie.js) **Database:** `TourBuilderOffline` (version 1) **Tables and Indexes:** ```javascript { assets: 'id, projectId, url, variantType, assetType, downloadedAt', projects: 'id, slug, status, lastSyncedAt', downloadQueue: 'id, projectId, status, priority, addedAt' } ``` **TypeScript Interfaces:** ```typescript // OfflineAsset - Large files stored in IndexedDB interface OfflineAsset { id: string; projectId: string; url: string; filename: string; variantType: AssetVariantType; assetType: AssetType; mimeType: string; sizeBytes: number; blob: Blob; downloadedAt: number; } // OfflineProject - Project offline status interface OfflineProject { id: string; slug: string; name: string; status: ProjectOfflineStatus; totalAssets: number; downloadedAssets: number; totalSizeBytes: number; downloadedSizeBytes: number; lastSyncedAt?: number; version?: string; } // DownloadQueueItem - Persistent download queue interface DownloadQueueItem { id: string; projectId: string; assetId: string; url: string; filename: string; status: PreloadJobStatus; priority: number; retryCount: number; bytesLoaded: number; totalBytes: number; addedAt: number; lastAttemptAt?: number; error?: string; } ``` ### Asset Storage Flow ``` storeAsset(blob, metadata) │ ├── Size < 5MB? │ │ │ YES → Create Response with headers │ → Store in Cache API │ └── Size ≥ 5MB? │ YES → Create OfflineAsset object → Store in IndexedDB ``` ### Asset Retrieval Flow ``` getAsset(url) [StorageManager.getAsset()] │ ├── Check IndexedDB first (large files) │ │ │ └── Found? → Return blob │ └── Check Cache API (small files) │ └── Found? → Return response blob ``` ### Service Worker (Serwist) Located at `frontend/src/sw.ts`, handles offline caching: | Resource Type | Strategy | Cache Name | |--------------|----------|------------| | Static Assets (images, fonts, css, js) | CacheFirst | `tour-builder-assets-v1` | | Videos | CacheFirst + Range | `tour-builder-assets-v1` | | API Responses | NetworkFirst | `api-cache` | | Pages | NetworkFirst | (default) | **Unified Cache Keys (cacheKeyWillBeUsed Plugin):** The Service Worker normalizes all asset URLs to storage keys using `cacheKeyWillBeUsed` plugin. This ensures preload (DownloadManager) and browser loads (SW interception) use the same cache keys: ```typescript // SW plugin for static assets handler cacheKeyWillBeUsed: async ({ request, mode }) => { const storagePath = extractStoragePathFromUrl(request.url); if (storagePath) { return new Request(storagePath); } return request; } ``` **Why this matters:** | Before | After | |--------|-------| | Preload stores: `assets/project/file.jpg` | Same | | Browser load stores: `https://s3.../file.jpg?X-Amz-Signature=abc` | `assets/project/file.jpg` | | New session with new presigned URL → cache MISS | cache HIT (same key) | **Range Request Support for Videos:** - Handles byte-range requests for video seeking - Extracts range from cached response - Returns partial content (HTTP 206) ### Asset Discovery (Unified Frontend) Located at `frontend/src/lib/assetCache/`, provides unified asset discovery for both online preload and offline download: ```typescript // Discover all assets for a project (used by offline mode) discoverProjectAssets(pages, pageLinks, elements): AssetToCache[] // Get prioritized assets for preloading (used by online preload) getPrioritizedAssets(currentPageId, pages, pageLinks, elements, neighbors): AssetToCache[] ``` **Asset Discovery Sources:** - Page backgrounds: `background_image_url`, `background_video_url`, `background_audio_url` - Element content_json: All URL fields from `PRELOAD_CONFIG.assetFields.all` - Transition videos: `transition.video_url` from page links - Nested arrays: `galleryCards[]`, `carouselSlides[]` **AssetToCache Structure:** ```typescript interface AssetToCache { storageKey: string; // Canonical key for caching originalUrl: string; // Original URL from data assetType: AssetType; // 'image' | 'video' | 'audio' | 'transition' | 'other' pageId: string; // Page the asset belongs to priority: number; // Download priority (higher = first) } ``` ### Navigation Data in ui_schema_json Navigation targets and transition videos are stored directly in `tour_pages.ui_schema_json`: ```typescript // Element with navigation and transition { type: "navigation_next", targetPageSlug: "gallery", // Slug-based navigation transitionVideoUrl: "assets/.../transition.mp4", // Inline transition transitionDurationSec: 1.5, } ``` The preload system extracts navigation targets from elements to build the neighbor graph, and transition video URLs are used directly for preloading. ### Key Files | File | Location | Purpose | |------|----------|---------| | `StorageManager.ts` | `frontend/src/lib/offline/` | Cache API + IndexedDB abstraction | | `DownloadManager.ts` | `frontend/src/lib/offline/` | Download queue with auto proxy fallback | | `OfflineDbManager.ts` | `frontend/src/lib/offlineDb/` | Dexie.js IndexedDB manager | | `schema.ts` | `frontend/src/lib/offlineDb/` | Dexie.js schema definition | | `AssetCacheService.ts` | `frontend/src/lib/assetCache/` | Unified asset caching service | | `assetDiscovery.ts` | `frontend/src/lib/assetCache/` | Shared asset extraction logic | | `offline.config.ts` | `frontend/src/config/` | Offline settings | | `sw.ts` | `frontend/src/` | Serwist service worker | | `useNeighborGraph.ts` | `frontend/src/hooks/` | Builds navigation graph from elements | --- ## Progress Tracking ### Event System The `DownloadEventBus` singleton emits events consumed by `usePreloadProgress`: ``` DownloadEventBus │ ├── preloadStart { jobId, assetId, url } ├── preloadProgress { jobId, progress, bytesLoaded, totalBytes } ├── preloadComplete { jobId, assetId } └── preloadError { jobId, assetId, error } │ ▼ usePreloadProgress Hook │ ├── Maintains jobs[] array ├── Calculates totalProgress ├── Auto-removes completed (3s) └── Auto-removes errors (10s) │ ▼ UI Components │ ├── OfflineToggle ├── OfflineStatusIndicator └── DownloadProgressPanel ``` --- ## Configuration Reference ### preload.config.ts ```typescript export const PRELOAD_CONFIG = { // Queue settings maxConcurrentDownloads: 3, maxRetries: 3, retryDelayMs: 1000, // Size thresholds largeFileThreshold: 5 * 1024 * 1024, // 5MB -> use IndexedDB videoChunkSize: 5 * 1024 * 1024, // 5MB chunks initialVideoBufferSeconds: 5, // Priority weights priority: { currentPage: 1000, neighborBase: 500, assetType: { transition: 150, // Highest - needed immediately on navigation click image: 100, audio: 50, video: 30, }, variant: { thumbnail: 50, preview: 40, webp: 35, mp4_low: 20, mp4_high: 10, original: 5, }, linkCountMultiplier: 10, maxLinkBonus: 50, }, // Storage thresholds storage: { warningPercent: 80, criticalPercent: 95, minFreeBuffer: 50 * 1024 * 1024, // 50MB }, // Auto-cleanup timeouts autoRemove: { completedMs: 3000, errorMs: 10000, }, // Neighbor graph traversal neighborGraph: { maxDepth: 1, // Only preload immediate neighbors (reduced from 2) constructorMaxDepth: 1, // Same as maxDepth for constructor }, // Asset URL field names for extraction assetFields: { all: [ 'iconUrl', 'imageUrl', 'mediaUrl', 'videoUrl', 'audioUrl', 'transitionVideoUrl', 'backgroundImageUrl', 'reverseVideoUrl', 'carouselPrevIconUrl', 'carouselNextIconUrl', 'src', 'url', 'poster', 'thumbnail' ], images: [ 'iconUrl', 'imageUrl', 'backgroundImageUrl', 'carouselPrevIconUrl', 'carouselNextIconUrl', 'src' ], nested: ['galleryCards', 'carouselSlides'], nestedUrlFields: ['imageUrl', 'videoUrl'], }, }; ``` ### offline.config.ts ```typescript export const OFFLINE_CONFIG = { // IndexedDB dbName: 'TourBuilderOffline', dbVersion: 1, // Cache names (for Cache API) cacheNames: { static: 'tour-builder-static-v1', dynamic: 'tour-builder-dynamic-v1', assets: 'tour-builder-assets-v1', }, // Events (EventEmitter event names) events: { preloadStart: 'asset-preload-start', preloadProgress: 'asset-preload-progress', preloadComplete: 'asset-preload-complete', preloadError: 'asset-preload-error', projectDownloadProgress: 'project-download-progress', projectDownloadComplete: 'project-download-complete', queueUpdate: 'queue-update', }, // Service worker settings serviceWorker: { scope: '/', updateInterval: 60 * 60 * 1000, // 1 hour }, // Storage settings storage: { cacheApiMaxSize: 5 * 1024 * 1024, // 5MB indexedDbMinSize: 5 * 1024 * 1024, // 5MB }, // Retry settings retry: { maxRetries: 3, backoffMs: 1000, maxBackoffMs: 30000, }, }; ``` --- ## Integration Examples ### Runtime Mode (Tour Playback) ```typescript // frontend/src/pages/runtime.tsx const preloadOrchestrator = usePreloadOrchestrator({ pages, pageLinks, // Includes transition.video_url from API elements, currentPageId: selectedPageId, pageHistory, enabled: !isLoading && !error, }); // Pass getCachedBlobUrl to reverse playback hook const { startReverse, stopReverse } = useReversePlayback({ videoRef: overlayVideoRef, onComplete: finishOverlayTransition, preloadedUrls: preloadOrchestrator.preloadedUrls, videoUrl: overlayTransition?.videoUrl, getCachedBlobUrl: preloadOrchestrator.getCachedBlobUrl, }); ``` ### RuntimePresentation Mode (Embedded/Standalone) ```typescript // frontend/src/components/RuntimePresentation.tsx // 1. Filter pages by environment (dev, stage, or production) const filteredPages = pages.filter(p => p.environment === environment); // 2. Extract navigation targets and preload elements from ui_schema_json const { pageLinks, preloadElements } = extractPageLinksAndElements(filteredPages); // 3. Initialize preload orchestrator const preloadOrchestrator = usePreloadOrchestrator({ pages: filteredPages, pageLinks, // Navigation links with transition videos elements: preloadElements, // All preloadable elements currentPageId: selectedPageId, pageHistory, enabled: !isLoading && !error, }); // 4. Initialize page switch with preload cache const pageSwitch = usePageSwitch({ preloadCache: preloadOrchestrator ? { getReadyBlobUrl: preloadOrchestrator.getReadyBlobUrl, // O(1) instant lookup getCachedBlobUrl: preloadOrchestrator.getCachedBlobUrl, preloadedUrls: preloadOrchestrator.preloadedUrls, } : undefined, }); // 5. Navigate using preloaded assets for instant display const handleNavigation = (targetPage, link) => { pageSwitch.switchToPage(targetPage, link); // Uses ready blob URLs }; // OfflineToggle for user-initiated downloads ``` **Environment Filtering:** Pages are filtered by environment (`dev`, `stage`, `production`) before preloading. This ensures: - Constructor always preloads `dev` environment pages - Stage preview preloads `stage` environment pages - Production runtime preloads `production` environment pages ### Constructor Mode (Editing) ```typescript // frontend/src/pages/constructor.tsx // Constructor always works with 'dev' environment pages const { pageLinks, preloadElements } = extractPageLinksAndElements(pages); const preloadOrchestrator = usePreloadOrchestrator({ pages, pageLinks, elements: preloadElements, currentPageId: activePageId, enabled: !isLoading && !!activePageId, // maxNeighborDepth defaults to 1 (same as runtime) }); // Page switch for preview navigation const pageSwitch = usePageSwitch({ preloadCache: preloadOrchestrator ? { getReadyBlobUrl: preloadOrchestrator.getReadyBlobUrl, getCachedBlobUrl: preloadOrchestrator.getCachedBlobUrl, preloadedUrls: preloadOrchestrator.preloadedUrls, } : undefined, }); ``` ### Offline Mode Hook ```typescript // frontend/src/components/Offline/OfflineToggle.tsx const { isOfflineCapable, isDownloaded, isDownloading, status, progress, startDownload, pauseDownload, resumeDownload, cancelDownload, deleteOfflineData, estimatedSize, formatSize, } = useOfflineMode({ projectId, projectSlug, projectName, pages, // Required: pages data for frontend asset discovery }); ``` ### Video Reverse Playback ```typescript // frontend/src/hooks/useReversePlayback.ts // Tries native playbackRate = -1 first (Chrome 141+, Safari 16+) // Falls back to frame-stepping with cached blob URL for better seeking const cachedUrl = await getCachedBlobUrl(videoUrl); if (cachedUrl) { video.src = cachedUrl; // Better seeking with local blob video.load(); } ``` --- ## Complete Data Flow Example **Scenario: User navigates from Page A to Page B** ``` 1. Page Change Detection └── currentPageId: "pageA" → "pageB" 2. Neighbor Discovery (BFS via useNeighborGraph, depth=1) └── pageB neighbors: [pageC (dist=1)] // Only immediate neighbors 3. Asset Extraction (via useNeighborGraph.getAssetsForPages) ├── pageB backgrounds (from page object) │ ├── background_image_url → [bg.jpg] │ ├── background_video_url → [bg-video.mp4] (if exists) │ └── background_audio_url → [bg-audio.mp3] (if exists) ├── pageB elements → [img1.webp, video1.mp4] ├── pageC backgrounds → [pageC-bg.jpg] ├── pageC elements → [img2.webp] └── pageLink B→C transition.video_url → [trans.mp4] 4. Priority Assignment ├── img1.webp → 1000 + 100 + 35 = 1135 ├── bg.jpg → 1000 + 100 + 5 = 1105 ├── video1.mp4 → 1000 + 30 + 20 = 1050 ├── trans.mp4 → 500 + 150 + 20 = 670 // Transition has highest type bonus └── img2.webp → 500 + 100 + 35 = 635 5. Network Check (useNetworkAware) └── 4g detected → concurrency = 3 6. Download Queue (sorted by priority) [img1.webp, bg.jpg, video1.mp4, trans.mp4, img2.webp] 7. Parallel Download (3 concurrent) ├── Download blob via presigned URL ├── Store in Cache API ├── Create blob URL ├── Decode (images only) └── Store in readyBlobUrlsRef Map 8. Progress Events └── DownloadEventBus → usePreloadProgress → UI 9. Navigation Click (instant) └── getReadyBlobUrl(url) → O(1) Map lookup → Pre-decoded blob URL └── Set as background → No delay, no flash ``` --- ## Storage Quota Management The system monitors storage usage via `navigator.storage.estimate()`: ```typescript const { usage, quota } = await navigator.storage.estimate(); const percentUsed = (usage / quota) * 100; const available = quota - usage - PRELOAD_CONFIG.storage.minFreeBuffer; if (percentUsed > 95) { // Critical: stop preloading } else if (percentUsed > 80) { // Warning: reduce preload depth } ``` The `useStorageQuota` hook provides reactive storage monitoring: ```typescript const { canStore, isWarning, isCritical } = useStorageQuota(); // Used in OfflineToggle to warn users before download if (isCritical) { alert('Storage space is critically low.'); return; } ``` --- ## Troubleshooting ### Assets Not Preloading 1. Check network status in DevTools 2. Verify `enabled` prop is `true` 3. Check Console for preload errors (look for `[PRELOAD]` prefix) 4. Verify Cache API storage quota 5. Check that `pageLinks` include `transition.video_url` ### Offline Mode Not Working 1. Ensure Service Worker is registered 2. Check IndexedDB for stored assets (`TourBuilderOffline` database) 3. Verify `pages` prop is passed to `OfflineToggle` / `useOfflineMode` 4. Check for CORS issues on asset URLs 5. Verify `OfflineToggle` component is rendered ### Video Seeking Issues Offline 1. Verify video is in cache with correct headers 2. Check Service Worker handles range requests (HTTP 206) 3. Verify `getCachedBlobUrl` returns blob URL 4. Try clearing cache and re-downloading ### Reverse Playback Not Working 1. Check if `getCachedBlobUrl` is passed to `useReversePlayback` 2. Verify transition video URL is preloaded 3. Check Console for `[REVERSE]` logs 4. Verify `preloadedUrls` Set contains the video URL --- ## Performance Considerations 1. **Storage Key Mapping**: Assets mapped by canonical storage key (e.g., `assets/vid.mp4`) enabling cache hits regardless of presigned URL changes 2. **Ready Blob URLs**: Use `getReadyBlobUrl(storageKey)` for O(1) instant lookup of pre-decoded blob URLs 3. **Post-Refresh Cache**: Assets stored in Cache API under storage key survive page refresh 4. **Image Decoding**: Blob URLs are pre-decoded to prevent white flash on navigation 5. **Neighbor Depth**: `maxNeighborDepth=1` for both editing and playback (reduced to prevent too many requests) 6. **Transition Priority**: Transitions have highest priority (150) - needed immediately on navigation click 7. **Large Videos**: Stored in IndexedDB (≥5MB) for reliable playback 8. **Network Adaptation**: Reduces concurrency on slow connections 9. **Memory Management**: Auto-cleans completed jobs from progress tracking; blob URLs revoked on unmount 10. **Environment Filtering**: Pages filtered by environment before preloading to avoid loading wrong content 11. **Service Worker Cache**: Same cache name (`tour-builder-assets-v1`) used by preloading and SW 12. **Blob URL Rendering**: Use native `` for blob URLs instead of Next.js Image to prevent re-fetching --- ## Blob URL Rendering Strategy When displaying preloaded blob URLs, use native `` tags instead of Next.js ``: ```tsx // Background image with conditional rendering {backgroundImageUrl.startsWith('blob:') ? ( markBackgroundReady()} /> ) : ( markBackgroundReady()} /> )} ``` **Why this matters:** - Next.js `` re-fetches `src` on every component re-render, even with `unoptimized` - Pages with frequent state updates (e.g., 100ms animation timers) cause thousands of requests - Blob URLs are already in-memory and don't benefit from Next.js optimization - Native `` with blob URLs is cached by browser and doesn't re-fetch **Where to apply:** - Background images in RuntimePresentation and Constructor - Element icons (tooltip, description, gallery, carousel) - Any image element receiving blob URLs from `getReadyBlobUrl()`