975 lines
36 KiB
Markdown
975 lines
36 KiB
Markdown
# Assets Preloading Feature - E2E Documentation
|
|
|
|
## Overview
|
|
|
|
The Tour Builder Platform implements a sophisticated dual-mode asset preloading system that optimizes user experience in both online and offline scenarios. The system intelligently prefetches assets based on navigation patterns, network conditions, and storage constraints.
|
|
|
|
---
|
|
|
|
## Architecture
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ Configuration Layer │
|
|
│ preload.config.ts │ offline.config.ts │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
│
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ Hook Layer │
|
|
│ usePreloadOrchestrator │ usePageNavigationState │ useNetworkAware│
|
|
│ usePreloadProgress │ usePWAPreload │ useOfflineMode │
|
|
│ useStorageQuota │ useIconPreload │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
│
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ Library Layer │
|
|
│ DownloadEventBus │ DownloadManager │ StorageManager │
|
|
│ OfflineDbManager │ extractPageLinks │ assetCache │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
│
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ Service Worker (Serwist) │
|
|
│ sw.ts - CacheFirst, NetworkFirst, Range Request Support │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
**Recent Refactor:** The preloading system was simplified to use a "stream-first" approach:
|
|
- **Online mode:** Preloads current page + outgoing transition videos only (no neighbor preloading)
|
|
- **Offline mode:** Full project assets downloaded via `useOfflineMode.startDownload()`
|
|
- **Video playback:** Streams on-demand, then caches for replay (no bandwidth competition)
|
|
- `useNeighborGraph` was removed (neighbor preloading caused excessive network requests)
|
|
- `usePageSwitch`, `useBackgroundTransition`, and other hooks were consolidated into `usePageNavigationState`
|
|
|
|
---
|
|
|
|
## Online Mode (Stream-First)
|
|
|
|
### How It Works
|
|
|
|
Online mode uses a **stream-first** approach orchestrated by `usePreloadOrchestrator.ts`:
|
|
- **Current page assets** are preloaded (backgrounds, element images)
|
|
- **Outgoing transition videos** are preloaded for instant playback
|
|
- **Other videos** stream on-demand using presigned URLs (browser handles buffering)
|
|
- **No neighbor page preloading** - eliminated to reduce network contention
|
|
|
|
### Asset Discovery Flow
|
|
|
|
1. **Page Change Detection** - Monitors `currentPageId` changes
|
|
2. **Current Page Asset Extraction** - Extracts URLs from:
|
|
- **Page backgrounds** - `background_image_url`, `background_video_url`, `background_audio_url`
|
|
- **Element content** - URLs from `ui_schema_json` elements
|
|
3. **Transition Video Extraction** - Gets `transitionVideoUrl` from outgoing page links
|
|
4. **Priority Assignment** - Assigns download priority based on asset type
|
|
|
|
### Asset URL Extraction Fields
|
|
|
|
The system extracts URLs from these `content_json` fields (configured in `PRELOAD_CONFIG.assetFields`):
|
|
|
|
**Direct Fields:**
|
|
`iconUrl`, `imageUrl`, `mediaUrl`, `videoUrl`, `audioUrl`, `transitionVideoUrl`, `backgroundImageUrl`, `reverseVideoUrl`, `carouselPrevIconUrl`, `carouselNextIconUrl`, `src`, `url`, `poster`, `thumbnail`
|
|
|
|
**Nested Arrays:**
|
|
- `galleryCards[].imageUrl`, `galleryCards[].videoUrl`
|
|
- `carouselSlides[].imageUrl`, `carouselSlides[].videoUrl`
|
|
|
|
**Page Links:**
|
|
- `pageLink.transition.video_url` (eagerly loaded from backend API)
|
|
|
|
### Priority Calculation
|
|
|
|
```
|
|
Priority = basePriority + assetTypeBonus + variantBonus
|
|
|
|
Current Page Assets: priority = 1000 + bonuses
|
|
Transition Videos: priority = 1000 + 150 = 1150 (highest)
|
|
```
|
|
|
|
**Asset Type Multipliers:**
|
|
| Type | Weight | Notes |
|
|
|------|--------|-------|
|
|
| Transition | 150 | Highest - needed immediately on navigation click |
|
|
| Image | 100 | Background and UI elements |
|
|
| Audio | 50 | Background audio tracks |
|
|
| Video | 30 | Lower priority since they stream |
|
|
|
|
**Note:** Neighbor page preloading was removed to simplify the system and reduce network contention. Videos stream on-demand using presigned URLs with browser-managed buffering.
|
|
|
|
**Variant Multipliers:**
|
|
| Variant | Weight |
|
|
|---------|--------|
|
|
| Thumbnail | 50 |
|
|
| Preview | 40 |
|
|
| WebP | 35 |
|
|
| MP4 Low | 20 |
|
|
| MP4 High | 10 |
|
|
| Original | 5 |
|
|
|
|
### Network-Aware Concurrency
|
|
|
|
The system adapts download concurrency based on network conditions via `useNetworkAware.ts`:
|
|
|
|
| Connection | Concurrent Downloads | Strategy |
|
|
|------------|---------------------|----------|
|
|
| 4g / Good | 3 | Aggressive |
|
|
| 3g | 2 | Normal |
|
|
| 2g / Slow | 1 | Conservative |
|
|
| Save-Data | 1 | Minimal |
|
|
|
|
**Network Thresholds:**
|
|
- Aggressive: 4g OR downlink ≥ 5Mbps
|
|
- Suggest offline: slow-2g OR 2g OR RTT > 500ms OR downlink < 0.5Mbps
|
|
- Low quality: save-data flag OR slow-2g OR downlink < 1Mbps
|
|
|
|
### Download Process
|
|
|
|
1. Fetch asset with progress tracking via ReadableStream
|
|
2. Emit progress events via `DownloadEventBus`
|
|
3. Store in Cache API (`tour-builder-assets-v1`) under download URL
|
|
4. **Store in Cache API under storage key** (enables post-refresh lookups)
|
|
5. **Create Blob URL**: `URL.createObjectURL(blob)` for local reference
|
|
6. **Image Decoding**: Call `image.decode()` on the blob URL to eliminate white flash on navigation
|
|
7. **Store Ready URLs**: Map both download URL AND storage key to blob URL in `readyBlobUrlsRef`
|
|
8. Track cached assets in memory Set for quick lookups
|
|
|
|
**Storage Key Mapping (Key Feature):**
|
|
|
|
The system maps assets by **canonical storage key** (e.g., `assets/project-123/video.mp4`) in addition to download URLs. This solves the presigned URL mismatch problem where different requests generate different signatures.
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────────────────────────────────┐
|
|
│ STORAGE KEY MAPPING │
|
|
│ │
|
|
│ Storage Key: "assets/project-123/video.mp4" ← CANONICAL IDENTIFIER (never changes) │
|
|
│ │ │
|
|
│ ┌──────────────┴──────────────┐ │
|
|
│ ▼ ▼ │
|
|
│ Download URL #1 Download URL #2 │
|
|
│ "https://s3...?Sig=ABC" "https://s3...?Sig=XYZ" │
|
|
│ │ │ │
|
|
│ ▼ ▼ │
|
|
│ ┌─────────────────────────────────────────────────────────────────────┐ │
|
|
│ │ readyBlobUrlsRef Map │ │
|
|
│ │ key: storageKey ────────────► blob://... │ │
|
|
│ │ key: downloadUrl ────────────► blob://... (same blob URL) │ │
|
|
│ └─────────────────────────────────────────────────────────────────────┘ │
|
|
│ │
|
|
│ Lookup: getReadyBlobUrl(storageKey) → O(1) instant → blob://... │
|
|
└─────────────────────────────────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
**Why storage key mapping matters:**
|
|
|
|
| Scenario | Download URL | Storage Key | Lookup Result |
|
|
|----------|-------------|-------------|---------------|
|
|
| Same session | `https://s3...?Sig=ABC` | `assets/vid.mp4` | ✅ Instant via storage key |
|
|
| New presigned URL | `https://s3...?Sig=XYZ` | `assets/vid.mp4` | ✅ Instant via storage key |
|
|
| Page refresh | N/A (in-memory cleared) | `assets/vid.mp4` | ✅ From Cache API by storage key |
|
|
|
|
**Ready Blob URL Flow:**
|
|
```
|
|
Download → Cache API (URL key) → Cache API (storage key) → Blob URL → Image Decode
|
|
↓
|
|
┌── readyBlobUrlsRef.set(downloadUrl, blobUrl)
|
|
└── readyBlobUrlsRef.set(storageKey, blobUrl)
|
|
└── readyUrlsVersion++ (triggers re-render)
|
|
↓
|
|
Navigation Click → getReadyBlobUrl(storageKey) → O(1) lookup → Instant display
|
|
```
|
|
|
|
**Re-render Trigger Mechanism:**
|
|
|
|
Since `readyBlobUrlsRef` is a ref (not state), updates don't trigger React re-renders. The `readyUrlsVersion` counter solves this:
|
|
|
|
1. When a blob URL becomes ready, `setReadyUrlsVersion(v => v + 1)` is called
|
|
2. Components using `readyUrlsVersion` in dependency arrays re-render
|
|
3. `resolveUrlWithBlob` callbacks are recreated with fresh lookups
|
|
4. UI switches from direct URLs to cached blob URLs
|
|
|
|
```typescript
|
|
// In constructor.tsx
|
|
const resolveUrlWithBlob = useCallback(
|
|
(url) => {
|
|
const blobUrl = preloadOrchestrator.getReadyBlobUrl(url);
|
|
if (blobUrl) return blobUrl;
|
|
return resolveAssetPlaybackUrl(url);
|
|
},
|
|
[preloadOrchestrator, preloadOrchestrator.readyUrlsVersion], // Re-render when ready
|
|
);
|
|
```
|
|
|
|
### Presigned URL Integration
|
|
|
|
For S3 storage, the preloader fetches presigned URLs before downloading assets for direct S3 access:
|
|
|
|
```typescript
|
|
// In usePreloadOrchestrator.ts
|
|
// 1. Collect storage paths that need presigning
|
|
const storagePaths = assets
|
|
.map(a => a.url)
|
|
.filter(isRelativeStoragePath);
|
|
|
|
// 2. Batch fetch presigned URLs (auto-batches and caches)
|
|
queuePresignedUrls(storagePaths)
|
|
.then(() => {
|
|
// 3. Add assets to queue with resolved URLs
|
|
assets.forEach(asset => {
|
|
const presignedUrl = getPresignedUrl(asset.url);
|
|
const resolvedUrl = presignedUrl || resolveAssetPlaybackUrl(asset.url);
|
|
addToQueue({ url: resolvedUrl, storageKey: asset.url, ... });
|
|
});
|
|
})
|
|
.catch(() => {
|
|
// Fallback to proxy URLs
|
|
addAssetsToQueue();
|
|
});
|
|
|
|
// 4. On successful download, mark presigned URLs as verified
|
|
// (enables resolveAssetPlaybackUrl to use presigned URLs)
|
|
if (isPresignedUrl(item.url)) {
|
|
markPresignedUrlsVerified();
|
|
}
|
|
```
|
|
|
|
**Key Functions (lib/assetUrl.ts):**
|
|
| Function | Purpose |
|
|
|----------|---------|
|
|
| `queuePresignedUrl()` | Queue single URL for presigning |
|
|
| `queuePresignedUrls()` | Batch queue URLs for presigning |
|
|
| `flushPresignedUrlQueue()` | Fetch queued presigned URLs |
|
|
| `getPresignedUrl()` | Get cached presigned URL (sync) |
|
|
| `resolveAssetPlaybackUrl()` | Resolve URL with presigned cache lookup |
|
|
| `markPresignedUrlFailed()` | Mark presigned URL as failed for specific key |
|
|
| `markPresignedUrlsVerified()` | Mark presigned URLs as verified (enables playback URL resolution) |
|
|
| `isRelativeStoragePath()` | Check if URL is a relative storage path (needs presigning) |
|
|
| `extractStoragePath()` | Extract storage path from full URL (handles S3, CDN, proxy URLs) |
|
|
| `setupPresignedUrlInterceptor()` | Setup Axios interceptor for CORS failure detection |
|
|
| `disablePresignedUrls()` | Globally disable presigned URLs (fallback to proxy) |
|
|
| `arePresignedUrlsDisabled()` | Check if presigned URLs are disabled |
|
|
| `clearPresignedUrlCache()` | Clear all cached presigned URLs |
|
|
|
|
**Key Functions (usePreloadOrchestrator):**
|
|
| Function | Purpose |
|
|
|----------|---------|
|
|
| `getReadyBlobUrl(key)` | O(1) instant lookup by storage key or resolved URL |
|
|
| `getCachedBlobUrl(key)` | Async lookup from Cache API by storage key or resolved URL |
|
|
| `isUrlPreloaded(key)` | Check if asset is ready for instant display |
|
|
|
|
**Lookup Priority (in usePageSwitch and useTransitionPlayback):**
|
|
```
|
|
1. getReadyBlobUrl(storageKey) → instant O(1) (same session)
|
|
2. getCachedBlobUrl(storageKey) → Cache API (~5ms, post-refresh)
|
|
3. getReadyBlobUrl(resolvedUrl) → fallback
|
|
4. getCachedBlobUrl(resolvedUrl) → fallback
|
|
5. Network fetch → last resort
|
|
```
|
|
|
|
**Fallback Behavior:**
|
|
- If presigned URL fails (CORS not configured), DownloadManager automatically retries with proxy URL
|
|
- This fallback is built into DownloadManager as single source of truth (no duplicate retry logic)
|
|
- Retry happens immediately with reset retry count
|
|
- Logger tracks presigned URL status changes: `[DownloadManager] Presigned URL failed, retrying with proxy`
|
|
|
|
### Key Files
|
|
|
|
| File | Location | Purpose |
|
|
|------|----------|---------|
|
|
| `usePreloadOrchestrator.ts` | `frontend/src/hooks/` | Main orchestrator with ready blob URL management |
|
|
| `usePageSwitch.ts` | `frontend/src/hooks/` | Page navigation using preloaded blob URLs |
|
|
| `useNeighborGraph.ts` | `frontend/src/hooks/` | BFS navigation graph, extracts page backgrounds + element assets |
|
|
| `useNetworkAware.ts` | `frontend/src/hooks/` | Network condition monitoring |
|
|
| `useIconPreload.ts` | `frontend/src/hooks/` | Constructor icon preloading to prevent flash |
|
|
| `extractPageLinks.ts` | `frontend/src/lib/` | Extract navigation targets and preload elements from pages |
|
|
| `preload.config.ts` | `frontend/src/config/` | Queue and priority settings |
|
|
| `assetUrl.ts` | `frontend/src/lib/` | URL resolution with presigned URL cache |
|
|
|
|
### Key Methods
|
|
|
|
| Method | Hook | Purpose |
|
|
|--------|------|---------|
|
|
| `getReadyBlobUrl(key)` | usePreloadOrchestrator | O(1) instant lookup by storage key or resolved URL |
|
|
| `getCachedBlobUrl(key)` | usePreloadOrchestrator | Async lookup from Cache API by storage key or URL |
|
|
| `isUrlPreloaded(key)` | usePreloadOrchestrator | Check if asset is ready for instant display |
|
|
| `preloadAsset(url)` | usePreloadOrchestrator | Manually trigger preload for a specific URL |
|
|
| `clearQueue()` | usePreloadOrchestrator | Clear the preload queue |
|
|
| `switchToPage(page, onSwitched?)` | usePageSwitch | Navigate with preloaded assets (resolves by storage key first) |
|
|
| `setBackgroundsDirectly(img, vid, aud)` | usePageSwitch | Set backgrounds without transition overlay (initial load) |
|
|
| `markBackgroundReady()` | usePageSwitch | Mark new background as ready (call from Image onLoad) |
|
|
| `clearPreviousBackground()` | usePageSwitch | Clear overlay after transition completes |
|
|
|
|
---
|
|
|
|
## Offline Mode
|
|
|
|
### Storage Strategy
|
|
|
|
The system uses a hybrid storage approach for optimal performance:
|
|
|
|
```
|
|
Assets < 5MB → Cache API (tour-builder-assets-v1)
|
|
Assets ≥ 5MB → IndexedDB (TourBuilderOffline)
|
|
```
|
|
|
|
**Why Hybrid:**
|
|
- Cache API has better browser support but size limitations
|
|
- IndexedDB provides reliable large file storage
|
|
- `StorageManager` abstracts both for transparent access
|
|
|
|
### IndexedDB Schema (Dexie.js)
|
|
|
|
**Database:** `TourBuilderOffline` (version 1)
|
|
|
|
**Tables and Indexes:**
|
|
```javascript
|
|
{
|
|
assets: 'id, projectId, url, variantType, assetType, downloadedAt',
|
|
projects: 'id, slug, status, lastSyncedAt',
|
|
downloadQueue: 'id, projectId, status, priority, addedAt'
|
|
}
|
|
```
|
|
|
|
**TypeScript Interfaces:**
|
|
|
|
```typescript
|
|
// OfflineAsset - Large files stored in IndexedDB
|
|
interface OfflineAsset {
|
|
id: string;
|
|
projectId: string;
|
|
url: string;
|
|
filename: string;
|
|
variantType: AssetVariantType;
|
|
assetType: AssetType;
|
|
mimeType: string;
|
|
sizeBytes: number;
|
|
blob: Blob;
|
|
downloadedAt: number;
|
|
}
|
|
|
|
// OfflineProject - Project offline status
|
|
interface OfflineProject {
|
|
id: string;
|
|
slug: string;
|
|
name: string;
|
|
status: ProjectOfflineStatus;
|
|
totalAssets: number;
|
|
downloadedAssets: number;
|
|
totalSizeBytes: number;
|
|
downloadedSizeBytes: number;
|
|
lastSyncedAt?: number;
|
|
version?: string;
|
|
}
|
|
|
|
// DownloadQueueItem - Persistent download queue
|
|
interface DownloadQueueItem {
|
|
id: string;
|
|
projectId: string;
|
|
assetId: string;
|
|
url: string;
|
|
filename: string;
|
|
status: PreloadJobStatus;
|
|
priority: number;
|
|
retryCount: number;
|
|
bytesLoaded: number;
|
|
totalBytes: number;
|
|
addedAt: number;
|
|
lastAttemptAt?: number;
|
|
error?: string;
|
|
}
|
|
```
|
|
|
|
### Asset Storage Flow
|
|
|
|
```
|
|
storeAsset(blob, metadata)
|
|
│
|
|
├── Size < 5MB?
|
|
│ │
|
|
│ YES → Create Response with headers
|
|
│ → Store in Cache API
|
|
│
|
|
└── Size ≥ 5MB?
|
|
│
|
|
YES → Create OfflineAsset object
|
|
→ Store in IndexedDB
|
|
```
|
|
|
|
### Asset Retrieval Flow
|
|
|
|
```
|
|
getAsset(url) [StorageManager.getAsset()]
|
|
│
|
|
├── Check IndexedDB first (large files)
|
|
│ │
|
|
│ └── Found? → Return blob
|
|
│
|
|
└── Check Cache API (small files)
|
|
│
|
|
└── Found? → Return response blob
|
|
```
|
|
|
|
### Service Worker (Serwist)
|
|
|
|
Located at `frontend/src/sw.ts`, handles offline caching:
|
|
|
|
| Resource Type | Strategy | Cache Name |
|
|
|--------------|----------|------------|
|
|
| Static Assets (images, fonts, css, js) | CacheFirst | `tour-builder-assets-v1` |
|
|
| Videos | CacheFirst + Range | `tour-builder-assets-v1` |
|
|
| API Responses | NetworkFirst | `api-cache` |
|
|
| Pages | NetworkFirst | (default) |
|
|
|
|
**Unified Cache Keys (cacheKeyWillBeUsed Plugin):**
|
|
|
|
The Service Worker normalizes all asset URLs to storage keys using `cacheKeyWillBeUsed` plugin. This ensures preload (DownloadManager) and browser loads (SW interception) use the same cache keys:
|
|
|
|
```typescript
|
|
// SW plugin for static assets handler
|
|
cacheKeyWillBeUsed: async ({ request, mode }) => {
|
|
const storagePath = extractStoragePathFromUrl(request.url);
|
|
if (storagePath) {
|
|
return new Request(storagePath);
|
|
}
|
|
return request;
|
|
}
|
|
```
|
|
|
|
**Why this matters:**
|
|
| Before | After |
|
|
|--------|-------|
|
|
| Preload stores: `assets/project/file.jpg` | Same |
|
|
| Browser load stores: `https://s3.../file.jpg?X-Amz-Signature=abc` | `assets/project/file.jpg` |
|
|
| New session with new presigned URL → cache MISS | cache HIT (same key) |
|
|
|
|
**Range Request Support for Videos:**
|
|
- Handles byte-range requests for video seeking
|
|
- Extracts range from cached response
|
|
- Returns partial content (HTTP 206)
|
|
|
|
### Asset Discovery (Unified Frontend)
|
|
|
|
Located at `frontend/src/lib/assetCache/`, provides unified asset discovery for both online preload and offline download:
|
|
|
|
```typescript
|
|
// Discover all assets for a project (used by offline mode)
|
|
discoverProjectAssets(pages, pageLinks, elements): AssetToCache[]
|
|
|
|
// Get prioritized assets for preloading (used by online preload)
|
|
getPrioritizedAssets(currentPageId, pages, pageLinks, elements, neighbors): AssetToCache[]
|
|
```
|
|
|
|
**Asset Discovery Sources:**
|
|
- Page backgrounds: `background_image_url`, `background_video_url`, `background_audio_url`
|
|
- Element content_json: All URL fields from `PRELOAD_CONFIG.assetFields.all`
|
|
- Transition videos: `transition.video_url` from page links
|
|
- Nested arrays: `galleryCards[]`, `carouselSlides[]`
|
|
|
|
**AssetToCache Structure:**
|
|
```typescript
|
|
interface AssetToCache {
|
|
storageKey: string; // Canonical key for caching
|
|
originalUrl: string; // Original URL from data
|
|
assetType: AssetType; // 'image' | 'video' | 'audio' | 'transition' | 'other'
|
|
pageId: string; // Page the asset belongs to
|
|
priority: number; // Download priority (higher = first)
|
|
}
|
|
```
|
|
|
|
### Navigation Data in ui_schema_json
|
|
|
|
Navigation targets and transition videos are stored directly in `tour_pages.ui_schema_json`:
|
|
|
|
```typescript
|
|
// Element with navigation and transition
|
|
{
|
|
type: "navigation_next",
|
|
targetPageSlug: "gallery", // Slug-based navigation
|
|
transitionVideoUrl: "assets/.../transition.mp4", // Inline transition
|
|
transitionDurationSec: 1.5,
|
|
}
|
|
```
|
|
|
|
The preload system extracts navigation targets from elements to build the neighbor graph, and transition video URLs are used directly for preloading.
|
|
|
|
### Key Files
|
|
|
|
| File | Location | Purpose |
|
|
|------|----------|---------|
|
|
| `StorageManager.ts` | `frontend/src/lib/offline/` | Cache API + IndexedDB abstraction |
|
|
| `DownloadManager.ts` | `frontend/src/lib/offline/` | Download queue with auto proxy fallback |
|
|
| `OfflineDbManager.ts` | `frontend/src/lib/offlineDb/` | Dexie.js IndexedDB manager |
|
|
| `schema.ts` | `frontend/src/lib/offlineDb/` | Dexie.js schema definition |
|
|
| `AssetCacheService.ts` | `frontend/src/lib/assetCache/` | Unified asset caching service |
|
|
| `assetDiscovery.ts` | `frontend/src/lib/assetCache/` | Shared asset extraction logic |
|
|
| `offline.config.ts` | `frontend/src/config/` | Offline settings |
|
|
| `sw.ts` | `frontend/src/` | Serwist service worker |
|
|
| `useNeighborGraph.ts` | `frontend/src/hooks/` | Builds navigation graph from elements |
|
|
|
|
---
|
|
|
|
## Progress Tracking
|
|
|
|
### Event System
|
|
|
|
The `DownloadEventBus` singleton emits events consumed by `usePreloadProgress`:
|
|
|
|
```
|
|
DownloadEventBus
|
|
│
|
|
├── preloadStart { jobId, assetId, url }
|
|
├── preloadProgress { jobId, progress, bytesLoaded, totalBytes }
|
|
├── preloadComplete { jobId, assetId }
|
|
└── preloadError { jobId, assetId, error }
|
|
│
|
|
▼
|
|
usePreloadProgress Hook
|
|
│
|
|
├── Maintains jobs[] array
|
|
├── Calculates totalProgress
|
|
├── Auto-removes completed (3s)
|
|
└── Auto-removes errors (10s)
|
|
│
|
|
▼
|
|
UI Components
|
|
│
|
|
├── OfflineToggle
|
|
├── OfflineStatusIndicator
|
|
└── DownloadProgressPanel
|
|
```
|
|
|
|
---
|
|
|
|
## Configuration Reference
|
|
|
|
### preload.config.ts
|
|
|
|
```typescript
|
|
export const PRELOAD_CONFIG = {
|
|
// Queue settings
|
|
maxConcurrentDownloads: 3,
|
|
maxRetries: 3,
|
|
retryDelayMs: 1000,
|
|
|
|
// Size thresholds
|
|
largeFileThreshold: 5 * 1024 * 1024, // 5MB -> use IndexedDB
|
|
videoChunkSize: 5 * 1024 * 1024, // 5MB chunks
|
|
initialVideoBufferSeconds: 5,
|
|
|
|
// Priority weights
|
|
priority: {
|
|
currentPage: 1000,
|
|
neighborBase: 500,
|
|
assetType: {
|
|
transition: 150, // Highest - needed immediately on navigation click
|
|
image: 100,
|
|
audio: 50,
|
|
video: 30,
|
|
},
|
|
variant: {
|
|
thumbnail: 50,
|
|
preview: 40,
|
|
webp: 35,
|
|
mp4_low: 20,
|
|
mp4_high: 10,
|
|
original: 5,
|
|
},
|
|
linkCountMultiplier: 10,
|
|
maxLinkBonus: 50,
|
|
},
|
|
|
|
// Storage thresholds
|
|
storage: {
|
|
warningPercent: 80,
|
|
criticalPercent: 95,
|
|
minFreeBuffer: 50 * 1024 * 1024, // 50MB
|
|
},
|
|
|
|
// Auto-cleanup timeouts
|
|
autoRemove: {
|
|
completedMs: 3000,
|
|
errorMs: 10000,
|
|
},
|
|
|
|
// Neighbor graph traversal
|
|
neighborGraph: {
|
|
maxDepth: 1, // Only preload immediate neighbors (reduced from 2)
|
|
constructorMaxDepth: 1, // Same as maxDepth for constructor
|
|
},
|
|
|
|
// Asset URL field names for extraction
|
|
assetFields: {
|
|
all: [
|
|
'iconUrl', 'imageUrl', 'mediaUrl', 'videoUrl', 'audioUrl',
|
|
'transitionVideoUrl', 'backgroundImageUrl', 'reverseVideoUrl',
|
|
'carouselPrevIconUrl', 'carouselNextIconUrl', 'src', 'url',
|
|
'poster', 'thumbnail'
|
|
],
|
|
images: [
|
|
'iconUrl', 'imageUrl', 'backgroundImageUrl',
|
|
'carouselPrevIconUrl', 'carouselNextIconUrl', 'src'
|
|
],
|
|
nested: ['galleryCards', 'carouselSlides'],
|
|
nestedUrlFields: ['imageUrl', 'videoUrl'],
|
|
},
|
|
};
|
|
```
|
|
|
|
### offline.config.ts
|
|
|
|
```typescript
|
|
export const OFFLINE_CONFIG = {
|
|
// IndexedDB
|
|
dbName: 'TourBuilderOffline',
|
|
dbVersion: 1,
|
|
|
|
// Cache names (for Cache API)
|
|
cacheNames: {
|
|
static: 'tour-builder-static-v1',
|
|
dynamic: 'tour-builder-dynamic-v1',
|
|
assets: 'tour-builder-assets-v1',
|
|
},
|
|
|
|
// Events (EventEmitter event names)
|
|
events: {
|
|
preloadStart: 'asset-preload-start',
|
|
preloadProgress: 'asset-preload-progress',
|
|
preloadComplete: 'asset-preload-complete',
|
|
preloadError: 'asset-preload-error',
|
|
projectDownloadProgress: 'project-download-progress',
|
|
projectDownloadComplete: 'project-download-complete',
|
|
queueUpdate: 'queue-update',
|
|
},
|
|
|
|
// Service worker settings
|
|
serviceWorker: {
|
|
scope: '/',
|
|
updateInterval: 60 * 60 * 1000, // 1 hour
|
|
},
|
|
|
|
// Storage settings
|
|
storage: {
|
|
cacheApiMaxSize: 5 * 1024 * 1024, // 5MB
|
|
indexedDbMinSize: 5 * 1024 * 1024, // 5MB
|
|
},
|
|
|
|
// Retry settings
|
|
retry: {
|
|
maxRetries: 3,
|
|
backoffMs: 1000,
|
|
maxBackoffMs: 30000,
|
|
},
|
|
};
|
|
```
|
|
|
|
---
|
|
|
|
## Integration Examples
|
|
|
|
### Runtime Mode (Tour Playback)
|
|
|
|
```typescript
|
|
// frontend/src/pages/runtime.tsx
|
|
const preloadOrchestrator = usePreloadOrchestrator({
|
|
pages,
|
|
pageLinks, // Includes transition.video_url from API
|
|
elements,
|
|
currentPageId: selectedPageId,
|
|
pageHistory,
|
|
enabled: !isLoading && !error,
|
|
});
|
|
|
|
// Pass getCachedBlobUrl to reverse playback hook
|
|
const { startReverse, stopReverse } = useReversePlayback({
|
|
videoRef: overlayVideoRef,
|
|
onComplete: finishOverlayTransition,
|
|
preloadedUrls: preloadOrchestrator.preloadedUrls,
|
|
videoUrl: overlayTransition?.videoUrl,
|
|
getCachedBlobUrl: preloadOrchestrator.getCachedBlobUrl,
|
|
});
|
|
```
|
|
|
|
### RuntimePresentation Mode (Embedded/Standalone)
|
|
|
|
```typescript
|
|
// frontend/src/components/RuntimePresentation.tsx
|
|
|
|
// 1. Filter pages by environment (dev, stage, or production)
|
|
const filteredPages = pages.filter(p => p.environment === environment);
|
|
|
|
// 2. Extract navigation targets and preload elements from ui_schema_json
|
|
const { pageLinks, preloadElements } = extractPageLinksAndElements(filteredPages);
|
|
|
|
// 3. Initialize preload orchestrator
|
|
const preloadOrchestrator = usePreloadOrchestrator({
|
|
pages: filteredPages,
|
|
pageLinks, // Navigation links with transition videos
|
|
elements: preloadElements, // All preloadable elements
|
|
currentPageId: selectedPageId,
|
|
pageHistory,
|
|
enabled: !isLoading && !error,
|
|
});
|
|
|
|
// 4. Initialize page switch with preload cache
|
|
const pageSwitch = usePageSwitch({
|
|
preloadCache: preloadOrchestrator
|
|
? {
|
|
getReadyBlobUrl: preloadOrchestrator.getReadyBlobUrl, // O(1) instant lookup
|
|
getCachedBlobUrl: preloadOrchestrator.getCachedBlobUrl,
|
|
preloadedUrls: preloadOrchestrator.preloadedUrls,
|
|
}
|
|
: undefined,
|
|
});
|
|
|
|
// 5. Navigate using preloaded assets for instant display
|
|
const handleNavigation = (targetPage, link) => {
|
|
pageSwitch.switchToPage(targetPage, link); // Uses ready blob URLs
|
|
};
|
|
|
|
// OfflineToggle for user-initiated downloads
|
|
<OfflineToggle
|
|
projectId={project?.id || null}
|
|
projectSlug={projectSlug}
|
|
projectName={project?.name}
|
|
pages={filteredPages} // Required: pages for asset discovery
|
|
showLabel={false}
|
|
size='small'
|
|
/>
|
|
```
|
|
|
|
**Environment Filtering:**
|
|
Pages are filtered by environment (`dev`, `stage`, `production`) before preloading. This ensures:
|
|
- Constructor always preloads `dev` environment pages
|
|
- Stage preview preloads `stage` environment pages
|
|
- Production runtime preloads `production` environment pages
|
|
|
|
### Constructor Mode (Editing)
|
|
|
|
```typescript
|
|
// frontend/src/pages/constructor.tsx
|
|
|
|
// Constructor always works with 'dev' environment pages
|
|
const { pageLinks, preloadElements } = extractPageLinksAndElements(pages);
|
|
|
|
const preloadOrchestrator = usePreloadOrchestrator({
|
|
pages,
|
|
pageLinks,
|
|
elements: preloadElements,
|
|
currentPageId: activePageId,
|
|
enabled: !isLoading && !!activePageId,
|
|
// maxNeighborDepth defaults to 1 (same as runtime)
|
|
});
|
|
|
|
// Page switch for preview navigation
|
|
const pageSwitch = usePageSwitch({
|
|
preloadCache: preloadOrchestrator
|
|
? {
|
|
getReadyBlobUrl: preloadOrchestrator.getReadyBlobUrl,
|
|
getCachedBlobUrl: preloadOrchestrator.getCachedBlobUrl,
|
|
preloadedUrls: preloadOrchestrator.preloadedUrls,
|
|
}
|
|
: undefined,
|
|
});
|
|
```
|
|
|
|
### Offline Mode Hook
|
|
|
|
```typescript
|
|
// frontend/src/components/Offline/OfflineToggle.tsx
|
|
const {
|
|
isOfflineCapable,
|
|
isDownloaded,
|
|
isDownloading,
|
|
status,
|
|
progress,
|
|
startDownload,
|
|
pauseDownload,
|
|
resumeDownload,
|
|
cancelDownload,
|
|
deleteOfflineData,
|
|
estimatedSize,
|
|
formatSize,
|
|
} = useOfflineMode({
|
|
projectId,
|
|
projectSlug,
|
|
projectName,
|
|
pages, // Required: pages data for frontend asset discovery
|
|
});
|
|
```
|
|
|
|
### Video Reverse Playback
|
|
|
|
```typescript
|
|
// frontend/src/hooks/useReversePlayback.ts
|
|
// Tries native playbackRate = -1 first (Chrome 141+, Safari 16+)
|
|
// Falls back to frame-stepping with cached blob URL for better seeking
|
|
|
|
const cachedUrl = await getCachedBlobUrl(videoUrl);
|
|
if (cachedUrl) {
|
|
video.src = cachedUrl; // Better seeking with local blob
|
|
video.load();
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Complete Data Flow Example
|
|
|
|
**Scenario: User navigates from Page A to Page B**
|
|
|
|
```
|
|
1. Page Change Detection
|
|
└── currentPageId: "pageA" → "pageB"
|
|
|
|
2. Neighbor Discovery (BFS via useNeighborGraph, depth=1)
|
|
└── pageB neighbors: [pageC (dist=1)] // Only immediate neighbors
|
|
|
|
3. Asset Extraction (via useNeighborGraph.getAssetsForPages)
|
|
├── pageB backgrounds (from page object)
|
|
│ ├── background_image_url → [bg.jpg]
|
|
│ ├── background_video_url → [bg-video.mp4] (if exists)
|
|
│ └── background_audio_url → [bg-audio.mp3] (if exists)
|
|
├── pageB elements → [img1.webp, video1.mp4]
|
|
├── pageC backgrounds → [pageC-bg.jpg]
|
|
├── pageC elements → [img2.webp]
|
|
└── pageLink B→C transition.video_url → [trans.mp4]
|
|
|
|
4. Priority Assignment
|
|
├── img1.webp → 1000 + 100 + 35 = 1135
|
|
├── bg.jpg → 1000 + 100 + 5 = 1105
|
|
├── video1.mp4 → 1000 + 30 + 20 = 1050
|
|
├── trans.mp4 → 500 + 150 + 20 = 670 // Transition has highest type bonus
|
|
└── img2.webp → 500 + 100 + 35 = 635
|
|
|
|
5. Network Check (useNetworkAware)
|
|
└── 4g detected → concurrency = 3
|
|
|
|
6. Download Queue (sorted by priority)
|
|
[img1.webp, bg.jpg, video1.mp4, trans.mp4, img2.webp]
|
|
|
|
7. Parallel Download (3 concurrent)
|
|
├── Download blob via presigned URL
|
|
├── Store in Cache API
|
|
├── Create blob URL
|
|
├── Decode (images only)
|
|
└── Store in readyBlobUrlsRef Map
|
|
|
|
8. Progress Events
|
|
└── DownloadEventBus → usePreloadProgress → UI
|
|
|
|
9. Navigation Click (instant)
|
|
└── getReadyBlobUrl(url) → O(1) Map lookup → Pre-decoded blob URL
|
|
└── Set as background → No delay, no flash
|
|
```
|
|
|
|
---
|
|
|
|
## Storage Quota Management
|
|
|
|
The system monitors storage usage via `navigator.storage.estimate()`:
|
|
|
|
```typescript
|
|
const { usage, quota } = await navigator.storage.estimate();
|
|
const percentUsed = (usage / quota) * 100;
|
|
const available = quota - usage - PRELOAD_CONFIG.storage.minFreeBuffer;
|
|
|
|
if (percentUsed > 95) {
|
|
// Critical: stop preloading
|
|
} else if (percentUsed > 80) {
|
|
// Warning: reduce preload depth
|
|
}
|
|
```
|
|
|
|
The `useStorageQuota` hook provides reactive storage monitoring:
|
|
|
|
```typescript
|
|
const { canStore, isWarning, isCritical } = useStorageQuota();
|
|
|
|
// Used in OfflineToggle to warn users before download
|
|
if (isCritical) {
|
|
alert('Storage space is critically low.');
|
|
return;
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Troubleshooting
|
|
|
|
### Assets Not Preloading
|
|
1. Check network status in DevTools
|
|
2. Verify `enabled` prop is `true`
|
|
3. Check Console for preload errors (look for `[PRELOAD]` prefix)
|
|
4. Verify Cache API storage quota
|
|
5. Check that `pageLinks` include `transition.video_url`
|
|
|
|
### Offline Mode Not Working
|
|
1. Ensure Service Worker is registered
|
|
2. Check IndexedDB for stored assets (`TourBuilderOffline` database)
|
|
3. Verify `pages` prop is passed to `OfflineToggle` / `useOfflineMode`
|
|
4. Check for CORS issues on asset URLs
|
|
5. Verify `OfflineToggle` component is rendered
|
|
|
|
### Video Seeking Issues Offline
|
|
1. Verify video is in cache with correct headers
|
|
2. Check Service Worker handles range requests (HTTP 206)
|
|
3. Verify `getCachedBlobUrl` returns blob URL
|
|
4. Try clearing cache and re-downloading
|
|
|
|
### Reverse Playback Not Working
|
|
1. Check if `getCachedBlobUrl` is passed to `useReversePlayback`
|
|
2. Verify transition video URL is preloaded
|
|
3. Check Console for `[REVERSE]` logs
|
|
4. Verify `preloadedUrls` Set contains the video URL
|
|
|
|
---
|
|
|
|
## Performance Considerations
|
|
|
|
1. **Storage Key Mapping**: Assets mapped by canonical storage key (e.g., `assets/vid.mp4`) enabling cache hits regardless of presigned URL changes
|
|
2. **Ready Blob URLs**: Use `getReadyBlobUrl(storageKey)` for O(1) instant lookup of pre-decoded blob URLs
|
|
3. **Post-Refresh Cache**: Assets stored in Cache API under storage key survive page refresh
|
|
4. **Image Decoding**: Blob URLs are pre-decoded to prevent white flash on navigation
|
|
5. **Neighbor Depth**: `maxNeighborDepth=1` for both editing and playback (reduced to prevent too many requests)
|
|
6. **Transition Priority**: Transitions have highest priority (150) - needed immediately on navigation click
|
|
7. **Large Videos**: Stored in IndexedDB (≥5MB) for reliable playback
|
|
8. **Network Adaptation**: Reduces concurrency on slow connections
|
|
9. **Memory Management**: Auto-cleans completed jobs from progress tracking; blob URLs revoked on unmount
|
|
10. **Environment Filtering**: Pages filtered by environment before preloading to avoid loading wrong content
|
|
11. **Service Worker Cache**: Same cache name (`tour-builder-assets-v1`) used by preloading and SW
|
|
12. **Blob URL Rendering**: Use native `<img>` for blob URLs instead of Next.js Image to prevent re-fetching
|
|
|
|
---
|
|
|
|
## Blob URL Rendering Strategy
|
|
|
|
When displaying preloaded blob URLs, use native `<img>` tags instead of Next.js `<Image>`:
|
|
|
|
```tsx
|
|
// Background image with conditional rendering
|
|
{backgroundImageUrl.startsWith('blob:') ? (
|
|
<img
|
|
src={backgroundImageUrl}
|
|
alt=""
|
|
className="absolute inset-0 w-full h-full object-cover"
|
|
onLoad={() => markBackgroundReady()}
|
|
/>
|
|
) : (
|
|
<NextImage
|
|
src={backgroundImageUrl}
|
|
fill
|
|
sizes="100vw"
|
|
className="object-cover"
|
|
onLoad={() => markBackgroundReady()}
|
|
/>
|
|
)}
|
|
```
|
|
|
|
**Why this matters:**
|
|
- Next.js `<Image>` re-fetches `src` on every component re-render, even with `unoptimized`
|
|
- Pages with frequent state updates (e.g., 100ms animation timers) cause thousands of requests
|
|
- Blob URLs are already in-memory and don't benefit from Next.js optimization
|
|
- Native `<img>` with blob URLs is cached by browser and doesn't re-fetch
|
|
|
|
**Where to apply:**
|
|
- Background images in RuntimePresentation and Constructor
|
|
- Element icons (tooltip, description, gallery, carousel)
|
|
- Any image element receiving blob URLs from `getReadyBlobUrl()`
|