# UI Element Processing & Neighbor Preloading Analysis

## Executive Summary

Deep analysis of how UI elements are processed throughout the Tour Builder Platform - from creation in the Constructor, to rendering in Runtime Presentations, across Online and Offline modes. This document traces each preloading thread step-by-step to verify robustness.

---

## 1. PRELOAD CONFIGURATION (Single Source of Truth)

**File:** `src/config/preload.config.ts`

### 1.1 Asset URL Fields Configuration

```typescript
assetFields: {
  // All 18 URL fields for preloading extraction
  all: [
    'iconUrl',                    // Base element icon
    'imageUrl',                   // Generic image reference
    'mediaUrl',                   // Video/audio player source
    'videoUrl',                   // Video element
    'audioUrl',                   // Audio element
    'transitionVideoUrl',         // Navigation transition video
    'backgroundImageUrl',         // Element background
    'reverseVideoUrl',            // Reverse transition video
    'carouselPrevIconUrl',        // Carousel prev button
    'carouselNextIconUrl',        // Carousel next button
    'galleryHeaderImageUrl',      // Gallery header background
    'galleryCarouselPrevIconUrl', // Gallery carousel prev
    'galleryCarouselNextIconUrl', // Gallery carousel next
    'galleryCarouselBackIconUrl', // Gallery carousel back
    'src', 'url', 'poster', 'thumbnail',  // Generic fallbacks
  ],

  // 10 Image-only fields for pre-decode optimization
  images: [
    'iconUrl', 'imageUrl', 'backgroundImageUrl',
    'carouselPrevIconUrl', 'carouselNextIconUrl',
    'galleryHeaderImageUrl', 'galleryCarouselPrevIconUrl',
    'galleryCarouselNextIconUrl', 'galleryCarouselBackIconUrl', 'src',
  ],

  // 3 Nested array fields containing assets
  nested: ['galleryCards', 'carouselSlides', 'galleryInfoSpans'],

  // 3 URL fields within nested items
  nestedUrlFields: ['imageUrl', 'videoUrl', 'iconUrl'],
}
```

### 1.2 Configuration Consumers (10 files)

| File | Uses | Purpose |
|------|------|---------|
| `extractPageLinks.ts` | all, nested, nestedUrlFields | Explicit field extraction |
| `useNeighborGraph.ts` | all | Recursive traversal (depth 5) |
| `usePreloadOrchestrator.ts` | all | Asset initialization |
| `imagePreDecode.ts` | images, nested, nestedUrlFields | Image-only pre-decode |
| `StorageManager.ts` | storage config | Storage thresholds |
| `DownloadManager.ts` | queue settings | Download concurrency |
| `useStorageQuota.ts` | storage config | Quota warnings |
| `usePreloadProgress.ts` | autoRemove | Progress cleanup |
| `DownloadContext.tsx` | various | Download UI state |

---

## 2. COMPLETE PRELOADING FLOW (Step-by-Step)

### 2.1 Entry Point: RuntimePresentation Mount

```
RuntimePresentation.tsx mounts
    ↓
extractPageLinksAndElements(pages)           [lib/extractPageLinks.ts:105-182]
    ├── Parse ui_schema_json for each page
    ├── extractAssetFields(element) → uses PRELOAD_CONFIG.assetFields
    │   ├── Extract top-level fields from assetFields.all
    │   └── Extract nested arrays (galleryCards, carouselSlides, galleryInfoSpans)
    │       └── Within each item, extract nestedUrlFields (imageUrl, videoUrl, iconUrl)
    ├── Build pageLinks[] for navigation graph
    └── Build preloadElements[] with content_json containing asset URLs
    ↓
usePreloadOrchestrator({ pages, pageLinks, elements, currentPageId })
```

### 2.2 Thread 1: Neighbor Graph Building

```
useNeighborGraph({ pages, pageLinks, elements, maxDepth: 1 })
    ↓
Build adjacencyList (Map<pageId, neighborPageIds[]>)     [line 119-141]
    ├── Initialize all pages in map
    └── Add edges from active pageLinks (from_pageId → to_pageId)
    ↓
getNeighbors(currentPageId, depth)                        [line 144-177]
    ├── BFS traversal from current page
    ├── Track visited pages to avoid cycles
    └── Return PreloadNeighborInfo[] sorted by distance
    ↓
getAssetsForPages(pageIds)                                [line 180-224]
    ├── For each page: filter elements by pageId
    ├── extractAssetsFromContent(content_json)            [line 56-106]
    │   ├── Parse JSON content
    │   ├── checkObject() recursive traversal (depth ≤ 5)
    │   │   └── For each field in PRELOAD_CONFIG.assetFields.all:
    │   │       └── If string value found, classify asset type and add to assets[]
    │   └── Return PreloadAssetInfo[] with { url, pageId, assetType, priority }
    └── Also extract transition videos from pageLinks
```

### 2.3 Thread 2: Priority Assignment

```
getPrioritizedAssets(currentPageId, maxDepth)             [line 228-274]
    ↓
Current page assets:
    priority = PRELOAD_CONFIG.priority.currentPage (1000)
             + PRELOAD_CONFIG.priority.assetType[type]

    Asset type priorities:
    ├── transition: +150 (highest - needed on navigation click)
    ├── image: +100 (backgrounds load during transition)
    ├── audio: +50
    └── video: +30
    ↓
Neighbor page assets:
    basePriority = PRELOAD_CONFIG.priority.neighborBase (500) / distance
    priority = basePriority + assetType priority
    ↓
Deduplicate by URL, keep highest priority
Sort descending by priority
```

### 2.4 Thread 3: URL Resolution (Presigned URLs)

```
usePreloadOrchestrator effect on currentPageId change     [line 669-924]
    ↓
Collect storage paths needing presigning:
    ├── Current page: background_image_url, background_video_url, background_audio_url
    ├── Element assets from neighborGraph.getPrioritizedAssets()
    └── Neighbor pages: background URLs
    ↓
queuePresignedUrls(storagePaths)                          [lib/assetUrl.ts]
    ├── POST /api/file/presign { urls: storagePaths[] }
    ├── Response: { [storageKey]: presignedUrl }
    └── Cache presigned URLs (1-hour expiry)
    ↓
resolveUrl(storageKey, presignedUrls)                     [line 761-771]
    ├── If presignedUrls[storageKey] exists → use presigned URL
    └── Fallback → resolveAssetPlaybackUrl (proxy URL)
```

### 2.5 Thread 4: Download Queue Processing

```
addToQueue(item: PreloadQueueItem)                        [line 494-530]
    ├── Skip if already in queue or preloaded
    ├── Insert in priority order (binary search insertion point)
    └── Trigger processQueue()
    ↓
processQueue()                                            [line 355-491]
    ├── Check: isOnline, queue not empty, not already processing
    ├── maxConcurrent = recommendedConcurrency (network-aware)
    ↓
    While queue has items AND activeDownloads < maxConcurrent:
        ├── Shift item from queue
        ├── Skip if already preloaded (preloadedUrls.has(url))
        ├── Skip if already cached (isUrlCached → StorageManager.hasAsset)
        │   └── If cached: createReadyBlobUrl() for instant display
        ↓
        preloadWithProgress(url, jobId, assetId)          [line 140-234]
            ├── Emit downloadEventBus.emitPreloadStart
            ├── fetch(url) with streaming progress
            ├── Collect chunks, emit progress events
            ├── Store in Cache API: caches.open(assets).put(url, response)
            └── Emit downloadEventBus.emitPreloadComplete
        ↓
        On success:
            ├── createReadyBlobUrl(url, storageKey)
            │   ├── StorageManager.getAsset(url) → blob
            │   ├── URL.createObjectURL(blob) → blobUrl
            │   ├── If image: decodeImage(blobUrl) → pre-paint ready
            │   └── readyBlobUrlsRef.set(url, blobUrl)
            └── Also cache under storageKey for post-refresh lookup
        ↓
        On failure (presigned URL CORS):
            ├── markPresignedUrlFailed(storageKey)
            ├── Build proxy URL: /api/file/download?privateUrl=...
            └── Retry with proxy URL
```

### 2.6 Thread 5: Ready State & Instant Lookup

```
getReadyBlobUrl(url): string | null                       [line 582-584]
    └── readyBlobUrlsRef.current.get(url) → O(1) Map lookup

Usage in RuntimePresentation:
    const resolveUrlWithBlob = (url) => {
        const blobUrl = preloadOrchestrator.getReadyBlobUrl(url);
        return blobUrl || resolveAssetPlaybackUrl(url);
    };

    <RuntimeElement resolveUrl={resolveUrlWithBlob} ... />
```

---

## 3. ELEMENT TYPE → URL FIELD MAPPING (Complete Audit)

### 3.1 All Element Types (11 total)

| Element Type | URL Fields | Nested Arrays |
|--------------|------------|---------------|
| `navigation_next/prev` | iconUrl, transitionVideoUrl, reverseVideoUrl | - |
| `gallery` | iconUrl, galleryHeaderImageUrl, galleryCarouselPrevIconUrl, galleryCarouselNextIconUrl, galleryCarouselBackIconUrl | galleryCards[].imageUrl, galleryInfoSpans[].iconUrl |
| `carousel` | iconUrl, carouselPrevIconUrl, carouselNextIconUrl | carouselSlides[].imageUrl |
| `video_player` | iconUrl, mediaUrl | - |
| `audio_player` | iconUrl, mediaUrl | - |
| `tooltip` | iconUrl | - |
| `description` | iconUrl | - |
| `spot` | iconUrl | - |
| `logo` | iconUrl | - |
| `popup` | iconUrl | - |

### 3.2 Cross-Reference Verification

**constructor.ts URL Fields vs preload.config.ts assetFields.all:**

| Field in constructor.ts | In assetFields.all? | Status |
|-------------------------|---------------------|--------|
| iconUrl | Yes | OK |
| mediaUrl | Yes | OK |
| backgroundImageUrl | Yes | OK |
| videoUrl | Yes | OK |
| audioUrl | Yes | OK |
| transitionVideoUrl | Yes | OK |
| reverseVideoUrl | Yes | OK |
| galleryHeaderImageUrl | Yes | OK |
| carouselPrevIconUrl | Yes | OK |
| carouselNextIconUrl | Yes | OK |
| galleryCarouselPrevIconUrl | Yes | OK |
| galleryCarouselNextIconUrl | Yes | OK |
| galleryCarouselBackIconUrl | Yes | OK |

**Nested Arrays vs nested config:**

| Nested Array | In nested config? | Status |
|--------------|-------------------|--------|
| galleryCards | Yes | OK |
| carouselSlides | Yes | OK |
| galleryInfoSpans | Yes | OK |

**Nested URL Fields vs nestedUrlFields:**

| Field in Nested Items | In nestedUrlFields? | Status |
|-----------------------|---------------------|--------|
| galleryCards[].imageUrl | Yes | OK |
| carouselSlides[].imageUrl | Yes | OK |
| galleryInfoSpans[].iconUrl | Yes | OK |

---

## 4. OFFLINE MODE PROCESSING

### 4.1 Storage Architecture

```
StorageManager                                [lib/offline/StorageManager.ts]
    ├── Threshold: OFFLINE_CONFIG.storage.indexedDbMinSize (5MB)
    ├── Small files (< 5MB): Cache API
    │   └── caches.open('vm-assets').put(url, response)
    └── Large files (≥ 5MB): IndexedDB via Dexie
        └── OfflineDbManager.storeAsset(asset)

hasAsset(url): Promise<boolean>
    1. Check IndexedDB: OfflineDbManager.hasAssetByUrl(url)
    2. Check Cache API: caches.open().match(url)

getAsset(url): Promise<Blob | null>
    1. Try IndexedDB: OfflineDbManager.getAssetByUrl(url)
    2. Try Cache API: caches.open().match(url).blob()
```

### 4.2 Service Worker Integration

```
sw.ts (Serwist-generated)
    ├── Precache: static assets (JS, CSS, fonts)
    ├── Runtime caching strategies:
    │   ├── API requests: NetworkFirst
    │   └── Assets: CacheFirst
    └── Offline fallback handling
```

---

## 5. IMAGE PRE-DECODE FLOW

### 5.1 extractPageImageUrls

```
extractPageImageUrls(page)                    [lib/imagePreDecode.ts:110-175]
    ↓
Extract background_image_url
    ↓
Parse ui_schema_json → elements[]
    ↓
For each element:
    ├── Direct image fields (assetFields.images):
    │   └── iconUrl, imageUrl, backgroundImageUrl, carousel*IconUrl, gallery*IconUrl, src
    └── Nested arrays (assetFields.nested):
        └── Filter nestedUrlFields to images only:
            ├── imageUrl (in images array)
            ├── iconUrl (in images array)
            └── videoUrl (not in images - correctly excluded)
```

### 5.2 waitForPageImages

```
waitForPageImages(page, timeoutMs, cacheProvider)
    ↓
extractPageImageUrls(page) → imageUrls[]
    ↓
decodeImages(imageUrls, timeoutMs, cacheProvider)
    ├── For each URL: decodeImage()
    │   ├── If cacheProvider: try getCachedBlobUrl() → blob URL (local, fast)
    │   └── new Image().decode() or onload fallback
    └── Promise.race([all decoded, timeout])
```

---

## 6. ROBUSTNESS VERIFICATION

### 6.1 All Asset Types Covered

| Asset Type | Extracted? | Preloaded? | Pre-decoded? | Cached? |
|------------|------------|------------|--------------|---------|
| Background images | Yes | Yes | Yes | Yes |
| Background videos | Yes | Yes | N/A | Yes |
| Background audio | Yes | Yes | N/A | Yes |
| Element icons | Yes | Yes | Yes | Yes |
| Transition videos | Yes | Yes | N/A | Yes |
| Reverse videos | Yes | Yes | N/A | Yes |
| Gallery cards images | Yes | Yes | Yes | Yes |
| Gallery info span icons | Yes | Yes | Yes | Yes |
| Gallery header image | Yes | Yes | Yes | Yes |
| Gallery carousel icons | Yes | Yes | Yes | Yes |
| Carousel slide images | Yes | Yes | Yes | Yes |
| Carousel nav icons | Yes | Yes | Yes | Yes |
| Media player URLs | Yes | Yes | N/A | Yes |

### 6.2 Extraction Algorithm Robustness

1. **extractPageLinks.ts** - Explicit field extraction
   - Uses `assetFields.all` for top-level fields
   - Uses `nested` + `nestedUrlFields` for nested arrays
   - Handles all configured fields

2. **useNeighborGraph.ts** - Recursive traversal
   - `checkObject()` recursively traverses to depth 5
   - Matches any field from `assetFields.all` at any nesting level
   - Handles deeply nested structures

3. **imagePreDecode.ts** - Image-only extraction
   - Filters `nestedUrlFields` against `assetFields.images`
   - Only pre-decodes actual images (not videos/audio)
   - Correctly filters non-image URLs

### 6.3 Edge Cases Handled

| Edge Case | Handling |
|-----------|----------|
| Empty ui_schema_json | Returns empty arrays safely |
| Missing nested arrays | Skipped gracefully |
| Null/undefined values | Filtered out |
| Already cached assets | Skipped download, creates blob URL |
| Presigned URL CORS failure | Retries with proxy URL |
| Network offline | Skips preloading, uses cache |
| Large files (>=5MB) | Stored in IndexedDB instead of Cache API |

---

## 7. CONCLUSION

The neighbor preloading system is **robust and comprehensive**:

1. **Central Configuration** - All asset URL fields defined in `preload.config.ts`
2. **Consistent Extraction** - All consumers use the same config
3. **Complete Coverage** - All 11 element types and their URL fields are covered
4. **Nested Arrays** - `galleryCards`, `carouselSlides`, `galleryInfoSpans` all handled
5. **Dual Storage** - Cache API for small files, IndexedDB for large files
6. **Fallback Handling** - Presigned URLs with proxy fallback
7. **Network Awareness** - Adaptive concurrency based on connection

**No gaps identified** - all asset types are properly extracted, preloaded, and cached.

---

## Related Documentation

- [assets-preloading.md](../../documentation/assets-preloading.md) - E2E preloading feature documentation
- [runtime-presentation.md](./runtime-presentation.md) - Runtime presentation viewer
- [constructor-page-editor.md](./constructor-page-editor.md) - Constructor page editor
- [hooks-module.md](./hooks-module.md) - Custom hooks reference
- [lib-module.md](./lib-module.md) - Library utilities reference