Dmitri 9f111d6226 added documentation

2026-07-03 16:11:24 +02:00

48 KiB

Raw Blame History

Offline PWA Mode - E2E Documentation

Overview

The Tour Builder Platform implements a sophisticated Progressive Web App (PWA) system enabling complete offline functionality. The system combines Service Workers, Cache API, and IndexedDB to provide seamless offline tour experiences with intelligent caching, download resume capability, and network adaptation.

Architecture

┌─────────────────────────────────────────────────────────────────┐
│                     Service Worker Layer                         │
│  Serwist (sw.ts) │ Precaching │ Runtime Caching │ Range Requests│
└─────────────────────────────────────────────────────────────────┘
                              │
┌─────────────────────────────────────────────────────────────────┐
│                      Storage Layer                               │
│  Cache API (< 5MB) │ IndexedDB (≥ 5MB) │ StorageManager         │
└─────────────────────────────────────────────────────────────────┘
                              │
┌─────────────────────────────────────────────────────────────────┐
│                     Download Layer                               │
│  DownloadManager │ DownloadEventBus │ Queue Persistence         │
│  assetUrl (presigned URLs + proxy fallback)                     │
└─────────────────────────────────────────────────────────────────┘
                              │
┌─────────────────────────────────────────────────────────────────┐
│                 Asset Cache Service Layer                        │
│  AssetCacheService │ assetDiscovery │ discoverProjectAssets     │
│  Unified asset discovery for online preload and offline mode    │
└─────────────────────────────────────────────────────────────────┘
                              │
┌─────────────────────────────────────────────────────────────────┐
│                     Hook Layer                                   │
│  usePreloadOrchestrator │ usePageSwitch │ useNeighborGraph      │
│  useNetworkAware │ useOfflineMode │ useStorageQuota             │
│  usePreloadProgress │ usePWAPreload                              │
└─────────────────────────────────────────────────────────────────┘
                              │
┌─────────────────────────────────────────────────────────────────┐
│                     UI Components                                │
│  OfflineToggle │ OfflineStatusIndicator │ StorageUsageDisplay   │
│  DownloadProgressPanel                                           │
└─────────────────────────────────────────────────────────────────┘

Service Worker (Serwist)

Configuration

Location: frontend/src/sw.ts

Built with Serwist (modern Workbox successor):

// next.config.mjs
{
  swSrc: 'src/sw.ts',
  swDest: 'public/sw.js',
  disable: process.env.NODE_ENV === 'development',
  maximumFileSizeToCacheInBytes: 5 * 1024 * 1024
}

The raised precache size limit keeps the main _app chunk in the Serwist precache. next.config.mjs also sets outputFileTracingRoot to the frontend directory so Next.js does not infer a parent workspace root when unrelated lockfiles exist above the project.

Caching Strategies

Resource Type	Strategy	Cache Name
Build assets	Precache	`serwist-precache-v2`
Images, fonts, CSS, JS	CacheFirst + cacheKeyWillBeUsed	`tour-builder-assets-v1`
Videos	CacheFirst + Range + cacheKeyWillBeUsed	`tour-builder-assets-v1`
API responses	NetworkFirst	`api-cache`
Audio files	CacheFirst + cacheKeyWillBeUsed	`tour-builder-assets-v1`

Unified Cache Keys (cacheKeyWillBeUsed Plugin)

All asset handlers use the cacheKeyWillBeUsed plugin to normalize URLs to storage keys. This ensures that:

Preload (DownloadManager) stores assets with storage keys: assets/project-id/file.jpg
Browser loads (SW interception) also use storage keys for cache lookup and storage
Cache hits occur regardless of presigned URL signature changes

// Applied to static, video, audio, and dynamic asset handlers
cacheKeyWillBeUsed: async ({ request, mode }) => {
  const storagePath = extractStoragePathFromUrl(request.url);
  if (storagePath) {
    console.log(`[SW] Using storagePath for ${mode}:`, storagePath.slice(-40));
    return new Request(storagePath);
  }
  return request;
}

Cacheable Extensions

Images: .png, .jpg, .jpeg, .gif, .webp, .svg, .ico
Video:  .mp4, .webm, .mov
Audio:  .mp3, .wav, .ogg, .m4a
Fonts:  .woff, .woff2, .ttf, .eot
Code:   .css, .js

Range Request Support

Enables video seeking in cached content via Serwist plugin:

// sw.ts - Video caching with Range support using Serwist plugin
{
  matcher: ({ request }) => isVideoRequest(request),
  handler: new CacheFirst({
    cacheName: OFFLINE_CONFIG.cacheNames.assets,
    plugins: [
      {
        // Handle range requests for video seeking
        cachedResponseWillBeUsed: async ({ cachedResponse, request }) => {
          if (!cachedResponse) return null;

          const rangeHeader = request.headers.get('range');
          if (!rangeHeader) return cachedResponse;

          // Parse range header
          const match = rangeHeader.match(/bytes=(\d+)-(\d*)/);
          if (!match) return cachedResponse;

          const start = parseInt(match[1], 10);
          const end = match[2] ? parseInt(match[2], 10) : undefined;

          // Get the full response body
          const blob = await cachedResponse.blob();
          const slicedBlob =
            end !== undefined
              ? blob.slice(start, end + 1)
              : blob.slice(start);

          // Return partial content
          return new Response(slicedBlob, {
            status: 206,
            statusText: 'Partial Content',
            headers: {
              'Content-Type':
                cachedResponse.headers.get('Content-Type') || 'video/mp4',
              'Content-Length': String(slicedBlob.size),
              'Content-Range': `bytes ${start}-${end !== undefined ? end : blob.size - 1}/${blob.size}`,
              'Accept-Ranges': 'bytes',
            },
          });
        },
        cacheWillUpdate: async ({ response }) => {
          if (response && response.status === 200) {
            return response;
          }
          return null;
        },
      },
    ],
  }),
},

Message API

Service worker receives commands from main thread:

Message	Payload	Description
`CACHE_ASSETS`	`{ urls: string[] }`	Cache specific asset URLs
`CACHE_VIDEO_CHUNK`	`{ url, chunk, contentType }`	Cache video chunks with streaming
`CLEAR_CACHE`	-	Clear all dynamic caches
`GET_CACHE_STATUS`	-	Query cache status
`SKIP_WAITING`	-	Force immediate activation

// Main thread → Service Worker
navigator.serviceWorker.controller.postMessage({
  type: 'CACHE_ASSETS',
  payload: { urls: ['https://example.com/video.mp4'] }
});

// Service Worker → Main thread (response to GET_CACHE_STATUS)
// { type: 'CACHE_STATUS', payload: { cachedCount: number, urls: string[] } }

Lifecycle

1. Registration → Next.js/Serwist auto-registers (production)
2. Installation → Caches all precache entries (build assets)
3. Activation  → Deletes old cache versions, claims clients
4. Fetch       → Intercepts requests, applies strategies
5. Update      → Checks for updates every 60 minutes

Cache Organization

Named Caches

// frontend/src/config/offline.config.ts
cacheNames: {
  static: 'tour-builder-static-v1',   // Precached build assets
  dynamic: 'tour-builder-dynamic-v1', // Runtime API responses (note: SW uses 'api-cache')
  assets: 'tour-builder-assets-v1',   // Project assets (used by SW and preloading)
}

Note: The actual API cache in the Service Worker is api-cache, not tour-builder-dynamic-v1. The dynamic config key is reserved for future use.

Cache Invalidation

Version bump clears old caches on SW activation:

tour-builder-static-v1 → tour-builder-static-v2
                         ↓
        Old v1 cache automatically deleted

Dual Storage Strategy

Storage Decision

┌─────────────────────────────────────────┐
│           File Size Check               │
└─────────────────────────────────────────┘
                  │
        ┌─────────┴─────────┐
        │                   │
        ▼                   ▼
   Size < 5MB          Size ≥ 5MB
        │                   │
        ▼                   ▼
┌───────────────┐   ┌───────────────┐
│   Cache API   │   │   IndexedDB   │
│  (Fast, SW)   │   │ (Large files) │
└───────────────┘   └───────────────┘

Why Dual Storage?

Storage	Pros	Cons
Cache API	Fast, SW integration, simple API	Size limits, no metadata
IndexedDB	Large files, rich metadata, queries	Slower, more complex

StorageManager

Location: frontend/src/lib/offline/StorageManager.ts

class StorageManager {
  // Storage quota check (returns StorageQuotaInfo with canStore function)
  static async getStorageQuota(): Promise<StorageQuotaInfo>;

  // Request persistent storage (won't be cleared by browser)
  static async requestPersistentStorage(): Promise<boolean>;

  // Determine storage location (5MB threshold)
  static shouldUseIndexedDB(sizeBytes: number): boolean {
    return sizeBytes >= OFFLINE_CONFIG.storage.indexedDbMinSize;
  }

  // Store asset (auto-selects Cache API or IndexedDB based on size)
  static async storeAsset(url: string, blob: Blob, metadata: AssetMetadata): Promise<void>;

  // Retrieve asset (checks IndexedDB first, then Cache API)
  static async getAsset(url: string): Promise<Blob | null>;

  // Check if asset exists (checks both storages)
  static async hasAsset(url: string): Promise<boolean>;

  // Delete asset from both storages
  static async deleteAsset(url: string, assetId?: string): Promise<void>;

  // Bulk delete for project
  static async deleteProjectAssets(projectId: string): Promise<void>;

  // Send URLs to SW for caching
  static async cacheViaServiceWorker(urls: string[]): Promise<void>;

  // Get SW cache status
  static async getServiceWorkerCacheStatus(): Promise<{ cachedCount: number; urls: string[] }>;

  // Get total storage used
  static async getTotalStorageUsed(): Promise<number>;

  // Clear all offline storage
  static async clearAll(): Promise<void>;
}

IndexedDB Schema

Database: `TourBuilderOffline` (v1)

Location: frontend/src/lib/offlineDb/schema.ts

Built with Dexie.js:

import Dexie, { type EntityTable } from 'dexie';

class OfflineDatabase extends Dexie {
  assets!: EntityTable<OfflineAsset, 'id'>;
  projects!: EntityTable<OfflineProject, 'id'>;
  downloadQueue!: EntityTable<DownloadQueueItem, 'id'>;

  constructor() {
    super(OFFLINE_CONFIG.dbName); // 'TourBuilderOffline'

    this.version(OFFLINE_CONFIG.dbVersion).stores({
      // Index definitions (comma-separated indexed fields)
      assets: 'id, projectId, url, variantType, assetType, downloadedAt',
      projects: 'id, slug, status, lastSyncedAt',
      downloadQueue: 'id, projectId, status, priority, addedAt',
    });
  }
}

Assets Table

interface OfflineAsset {
  id: string;
  projectId: string;
  url: string;
  filename: string;
  variantType: AssetVariantType;
  assetType: AssetType;
  mimeType: string;
  sizeBytes: number;
  blob: Blob;              // Actual file data
  downloadedAt: number;    // Timestamp
}

Projects Table

interface OfflineProject {
  id: string;
  slug: string;
  name: string;             // Project display name
  status: ProjectOfflineStatus;
  totalAssets: number;
  downloadedAssets: number;
  totalSizeBytes: number;
  downloadedSizeBytes: number;
  lastSyncedAt?: number;    // Optional timestamp
  version?: string;         // Manifest version for update detection
}

type ProjectOfflineStatus =
  | 'not_downloaded'
  | 'downloading'
  | 'downloaded'
  | 'outdated'
  | 'error';

Download Queue Table

interface DownloadQueueItem {
  id: string;
  projectId: string;
  assetId: string;
  url: string;
  filename: string;
  status: PreloadJobStatus;  // 'queued' | 'downloading' | 'completed' | 'error' | 'paused'
  priority: number;
  retryCount: number;
  bytesLoaded: number;
  totalBytes: number;
  addedAt: number;
  lastAttemptAt?: number;
  error?: string;
}

OfflineDbManager

Location: frontend/src/lib/offlineDb/OfflineDbManager.ts (~342 LOC)

class OfflineDbManager {
  // Asset operations
  static async storeAsset(asset: OfflineAsset): Promise<void>;
  static async getAsset(id: string): Promise<OfflineAsset | undefined>;
  static async getAssetByUrl(url: string): Promise<OfflineAsset | undefined>;
  static async getProjectAssets(projectId: string): Promise<OfflineAsset[]>;
  static async deleteAsset(id: string): Promise<void>;
  static async deleteProjectAssets(projectId: string): Promise<number>;
  static async hasAsset(id: string): Promise<boolean>;
  static async hasAssetByUrl(url: string): Promise<boolean>;
  static async getTotalAssetsSize(): Promise<number>;
  static async getProjectAssetsSize(projectId: string): Promise<number>;

  // Project tracking
  static async upsertProject(project: OfflineProject): Promise<void>;
  static async getProject(id: string): Promise<OfflineProject | undefined>;
  static async getProjectBySlug(slug: string): Promise<OfflineProject | undefined>;
  static async getAllProjects(): Promise<OfflineProject[]>;
  static async updateProjectStatus(id: string, status: ProjectOfflineStatus): Promise<void>;
  static async updateProjectProgress(id: string, downloadedAssets: number, downloadedSizeBytes: number): Promise<void>;
  static async deleteProject(id: string): Promise<void>;

  // Queue management
  static async addToQueue(item: DownloadQueueItem): Promise<void>;
  static async getQueueItem(id: string): Promise<DownloadQueueItem | undefined>;
  static async getProjectQueue(projectId: string): Promise<DownloadQueueItem[]>;
  static async getPendingQueue(): Promise<DownloadQueueItem[]>;
  static async updateQueueStatus(id: string, status: PreloadJobStatus, error?: string): Promise<void>;
  static async updateQueueProgress(id: string, bytesLoaded: number, totalBytes: number): Promise<void>;
  static async incrementRetry(id: string): Promise<number>;
  static async removeFromQueue(id: string): Promise<void>;
  static async clearProjectQueue(projectId: string): Promise<number>;
  static async clearQueue(): Promise<void>;
  static async resetFailedItems(projectId?: string): Promise<number>;

  // Utility
  static async clearAll(): Promise<void>;
  static async getStats(): Promise<{
    projectCount: number;
    assetCount: number;
    queueCount: number;
    totalSizeBytes: number;
  }>;
}

Download Manager

Location: frontend/src/lib/offline/DownloadManager.ts

Features

Priority Queue - Downloads sorted by importance
Concurrent Downloads - Configurable max (default: 3)
Streaming Progress - Uses response.body.getReader()
Retry Logic - 3 retries with linear backoff
Queue Persistence - Resumes after page reload via IndexedDB
Pause/Resume - Pause all or individual downloads via AbortController
Cancellation - Cancel downloads by project or individually
Presigned URL → Proxy Fallback - Automatic fallback when S3 presigned URLs fail (CORS)

Priority Calculation

priority = assetTypePriority + variantPriority

// Asset type weights (PRELOAD_CONFIG.priority.assetType)
transition: 150  // Highest - needed immediately on navigation click
image: 100
audio: 50
video: 30

// Variant weights (PRELOAD_CONFIG.priority.variant)
thumbnail: 50
preview: 40
webp: 35
mp4_low: 20
mp4_high: 10
original: 5

Download Flow

┌─────────────────────────────────────────┐
│  1. Add assets to queue                 │
│     └── downloadManager.addJob(params)  │
└─────────────────────────────────────────┘
                  │
                  ▼
┌─────────────────────────────────────────┐
│  2. Persist to IndexedDB                │
│     └── Resume capability on reload     │
└─────────────────────────────────────────┘
                  │
                  ▼
┌─────────────────────────────────────────┐
│  3. Sort by priority                    │
│     └── Higher priority = download first│
└─────────────────────────────────────────┘
                  │
                  ▼
┌─────────────────────────────────────────┐
│  4. Process queue (max 3 concurrent)    │
│     └── fetch() with AbortController    │
└─────────────────────────────────────────┘
                  │
                  ▼
┌─────────────────────────────────────────┐
│  5. Stream response body                │
│     └── response.body.getReader()       │
│     └── Track bytesLoaded / totalBytes  │
│     └── Emit progress events            │
└─────────────────────────────────────────┘
                  │
           ┌──────┴──────┐
           ▼             ▼
┌──────────────────┐  ┌─────────────────────────┐
│  Download OK     │  │  Download FAILED        │
│                  │  │  (e.g., CORS error)     │
└────────┬─────────┘  └───────────┬─────────────┘
         │                        │
         │                        ▼
         │            ┌─────────────────────────┐
         │            │  6a. Presigned URL?     │
         │            │  → Retry with proxy URL │
         │            │  /api/file/download?... │
         │            └───────────┬─────────────┘
         │                        │
         ▼                        ▼
┌─────────────────────────────────────────┐
│  7. Store in appropriate storage        │
│     └── < 5MB → Cache API               │
│     └── ≥ 5MB → IndexedDB               │
│     └── With isPartial flag if partial  │
└─────────────────────────────────────────┘
                  │
                  ▼
┌─────────────────────────────────────────┐
│  8. Emit completion event               │
│     └── emitComplete({ storageKey })    │
└─────────────────────────────────────────┘

Retry Configuration

// From PRELOAD_CONFIG
retry: {
  maxRetries: 3,
  retryDelayMs: 1000,  // Delay before retry
}

// Linear backoff: retryDelayMs * retryCount
// 1s → 2s → 3s

Resume Capability

Queue state persisted in IndexedDB:

// On page load
await downloadManager.restoreQueue();

// Restores pending/paused items from downloadQueue table
// Re-adds to in-memory queue sorted by priority
// Continues processing automatically

API

const downloadManager = new DownloadManagerClass();

// Add job
await downloadManager.addJob({
  assetId: string;
  projectId: string;
  url: string;
  filename: string;
  variantType: AssetVariantType;
  assetType: AssetType;
  priority?: number;  // Auto-calculated if not provided
});

// Control
downloadManager.pauseAll();
downloadManager.resumeAll();
downloadManager.cancelJob(jobId: string);
downloadManager.cancelProjectDownloads(projectId: string);
downloadManager.clearQueue();

// Status
downloadManager.getStatus(); // { queueLength, activeCount, isPaused }

// Restore
await downloadManager.restoreQueue();

S3 Presigned URL Integration

For S3 storage, the preload system uses presigned URLs for direct downloads, bypassing the backend proxy.

Flow

1. Batch fetch presigned URLs
   └── POST /api/file/presign { urls: [...] }

2. Download directly from S3
   └── fetch(presignedUrl) with progress tracking

3. Store in Cache API (dual key)
   └── caches.put(downloadUrl, blob)     // By presigned URL
   └── caches.put(storageKey, blob)      // By canonical path (e.g., "assets/vid.mp4")

4. Create blob URL
   └── URL.createObjectURL(blob)

5. Pre-decode (images only)
   └── new Image().decode() on blob URL

6. Store ready blob URL (dual key)
   └── readyBlobUrlsRef.set(downloadUrl, blobUrl)
   └── readyBlobUrlsRef.set(storageKey, blobUrl)  // Enables lookup after URL change

Storage Key Mapping: Assets are stored under both download URL AND storage key (canonical path). This ensures cache hits even when presigned URLs change signatures or after page refresh:

Scenario	Lookup Key	Result
Same session	`storageKey`	✅ Instant from readyBlobUrlsRef
After URL regeneration	`storageKey`	✅ Instant (signature doesn't matter)
After page refresh	`storageKey`	✅ From Cache API

Key Functions

Function	Source	Purpose
`queuePresignedUrls()`	`lib/assetUrl.ts`	Batch queue URLs for presigning
`flushPresignedUrlQueue()`	`lib/assetUrl.ts`	Fetch queued presigned URLs
`resolveAssetPlaybackUrl()`	`lib/assetUrl.ts`	Resolve URL with presigned cache
`getReadyBlobUrl()`	`usePreloadOrchestrator`	O(1) instant lookup for pre-decoded blob URL
`getCachedBlobUrl()`	`usePreloadOrchestrator`	Async fallback - creates blob URL from Cache API

Presigned URL Expiry

URLs expire after 1 hour (3600 seconds)
Frontend caches with 1-hour TTL, refreshes 5 minutes before expiry
On CORS failure, automatically falls back to backend proxy (/api/file/download)

Download Event Bus

Location: frontend/src/lib/offline/DownloadEventBus.ts

Browser-native EventEmitter (no Socket.IO dependency):

Events

Event	Payload Interface
`asset-preload-start`	`PreloadStartEvent`
`asset-preload-progress`	`PreloadProgressEvent`
`asset-preload-complete`	`PreloadCompleteEvent`
`asset-preload-error`	`PreloadErrorEvent`
`project-download-progress`	`ProjectDownloadProgressEvent`
`project-download-complete`	`ProjectDownloadCompleteEvent`
`queue-update`	`void` (no payload)

Event Payloads

interface PreloadStartEvent {
  jobId: string;
  assetId: string;
  url: string;
}

interface PreloadProgressEvent {
  jobId: string;
  progress: number;        // 0-100
  bytesLoaded: number;
  totalBytes: number;
}

interface PreloadCompleteEvent {
  jobId: string;
  assetId: string;
  storageKey: string;  // Canonical storage key for cache lookup
}

interface PreloadErrorEvent {
  jobId: string;
  assetId: string;
  error: string;
}

interface ProjectDownloadProgressEvent {
  projectId: string;
  progress: number;           // 0-100
  downloadedAssets: number;
  totalAssets: number;
  downloadedBytes: number;
  totalBytes: number;
}

interface ProjectDownloadCompleteEvent {
  projectId: string;
}

Usage

import { downloadEventBus } from '@/lib/offline/DownloadEventBus';
import { OFFLINE_CONFIG } from '@/config/offline.config';

// Subscribe to events
const unsubscribe = downloadEventBus.on(
  OFFLINE_CONFIG.events.preloadProgress,  // 'asset-preload-progress'
  (event: PreloadProgressEvent) => {
    console.log(`Downloaded ${event.progress}%`);
    updateProgressUI(event.progress);
  }
);

// Unsubscribe when done
unsubscribe();

// Or manually unsubscribe
downloadEventBus.off(OFFLINE_CONFIG.events.preloadProgress, handler);

// One-time subscription
downloadEventBus.once(OFFLINE_CONFIG.events.preloadComplete, handler);

// Remove all listeners
downloadEventBus.removeAllListeners();

Convenience Methods

// Pre-built emit methods on downloadEventBus
downloadEventBus.emitPreloadStart(data: PreloadStartEvent);
downloadEventBus.emitPreloadProgress(data: PreloadProgressEvent);
downloadEventBus.emitPreloadComplete(data: PreloadCompleteEvent);
downloadEventBus.emitPreloadError(data: PreloadErrorEvent);
downloadEventBus.emitProjectProgress(data: ProjectDownloadProgressEvent);
downloadEventBus.emitProjectComplete(data: ProjectDownloadCompleteEvent);
downloadEventBus.emitQueueUpdate();

Hooks

useOfflineMode

Location: frontend/src/hooks/useOfflineMode.ts

Orchestrates offline mode for a specific project. Uses frontend asset discovery (same as online preloading) - no backend manifest dependency.

interface UseOfflineModeOptions {
  projectId: string | null;
  projectSlug?: string;
  projectName?: string;
  /** Pages data for frontend asset discovery (required for offline download) */
  pages?: PreloadPage[];
  enabled?: boolean;
}

interface UseOfflineModeResult {
  // Status
  isOfflineCapable: boolean;     // SW + caches supported
  isDownloaded: boolean;         // status === 'downloaded'
  isDownloading: boolean;        // status === 'downloading' && !isPaused
  status: ProjectOfflineStatus;
  progress: number;              // 0-100
  downloadedAssets: number;
  totalAssets: number;
  downloadedBytes: number;
  totalBytes: number;
  error: string | null;

  // Actions
  startDownload: () => Promise<void>;   // Discover assets from pages, queue downloads
  pauseDownload: () => void;
  resumeDownload: () => void;
  cancelDownload: () => void;
  deleteOfflineData: () => Promise<void>;
  checkForUpdates: () => Promise<boolean>;

  // Info
  projectInfo: OfflineProject | null;
  estimatedSize: number;
  formatSize: (bytes: number) => string;
}

// Usage
const {
  isDownloaded,
  isDownloading,
  progress,
  startDownload,
  deleteOfflineData,
  formatSize,
  estimatedSize,
} = useOfflineMode({
  projectId: '123',
  projectSlug: 'my-tour',
  projectName: 'My Tour',
  pages, // Required: pages data for asset discovery
});

useStorageQuota

Location: frontend/src/hooks/useStorageQuota.ts

Monitors storage quota and usage:

interface UseStorageQuotaResult extends StorageQuotaInfo {
  isLoading: boolean;
  error: string | null;
  refresh: () => Promise<void>;
  requestPersistence: () => Promise<boolean>;
  isPersisted: boolean;
  isWarning: boolean;   // percentUsed >= 80%
  isCritical: boolean;  // percentUsed >= 95%
  formatSize: (bytes: number) => string;
}

// Usage
const {
  usage,
  quota,
  percentUsed,
  available,
  canStore,
  isWarning,
  isCritical,
  isPersisted,
  requestPersistence,
  formatSize,
} = useStorageQuota();

// Check if we can store 100MB
if (canStore(100 * 1024 * 1024)) {
  // Proceed with download
}

useNetworkAware

Location: frontend/src/hooks/useNetworkAware.ts

Network detection and adaptive behavior:

interface NetworkInfo {
  isOnline: boolean;
  effectiveType?: 'slow-2g' | '2g' | '3g' | '4g';
  downlink?: number;      // Mbps
  rtt?: number;           // Round-trip time (ms)
  saveData?: boolean;     // Data saver mode
}

interface UseNetworkAwareResult {
  networkInfo: NetworkInfo;
  shouldPreloadAggressively: boolean;  // 4g or downlink >= 5Mbps
  preferLowQuality: boolean;           // slow-2g, 2g, or saveData
  recommendedConcurrency: number;      // 1-3 based on connection
  suggestOfflineMode: boolean;         // Poor connection detected
}

// Usage
const {
  networkInfo,
  shouldPreloadAggressively,
  preferLowQuality,
  recommendedConcurrency,
} = useNetworkAware();

if (shouldPreloadAggressively) {
  // Preload neighbor pages
}

Concurrency Adaptation

Connection	Concurrent Downloads
4g / Good	3
3g	2
2g / Slow	1
Save-Data	1

UI Components

Location: frontend/src/components/Offline/

OfflineToggle

Button component for toggling offline mode on a project:

interface OfflineToggleProps {
  projectId: string | null;
  projectSlug?: string;
  projectName?: string;
  /** Pages data for frontend asset discovery (required for offline download) */
  pages?: PreloadPage[];
  className?: string;
  showLabel?: boolean;     // Default: true
  size?: 'small' | 'medium' | 'large';  // Default: 'medium'
}

// Usage
<OfflineToggle
  projectId={project.id}
  projectSlug={project.slug}
  projectName={project.name}
  pages={filteredPages}  // Required: pages for asset discovery
  showLabel={true}
  size="medium"
/>

Features:

Shows download status with icons (cloud download, cloud check, cloud off)
Displays size estimate before download
Shows progress percentage during download
Downloaded state switches the same runtime control into its active/remove state; deleting offline data uses the active offline button with confirmation
Storage quota warnings
Confirmation dialogs for deletion

OfflineStatusIndicator

Displays current offline/online status with visual indicator.

StorageUsageDisplay

Shows storage quota usage with progress bar and statistics.

DownloadProgressPanel

Panel showing active downloads with individual progress bars and controls.

Frontend Asset Discovery

Overview

Asset discovery is handled entirely in the frontend using the same logic for both online preloading and offline downloads. This ensures consistency and eliminates backend dependencies.

Location: frontend/src/lib/assetCache/

Key Components

File	Purpose
`AssetCacheService.ts`	Unified entry point for asset caching
`assetDiscovery.ts`	Shared asset extraction logic
`index.ts`	Module exports

Asset Discovery Functions

// Discover all assets for a project (offline download)
discoverProjectAssets(
  pages: PreloadPage[],
  pageLinks: PreloadPageLink[],
  elements: PreloadElement[],
  options?: AssetDiscoveryOptions,
): AssetToCache[]

// Get prioritized assets for current page + neighbors (online preload)
getPrioritizedAssets(
  currentPageId: string,
  pages: PreloadPage[],
  pageLinks: PreloadPageLink[],
  elements: PreloadElement[],
  neighborPageIds: Array<{ pageId: string; distance: number }>,
  options?: AssetDiscoveryOptions,
): AssetToCache[]

AssetToCache Interface

interface AssetToCache {
  storageKey: string;      // Canonical storage key for caching
  originalUrl: string;     // Original URL from page/element data
  assetType: AssetType;    // 'image' | 'video' | 'audio' | 'transition' | 'other'
  pageId: string;          // Page this asset belongs to
  priority: number;        // Download priority (higher = first)
  sizeBytes?: number;      // Estimated size in bytes
  isTransition?: boolean;  // True for transition videos
}

Asset Sources

The discovery system extracts assets from:

Page backgrounds - background_image_url, background_video_url, background_audio_url
Element content_json - All URL fields defined in PRELOAD_CONFIG.assetFields.all
Transition videos - transition.video_url from page links
Nested assets - galleryCards[].imageUrl, carouselSlides[].videoUrl, etc.

URL Fields Scanned

// From PRELOAD_CONFIG.assetFields.all
[
  'iconUrl', 'imageUrl', 'mediaUrl', 'videoUrl', 'audioUrl',
  'transitionVideoUrl', 'backgroundImageUrl', 'reverseVideoUrl',
  'carouselPrevIconUrl', 'carouselNextIconUrl', 'src', 'url',
  'poster', 'thumbnail'
]

AssetCacheService

Unified service for both online preload and offline download:

class AssetCacheService {
  // Discover assets (single source of truth)
  static discoverProjectAssets(pages, pageLinks, elements, options?): AssetToCache[]
  static getPrioritizedAssets(currentPageId, pages, pageLinks, elements, neighbors, options?): AssetToCache[]

  // Check cache status
  static async getAssetInfo(storageKey): Promise<CachedAssetInfo | null>
  static async checkCacheStatus(storageKeys): Promise<Map<string, CachedAssetInfo>>
  static async filterAssetsNeedingDownload(assets, mode): Promise<AssetToCache[]>

  // Queue downloads
  static async queueDownloads(assets, options: QueueDownloadOptions): Promise<void>

  // Resolve download URL (presigned or proxy)
  static resolveDownloadUrl(storageKey, presignedUrls?): string

  // Cache utilities
  static async clearProjectCache(projectId): Promise<void>
  static getReadyBlobUrl(url): string | null
  static hasReadyBlobUrl(url): boolean
}

CachedAssetInfo Interface

interface CachedAssetInfo {
  storageKey: string;
  exists: boolean;
  isPartial: boolean;    // True for partial downloads (need full for offline)
  sizeBytes: number;
  downloadedAt?: number;
}

Usage in useOfflineMode

// frontend/src/hooks/useOfflineMode.ts

const discoverAssets = useCallback((): AssetToCache[] => {
  if (!pages || pages.length === 0) return [];

  // Extract pageLinks and elements from all pages
  const { pageLinks, preloadElements } = extractPageLinksAndElements(pages);

  // Use shared asset discovery (same as online preload)
  return discoverProjectAssets(pages, pageLinks, preloadElements);
}, [pages]);

const startDownload = useCallback(async () => {
  const assets = discoverAssets();
  // ... queue downloads using discovered assets
}, [discoverAssets]);

Offline Types

Location: frontend/src/types/offline.ts

// Project status
type ProjectOfflineStatus =
  | 'not_downloaded'
  | 'downloading'
  | 'downloaded'
  | 'outdated'
  | 'error';

// Asset variant types
type AssetVariantType =
  | 'thumbnail'
  | 'preview'
  | 'webp'
  | 'mp4_low'
  | 'mp4_high'
  | 'original';

// Asset types
type AssetType =
  | 'image'
  | 'video'
  | 'audio'
  | 'transition'
  | 'other';

// Preload job status
type PreloadJobStatus =
  | 'queued'
  | 'downloading'
  | 'completed'
  | 'error'
  | 'paused';

// Storage quota info
interface StorageQuotaInfo {
  usage: number;
  quota: number;
  percentUsed: number;
  available: number;
  canStore: (bytes: number) => boolean;
}

// Network information
interface NetworkInfo {
  isOnline: boolean;
  effectiveType?: 'slow-2g' | '2g' | '3g' | '4g';
  downlink?: number;
  rtt?: number;
  saveData?: boolean;
}

// Cache status for an asset (from StorageManager.getAssetInfo)
interface CachedAssetInfo {
  storageKey: string;
  exists: boolean;
  isPartial: boolean;    // True for partial downloads
  sizeBytes: number;
  downloadedAt?: number;
}

// Asset to cache (from assetDiscovery)
interface AssetToCache {
  storageKey: string;
  originalUrl: string;
  assetType: AssetType;
  pageId: string;
  priority: number;
  sizeBytes?: number;
  isTransition?: boolean;
}

// Preload state
interface PreloadState {
  isActive: boolean;
  currentPageId: string | null;
  queuedAssets: string[];
  loadingAssets: string[];
  completedAssets: string[];
  failedAssets: string[];
  totalProgress: number;
}

// Neighbor graph
interface NeighborGraph {
  connections: Map<string, Array<{ pageId: string; distance: number }>>;
  getNeighbors: (pageId: string, maxDepth: number) => string[];
  getAssetsForPages: (pageIds: string[]) => string[];
}

Key Workflows

A. Project Download for Offline Use

1. User clicks OfflineToggle button
   └── useOfflineMode.startDownload()

2. Discover assets from pages (frontend-only, no backend call)
   └── discoverProjectAssets(pages, pageLinks, preloadElements)
   └── Same asset discovery as online preload

3. Check which assets need downloading
   └── StorageManager.getAssetInfo(storageKey) for each
   └── Skip fully cached, re-download partial for offline

4. Check storage quota
   └── StorageManager.getStorageQuota()
   └── Ensure sufficient space available

5. Create project record in IndexedDB
   └── OfflineDbManager.upsertProject()

6. Fetch presigned URLs for S3 assets
   └── queuePresignedUrls(storagePaths)
   └── Falls back to proxy if presigning fails

7. Initialize download queue
   └── downloadManager.addJob() for each asset
   └── Full downloads (no maxBytes) for offline

8. Process downloads
   └── Priority-based concurrent downloads (max 3)
   └── Stream with progress tracking
   └── Auto-fallback: presigned URL → proxy on CORS error

9. Store assets
   └── < 5MB → Cache API
   └── ≥ 5MB → IndexedDB
   └── With isPartial: false for offline completeness

10. Track progress via events
    └── Update project status in IndexedDB
    └── Emit events via DownloadEventBus (includes storageKey)

11. Complete
    └── Mark project as 'downloaded'
    └── Emit projectDownloadComplete event

B. Offline Asset Retrieval

1. App requests asset (by storage path)
   └── e.g., storagePath = 'assets/project-123/video.mp4'

2. Try in-memory lookup first (O(1) instant)
   └── getReadyBlobUrl(storagePath)
   └── Returns pre-decoded blob URL if available

3. Try Cache API by storage key
   └── getCachedBlobUrl(storagePath)
   └── Works after page refresh (in-memory cleared)

4. Try resolved URL lookups (fallback)
   └── getReadyBlobUrl(resolvedUrl)
   └── getCachedBlobUrl(resolvedUrl)

5. If found
   └── Create blob URL: URL.createObjectURL(blob)
   └── Return for playback

6. If not found
   └── Try network fetch (if online)
   └── Or show offline error

Lookup Priority:

1. getReadyBlobUrl(storageKey)   → O(1) instant (same session)
2. getCachedBlobUrl(storageKey)  → Cache API (~5ms, post-refresh)
3. getReadyBlobUrl(resolvedUrl)  → fallback
4. getCachedBlobUrl(resolvedUrl) → fallback
5. Network fetch                 → last resort

C. Resume Interrupted Downloads

1. Page loads
   └── downloadManager.restoreQueue()

2. Query IndexedDB
   └── OfflineDbManager.getPendingQueue()
   └── Get pending/paused queue items

3. Resume downloads
   └── Re-add to in-memory queue
   └── Sort by priority
   └── Continue processing

4. Update progress
   └── Emit events for UI

D. Check for Updates

1. App calls useOfflineMode.checkForUpdates()

2. Discover current assets from pages
   └── discoverProjectAssets(pages, pageLinks, preloadElements)

3. Compare with stored project info
   └── If asset count changed, set status to 'outdated'

4. User can re-download
   └── deleteOfflineData() then startDownload()

Configuration Reference

Preload Configuration

Location: frontend/src/config/preload.config.ts

export const PRELOAD_CONFIG = {
  // Queue settings
  maxConcurrentDownloads: 3,
  maxRetries: 3,
  retryDelayMs: 1000,

  // Size thresholds
  largeFileThreshold: 5 * 1024 * 1024,  // 5MB -> use IndexedDB
  videoChunkSize: 5 * 1024 * 1024,       // 5MB chunks
  initialVideoBufferSeconds: 5,

  // Priority weights (higher = load first)
  priority: {
    currentPage: 1000,
    neighborBase: 500,
    assetType: {
      transition: 150,  // Highest - needed immediately on navigation click
      image: 100,
      audio: 50,
      video: 30,
    },
    variant: {
      thumbnail: 50,
      preview: 40,
      webp: 35,
      mp4_low: 20,
      mp4_high: 10,
      original: 5,
    },
    linkCountMultiplier: 10,
    maxLinkBonus: 50,
  },

  // Storage warnings
  storage: {
    warningPercent: 80,
    criticalPercent: 95,
    minFreeBuffer: 50 * 1024 * 1024,  // 50MB
  },

  // Auto-cleanup timeouts
  autoRemove: {
    completedMs: 3000,
    errorMs: 10000,
  },

  // Neighbor graph traversal
  neighborGraph: {
    maxDepth: 1,              // Only preload immediate neighbors (reduced from 2)
    constructorMaxDepth: 1,   // Same as maxDepth for constructor
  },

  // Asset URL field names in element content_json
  assetFields: {
    all: [
      'iconUrl', 'imageUrl', 'mediaUrl', 'videoUrl', 'audioUrl',
      'transitionVideoUrl', 'backgroundImageUrl', 'reverseVideoUrl',
      'carouselPrevIconUrl', 'carouselNextIconUrl',
      'src', 'url', 'poster', 'thumbnail',
    ],
    images: [
      'iconUrl', 'imageUrl', 'backgroundImageUrl',
      'carouselPrevIconUrl', 'carouselNextIconUrl', 'src',
    ],
    nested: ['galleryCards', 'carouselSlides'],
    nestedUrlFields: ['imageUrl', 'videoUrl'],
  },
};

Offline Configuration

Location: frontend/src/config/offline.config.ts

export const OFFLINE_CONFIG = {
  // IndexedDB
  dbName: 'TourBuilderOffline',
  dbVersion: 1,

  // Cache names (for Cache API)
  cacheNames: {
    static: 'tour-builder-static-v1',
    dynamic: 'tour-builder-dynamic-v1',
    assets: 'tour-builder-assets-v1',
  },

  // Events (EventEmitter event names)
  events: {
    preloadStart: 'asset-preload-start',
    preloadProgress: 'asset-preload-progress',
    preloadComplete: 'asset-preload-complete',
    preloadError: 'asset-preload-error',
    projectDownloadProgress: 'project-download-progress',
    projectDownloadComplete: 'project-download-complete',
    queueUpdate: 'queue-update',
  },

  // Service worker settings
  serviceWorker: {
    scope: '/',
    updateInterval: 60 * 60 * 1000,  // 1 hour
  },

  // Storage settings
  storage: {
    cacheApiMaxSize: 5 * 1024 * 1024,   // 5MB
    indexedDbMinSize: 5 * 1024 * 1024,  // 5MB
  },

  // Retry settings
  retry: {
    maxRetries: 3,
    backoffMs: 1000,
    maxBackoffMs: 30000,
  },
};

Key Files Reference

File	Location	Purpose
`sw.ts`	`frontend/src/`	Service worker implementation
`offline.config.ts`	`frontend/src/config/`	Offline settings
`preload.config.ts`	`frontend/src/config/`	Preload settings
`StorageManager.ts`	`frontend/src/lib/offline/`	Dual-storage abstraction
`DownloadManager.ts`	`frontend/src/lib/offline/`	Download queue with auto proxy fallback
`DownloadEventBus.ts`	`frontend/src/lib/offline/`	Event emitter
`OfflineDbManager.ts`	`frontend/src/lib/offlineDb/`	IndexedDB operations
`schema.ts`	`frontend/src/lib/offlineDb/`	Dexie schema
`AssetCacheService.ts`	`frontend/src/lib/assetCache/`	Unified asset caching service
`assetDiscovery.ts`	`frontend/src/lib/assetCache/`	Shared asset extraction logic
`assetUrl.ts`	`frontend/src/lib/`	Presigned URL cache and resolution
`extractPageLinks.ts`	`frontend/src/lib/`	Extract navigation targets from pages
`usePreloadOrchestrator.ts`	`frontend/src/hooks/`	Asset preloading with ready blob URLs
`usePageSwitch.ts`	`frontend/src/hooks/`	Page navigation using preloaded assets
`useOfflineMode.ts`	`frontend/src/hooks/`	Project offline orchestration (frontend discovery)
`useStorageQuota.ts`	`frontend/src/hooks/`	Storage quota monitoring
`useNetworkAware.ts`	`frontend/src/hooks/`	Network detection
`OfflineToggle.tsx`	`frontend/src/components/Offline/`	Offline toggle button
`OfflineStatusIndicator.tsx`	`frontend/src/components/Offline/`	Offline status display
`StorageUsageDisplay.tsx`	`frontend/src/components/Offline/`	Storage quota visualization
`DownloadProgressPanel.tsx`	`frontend/src/components/Offline/`	Download progress panel
`usePWAPreload.ts`	`frontend/src/hooks/`	PWA preload orchestration
`offline.ts`	`frontend/src/types/`	Type definitions

Browser Requirements

Feature	Support
Service Workers	All modern browsers
IndexedDB	All modern browsers (Dexie handles differences)
Cache API	All modern browsers
Network Information API	Chrome, Edge, Firefox (graceful degradation)
Blob URLs	All modern browsers
navigator.storage.persist()	Most modern browsers
Response streaming	All modern browsers

Troubleshooting

Downloads Not Starting

Check network connectivity
Verify storage quota available (useStorageQuota)
Check browser console for errors
Ensure Service Worker registered
Verify project ID is valid

Assets Not Loading Offline

Verify assets in IndexedDB/Cache API (DevTools → Application tab)
Check Service Worker fetch handler
Look for CORS issues
Verify blob URL creation
Check StorageManager.hasAsset(url) returns true

Storage Quota Exceeded

Clear old project downloads (deleteOfflineData)
Delete unused cached assets
Request persistent storage (requestPersistence)
Reduce asset quality (use mobile variant)
Check isCritical flag from useStorageQuota

Download Resume Not Working

Check IndexedDB downloadQueue table (DevTools → Application → IndexedDB)
Verify restoreQueue() called on load
Check for corrupted queue items
Clear queue and re-download

Video Seeking Issues Offline

Verify video is in cache (check DevTools → Application → Cache Storage)
Check video cached with proper Content-Type header
Ensure Range request handler working (check Network tab for 206 responses)
Test with smaller video file
Check if video is in IndexedDB (for large files > 5MB)

Progress Not Updating

Verify DownloadEventBus subscriptions
Check for event name mismatches (use OFFLINE_CONFIG.events.*)
Ensure handler is not unsubscribed prematurely
Check for errors in event handler (caught silently by bus)

useOfflineMode Not Working

Verify projectId is not null
Check enabled option (default: true)
Verify pages prop is provided (required for asset discovery)
Verify isOfflineCapable is true (SW + caches supported)
Check browser console for [useOfflineMode] logs

48 KiB Raw Blame History

Offline PWA Mode - E2E Documentation

Overview

Architecture

Service Worker (Serwist)

Configuration

Caching Strategies

Unified Cache Keys (cacheKeyWillBeUsed Plugin)

Cacheable Extensions

Range Request Support

Message API

Lifecycle

Cache Organization

Named Caches

Cache Invalidation

Dual Storage Strategy

Storage Decision

Why Dual Storage?

StorageManager

IndexedDB Schema

Database: TourBuilderOffline (v1)

Assets Table

Projects Table

Download Queue Table

OfflineDbManager

Download Manager

Features

Priority Calculation

Download Flow

Retry Configuration

Resume Capability

API

S3 Presigned URL Integration

Flow

Key Functions

Presigned URL Expiry

Download Event Bus

Events

Event Payloads

Usage

Convenience Methods

Hooks

useOfflineMode

useStorageQuota

useNetworkAware

Concurrency Adaptation

UI Components

OfflineToggle

OfflineStatusIndicator

StorageUsageDisplay

DownloadProgressPanel

Frontend Asset Discovery

Overview

Key Components

Asset Discovery Functions

AssetToCache Interface

Asset Sources

URL Fields Scanned

AssetCacheService

CachedAssetInfo Interface

Usage in useOfflineMode

Offline Types

Key Workflows

A. Project Download for Offline Use

B. Offline Asset Retrieval

C. Resume Interrupted Downloads

D. Check for Updates

Configuration Reference

Preload Configuration

Offline Configuration

Key Files Reference

Browser Requirements

Troubleshooting

Downloads Not Starting

Assets Not Loading Offline

Storage Quota Exceeded

Download Resume Not Working

Video Seeking Issues Offline

Progress Not Updating

useOfflineMode Not Working

48 KiB

Raw Blame History

Database: `TourBuilderOffline` (v1)