39948-vm/frontend/docs/ui-adaptivity-system.md

UI Adaptivity System

Comprehensive documentation for the Tour Builder Platform's responsive canvas scaling and UI adaptivity system.

Overview

The platform uses a Canvas Units system to ensure UI elements scale proportionally across all viewport sizes while maintaining a consistent design. This system allows content authored at a design resolution (e.g., 1920×1080) to display correctly on any screen size.

┌─────────────────────────────────────────────────────────────────────────┐
│                    UI Adaptivity Architecture                            │
│                                                                          │
│  ┌──────────────────────────────────────────────────────────────────┐  │
│  │                    Configuration Layer                            │  │
│  │                                                                   │  │
│  │   canvas.config.ts                                                │  │
│  │   ├── defaults: { width: 1920, height: 1080 }                    │  │
│  │   ├── scaling: { mode: 'fit', minScale: 0.1, maxScale: 4.0 }     │  │
│  │   └── cssVars: { scale: '--canvas-scale', unit: '--cu', ... }    │  │
│  └───────────────────────────┬──────────────────────────────────────┘  │
│                              │                                          │
│  ┌──────────────────────────────────────────────────────────────────┐  │
│  │                     Calculation Layer                             │  │
│  │                                                                   │  │
│  │   canvasScale.ts (utilities)                                      │  │
│  │   ├── calculateCanvasScale(viewport, design) → scale factor      │  │
│  │   ├── toCU(designPixels) → "calc(N * var(--cu, 1px))"            │  │
│  │   ├── normalizeToCanvasUnits(value) → canvas unit expression     │  │
│  │   └── getCanvasCssVars(scale) → CSS custom properties object     │  │
│  └───────────────────────────┬──────────────────────────────────────┘  │
│                              │                                          │
│  ┌──────────────────────────────────────────────────────────────────┐  │
│  │                      Hook/Context Layer                           │  │
│  │                                                                   │  │
│  │   useCanvasScale (hook)    │    CanvasScaleContext (context)     │  │
│  │   ├── scale               │    ├── Same properties               │  │
│  │   ├── cssVars             │    ├── Provider pattern              │  │
│  │   ├── letterboxStyles     │    └── Optional hook variant         │  │
│  │   └── showRotatePrompt    │                                      │  │
│  └───────────────────────────┬──────────────────────────────────────┘  │
│                              │                                          │
│  ┌──────────────────────────────────────────────────────────────────┐  │
│  │                     Application Layer                             │  │
│  │                                                                   │  │
│  │   RuntimePresentation     │    Constructor                        │  │
│  │   ├── cssVars on root     │    ├── cssVars on canvas             │  │
│  │   ├── letterboxStyles     │    ├── letterboxStyles               │  │
│  │   └── Elements use --cu   │    └── Elements use --cu             │  │
│  └──────────────────────────────────────────────────────────────────┘  │
└─────────────────────────────────────────────────────────────────────────┘

Core Concepts

Canvas Units (--cu)

The --cu CSS custom property is the foundation of the adaptivity system. It represents "1 design pixel" that scales with the viewport.

At design resolution (1920×1080 on 1920×1080 viewport):
  --cu = 1px

At 4K (1920×1080 on 3840×2160 viewport):
  --cu = 2px (elements render 2x larger)

At half-resolution (1920×1080 on 960×540 viewport):
  --cu = 0.5px (elements render at half size)

Usage:

/* Direct CSS usage */
.element {
  font-size: calc(24 * var(--cu, 1px));  /* 24px at design scale */
  padding: calc(16 * var(--cu, 1px));
  border-radius: calc(8 * var(--cu, 1px));
}

// JavaScript usage via toCU()
import { toCU } from '../lib/canvasScale';

const style = {
  fontSize: toCU(24),      // "calc(24 * var(--cu, 1px))"
  padding: toCU(16),       // "calc(16 * var(--cu, 1px))"
  borderRadius: toCU(8),   // "calc(8 * var(--cu, 1px))"
};

Scale Factor Calculation

The scale factor is calculated to fit the design canvas within the viewport while maintaining aspect ratio:

// canvasScale.ts
export function calculateCanvasScale(
  viewportWidth: number,
  viewportHeight: number,
  designWidth: number = 1920,
  designHeight: number = 1080,
): number {
  const scaleX = viewportWidth / designWidth;
  const scaleY = viewportHeight / designHeight;

  // Use min() to fit content within viewport (letterbox/pillarbox)
  const scale = Math.min(scaleX, scaleY);

  // Clamp to configured min/max (0.1 - 4.0)
  return Math.max(0.1, Math.min(4.0, scale));
}

Letterbox Mode

When the viewport aspect ratio doesn't match the design aspect ratio, the system creates black bars (letterbox for horizontal bars, pillarbox for vertical bars) to maintain content proportions.

// Generated letterbox styles
const letterboxStyles: CSSProperties = {
  width: designWidth * scale,      // Actual canvas width
  height: designHeight * scale,    // Actual canvas height
  position: 'absolute',
  left: '50%',
  top: '50%',
  transform: 'translate(-50%, -50%)',  // Center in viewport
};

Visual Example:

┌─────────────────────────────────────────┐
│░░░░░░░░░░░░░ Letterbox Bar ░░░░░░░░░░░░│  ← Black bar (wider viewport)
├─────────────────────────────────────────┤
│                                         │
│              Canvas Content             │  ← Design content (1920×1080)
│           (maintains 16:9 ratio)        │
│                                         │
├─────────────────────────────────────────┤
│░░░░░░░░░░░░░ Letterbox Bar ░░░░░░░░░░░░│  ← Black bar
└─────────────────────────────────────────┘

Global UI Controls

Fullscreen, global sound, and offline controls are positioned against the same visible canvas rectangle as page elements. Their coordinates are stored as xPercent/yPercent. Their dimensions are stored as canvas-width-relative percentages (buttonSizePercent, iconSizePercent, borderRadiusPercent), so they scale with the displayed canvas instead of remaining fixed CSS pixels.

Vertical clamping accounts for the canvas aspect ratio because button height is also derived from canvas width. This keeps controls fully inside the canvas for 16:9, 4:3, ultra-wide, and custom project ratios.

File Reference

Configuration

File: frontend/src/config/canvas.config.ts

export const CANVAS_CONFIG = {
  // Default design dimensions
  defaults: {
    width: 1920,
    height: 1080,
  },

  // Common presets for project settings
  presets: [
    { name: 'HD 16:9', width: 1920, height: 1080 },
    { name: '4K 16:9', width: 3840, height: 2160 },
    { name: 'HD 4:3', width: 1440, height: 1080 },
    { name: 'Ultra-wide 21:9', width: 2560, height: 1080 },
  ],

  // Scaling behavior
  scaling: {
    mode: 'fit' as const,  // Fit within viewport, may letterbox
    minScale: 0.1,
    maxScale: 4.0,
  },

  // Portrait orientation handling
  orientation: {
    showRotatePrompt: true,
    minAspectRatioForPrompt: 0.8,
  },

  // CSS custom property names
  cssVars: {
    scale: '--canvas-scale',
    unit: '--cu',
    designWidth: '--design-width',
    designHeight: '--design-height',
  },
};

Utilities

File: frontend/src/lib/canvasScale.ts

Function	Purpose
calculateCanvasScale(vw, vh, dw, dh)	Calculate scale factor for viewport
toCU(designPixels)	Convert design pixels to calc() expression
isLegacyUnit(value)	Check if value uses px/rem/vw/vh units
normalizeToCanvasUnits(value, property)	Convert legacy units to canvas units
getCanvasCssVars(scale, dw, dh)	Generate CSS custom properties object
vwToDesignPx(vw)	Convert viewport width to design pixels
vhToDesignPx(vh)	Convert viewport height to design pixels
remToDesignPx(rem)	Convert rem to design pixels (16px base)

Hook

File: frontend/src/hooks/useCanvasScale.ts

interface UseCanvasScaleOptions {
  designWidth?: number;   // From project.design_width
  designHeight?: number;  // From project.design_height
}

interface CanvasScaleResult {
  scale: number;                    // Current scale factor (1.0 = design size)
  designWidth: number;              // Design canvas width
  designHeight: number;             // Design canvas height
  canvasWidth: number;              // Calculated width at current scale
  canvasHeight: number;             // Calculated height at current scale
  isPortrait: boolean;              // Viewport is portrait orientation
  showRotatePrompt: boolean;        // Should show "rotate device" prompt
  cssVars: CSSProperties;           // CSS custom properties object
  letterboxStyles: CSSProperties;   // Styles for letterbox container
}

// Usage
const {
  scale,
  cssVars,
  letterboxStyles,
  isPortrait,
  showRotatePrompt,
} = useCanvasScale({
  designWidth: project.design_width,
  designHeight: project.design_height,
});

Context (Optional)

File: frontend/src/context/CanvasScaleContext.tsx

For deeply nested components that need access to canvas scale without prop drilling:

// Provider setup
<CanvasScaleProvider
  designWidth={project.design_width}
  designHeight={project.design_height}
>
  {children}
</CanvasScaleProvider>

// Consumer hook
const { scale, cssVars } = useCanvasScaleContext();

// Optional variant (returns null if no provider)
const scaleContext = useCanvasScaleContextOptional();

Element Styling Integration

elementStyles.ts

File: frontend/src/lib/elementStyles.ts

The element styles library automatically converts values to canvas units:

// Normalization functions
normalizePixelValue('24')      // → "calc(24 * var(--cu, 1px))"
normalizePixelValue('24px')    // → "calc(24 * var(--cu, 1px))"
normalizeViewportWidth('50vw') // → "calc(960 * var(--cu, 1px))" (50% of 1920)
normalizeViewportHeight('25vh')// → "calc(270 * var(--cu, 1px))" (25% of 1080)

// Build complete style object
const style = buildElementStyle({
  width: '200',
  height: '100',
  fontSize: '16',
  borderRadius: '8',
});
// Result: all values converted to calc() expressions with --cu

useElementWrapperStyle Hook

File: frontend/src/components/UiElements/shared/useElementWrapperStyle.ts

Provides consistent styling for UI elements across constructor and runtime:

const { className, style } = useElementWrapperStyle({
  element,
  isSelected: false,
  isEditMode: false,
});

// Returns:
// - className: Tailwind classes for appearance
// - style: CSSProperties with canvas unit values

Usage Patterns

RuntimePresentation

// RuntimePresentation.tsx
const { cssVars, letterboxStyles } = useCanvasScale({
  designWidth: project.design_width,
  designHeight: project.design_height,
});

return (
  <div className="relative w-screen h-screen overflow-hidden bg-black">
    {/* Inner canvas: maintains aspect ratio */}
    <div
      className="overflow-hidden"
      style={{
        ...cssVars,           // Sets --cu, --canvas-scale, etc.
        ...letterboxStyles,   // Centers and sizes canvas
      }}
    >
      {/* All child elements use --cu for sizing */}
      {elements.map(el => <RuntimeElement key={el.id} element={el} />)}
    </div>
  </div>
);

Constructor

// constructor.tsx
const { cssVars, letterboxStyles } = useCanvasScale({
  designWidth: project?.design_width,
  designHeight: project?.design_height,
});

return (
  <div
    className="overflow-hidden"
    style={{
      ...cssVars,
      ...letterboxStyles,
    }}
  >
    <CanvasBackground ... />
    <div className="absolute inset-0 z-10">
      {elements.map(el => <CanvasElement key={el.id} element={el} />)}
    </div>
  </div>
);

Individual Elements

// Element component using canvas units
import { toCU } from '../../lib/canvasScale';

const ButtonElement = ({ element }) => {
  return (
    <button
      style={{
        fontSize: toCU(element.fontSize || 16),
        padding: `${toCU(8)} ${toCU(16)}`,
        borderRadius: toCU(element.borderRadius || 4),
      }}
    >
      {element.label}
    </button>
  );
};

CSS Custom Properties

The system sets these CSS custom properties on the canvas container:

Property	Description	Example Value
--cu	Canvas unit (1 design pixel)	calc(1px * 0.75)
--canvas-scale	Current scale factor	0.75
--design-width	Design canvas width	1920
--design-height	Design canvas height	1080

CSS Usage:

.element {
  /* Use --cu for scalable dimensions */
  font-size: calc(18 * var(--cu, 1px));

  /* Use --canvas-scale for transforms */
  transform: scale(var(--canvas-scale, 1));

  /* Use design dimensions for calculations */
  width: calc(var(--design-width) * 0.5 * var(--cu, 1px));
}

Legacy Unit Migration

When encountering legacy units (px, vw, vh, rem), use the normalization utilities:

import { normalizeToCanvasUnits } from '../lib/canvasScale';

// Convert various legacy formats
normalizeToCanvasUnits('24px', 'fontSize');     // → "calc(24 * var(--cu, 1px))"
normalizeToCanvasUnits('50vw', 'width');        // → "calc(960 * var(--cu, 1px))"
normalizeToCanvasUnits('25vh', 'height');       // → "calc(270 * var(--cu, 1px))"
normalizeToCanvasUnits('1.5rem', 'fontSize');   // → "calc(24 * var(--cu, 1px))"
normalizeToCanvasUnits(100, 'width');           // → "calc(100 * var(--cu, 1px))"

Orientation Handling

The system detects portrait orientation and can prompt users to rotate their device:

const { isPortrait, showRotatePrompt } = useCanvasScale({
  designWidth: 1920,
  designHeight: 1080,
});

// showRotatePrompt is true when:
// 1. Device is in portrait mode (height > width)
// 2. orientation.showRotatePrompt is enabled in config
// 3. Aspect ratio < minAspectRatioForPrompt (0.8)

{showRotatePrompt && (
  <RotateDeviceOverlay />
)}

Integration Points

Component	Uses	Purpose
RuntimePresentation	useCanvasScale	Full-screen tour playback
constructor.tsx	useCanvasScale	Tour editing canvas
TransitionPreviewOverlay	letterboxStyles prop	Transition videos within canvas
CanvasBackground	Inherits from parent	Background media display
RuntimeElement	buildElementStyle	Element rendering
CanvasElement	useElementWrapperStyle	Element editing
CarouselElement	toCU()	Gallery styling
gallerySectionStyles	toCU()	Gallery section dimensions

Performance Considerations

1. Resize listener - Uses single resize event listener, debounced via React state
2. Memoization - Scale calculations wrapped in useMemo to prevent recalculation
3. CSS calc() - Browser handles scaling efficiently via CSS custom properties
4. No layout thrashing - Scale updates don't trigger element-by-element recalculation

Troubleshooting

Elements Not Scaling

1. Verify cssVars is applied to a parent container
2. Check that values use toCU() or normalization functions
3. Ensure --cu is not overridden by other styles

Letterbox Not Appearing

1. Parent container must have overflow: hidden and bg-black
2. letterboxStyles must be applied to inner container
3. Check that viewport dimensions are being detected (resize listener)

Scale Factor Too Small/Large

1. Check minScale and maxScale in canvas.config.ts
2. Verify project's design_width and design_height are set correctly
3. Test with different viewport sizes to confirm scaling behavior