510 lines
19 KiB
Markdown
510 lines
19 KiB
Markdown
# Search System
|
||
|
||
Complete documentation for the Tour Builder Platform's global full-text search system with permission-based filtering.
|
||
|
||
## Overview
|
||
|
||
The platform implements a global search service that performs full-text search across multiple database tables simultaneously. Results are filtered based on the current user's permissions, ensuring users only see data they have access to.
|
||
|
||
```
|
||
┌─────────────────────────────────────────────────────────────────────────────┐
|
||
│ Search System Architecture │
|
||
│ │
|
||
│ ┌────────────────────────────────────────────────────────────────────────┐ │
|
||
│ │ Search Request │ │
|
||
│ │ │ │
|
||
│ │ POST /api/search │ │
|
||
│ │ { searchQuery: "vacation" } │ │
|
||
│ │ │ │ │
|
||
│ │ ▼ │ │
|
||
│ │ ┌─────────────────────────────────────────────────────────────────┐ │ │
|
||
│ │ │ Permission Check (checkCrudPermissions) │ │ │
|
||
│ │ │ │ │ │
|
||
│ │ │ READ_SEARCH permission required │ │ │
|
||
│ │ └──────────────────────────┬──────────────────────────────────────┘ │ │
|
||
│ │ ▼ │ │
|
||
│ │ ┌─────────────────────────────────────────────────────────────────┐ │ │
|
||
│ │ │ SearchService.search() │ │ │
|
||
│ │ │ │ │ │
|
||
│ │ │ For each table: │ │ │
|
||
│ │ │ 1. Check READ_<TABLE> permission │ │ │
|
||
│ │ │ 2. If allowed, search text + numeric columns │ │ │
|
||
│ │ │ 3. Return up to 50 matching records │ │ │
|
||
│ │ └──────────────────────────┬──────────────────────────────────────┘ │ │
|
||
│ │ ▼ │ │
|
||
│ │ ┌─────────────────────────────────────────────────────────────────┐ │ │
|
||
│ │ │ Search Results │ │ │
|
||
│ │ │ │ │ │
|
||
│ │ │ [{ id, tableName, matchAttribute, ...record }] │ │ │
|
||
│ │ └─────────────────────────────────────────────────────────────────┘ │ │
|
||
│ └────────────────────────────────────────────────────────────────────────┘ │
|
||
└─────────────────────────────────────────────────────────────────────────────┘
|
||
```
|
||
|
||
## API Reference
|
||
|
||
### Endpoint
|
||
|
||
**URL:** `POST /api/search`
|
||
|
||
**Authentication:** Required (JWT)
|
||
|
||
**Permission:** `READ_SEARCH`
|
||
|
||
**Source:** `backend/src/routes/search.ts`
|
||
|
||
### Request Format
|
||
|
||
```json
|
||
POST /api/search
|
||
Content-Type: application/json
|
||
Authorization: Bearer <jwt-token>
|
||
|
||
{
|
||
"searchQuery": "vacation tour"
|
||
}
|
||
```
|
||
|
||
### Response Format
|
||
|
||
```json
|
||
[
|
||
{
|
||
"id": "uuid-1",
|
||
"name": "Vacation Paradise Tour",
|
||
"slug": "vacation-paradise",
|
||
"tableName": "projects",
|
||
"matchAttribute": ["name", "slug"]
|
||
},
|
||
{
|
||
"id": "uuid-2",
|
||
"name": "Beach Vacation Page",
|
||
"slug": "beach-vacation",
|
||
"tableName": "tour_pages",
|
||
"matchAttribute": ["name"]
|
||
}
|
||
]
|
||
```
|
||
|
||
### Response Fields
|
||
|
||
| Field | Type | Description |
|
||
|-------|------|-------------|
|
||
| `id` | UUID | Record's unique identifier |
|
||
| `tableName` | string | Source table name |
|
||
| `matchAttribute` | string[] | Array of field names that matched the query |
|
||
| `...record` | object | All searchable fields from the record |
|
||
|
||
### Error Responses
|
||
|
||
| Status | Error | Description |
|
||
|--------|-------|-------------|
|
||
| 400 | "Please enter a search query" | Missing searchQuery in request body |
|
||
| 401 | Unauthorized | Missing or invalid JWT token |
|
||
| 403 | Forbidden | User lacks READ_SEARCH permission |
|
||
| 500 | "Internal Server Error" | Database or server error |
|
||
|
||
## Search Service
|
||
|
||
**Source:** `backend/src/services/search.ts`
|
||
|
||
### Core Logic
|
||
|
||
```typescript
|
||
export default class SearchService {
|
||
static async search(searchQuery: string, currentUser: CurrentUser | undefined): Promise<SearchResult> {
|
||
// 1. Validate search query
|
||
if (!searchQuery) {
|
||
throw new ValidationError('iam.errors.searchQueryRequired');
|
||
}
|
||
|
||
// 2. Get user's permissions
|
||
const permissionSet = await getUserPermissions(currentUser);
|
||
|
||
// 3. Search each table in parallel
|
||
const searchTasks = searchTables.map(async ({ tableName, textColumns }) => {
|
||
// Skip if user lacks READ permission for this table
|
||
if (!hasPermission(permissionSet, `READ_${tableName.toUpperCase()}`)) {
|
||
return [];
|
||
}
|
||
|
||
// Perform case-insensitive search
|
||
const model = db[tableName];
|
||
if (!isSearchModel(model)) {
|
||
return [];
|
||
}
|
||
|
||
const foundRecords = await model.findAll({
|
||
where: { [Op.or]: [...searchConditions] },
|
||
limit: 50,
|
||
});
|
||
|
||
return foundRecords.map(record => ({
|
||
...record.get(),
|
||
matchAttribute,
|
||
tableName,
|
||
}));
|
||
});
|
||
|
||
return (await Promise.all(searchTasks)).flat();
|
||
}
|
||
}
|
||
```
|
||
|
||
### Search Configuration
|
||
|
||
```javascript
|
||
const SEARCH_LIMIT_PER_TABLE = 50; // Max results per table
|
||
```
|
||
|
||
## Searchable Tables
|
||
|
||
### Text Columns (String Search)
|
||
|
||
| Table | Searchable Columns |
|
||
|-------|-------------------|
|
||
| `users` | firstName, lastName, phoneNumber, email |
|
||
| `projects` | name, slug, description, logo_url, favicon_url, og_image_url |
|
||
| `assets` | name, cdn_url, storage_key, mime_type, checksum |
|
||
| `asset_variants` | cdn_url |
|
||
| `presigned_url_requests` | requested_key, mime_type, status |
|
||
| `tour_pages` | source_key, name, slug, background_image_url, background_video_url, background_audio_url, ui_schema_json |
|
||
| `project_audio_tracks` | source_key, name, slug, url |
|
||
| `publish_events` | error_message |
|
||
| `pwa_caches` | cache_version, manifest_json, asset_list_json |
|
||
| `access_logs` | path, ip_address, user_agent |
|
||
|
||
### Numeric Columns (Cast to String)
|
||
|
||
| Table | Searchable Columns |
|
||
|-------|-------------------|
|
||
| `assets` | size_mb, width_px, height_px, duration_sec |
|
||
| `asset_variants` | width_px, height_px, size_mb |
|
||
| `presigned_url_requests` | requested_size_mb |
|
||
| `tour_pages` | sort_order |
|
||
| `project_audio_tracks` | volume, sort_order |
|
||
| `publish_events` | pages_copied, transitions_copied, audios_copied |
|
||
|
||
## Permission-Based Filtering
|
||
|
||
### How It Works
|
||
|
||
The search system implements two levels of permission checking:
|
||
|
||
1. **Route-Level:** `READ_SEARCH` permission required to access the endpoint
|
||
2. **Table-Level:** `READ_<TABLE>` permission required for each table
|
||
|
||
```
|
||
┌─────────────────────────────────────────────────────────────────────────────┐
|
||
│ Permission-Based Search Filtering │
|
||
│ │
|
||
│ User with permissions: [READ_SEARCH, READ_PROJECTS, READ_TOUR_PAGES] │
|
||
│ │
|
||
│ Search Query: "beach" │
|
||
│ │
|
||
│ ┌──────────────────────────────────────────────────────────────────────┐ │
|
||
│ │ Table │ Permission Required │ Searched? │ Results │ │
|
||
│ ├──────────────────────────────────────────────────────────────────────┤ │
|
||
│ │ users │ READ_USERS │ ✗ Skipped │ - │ │
|
||
│ │ projects │ READ_PROJECTS │ ✓ Yes │ 3 matches │ │
|
||
│ │ assets │ READ_ASSETS │ ✗ Skipped │ - │ │
|
||
│ │ tour_pages │ READ_TOUR_PAGES │ ✓ Yes │ 5 matches │ │
|
||
│ │ project_audio_tracks│ READ_PROJECT_AUDIO_TRACKS│ ✗ Skipped │ - │ │
|
||
│ │ ... │ ... │ ... │ ... │ │
|
||
│ └──────────────────────────────────────────────────────────────────────┘ │
|
||
│ │
|
||
│ Total Results: 8 (only from projects + tour_pages) │
|
||
└─────────────────────────────────────────────────────────────────────────────┘
|
||
```
|
||
|
||
### Permission Resolution
|
||
|
||
**Source:** `backend/src/services/search.ts`
|
||
|
||
```javascript
|
||
async function getUserPermissions(currentUser) {
|
||
if (!currentUser) {
|
||
throw new ValidationError('auth.unauthorized');
|
||
}
|
||
|
||
const permissions = new Set(
|
||
(currentUser.custom_permissions || [])
|
||
.map((cp) => cp.name)
|
||
.filter(Boolean),
|
||
);
|
||
|
||
if (!currentUser.app_role) {
|
||
throw new ValidationError('auth.forbidden');
|
||
}
|
||
|
||
// Add role-based permissions
|
||
const rolePermissions = await currentUser.app_role.getPermissions();
|
||
for (const permission of rolePermissions) {
|
||
if (permission?.name) {
|
||
permissions.add(permission.name);
|
||
}
|
||
}
|
||
|
||
return permissions;
|
||
}
|
||
```
|
||
|
||
### Permission Sources
|
||
|
||
| Source | Description |
|
||
|--------|-------------|
|
||
| `currentUser.custom_permissions` | User-specific permission overrides |
|
||
| `currentUser.app_role.getPermissions()` | Permissions inherited from assigned role |
|
||
|
||
## Search Algorithm
|
||
|
||
### Case-Insensitive Matching
|
||
|
||
The search uses PostgreSQL's `ILIKE` operator for case-insensitive partial matching:
|
||
|
||
```javascript
|
||
// Text columns: direct ILIKE
|
||
{
|
||
[attribute]: {
|
||
[Op.iLike]: `%${searchQuery}%`,
|
||
},
|
||
}
|
||
|
||
// Numeric columns: cast to varchar then ILIKE
|
||
Sequelize.where(
|
||
Sequelize.cast(Sequelize.col(`${tableName}.${attribute}`), 'varchar'),
|
||
{ [Op.iLike]: `%${searchQuery}%` },
|
||
)
|
||
```
|
||
|
||
### Match Detection
|
||
|
||
After finding records, the service identifies which attributes matched:
|
||
|
||
```javascript
|
||
const matchAttribute = [];
|
||
|
||
// Check text columns
|
||
for (const attribute of attributesToSearch) {
|
||
if (record[attribute]?.toLowerCase()?.includes(normalizedSearchQuery)) {
|
||
matchAttribute.push(attribute);
|
||
}
|
||
}
|
||
|
||
// Check numeric columns (cast to string)
|
||
for (const attribute of attributesIntToSearch) {
|
||
const castedValue = String(record[attribute]);
|
||
if (castedValue && castedValue.toLowerCase().includes(normalizedSearchQuery)) {
|
||
matchAttribute.push(attribute);
|
||
}
|
||
}
|
||
```
|
||
|
||
## Middleware Chain
|
||
|
||
### Route Configuration
|
||
|
||
```typescript
|
||
// backend/src/routes/search.ts
|
||
import { checkCrudPermissions } from '../middlewares/check-permissions.ts';
|
||
router.use(checkCrudPermissions('search'));
|
||
|
||
router.post('/', async (req, res) => {
|
||
const foundMatches = await SearchService.search(searchQuery, req.currentUser);
|
||
res.json(foundMatches);
|
||
});
|
||
```
|
||
|
||
### Middleware Flow
|
||
|
||
```
|
||
Request → JWT Auth → checkCrudPermissions('search') → SearchService.search()
|
||
│
|
||
▼
|
||
Checks READ_SEARCH permission
|
||
using currentUser.app_role
|
||
```
|
||
|
||
## Role Requirements
|
||
|
||
### Minimum Permissions for Search
|
||
|
||
| Permission | Required For |
|
||
|------------|--------------|
|
||
| `READ_SEARCH` | Access to search endpoint |
|
||
| `READ_<TABLE>` | See results from specific table |
|
||
|
||
### Example Role Configuration
|
||
|
||
```javascript
|
||
// Role: "ContentEditor" with search access to projects and pages
|
||
{
|
||
name: "ContentEditor",
|
||
permissions: [
|
||
"READ_SEARCH",
|
||
"READ_PROJECTS",
|
||
"READ_TOUR_PAGES",
|
||
"READ_PAGE_ELEMENTS",
|
||
"READ_ASSETS"
|
||
]
|
||
}
|
||
```
|
||
|
||
## Performance Characteristics
|
||
|
||
### Parallel Execution
|
||
|
||
All table searches execute in parallel using `Promise.all`:
|
||
|
||
```javascript
|
||
const searchTasks = Object.keys(tableColumns).map(async (tableName) => {
|
||
// Each table search runs concurrently
|
||
});
|
||
const resultsByTable = await Promise.all(searchTasks);
|
||
return resultsByTable.flat();
|
||
```
|
||
|
||
### Limits
|
||
|
||
| Limit | Value | Description |
|
||
|-------|-------|-------------|
|
||
| Results per table | 50 | Maximum records returned from each table |
|
||
| Total tables | 10 | Number of searchable tables |
|
||
| Max total results | 500 | Theoretical maximum (50 × 10 tables) |
|
||
|
||
## File Structure
|
||
|
||
```
|
||
backend/src/
|
||
├── routes/
|
||
│ └── search.ts # Search route with permission check
|
||
├── services/
|
||
│ └── search.ts # SearchService with table definitions
|
||
└── middlewares/
|
||
└── check-permissions.ts # Permission checking middleware
|
||
|
||
frontend/src/
|
||
├── components/
|
||
│ ├── Search.tsx # Global search input in navbar
|
||
│ └── SearchResults.tsx # Search results display component
|
||
├── pages/
|
||
│ └── search.tsx # Search results page
|
||
└── layouts/
|
||
└── Authenticated.tsx # Includes Search component in navbar
|
||
```
|
||
|
||
## Frontend Integration
|
||
|
||
### Global Search Input
|
||
|
||
**Source:** `frontend/src/components/Search.tsx`
|
||
|
||
The navbar includes a global search input that:
|
||
- Validates minimum 2 characters
|
||
- Redirects to `/search?query=<searchQuery>`
|
||
|
||
```tsx
|
||
<Formik
|
||
onSubmit={(values) => {
|
||
router.push(`/search?query=${values.search}`);
|
||
}}
|
||
>
|
||
<Field name='search' placeholder='Search' validate={validateSearch} />
|
||
</Formik>
|
||
```
|
||
|
||
### Search Results Page
|
||
|
||
**Source:** `frontend/src/pages/search.tsx`
|
||
|
||
**URL:** `/search?query=<searchQuery>`
|
||
|
||
**Permission:** `CREATE_SEARCH` (via LayoutAuthenticated wrapper)
|
||
|
||
> **Note:** The page uses `CREATE_SEARCH` permission while the API requires `READ_SEARCH`.
|
||
|
||
The search results page:
|
||
- Extracts query from URL parameters
|
||
- Calls `POST /search` with searchQuery
|
||
- Groups results by tableName
|
||
- Displays results using SearchResults component
|
||
|
||
### Search Results Component
|
||
|
||
**Source:** `frontend/src/components/SearchResults.tsx`
|
||
|
||
Displays search results grouped by table:
|
||
- Shows table name as section header (humanized)
|
||
- Renders results in a table with all searchable columns
|
||
- Clickable rows navigate to `/<tableName>/<tableName>-view/?id=<id>`
|
||
|
||
```tsx
|
||
// Results grouped by table
|
||
{Object.keys(searchResults).map((tableName) => (
|
||
<CardBox>
|
||
<table>
|
||
{/* Headers from result keys */}
|
||
{/* Clickable rows linking to entity view */}
|
||
</table>
|
||
</CardBox>
|
||
))}
|
||
```
|
||
|
||
## Known Considerations
|
||
|
||
1. **No Pagination:** The search returns all results up to the per-table limit. For large result sets, consider implementing pagination.
|
||
|
||
2. **JSON Column Search:** Searching JSON columns (ui_schema_json, manifest_json, asset_list_json) searches the raw JSON text, not parsed values.
|
||
|
||
3. **Permission Inconsistency:** The search results page uses `CREATE_SEARCH` permission wrapper, but the API endpoint checks `READ_SEARCH`. Users need both permissions to use frontend search.
|
||
|
||
4. **Numeric Search:** Numbers are cast to strings for matching. Searching "10" will match "100", "210", etc.
|
||
|
||
5. **Permission Caching:** User permissions are fetched fresh for each search request. No caching is implemented at the service level.
|
||
|
||
6. **No Role Error:** If user has no app_role, the service throws `ValidationError('auth.forbidden')` - there is no fallback to a public role.
|
||
|
||
7. **Search Query Required:** Empty or missing search queries return a 400 error, not empty results.
|
||
|
||
8. **Minimum Query Length:** Frontend validates minimum 2 characters, but API accepts any non-empty string.
|
||
|
||
9. **Excluded Tables:** The following tables are intentionally not included in global search:
|
||
- `permissions` - System configuration, not user content
|
||
- `roles` - System configuration, not user content
|
||
- `project_memberships` - Access control data, not searchable content
|
||
- `element_type_defaults` - System configuration for default element settings
|
||
- `project_element_defaults` - Project-specific element settings (managed separately)
|
||
|
||
## Usage Examples
|
||
|
||
### cURL Example
|
||
|
||
```bash
|
||
curl -X POST http://localhost:3000/api/search \
|
||
-H "Content-Type: application/json" \
|
||
-H "Authorization: Bearer <jwt-token>" \
|
||
-d '{"searchQuery": "vacation"}'
|
||
```
|
||
|
||
### JavaScript Example
|
||
|
||
```javascript
|
||
const response = await fetch('/api/search', {
|
||
method: 'POST',
|
||
headers: {
|
||
'Content-Type': 'application/json',
|
||
'Authorization': `Bearer ${token}`,
|
||
},
|
||
body: JSON.stringify({ searchQuery: 'vacation' }),
|
||
});
|
||
|
||
const results = await response.json();
|
||
|
||
// Group results by table
|
||
const byTable = results.reduce((acc, result) => {
|
||
if (!acc[result.tableName]) acc[result.tableName] = [];
|
||
acc[result.tableName].push(result);
|
||
return acc;
|
||
}, {});
|
||
```
|