576 lines
16 KiB
Markdown
576 lines
16 KiB
Markdown
# DB Seeders Module
|
||
|
||
## Overview
|
||
|
||
The DB Seeders module provides initial data population for the database using
|
||
the typed **Umzug v3** runner in `backend/src/db/umzug.ts`. Seeders create
|
||
essential system data (users, roles, permissions) and optional sample data for
|
||
development and testing.
|
||
|
||
**Location:** `backend/src/db/seeders/`
|
||
|
||
**Files:** 3 TypeScript ESM seeder entries. There are no JavaScript seeder
|
||
bridges after the backend ESM migration.
|
||
|
||
---
|
||
|
||
## Architecture
|
||
|
||
```
|
||
backend/src/db/seeders/
|
||
├── 20200430130759-admin-user.ts # Initial admin users
|
||
├── 20200430130760-user-roles.ts # RBAC roles + permissions
|
||
└── 20231127130745-sample-data.ts # Sample data, opt-in
|
||
```
|
||
|
||
**Execution Order:** Seeders run in timestamp order (oldest first).
|
||
|
||
---
|
||
|
||
## Configuration
|
||
|
||
### NPM Scripts
|
||
```bash
|
||
# Run pending seeders
|
||
npm run db:seed
|
||
|
||
# Undo all seeders
|
||
npm run db:seed:undo
|
||
|
||
# Full database reset (includes seeding)
|
||
npm run db:reset
|
||
```
|
||
|
||
### Server Startup
|
||
Seeders run automatically via `npm start`:
|
||
```json
|
||
{
|
||
"scripts": {
|
||
"start": "npm run db:migrate && npm run db:seed && npm run watch"
|
||
}
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
## Seeder Files
|
||
|
||
### 1. Admin User Seeder (`20200430130759-admin-user.ts`)
|
||
|
||
**Purpose:** Create initial system users.
|
||
|
||
The implementation uses reusable contracts from
|
||
`backend/src/types/db-seeders.ts`. Umzug executes the TypeScript source in
|
||
development and the compiled JavaScript file from `dist/` in production builds.
|
||
|
||
**Users Created:**
|
||
|
||
| User | Email | Role Assignment |
|
||
|------|-------|-----------------|
|
||
| Admin | `config.admin_email` | Administrator |
|
||
| John | john@doe.com | Account Manager |
|
||
| Client | client@hello.com | Platform Owner |
|
||
|
||
**Key Features:**
|
||
- Uses bcrypt for password hashing with configured salt rounds
|
||
- Hardcoded UUIDs for consistent user IDs
|
||
- Reads credentials from `config.ts` (environment variables)
|
||
- Idempotent on repeated `db:seed`: existing seed user IDs are selected first,
|
||
and only missing rows are inserted. This prevents duplicate-key failures when
|
||
`SequelizeData` lacks the seeder entry but users already exist.
|
||
|
||
```typescript
|
||
const existingRows =
|
||
await queryInterface.sequelize.query<AdminUserSeedExistingIdRow>(
|
||
'SELECT "id" FROM "users" WHERE "id" IN (:ids)',
|
||
{
|
||
replacements: { ids: seedUserIds },
|
||
type: QueryTypes.SELECT,
|
||
},
|
||
);
|
||
|
||
const existingIds = new Set(existingRows.map((row) => row.id));
|
||
const rowsToInsert = createAdminUserRows().filter(
|
||
(row) => !existingIds.has(row.id),
|
||
);
|
||
|
||
if (rowsToInsert.length > 0) {
|
||
await queryInterface.bulkInsert('users', rowsToInsert);
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
### 2. User Roles Seeder (`20200430130760-user-roles.ts`)
|
||
|
||
**Purpose:** Create the complete RBAC (Role-Based Access Control) system.
|
||
|
||
The implementation uses reusable contracts from
|
||
`backend/src/types/db-seeders.ts`. Umzug stores stable seeder execution names
|
||
in `SequelizeData` so repeated startup seeding does not rerun completed seed
|
||
data.
|
||
|
||
**Data Created:**
|
||
|
||
| Data Type | Count | Description |
|
||
|-----------|-------|-------------|
|
||
| Roles | 7 | User role definitions |
|
||
| Permissions | 54 | CRUD permissions for 13 entities + special |
|
||
| Role-Permission Links | 200+ | M:N relationships |
|
||
| Join Table | 1 | `rolesPermissionsPermissions` table |
|
||
|
||
**Key Features:**
|
||
- Uses stable named role and permission definitions, then reuses existing DB IDs
|
||
when the same role/permission name is already present.
|
||
- Inserts only missing roles, permissions, and role-permission links. This keeps
|
||
repeated `db:seed` safe.
|
||
- Keeps the existing startup behavior that assigns system users to seeded roles
|
||
by email after RBAC data exists.
|
||
|
||
#### Roles
|
||
```javascript
|
||
const roles = [
|
||
'Administrator', // Full system access
|
||
'PlatformOwner', // Full project/content access
|
||
'AccountManager', // User and project management
|
||
'TourDesigner', // Content creation and editing
|
||
'ContentReviewer', // Read + limited update access
|
||
'AnalyticsViewer', // Read-only access
|
||
'Public', // Public/unauthenticated access
|
||
];
|
||
```
|
||
|
||
#### Permission Generation Pattern
|
||
```javascript
|
||
// Generates CREATE, READ, UPDATE, DELETE permissions per entity
|
||
function createPermissions(name) {
|
||
return [
|
||
{ name: `CREATE_${name.toUpperCase()}` },
|
||
{ name: `READ_${name.toUpperCase()}` },
|
||
{ name: `UPDATE_${name.toUpperCase()}` },
|
||
{ name: `DELETE_${name.toUpperCase()}` },
|
||
];
|
||
}
|
||
|
||
const entities = [
|
||
'users', 'roles', 'permissions', 'projects', 'project_memberships',
|
||
'assets', 'asset_variants', 'presigned_url_requests', 'tour_pages',
|
||
'project_audio_tracks', 'publish_events', 'pwa_caches', 'access_logs'
|
||
];
|
||
|
||
// Creates 52 permissions (13 entities × 4 CRUD operations)
|
||
await queryInterface.bulkInsert('permissions', entities.flatMap(createPermissions));
|
||
|
||
// Plus special permissions
|
||
await queryInterface.bulkInsert('permissions', [
|
||
{ name: 'READ_API_DOCS' },
|
||
{ name: 'CREATE_SEARCH' },
|
||
]);
|
||
```
|
||
|
||
#### ID Map Pattern
|
||
```javascript
|
||
// Consistent UUID generation using key-based map
|
||
const idMap = new Map();
|
||
|
||
function getId(key) {
|
||
if (idMap.has(key)) {
|
||
return idMap.get(key);
|
||
}
|
||
const id = uuid();
|
||
idMap.set(key, id);
|
||
return id;
|
||
}
|
||
|
||
// Usage - same key always returns same UUID within seeder run
|
||
getId('Administrator') // Returns consistent UUID
|
||
getId('CREATE_USERS') // Returns consistent UUID
|
||
```
|
||
|
||
#### Permission Matrix
|
||
|
||
| Role | Users | Projects | Assets | Tour Pages | Access Logs |
|
||
|------|-------|----------|--------|------------|-------------|
|
||
| **Administrator** | CRUD | CRUD | CRUD | CRUD | CRUD |
|
||
| **PlatformOwner** | CRUD | CRUD | CRUD | CRUD | CRUD |
|
||
| **AccountManager** | RU | CRU | CRU | CRU | R |
|
||
| **TourDesigner** | R | RU | CRU | CRU | R |
|
||
| **ContentReviewer** | R | RU | RU | RU | R |
|
||
| **AnalyticsViewer** | R | R | R | R | R |
|
||
| **Public** | - | - | - | - | - |
|
||
|
||
**Legend:** C=Create, R=Read, U=Update, D=Delete
|
||
|
||
#### Join Table Creation
|
||
```javascript
|
||
// Creates M:N relationship table directly in seeder
|
||
await queryInterface.sequelize.query(`
|
||
CREATE TABLE IF NOT EXISTS "rolesPermissionsPermissions" (
|
||
"createdAt" timestamp with time zone not null,
|
||
"updatedAt" timestamp with time zone not null,
|
||
"roles_permissionsId" uuid not null,
|
||
"permissionId" uuid not null,
|
||
primary key ("roles_permissionsId", "permissionId"),
|
||
constraint "rolesPermissionsPermissions_roles_permissions_fk"
|
||
foreign key ("roles_permissionsId") references "roles"("id")
|
||
on delete cascade on update cascade,
|
||
constraint "rolesPermissionsPermissions_permission_fk"
|
||
foreign key ("permissionId") references "permissions"("id")
|
||
on delete cascade on update cascade
|
||
);
|
||
`);
|
||
```
|
||
|
||
---
|
||
|
||
### 3. Sample Data Seeder (`20231127130745-sample-data.ts`)
|
||
|
||
**Purpose:** Create sample data for development and testing.
|
||
|
||
The implementation lives in `20231127130745-sample-data.ts`. Demo-only
|
||
Sequelize operations use reusable `SampleDataModel` and
|
||
`SampleDataAssociationRecord` contracts from
|
||
`backend/src/types/db-seeders.ts`; the production service model overloads stay
|
||
separate. Umzug records the seeder under its legacy `.js` name for storage
|
||
compatibility only.
|
||
|
||
**Opt-In Activation:**
|
||
```bash
|
||
# Enable sample data seeding
|
||
export ENABLE_SAMPLE_DATA=true
|
||
npm run db:seed
|
||
```
|
||
|
||
**Check in Code:**
|
||
```typescript
|
||
const sampleDataSeeder: SequelizeSeeder = {
|
||
async up() {
|
||
if (process.env.ENABLE_SAMPLE_DATA !== 'true') return;
|
||
// ... seed data
|
||
},
|
||
};
|
||
```
|
||
|
||
**Data Created:**
|
||
|
||
| Entity | Records | Description |
|
||
|--------|---------|-------------|
|
||
| Projects | 3 | Sample tour projects |
|
||
| Project Memberships | 3 | User-project associations |
|
||
| Assets | 3 | Images, videos, audio |
|
||
| Asset Variants | 3 | Thumbnail/preview variants |
|
||
| Presigned URL Requests | 3 | Upload/download requests |
|
||
| Tour Pages | 3 | Sample tour pages |
|
||
| Project Audio Tracks | 3 | Background audio |
|
||
| Publish Events | 3 | Deployment history |
|
||
| PWA Caches | 3 | Offline cache configs |
|
||
| Access Logs | 3 | Visitor tracking |
|
||
|
||
#### Sample Projects
|
||
```javascript
|
||
const ProjectsData = [
|
||
{
|
||
name: 'Cardiff Arena Tour',
|
||
slug: 'cardiff-arena',
|
||
description: 'Interactive arena tour for visitors and event planners.',
|
||
logo_url: 'https://cdn.platform.com/cardiff/logo.png',
|
||
favicon_url: 'https://cdn.platform.com/cardiff/favicon.ico',
|
||
og_image_url: 'https://cdn.platform.com/cardiff/og.jpg',
|
||
},
|
||
{
|
||
name: 'Riverside Park Walkthrough',
|
||
slug: 'riverside-park',
|
||
description: 'Offline-ready guided walkthrough for the city park.',
|
||
// ...
|
||
},
|
||
{
|
||
name: 'Mall Central Experience',
|
||
slug: 'mall-central',
|
||
description: 'Retail complex presentation with navigation and galleries.',
|
||
// ...
|
||
},
|
||
];
|
||
```
|
||
|
||
#### Association Helper Pattern
|
||
```javascript
|
||
// Associates records after bulk creation using Sequelize model methods
|
||
async function associateAssetWithProject() {
|
||
const relatedProject = await Projects.findOne({
|
||
offset: Math.floor(Math.random() * (await Projects.count())),
|
||
});
|
||
const asset = await Assets.findOne({
|
||
order: [['id', 'ASC']],
|
||
offset: 0,
|
||
});
|
||
if (asset?.setProject) {
|
||
await asset.setProject(relatedProject);
|
||
}
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
## Seeder Patterns
|
||
|
||
### 1. bulkInsert Pattern
|
||
**Purpose:** Insert multiple records efficiently.
|
||
|
||
```javascript
|
||
await queryInterface.bulkInsert('tableName', [
|
||
{
|
||
id: uuid(),
|
||
field1: 'value1',
|
||
createdAt: new Date(),
|
||
updatedAt: new Date(),
|
||
},
|
||
// ... more records
|
||
]);
|
||
```
|
||
|
||
### 2. bulkDelete Pattern
|
||
**Purpose:** Remove seeded data during rollback.
|
||
|
||
```javascript
|
||
async down(queryInterface, Sequelize) {
|
||
await queryInterface.bulkDelete('users', {
|
||
id: { [Sequelize.Op.in]: ids }
|
||
});
|
||
}
|
||
```
|
||
|
||
### 3. Conditional Execution Pattern
|
||
**Purpose:** Enable/disable seeders based on environment.
|
||
|
||
Seeder-only environment gates are intentionally allowed to read `process.env`
|
||
directly. Runtime app configuration must go through `backend/src/config.ts`,
|
||
but seeders and scripts are bootstrap/data-loading entrypoints and are listed
|
||
as exceptions in `AGENTS.md`.
|
||
|
||
The resilience/config hardening work does not require seeder changes because it
|
||
does not introduce database schema, RBAC permissions, seed users, roles, default
|
||
records, or sample-data entities.
|
||
|
||
```javascript
|
||
up: async () => {
|
||
if (process.env.ENABLE_SAMPLE_DATA !== 'true') {
|
||
return; // Skip seeding
|
||
}
|
||
// ... proceed with seeding
|
||
}
|
||
```
|
||
|
||
### 4. ID Consistency Pattern
|
||
**Purpose:** Use deterministic IDs for reliable down() migrations.
|
||
|
||
```javascript
|
||
// Hardcoded IDs for rollback capability
|
||
const ids = [
|
||
'193bf4b5-9f07-4bd5-9a43-e7e41f3e96af',
|
||
'af5a87be-8f9c-4630-902a-37a60b7005ba',
|
||
];
|
||
|
||
// OR: Map-based consistent generation
|
||
const idMap = new Map();
|
||
function getId(key) {
|
||
if (!idMap.has(key)) {
|
||
idMap.set(key, uuid());
|
||
}
|
||
return idMap.get(key);
|
||
}
|
||
```
|
||
|
||
### 5. Model-Based Association Pattern
|
||
**Purpose:** Create relationships using Sequelize models after bulk insert.
|
||
|
||
```javascript
|
||
// Use model methods for associations (more readable)
|
||
const project = await Projects.findOne({ where: { slug: 'test' } });
|
||
const asset = await Assets.findOne({ order: [['id', 'ASC']] });
|
||
await asset.setProject(project);
|
||
```
|
||
|
||
### 6. Raw SQL Pattern
|
||
**Purpose:** Create structures not managed by Sequelize models.
|
||
|
||
```javascript
|
||
// Create join table directly via SQL
|
||
await queryInterface.sequelize.query(`
|
||
CREATE TABLE IF NOT EXISTS "rolesPermissionsPermissions" (
|
||
...
|
||
);
|
||
`);
|
||
|
||
// Create indexes
|
||
await queryInterface.sequelize.query(
|
||
'CREATE INDEX IF NOT EXISTS "index_name" ON "tableName" ("columnName");'
|
||
);
|
||
```
|
||
|
||
---
|
||
|
||
## Execution Flow
|
||
|
||
```
|
||
npm run db:seed
|
||
│
|
||
▼
|
||
┌────────────────────────────────────┐
|
||
│ 20200430130759-admin-user.ts │
|
||
│ ├─ Typed ESM source │
|
||
│ ├─ Hash passwords │
|
||
│ └─ Insert missing seed users │
|
||
└────────────────────────────────────┘
|
||
│
|
||
▼
|
||
┌────────────────────────────────────┐
|
||
│ 20200430130760-user-roles.ts │
|
||
│ ├─ Typed ESM source │
|
||
│ ├─ Insert missing roles │
|
||
│ ├─ Insert missing permissions │
|
||
│ ├─ Create join table │
|
||
│ ├─ Insert missing RBAC links │
|
||
│ └─ Update user app_roleId │
|
||
└────────────────────────────────────┘
|
||
│
|
||
▼
|
||
┌────────────────────────────────────┐
|
||
│ 20231127130745-sample-data.ts │
|
||
│ ├─ Check ENABLE_SAMPLE_DATA │
|
||
│ ├─ Skip if not enabled │
|
||
│ ├─ Insert sample records │
|
||
│ └─ Create associations │
|
||
└────────────────────────────────────┘
|
||
```
|
||
|
||
---
|
||
|
||
## Best Practices
|
||
|
||
### 1. Use Consistent IDs
|
||
```javascript
|
||
// Good - allows rollback
|
||
const ids = ['uuid-1', 'uuid-2'];
|
||
await queryInterface.bulkInsert('table', records.map((r, i) => ({ id: ids[i], ...r })));
|
||
|
||
// Down migration can target specific IDs
|
||
await queryInterface.bulkDelete('table', { id: { [Op.in]: ids } });
|
||
```
|
||
|
||
### 2. Always Include Timestamps
|
||
```javascript
|
||
{
|
||
field: 'value',
|
||
createdAt: new Date(),
|
||
updatedAt: new Date(),
|
||
}
|
||
```
|
||
|
||
### 3. Handle Errors
|
||
```javascript
|
||
try {
|
||
await queryInterface.bulkInsert('users', [...]);
|
||
} catch (error) {
|
||
console.error('Error during bulkInsert:', error);
|
||
throw error;
|
||
}
|
||
```
|
||
|
||
### 4. Environment-Aware Seeding
|
||
```javascript
|
||
// Production - only essential data
|
||
// Development - include sample data
|
||
if (process.env.ENABLE_SAMPLE_DATA !== 'true') {
|
||
return;
|
||
}
|
||
```
|
||
|
||
### 5. Idempotent Where Possible
|
||
```javascript
|
||
// Use IF NOT EXISTS for table/index creation
|
||
await queryInterface.sequelize.query(`
|
||
CREATE TABLE IF NOT EXISTS "tableName" (...)
|
||
`);
|
||
|
||
await queryInterface.sequelize.query(`
|
||
CREATE INDEX IF NOT EXISTS "indexName" ON "tableName" (...)
|
||
`);
|
||
```
|
||
|
||
---
|
||
|
||
## Data Dependencies
|
||
|
||
```
|
||
admin-user.ts
|
||
│
|
||
│ Creates missing users with stable IDs
|
||
▼
|
||
user-roles.ts
|
||
│
|
||
│ Creates/reuses roles and permissions, then references user emails to assign roles
|
||
│ UPDATE users SET app_roleId = '...' WHERE email = '...'
|
||
▼
|
||
sample-data.ts
|
||
│
|
||
│ Uses Sequelize models to find existing users/projects
|
||
│ Creates associations via model methods
|
||
▼
|
||
```
|
||
|
||
---
|
||
|
||
## Running Seeders
|
||
|
||
### Development Setup
|
||
```bash
|
||
cd backend
|
||
npm run db:seed
|
||
```
|
||
|
||
### With Sample Data
|
||
```bash
|
||
export ENABLE_SAMPLE_DATA=true
|
||
npm run db:seed
|
||
```
|
||
|
||
### Fresh Database
|
||
```bash
|
||
npm run db:reset # drop, create, migrate, seed
|
||
```
|
||
|
||
### Undo Seeders
|
||
```bash
|
||
npm run db:seed:undo # Runs all down() methods in reverse order
|
||
```
|
||
|
||
---
|
||
|
||
## Seeder Inventory
|
||
|
||
| # | Timestamp | Name | Records | Required |
|
||
|---|-----------|------|---------|----------|
|
||
| 1 | 20200430130759 | admin-user | 3 users | Yes |
|
||
| 2 | 20200430130760 | user-roles | 7 roles, 54 permissions, 200+ links | Yes |
|
||
| 3 | 20231127130745 | sample-data | 30+ sample records | No (opt-in) |
|
||
|
||
---
|
||
|
||
## Environment Variables
|
||
|
||
| Variable | Purpose | Default |
|
||
|----------|---------|---------|
|
||
| `ENABLE_SAMPLE_DATA` | Enable sample data seeder | `false` |
|
||
| `ADMIN_EMAIL` | Admin user email | (from config) |
|
||
| `ADMIN_PASS` | Admin user password | (from config) |
|
||
| `USER_PASS` | Default user password | (from config) |
|
||
|
||
---
|
||
|
||
## Related Documentation
|
||
|
||
- [DB Migrations](./db-migrations.md) - Schema evolution
|
||
- [DB Models](./db-models.md) - Sequelize model definitions
|
||
- [RBAC System](../../../documentation/rbac-system.md) - Role-based access control details
|
||
- [Authentication](./auth.md) - User authentication
|