40227-vm/docs/full-integration-refactor-plan.md

# Full Integration Refactor Plan

## Purpose

This document is the active cross-application backlog for integrating `frontend/`, `backend/`, and the PostgreSQL database.

It must track only unfinished integration work and hard implementation rules. Completed migration history belongs in the feature-specific docs under `backend/docs/`, `frontend/docs/`, and supporting audit docs.

When an item is completed, remove it from this plan and update the owning feature documentation instead of adding a completed checklist entry here.

## Current Application Shape

- `frontend/` is the only product frontend.
- `backend/` is the source of truth for auth, tenant ownership, roles, permissions, users, staff profiles, campuses, persisted workflows, content catalogs, email, files, and database access.
- `ref-frontend/` is a temporary generated reference. It is not runtime code and must be deleted after all needed endpoint-contract reference value is exhausted.

## Non-Negotiable Rules

- No frontend persisted workflow may use mock, sample, seed, or fallback records.
- No frontend runtime code may import backend seed files.
- Frontend runtime constants may contain only UI config, route/module metadata, query keys, timing values, style tokens, static UI labels, and intentionally static product copy.
- Backend seeds are for database seeding only.
- Frontend seed data is allowed only in tests or `frontend/src/test-seeds/`.
- Backend owns tenant scoping and permissions. Frontend route hiding is UX only.
- All frontend backend calls go through typed modules in `frontend/src/shared/api/`.
- New frontend workflows must follow `View -> Business Logic -> API/Data Access -> Backend`.
- View components must not import `frontend/src/shared/api/`.
- Business logic must not import view components.
- API modules must not import business or view code.
- All imports use the `@` alias.
- No `any`, unsafe casts, disabled TypeScript rules, disabled ESLint rules, compatibility bypasses, silent failures, or legacy re-export surfaces.
- Frontend auth must use backend-owned HttpOnly cookies only. Access and refresh tokens must never be stored in frontend browser storage.
- Secrets live only in backend `.env` or deployment environment variables. Frontend env values must be public browser-safe values only.
- New backend modules must include migration/model/service/route, tenant and role enforcement, docs, and focused verification.
- Every changed workflow must update the relevant docs before it is considered complete.

## Current Verification Baseline

Frontend current baseline:

- `npm run typecheck` passes.
- `npm run lint` passes.
- `npm run test` passes with 51 files and 198 tests.
- `npm run build` passes.
- `npm run test:e2e` passes with 4 backend-free Playwright smoke tests.
- `npm run test:e2e:content` exists for backend-seeded content catalog integration tests. It requires backend migrations, backend seeders, and a running backend server.

Backend current baseline:

- Runtime code is 100% TypeScript + native ESM; the JS->TS / CJS->ESM migration is complete.
- `npm run typecheck` (`tsc --noEmit`) passes.
- `npm run lint` (`eslint .`) passes with no broad ignores.
- `npm test` (`node --test` via `tsx`) passes: 2 files, 15 tests (error-handler + import-boundaries).
- `npm run verify` (typecheck + lint + test) is the combined gate and is green.
- `npm run build` (`tsc` + `tsc-alias -f` + email-template asset copy) produces a runnable `dist/`.
- Migrations/seeders run via Umzug (`npm run db:migrate`, `npm run db:seed`); a run against the configured local database is still pending (Workstream 1).

## Active Workstreams

The workstreams are numbered in dependency/execution order and grouped into five phases. Earlier phases unblock later ones; within a phase, items may run in parallel unless a dependency is stated. Renumbered from the previous flat backlog: the former Role Model Decision, Product Onboarding Contract, and Permission-Based Frontend Authorization workstreams are now merged into Workstream 3 (RBAC); the former standalone seed-verification step is merged into Workstream 4.

**Phase 1 — Foundation (everything else depends on a backend that boots, migrates, and serves routes).**

### 1. Backend Migration And Runtime Boot

Status: open.

Problem:

Backend product modules exist, but the configured local database migration/seed run has not been verified as a passing gate in this plan.

Required work:

1. Run `npm run db:migrate` against the configured local database.
2. Run `npm run db:seed` (current seed set; the final RBAC seed content is delivered in Workstream 4, which re-verifies this step).
3. Start backend with the documented env file.
4. Verify public content catalog routes, auth routes, and product module routes respond.
5. Record exact failing migration/seed/runtime errors if any.

Acceptance criteria:

- `npm run db:migrate` passes.
- `npm run db:seed` passes.
- Backend starts without generated default secrets.
- `GET /api/public/content-catalog/:contentType` works for the required seeded content catalog types.
- Any remaining backend runtime blocker is captured as a specific follow-up with file/error references.

**Phase 2 — Identity, Tenancy & Access Core (the security spine: tenant isolation, the role/permission model, the seeded fixtures, and the finalized protected API surface). Workstreams 5–7 finalize what is public, what exists, and what is permission-gated, so they belong with the access model rather than after it.**

### 2. Tenant Boundary Audit And Tests

Status: open and high priority. Co-designed with Workstream 3: the campus/organization scoping here and the `campusId`-on-`users` model in §3.1 are the same scoping layer.

Problem:

Generated backend code has partial tenant scoping. Multi-tenant correctness cannot rely on frontend filtering.

Required work:

1. Audit all tenant-owned generated and product routes for organization scoping.
2. ~~Resolve inconsistent `organizationsId` versus `organizationId` usage~~ — done: unified on `organizationId` for all models (renamed the `users.organizationsId` column via migration `20260609000000-rename-users-organizationsid-to-organizationid`, updated db/api scoping, the auth DTO, and the frontend `CurrentUser` type). Verify the tenant-scoping `where` now correctly targets each model's `organizationId`.
3. Ensure create/update/delete paths cannot accept another tenant's organization or campus.
4. Ensure list/count/autocomplete endpoints are tenant-scoped.
5. Add backend tests proving cross-tenant records are not visible or mutable.

Acceptance criteria:

- Tenant isolation tests cover users, campuses, staff, students, attendance, messages, documents, and product module tables.
- A non-global user cannot read or mutate another tenant's data.
- Campus-scoped users cannot mutate another campus unless their backend permission explicitly allows it.

### 3. RBAC: Role Hierarchy, Scoped Provisioning, And Guards

Status: open and high priority. Umbrella workstream for the platform's user/role model. **Merged here:** the former Role Model Decision (role-model choice), the former Product Onboarding Contract (provisioning, §3.4), and the former Permission-Based Frontend Authorization (frontend guards, §3.6). Consumes the existing permission engine in `middlewares/check-permissions.ts`.

Decisions recorded (owner-approved):

- Role model: **first-class persisted roles** in the `roles` table, each carrying a `scope`, with preset permission sets and code-enforced relational constraints. The `GENERATED_ROLE_TO_PRODUCT_ROLE` / `STAFF_TYPE_TO_PRODUCT_ROLE` mappings and the 5-value `productRole` are retired.
- Frontend: **permission-based guards plus an authentication guard**; forbidden direct-URL access redirects to the 404 page; menu items the user cannot access are hidden.

#### 3.0 Current state (analysis, verified)

- Roles today (`backend/src/db/seeders/20200430130760-user-roles.ts`, `backend/src/shared/constants/roles.ts`): 7 generated roles (`Super Administrator`, `Administrator`, `Platform Owner`, `Tenant Director`, `Campus Manager`, `Academic Coordinator`, `Finance Officer`) + `Public` + a `User` default-role name referenced in code. `globalAccess: true` is set only on `Super Administrator` and `Administrator`. These map down to 5 `productRole` values (`teacher | para | office | director | superintendent`).
- Authorization engine: `middlewares/check-permissions.ts` grants a request when (1) `currentUser.id === req.params.id || req.body?.id` (self-access), (2) a `custom_permissions` row matches, or (3) the effective role (assigned `app_role`, else cached `Public`) has the `${METHOD}_${ENTITY}` permission. CRUD routers wire `checkCrudPermissions(entity)`. This engine is sound and is kept.
- Permission catalog: CRUD × 27 entities + `READ_API_DOCS` + `CREATE_SEARCH`. **Product modules are not represented** (no permission exists for the FRAME, walkthrough, quizzes, director-dashboard, attendance-fill, ESA/vocational/community pages, classrooms, etc.).
- Multi-tenancy: `organizationId` on most models (unified, per Workstream 2). Campus link for a user is only indirect, via the `staff` row's `campusId`; `staff_type` is `teacher | admin | support`. There is **no `campusId` on `users`** and no first-class concept of director/owner/student/guardian.
- Seeders: **no organization row is seeded at all**, and the seeded campuses (`shared/constants/campuses.ts`, 6 rows) have **no `organizationId`**. `20200430130760` assigns roles by raw SQL: `super_admin@flatlogic.com`→SuperAdmin, `admin@flatlogic.com`→Administrator, `client@hello.com`→PlatformOwner, `john@doe.com`→TenantDirector. **This contradicts the product spec**, which states `admin@flatlogic.com` (`SEED_ADMIN_EMAIL`) is the super admin. No staff profiles are seeded.
- Frontend: `ModuleRouteGuard` + sidebar filter gate purely on the 5-value `productRole` (`shared/constants/appData.ts` `MODULES[].roles`, `shared/constants/moduleRoutes.ts`). `CurrentUser.permissions` exists but is unused. There is **no authentication guard** (an unauthenticated user reaching the shell is not redirected to `/login`), and `canUserRoleAccessModuleRoute` returns `true` for unknown paths (forbidden routes fall through to the catch-all rather than failing at the guard). Only `/login` and `*` (404) are public; there is no signup route.

#### 3.1 Target role hierarchy and scope

Eleven roles across five scopes (guest is the unauthenticated case, not a stored user):

1. `super_admin` — scope `system`. Unlimited across the whole platform.
2. `system_admin` — scope `system`. Unlimited platform-wide **except** creating/deleting `super_admin` or other `system_admin`.
3. `owner` — scope `organization`. Unlimited inside its own company.
4. `superintendent` — scope `organization`. Unlimited inside the company **except** creating/deleting `owner` or other `superintendent`, and **except** deleting the company profile.
5. `director` — scope `campus`. Unlimited inside its assigned campus; creates campus users and classrooms; grants extra per-user permissions.
6. `office_manager` — scope `campus`. Reads all campus pages **except** the director dashboard; takes quizzes; leaves read receipts; may fill the attendance page.
7. `teacher` — scope `campus`. Reads all campus pages except the director dashboard.
8. `support_staff` — scope `campus`. Reads all campus pages except the director dashboard. (Replaces the current `para` value.)
9. `student` — scope `external`. Reads only `/community-partnerships`, `/vocational-opportunities`, `/esa-funding`.
10. `guardian` — scope `external`. Same external pages as `student`.
11. `guest` — unauthenticated. Public routes only (`/login`, `/signup`, error page); any other URL redirects to 404.

Required work:

1. Add a `scope` column to the `roles` model/migration: enum `system | organization | campus | external | guest`. Keep `globalAccess` for the two system roles (it already short-circuits tenant filtering in the CRUD layer).
2. Replace the seeded role set with these 11 (plus keep `Public` as the unauthenticated fallback mapped to `guest`). Write a data migration mapping legacy roles → new roles (`Super Administrator`→`super_admin`, `Administrator`→`system_admin`, `Platform Owner`→`owner`, `Tenant Director`→`superintendent`, `Campus Manager`→`director`, `Academic Coordinator`→`teacher`, `Finance Officer`→`office_manager`; `para`/`support` HR type → `support_staff`).
3. Retire `GENERATED_ROLE_TO_PRODUCT_ROLE`, `STAFF_TYPE_TO_PRODUCT_ROLE`, and `PRODUCT_ROLE_VALUES`; the role name + scope is now the source of truth. Replace `productRole` in the `/api/auth/me` DTO with `role: { name, scope }` (keep a stable string the frontend can switch on).
4. Add a campus link for campus-scoped users. Recommended: a nullable `campusId` on `users` (authorization scope), independent of the HR `staff` row. `organizationId` remains the tenant key; `campusId` is required for `director`/`office_manager`/`teacher`/`support_staff` and optional for `student`/`guardian`; null for `system`/`organization` scopes. (Same scoping layer as Workstream 2.)

#### 3.2 Permission catalog expansion

The frontend can only gate on permissions the backend actually checks, so every guarded page/action needs a permission name.

1. Keep the CRUD × entity permissions; add `classrooms` (alias of `classes` if the same table, otherwise a new entity) to CRUD.
2. Add product-feature permissions for the module routes and special actions, named in the existing `${VERB}_${THING}` style — e.g. `READ_DIRECTOR_DASHBOARD`, `FILL_ATTENDANCE`, `TAKE_QUIZ`, `ACK_READ_RECEIPT`, `READ_COMMUNITY_PARTNERSHIPS`, `READ_VOCATIONAL_OPPORTUNITIES`, `READ_ESA_FUNDING`, plus one per remaining campus module page.
3. Define the **role → permission preset matrix** (the "predefined permissions per role" from the spec) and seed it into `rolesPermissionsPermissions`. Directors additionally hand out per-user `custom_permissions`.
4. Document the matrix in `backend/docs/auth-profile.md` (or a new `backend/docs/rbac.md`).

#### 3.3 Relational role constraints (the "except …" rules)

The constraints are about *who the target is*, not just *what verb on what entity*, so a flat `${METHOD}_${ENTITY}` permission cannot express them. Implement a policy layer enforced in the **service layer** (authoritative), with a middleware in front for early rejection:

1. On user create/update/delete, compare actor scope+role against the target's role: `system_admin` cannot create/delete `super_admin` or `system_admin`; `superintendent` cannot create/delete `owner` or `superintendent`; campus roles cannot escalate above their scope; a director may only create/manage users within its own `campusId`.
2. On organization delete, require `super_admin`/`system_admin`/`owner` (a `superintendent` is blocked).
3. Cross-scope tenancy: every actor may only act within its `organizationId` (and `campusId` for campus scope) unless `globalAccess`.
4. Fix the self-access bypass security hole: `currentUser.id === req.body?.id` (`middlewares/check-permissions.ts`, the self-access branch) lets any user pass a guard by putting their own id in the request body; restrict self-access to read/update of the caller's own profile only.

#### 3.4 Scoped provisioning / onboarding contract (former Product Onboarding Contract)

Auto-provisioning chain, all backend-owned, each step sending an invitation email with a login link (`services/email/list/invitation.ts` already exists). No-go rules carried over from the former onboarding workstream: do not implement frontend self-registration; do not build profile creation/editing UI ahead of the backend contract; do not treat the generated `signup`/`profile` endpoints as the product contract; no temporary compatibility paths.

1. `super_admin`/`system_admin` create an `owner` user → **auto-create the company (organization)** and link the owner; both start minimal (owner has only email+password; company has only the owner link).
2. `owner`/`superintendent` create superintendents, other org users, and campuses; assign a `director` to a campus.
3. `director` creates campus users (`teacher`, `support_staff`, `office_manager`, `student`, `guardian`) and classrooms, and grants extra `custom_permissions`.
4. Every created user receives a login-link email; on first login they land on their own editable profile.
5. Define which profile fields each role may self-edit vs. which require a higher role.

#### 3.5 Backend guard coverage audit

1. Confirm **every** route mounts authentication + `checkCrudPermissions`/`checkPermissions` (cross-reference Workstream 5 public-route audit so only intended routes are public).
2. Add the §3.3 relational policy to the user, organization, campus, staff, and classroom write paths.
3. Add the product-feature permission checks (§3.2) to the feature routes (FRAME, walkthrough, quizzes, attendance, communications, content-catalog, etc.).
4. Ensure list/count/autocomplete are tenant- and campus-scoped (ties to Workstream 2).

#### 3.6 Frontend guards (former Permission-Based Frontend Authorization)

Context: the backend already authorizes every request by **permission**, not role; the user DTO already carries `permissions: string[]`, but the frontend ignores it and gates on the 5-value `productRole`. The goal is to gate UI affordances (routes/menu items/buttons/request triggers) by **permission** — using the same `${METHOD}_${ENTITY}` names the backend checks — while the backend stays the sole source of truth (frontend gating is UX-only and must never be the only enforcement).

1. **AuthGuard.** Wrap the shell: unauthenticated → redirect to `/login`; only `/login`, `/signup`, and the error page are reachable without a session.
2. **Expose permissions in the auth contract.** Confirm `GET /api/auth/me` and the sign-in response include the resolved effective `permissions: string[]` (role permissions ∪ `custom_permissions`); merge `custom_permissions` if not already; document the field in `backend/docs/auth-profile.md`.
3. **Shared permission vocabulary.** Add a typed catalog on the frontend (`frontend/src/shared/auth/permissions.ts`) mirroring backend `${METHOD}_${ENTITY}` names; keep it as `shared/constants`-style UI config; do not import backend code.
4. **Permission selector/hook.** Implement `hasPermission/hasAnyPermission/hasAllPermissions` in `frontend/src/business/auth/` (pure functions over `CurrentUser.permissions`) plus a `usePermissions()` hook.
5. **Gate routes by permission.** Replace `productRole`-based `canUserRoleAccessModuleRoute`/`getAccessibleModules` with permission checks; each `MODULES[]` entry declares the permission(s) it requires. Keep a role concept only where genuinely role-specific (e.g. dashboards), not for resource access.
6. **Forbidden direct-URL access redirects to the 404 page** (change `canUserRoleAccessModuleRoute`'s unknown-path `return true` and the guard's redirect target); inaccessible sidebar items are hidden.
7. **Encode the role-specific page sets.** `office_manager` sees all campus pages except the director dashboard (plus quiz/read-receipt/attendance-fill affordances); `teacher`/`support_staff` see all campus pages except the director dashboard; `student`/`guardian` see only `/community-partnerships`, `/vocational-opportunities`, `/esa-funding`; `director`/`superintendent`/`owner` see their full scope; `guest` sees only public routes.
8. **Gate affordances + handle 403.** Hide/disable create/edit/delete triggers by the matching permission (e.g. hide "Add campus" without `CREATE_CAMPUSES`); surface backend `forbidden` responses through one handler (toast + no crash) so UI/permission drift degrades gracefully.
9. **Roles/permissions admin UI.** Let an admin/director create a role, attach/detach permissions, and assign `custom_permissions` to a user — backed by the existing `roles`/`permissions`/`users` endpoints.
10. **Docs.** Update `frontend/docs/frontend-architecture.md` and `backend/docs/auth-profile.md` to describe the permission-based frontend model and the `${METHOD}_${ENTITY}` contract.

#### 3.7 Tests

1. Backend: relational-constraint tests (`system_admin` cannot delete `super_admin`; `superintendent` cannot delete `owner`/`superintendent` or the company; director scoped to its campus), tenant/campus isolation, and the provisioning chain (owner-create auto-creates company).
2. Frontend: per-role route-guard tests (each role can/cannot open the right pages), sidebar-visibility tests, `hasPermission`/selector unit tests, unauthenticated→`/login`, and forbidden-URL→404.
3. Backend-seeded authenticated e2e using the Workstream 4 fixtures (feeds Workstream 8) proving UI affordances match backend enforcement for at least one CRUD entity.

Acceptance criteria:

- The 11 roles exist as first-class scoped roles; the legacy mapping and 5-value `productRole` are gone.
- Backend is the sole authority: every route enforces authentication + permission + (where relational) the §3.3 policy; removing a frontend check never grants real access (backend still returns 403).
- The "except …" constraints hold in tests for `system_admin`, `superintendent`, and `director`.
- Frontend route/menu/affordance visibility is permission-driven; unauthenticated users are redirected to `/login`; forbidden direct URLs land on 404; guest sees only public routes.
- An admin can create a role, assign permissions, and the change is reflected in both backend enforcement and frontend UI for affected users.
- Owner-creation auto-creates the company; each provisioning step emails a login link.
- Backend and frontend `typecheck`/`lint`/`test` pass; the Workstream 4 seed runs clean and backs an authenticated e2e.

### 4. RBAC Seed Data And Fixtures

Status: open — depends on Workstream 3 (run after the role/permission/model changes land). Implements the spec's seed requirement: a preset company with one campus and staff covering every role, plus one test user per role. Also re-verifies the `npm run db:seed` gate from Workstream 1 against the final seed content.

Required work:

1. **Reconcile the admin identity with the spec.** Make `admin@flatlogic.com` (`SEED_ADMIN_EMAIL`) the **`super_admin`** (the spec's superadmin), not `Administrator`. Add a distinct `system_admin` seed user. Repurpose or delete the leftover template users (`john@doe.com`, `client@hello.com`, `super_admin@flatlogic.com`) so the seed set is exactly the role fixtures below.
2. **Seed one organization (company)** and **one campus** linked to it (give the existing `campuses` seed rows an `organizationId`, or seed a dedicated test campus).
3. **Seed 10 users, one per stored role** — `super_admin`, `system_admin`, `owner`, `superintendent`, `director`, `office_manager`, `teacher`, `support_staff`, `student`, `guardian` — each with `app_roleId` set, `organizationId` set (except the two `system` users), and `campusId` set for the campus-scoped and external users as applicable. (`guest` = no user.) Passwords come from `SEED_ADMIN_PASSWORD`/`SEED_USER_PASSWORD` (no plaintext in the repo); test credentials live only in ignored local env or a dedicated test-seed config (per Workstream 8).
4. **Seed staff profiles** for the campus staff roles (`director`, `office_manager`, `teacher`, `support_staff`) so the campus has staff covering every staff role, each linked to the org, the campus, and its user.
5. Make the seed **idempotent** and reversible (clean `down`), and ordered after the role/permission seeder.
6. Re-verify the seeded data: one company, one campus under it, staff with all campus roles, and exactly one loginable user per role. Add a seed/integration check.

Acceptance criteria:

- `npm run db:seed` produces exactly one company, one campus under that company, staff covering every campus role, and one user per role with correct role/org/campus links.
- `admin@flatlogic.com` is the `super_admin` and can log in; each seeded role can log in and lands on the access surface defined in §3.6.
- No template leftover users remain; no plaintext credentials are committed.

### 5. Public Backend Route Audit

Status: open. Feeds Workstream 3 §3.5 (guard coverage): the guard audit can only assert "every non-public route is authenticated" once the public set is fixed.

Problem:

Public routes must be intentionally public. This includes public content catalog routes and any generated/template public integrations.

Required work:

1. List all routes without auth middleware.
2. Mark each as public-by-design or requiring auth.
3. Add auth where required.
4. Document intentionally public routes.

Acceptance criteria:

- Every unauthenticated backend route is documented.
- No tenant-owned data is exposed through accidental public routes.

### 6. API Surface Coverage And Dead-Endpoint Decision

Status: open (analysis complete; decision pending). Belongs with the access core: the permission catalog (§3.2) and guard audit (§3.5) must cover exactly the endpoints that survive this classification.

Problem:

The backend exposes the full Flatlogic-generated CRUD surface for all 39 models plus
template auth/file/search routes, but the product frontend only consumes a small set of
custom feature endpoints. The unused surface is dead code and attack surface; it must be
either pruned or intentionally wired.

Method (how this was established):

- Enumerated every backend route from `backend/src/routes/*` plus its mount prefix in `backend/src/index.ts`.
- Enumerated every frontend HTTP call. All frontend HTTP goes through `frontend/src/shared/api/*` over `httpClient` (`fetch`); there are no stray `fetch`/`axios`/`XMLHttpRequest` calls elsewhere, so the frontend call list is exhaustive.

Findings:

- **Frontend is fully wired:** every `shared/api` method targets a real, mounted backend endpoint. There are **0 orphan/broken frontend calls**.
- **Backend is only partially consumed:** of ~278 endpoints, the frontend uses **~35 (~13%)**; **~243 (~87%) have no frontend consumer.**
- The mismatch is one-directional: the frontend calls nothing extra; the backend carries a large unused layer.

Consumed endpoints (the only wired surface — 35):

- `auth` (4 of 16): `POST /api/auth/signin/local`, `GET /api/auth/me`, `POST /api/auth/refresh`, `POST /api/auth/signout`.
- `campuses` (1): `GET /api/public/campuses` (the authenticated `/api/campuses` CRUD is NOT used).
- `content-catalog` (6): `GET /api/public/content-catalog/:contentType`, `GET /api/content-catalog`, `GET /api/content-catalog/:contentType`, `POST /api/content-catalog`, `PUT /api/content-catalog/:contentType`, `DELETE /api/content-catalog/:contentType`.
- `campus_attendance` (4): `GET /configs`, `PUT /configs/:campusKey`, `GET /summaries`, `PUT /summaries/:campusKey/:date`.
- `communications` (4): `GET /parent-messages`, `POST /parent-messages`, `GET /events`, `POST /events`.
- `frame_entries` (3): `GET /`, `POST /`, `PUT /:id`.
- `personality_quiz_results` (3): `GET /me`, `PUT /me`, `GET /distribution`.
- `safety_quiz_results` (2): `GET /`, `POST /`.
- `staff_attendance` (2): `GET /records`, `GET /summary`.
- `user_progress` (3): `GET /`, `POST /`, `DELETE /by-item`.
- `walkthrough_checkins` (3): `GET /`, `POST /`, `DELETE /:id`.

Unused backend endpoints (no frontend consumer):

1. **Generic CRUD template — 25 route groups × 9 endpoints = 225, none called:**
   `academic_years`, `assessments`, `assessment_results`, `attendance_records`,
   `attendance_sessions`, `campuses` (the `/api/campuses` CRUD), `classes`,
   `class_enrollments`, `class_subjects`, `fee_plans`, `grades`, `guardians`,
   `invoices`, `message_recipients`, `messages`, `organizations`, `payments`,
   `permissions`, `roles`, `staff`, `students`, `subjects`, `timetable_periods`,
   `timetables`, `users`.
   Each exposes the identical shape: `POST /`, `POST /bulk-import`, `PUT /:id`,
   `DELETE /:id`, `POST /deleteByIds`, `GET /`, `GET /count`, `GET /autocomplete`,
   `GET /:id`.
2. **`auth` extras — 12, not called:** `POST /api/auth/signup`, `PUT /api/auth/profile`,
   `PUT /api/auth/password-reset`, `PUT /api/auth/password-update`,
   `POST /api/auth/send-password-reset-email`,
   `POST /api/auth/send-email-address-verification-email`,
   `PUT /api/auth/verify-email`, `GET /api/auth/email-configured`,
   `GET /api/auth/signin/google` (+ `/callback`), `GET /api/auth/signin/microsoft` (+ `/callback`).
3. **`file` — 2, not called:** `GET /api/file/download`, `POST /api/file/upload/:table/:field`
   (the frontend never uploads; `DocumentMutationDto` has no `file`).
4. **`search` — 1, not called:** `GET /api/search`.

**Decision (owner, recorded):** the generic CRUD layer is **WIRE — kept, not
pruned.** These ~24 groups are not dead code; they will be used and integrated
with the frontend later (likely modeled on `ref-frontend`). No generic CRUD group
is to be removed. The `auth`/`file`/`search` extras (items 2–4 below) remain
individually decision-gated.

Required work:

1. Generic CRUD groups: **WIRE** (decided above) — build the frontend that uses
   them (e.g. real `students`/`staff`/`guardians`/`invoices` management) and bring
   each group up to the target backend architecture. Do **not** prune them.
2. `auth` extras: keep `signup`/password-reset/verify-email only if onboarding/recovery is
   in scope (see Workstream 3 §3.4 provisioning); otherwise prune. OAuth callbacks tie to Workstream 15.
3. `file`: keep only if document/avatar upload is on the roadmap (ties to Workstream 7);
   otherwise prune.
4. `search`: prune unless a search UI is planned.
5. After any prune, re-run backend `typecheck`/`lint`, and regenerate
   `database-schema.md` only if models change.

Cross-references: Workstream 7 (file upload), 5 (public route audit), 15 (OAuth), 3 (RBAC / permission-based authorization).

Acceptance criteria:

- Every backend endpoint is classified as **used**, **prune**, or **wire (planned)**, with no "unclassified" remainder.
- Pruned routes are removed cleanly (route + service + db/api + permission wiring) with `typecheck`/`lint` green and no dangling imports.
- The frontend remains fully wired (0 orphan calls) after changes.
- This document reflects the final surface.

Performance hardening applied (owner chose **wire**, not prune, for the generic CRUD layer — it will back near-term management UIs modeled on `ref-frontend`):

- `findBy` in all 24 generic-CRUD `db/api/*.ts` now loads its associations via a single `Promise.all` instead of sequential awaited getters (detail-endpoint latency drops from sum to max of the association queries). `users.findBy` was done earlier.
- List pagination now has shared defaults via `@/shared/constants/pagination` (`resolvePagination`, default page size 10 to match the `ref-frontend` grids, capped at 100). Applied to every generic-CRUD `findAll` and to the feature lists (`user_progress`, `safety_quiz_results`, `walkthrough_checkins`, `frame_entries`, `content_catalog`, `communications` parent-messages/events, `campus_attendance` configs), which were reverted from `findAll` back to `findAndCountAll` so `count` is the true total for the pager. `staff_attendance /records` and `campus_attendance /summaries` keep their pre-existing per-endpoint limits.
- Fixed a latent CRUD tenant-scoping bug: routes and `db/api` create/update read `currentUser.organization?.id` (singular), but `findBy` only ever populates `organizations` (plural) + the `organizationId` scalar, so that read was always `undefined` (non-global creates silently set no organization). All 22 `db/api` + 24 route reads now use `currentUser.organizationId`; the dead 3-field fallback term was dropped from the 4 feature services; and the non-existent `organization?: { id }` field was removed from the `CurrentUser` type so the mistake cannot recur.
- The redundant existence-check `findBy` in the 22 CRUD `update` services was removed; they now rely on `DbApi.update` returning `null` (avoids loading every association just to validate existence). `remove` already had no pre-check.
- The per-request `UsersDBApi.findBy` (passport JWT strategy → `req.currentUser`, read by every guarded route) was collapsed from `findOne` + 4 parallel association getters + a `getPermissions()` into a **single** eager-loaded query (`findOne` + `include` of `app_role`+`permissions`, `staff_user`, `custom_permissions`, `organizations`). Its returned `app_role` now carries `permissions`, and `middlewares/check-permissions.ts` was reordered to read that eager-loaded `permissions` array before falling back to `getPermissions()` — removing the extra per-request permissions query. Same returned shape/fields as before (`AuthenticatedUser`); ~6–7 queries per request → 1. The cached `Public` role still works (its record carries `permissions`).
- `AuthService.currentUserProfile` (the `GET /me`, signin and refresh responses) now uses a dedicated `UsersDBApi.findProfileById` — a single eager-loaded query (`findByPk` + scoped `include`/`attributes`) returning only the columns and relations the profile DTO reads — instead of the heavy generic `findBy` (1 `findOne` + parallel association getters + `getPermissions`) plus a separate `getCampus`. Required idiomatic `NonAttribute` association declarations on the `Users`/`Roles`/`Staff` models so the include is type-safe without casts. The per-request passport `findBy` that populates `req.currentUser` is unchanged (it is the auth gate read by every guard); `/me` still performs that auth fetch plus this one lean profile query.

### 7. File Upload And Download Permissions

Status: open. Permission-gated surface; aligns with Workstream 3 §3.2/§3.5 and Workstream 6 (`file` extras).

Problem:

Backend file and document routes exist. Product file workflows need explicit permission verification before new upload/download UI is added.

Required work:

1. Audit upload permissions.
2. Audit download permissions.
3. Verify tenant and document ownership checks.
4. Add typed frontend upload client only for approved workflows.

Acceptance criteria:

- Unauthorized users cannot access another tenant's files.
- Upload/download behavior is documented and tested before new UI is added.

**Phase 3 — Verification & Quality (locks in correctness once the access model and fixtures exist).**

### 8. Backend-Seeded Authenticated E2E

Status: depends on Workstream 4 fixtures and Workstream 3 guards.

Problem:

Current Playwright coverage includes backend-free smoke tests and backend-seeded content catalog tests. Authenticated persisted workflows need backend-seeded users, roles, campuses, staff profiles, and known credentials.

Required prerequisites:

1. Product onboarding/profile contract (Workstream 3 §3.4).
2. Backend-seeded auth fixtures (Workstream 4).
3. Tenant-scoped campus/staff fixtures (Workstream 4).
4. Stable test credentials stored only in ignored local env or dedicated test seed config.

Required workflows after prerequisites:

1. Login to dashboard.
2. Director creates or edits a FRAME entry and sees it after reload.
3. Staff completes QBS quiz and director/superintendent sees compliance.
4. Office enters campus attendance and superintendent sees aggregate.
5. Director submits walkthrough and sees summary update.
6. Staff marks a sign learned and progress persists after reload.

Acceptance criteria:

- Authenticated e2e tests use backend seeds, not frontend mock data.
- Tests do not require production secrets.
- Tests are documented and repeatable.

### 9. API Documentation Hardening

Status: open. Run after the route surface is finalized (Workstreams 3, 5, 6) so the docs describe the real endpoints.

Problem:

Markdown docs exist for migrated modules, but Swagger/OpenAPI coverage for product-specific and cookie-session endpoints is incomplete.

Required work:

1. Document `/api/auth/signin/local`.
2. Document `/api/auth/refresh`.
3. Document `/api/auth/signout`.
4. Document `/api/auth/me`.
5. Document product module endpoints that are not covered by generated Swagger output.
6. Document response and error shapes per endpoint.

Acceptance criteria:

- API docs match actual route payloads and response shapes.
- Cookie auth behavior is explicit.
- Frontend API contract tests remain aligned with docs.

### 10. Accessibility Test Coverage

Status: open.

Required work:

1. Add axe/Playwright accessibility checks for login.
2. Add axe/Playwright checks for dashboard.
3. Add checks for sidebar navigation.
4. Add checks for modal dialogs.
5. Add checks for forms with validation.
6. Add checks for tables/reports.

Acceptance criteria:

- Accessibility tests run in a documented command.
- Critical violations block completion of the refactor.

**Phase 4 — Product Contracts (each gated by a customer/provider decision; independent of one another and parallelizable once the access core is in place).**

### 11. Policy And Safety Acknowledgment Persistence

Status: open pending product contract.

Problem:

Policy content is document-backed, but policy/protocol acknowledgments are not yet persisted.

Required decisions:

1. Which policies/protocols require acknowledgment.
2. Which roles must acknowledge.
3. Whether acknowledgments are per document version.
4. Who can report acknowledgment status.

Required work after decision:

1. Add acknowledgment backend model/migration/service/route.
2. Enforce tenant and role scope.
3. Add frontend typed API and business workflow.
4. Add report views only if required.

Acceptance criteria:

- Acknowledgments survive reload.
- Acknowledgment status is tenant-scoped.
- Unauthorized roles cannot view individual acknowledgment records.

### 12. Attendance Source Contracts

Status: partially open.

Current state:

- Campus attendance daily aggregate summaries are implemented for the current UI.
- Staff attendance snapshot/reporting is read-only.
- Student/class attendance source-of-truth workflow is not defined.

Open decisions:

1. Whether campus attendance aggregates are manually entered, imported, or derived from student attendance records.
2. Which external or internal source owns staff attendance writes/imports.
3. Whether student-level attendance UI is required.

Required work after decision:

1. Add write/import endpoints only after source contract exists.
2. Keep derived summaries server-side if summaries are derived.
3. Add backend tests for source-of-truth calculations.
4. Add frontend workflows only after backend contracts exist.

Acceptance criteria:

- Attendance source of truth is documented.
- UI values can be traced to backend records or server-side derivation.
- No frontend-only attendance source remains in persisted workflows.

### 13. Generated Audio Provider Contract

Status: open pending provider decision.

Problem:

Classroom timer uses built-in Web Audio sounds. AI-generated audio UI is intentionally not exposed because no backend audio provider contract exists.

No-go rules:

- Do not add generated audio UI.
- Do not call external audio providers from frontend runtime.
- Do not add frontend API keys or provider secrets.

Required work after decision:

1. Define backend provider contract.
2. Keep provider secrets in backend env.
3. Add backend service with native provider errors where required.
4. Add typed frontend API and explicit loading/error states.

Acceptance criteria:

- Generated audio is backend-mediated.
- No provider secret reaches the browser.
- Provider failures remain visible.

**Phase 5 — Operations, Modernization & Cleanup (low coupling; can run any time after Phase 2, scheduled last to avoid mixing with the access-model changes).**

### 14. Refresh Token Maintenance

Status: open.

Problem:

Cookie auth and refresh rotation exist, but scheduled cleanup for expired/revoked refresh-token rows remains unresolved.

Required work:

1. Define refresh-token retention period.
2. Add cleanup job or operational command.
3. Ensure cleanup is observable.
4. Document operational usage.

Acceptance criteria:

- Expired and revoked refresh-token rows are cleaned after the approved retention window.
- Cleanup failures are visible and not silent.
- Auth behavior remains unchanged for valid sessions.

### 15. OAuth Provider Strategy Modernization

Status: open. Deferred until after the backend TypeScript/ESM migration (now complete), to keep auth-flow changes separate from the language/module migration.

Problem:

Social login uses `passport-google-oauth2` (0.2.0, last published 2022) and `passport-microsoft` (2.1.0). The Google strategy is low-maintenance and should be modernized, ideally consolidating both providers on a single maintained OAuth/OIDC library.

Required work:

1. Choose target: minimal swap to `passport-google-oauth20`, or consolidate both providers on `openid-client`.
2. Replace the Google (and optionally Microsoft) passport strategy in `backend/src/auth/auth.ts`.
3. Keep the existing cookie-based session and JWT issuance unchanged.
4. Verify callback URLs, scopes, and the social-signup user flow end to end.
5. Update auth docs (`backend/docs/auth-profile.md`, `cookie-auth.md`).

Acceptance criteria:

- Google and Microsoft sign-in work end to end with the chosen library.
- No deprecated/unmaintained OAuth strategy remains in dependencies.
- Cookie/JWT auth behavior is unchanged for existing sessions.
- This change is isolated from the language/module migration (separate PR/task).

### 16. `ref-frontend/` Removal

Status: open.

Required work:

1. Confirm no active docs require `ref-frontend/` for endpoint contract reference.
2. Confirm no scripts import or run `ref-frontend/`.
3. Delete `ref-frontend/`.
4. Update root docs and any setup docs.

Acceptance criteria:

- Only `frontend/` and `backend/` remain as active application code.
- No docs describe `ref-frontend/` as needed for normal development.

## Phased Execution Order

Run in workstream-number order unless the user explicitly reprioritizes. The phases are the dependency gates; within a phase, items may proceed in parallel.

1. **Phase 1 — Foundation:** Workstream 1 (backend migrates, boots, serves routes).
2. **Phase 2 — Identity, Tenancy & Access Core:** Workstream 2 (tenant isolation) → Workstream 3 (RBAC: roles, permissions, constraints, provisioning, guards) → Workstream 4 (RBAC seed/fixtures), with Workstream 5 (public route audit), Workstream 6 (API surface classify/wire), and Workstream 7 (file permissions) finalizing the protected surface alongside Workstream 3.
3. **Phase 3 — Verification & Quality:** Workstream 8 (authenticated seeded e2e) → Workstream 9 (API docs) → Workstream 10 (accessibility).
4. **Phase 4 — Product Contracts (customer-gated, parallel):** Workstreams 11 (acknowledgments), 12 (attendance source), 13 (generated audio).
5. **Phase 5 — Operations, Modernization & Cleanup:** Workstreams 14 (refresh-token cleanup), 15 (OAuth modernization), 16 (`ref-frontend/` removal).

## Definition Of Done

The integration refactor is complete only when all of the following are true:

- Backend migrations and seeders pass on the configured local database.
- Backend lint passes without broad ignores.
- Backend tenant isolation tests prove cross-tenant data is not visible or mutable.
- The 11-role scoped hierarchy (Workstream 3) is enforced backend-side, including the relational "except …" constraints for `system_admin`, `superintendent`, and `director`; the legacy generated-role mapping and 5-value `productRole` are removed.
- Backend role/permission tests cover product staff, director, and superintendent paths.
- Frontend gating is permission-driven with an authentication guard; unauthenticated users redirect to `/login`, forbidden direct URLs land on 404, and guest sees only public routes.
- The RBAC seed (Workstream 4) provisions one company, one campus with staff covering every campus role, and one loginable user per role, with `admin@flatlogic.com` as the super admin.
- Every unauthenticated backend route is documented as public-by-design.
- Product onboarding contract is implemented per Workstream 3 §3.4 or explicitly excluded from release scope.
- Every visible frontend workflow is backed by a typed backend API contract.
- No frontend runtime workflow depends on mock, sample, seed, or fallback records for persisted behavior.
- Frontend `typecheck`, `lint`, `test`, `build`, `test:e2e`, and documented seeded e2e suites pass in their required environments.
- Backend auth/API docs match actual route behavior.
- Required accessibility checks pass.
- `ref-frontend/` is deleted or a dated, explicit exception explains why it is still needed.