38686-vm/CLAUDE.md

# FoxFitt LabourPay v5

## Coding Style
- Always add clear section header comments using the format: # === SECTION NAME ===
- Add plain English comments explaining what complex logic does
- The project owner is not a programmer — comments should be understandable by a non-technical person
- When creating or editing code, maintain the existing comment structure

## Project Overview
Django payroll management system for FoxFitt Construction, a civil works contractor specializing in solar farm foundation installations. Manages field worker attendance, payroll processing, employee loans, and business expenses for solar farm projects.

This is v5 — a fresh export from Flatlogic/AppWizzy, rebuilt from the v2 codebase with simplified models and cleaner structure.

## Tech Stack
- Django 5.2.7, Python 3.13, MySQL (production on Flatlogic Cloud Run) / SQLite (local dev)
- Bootstrap 5.3.3 (CDN), Font Awesome 6.5.1 (CDN), Google Fonts (Inter + Poppins)
- WeasyPrint for PDF generation (payroll report, payslips, receipts) — migrated from xhtml2pdf; browser-grade HTML/CSS rendering with flexbox, grid, @font-face, shadows, and proper CSS cascade
- Gmail SMTP for automated document delivery
- Hosted on Flatlogic/AppWizzy platform (Apache on Debian, e2-micro VM)

## Project Structure
```
config/          — Django project settings, URLs, WSGI/ASGI
core/            — Single main app: ALL business logic, models, views, forms, templates
  context_processors.py — Injects deployment_timestamp (cache-busting), Flatlogic branding vars
  forms.py       — AttendanceLogForm, PayrollAdjustmentForm, ExpenseReceiptForm + formset
  models.py      — All 10 database models
  utils.py       — render_to_pdf() helper (lazy WeasyPrint import + Windows GTK3 DLL registration)
  views.py       — All view functions (~52 functions, ~3,800 lines) — dashboard, attendance, payroll, reports, worker/team/project CRUD
  forms.py       — All form classes + validators (WorkerForm, TeamForm, ProjectForm, AttendanceLogForm, PayrollAdjustmentForm, ExpenseReceiptForm, WorkerCertificate/WarningFormSet, 5MB file validator)
  admin.py       — Django admin registrations for all core models + WorkerCertificate/Warning inlines on Worker
  templatetags/  — format_tags.py (money filter for ZAR formatting)
  management/commands/ — setup_groups, setup_test_data, import_production_data
  templates/
    base.html    — App shell (topbar + mobile menu + bottom tab bar)
    core/        — Page templates: index, attendance_log, work_history, payroll_dashboard,
                   report, create_receipt, payslip, login, _report_config_modal (partial)
    core/workers/ — 4 templates: list, detail, edit, batch_report
    core/teams/   — 4 templates: list, detail, edit, batch_report
    core/projects/— 4 templates: list, detail, edit, batch_report
    core/pdf/    — 4 PDF templates: report_pdf, payslip_pdf, receipt_pdf, workers_report_pdf
    core/email/  — 2 HTML email templates
    admin/       — base_site.html override (adds admin CSS tweaks, e.g. taller M2M pickers)
ai/              — Flatlogic AI proxy client (not used in app logic)
static/css/      — custom.css (CSS variables, component styles, tooltip overrides)
staticfiles/     — Collected static assets (Bootstrap, admin) — NOT in git (build artifact, regenerated by collectstatic)
```

## Key Models
- **UserProfile** — extends Django User (OneToOne); minimal, no extra fields in v5
- **Project** — work sites with supervisor assignments (M2M User), start/end dates, active flag
- **Worker** — profiles with salary, `daily_rate` property (monthly_salary / 20), photo, ID doc, PPE sizing (shoe, overall top, pants, tshirt), drivers license (boolean + file upload)
- **Team** — groups of workers under a supervisor, with optional pay schedule (`pay_frequency`: weekly/fortnightly/monthly, `pay_start_date`: anchor date)
- **WorkLog** — daily attendance: date, project, team, workers (M2M), supervisor, overtime, `priced_workers` (M2M)
- **PayrollRecord** — completed payments linked to WorkLogs (M2M) and Worker (FK)
- **PayrollAdjustment** — bonuses, deductions, overtime, loans; linked to project (FK, optional), worker, and optionally to a Loan or WorkLog
- **Loan** — worker loans AND advances with principal, remaining_balance, `loan_type` ('loan' or 'advance')
- **ExpenseReceipt / ExpenseLineItem** — business expense records with VAT handling
- **WorkerCertificate** — per-worker certifications (Skills, PDP, First Aid, Medical, Work at Height) with `valid_until` expiry and optional document upload. Unique per (worker, cert_type). Has `is_expired` and `expires_soon` (≤30 days) properties.
- **WorkerWarning** — disciplinary records per worker with severity (verbal/written/final), reason, optional document. Ordered -date.

### Schema name-drifts to remember
Fields / accessors that differ from what you'd guess. Each has bitten multiple
sessions; grep `core/models.py` before using any field you haven't used before:

- `PayrollAdjustment.description` — NOT `reason`
- `log.adjustments_by_work_log` (reverse accessor for PayrollAdjustment.work_log FK) — NOT `payrolladjustment_set` (the FK has `related_name` set)
- `log.overtime_amount` (DecimalField, default 0.00) — NOT `log.overtime`

## Key Business Rules
- All business logic lives in the `core/` app — do not create additional Django apps
- Workers have a `daily_rate` property: `monthly_salary / Decimal('20.00')`
- Admin = `is_staff` or `is_superuser` (checked via `is_admin(user)` helper in views.py)
- Supervisors see only their assigned projects, teams, and workers
- Admins have full access to payroll, adjustments, and resource management
- WorkLog is the central attendance record — links workers to projects on specific dates
- Attendance logging includes conflict detection (prevents double-logging same worker+date+project)
- Loans have automated repayment deductions during payroll processing
- Cascading deletes use SET_NULL for supervisors/teams to preserve historical data

## Payroll Constants
Defined at top of views.py — used in dashboard calculations and payment processing:
- **ADDITIVE_TYPES** = `['Bonus', 'Overtime', 'New Loan', 'Advance Payment']` — increase worker's net pay
- **DEDUCTIVE_TYPES** = `['Deduction', 'Loan Repayment', 'Advance Repayment']` — decrease net pay

## Django ORM gotcha — M2M filter + aggregate inflation
Chained `.filter(m2m__field=X).filter(m2m__other=Y)` creates **separate JOIN
aliases**, producing a cartesian product of rows. `.aggregate(Sum(...))` dedupes
via subquery when `distinct()` is present; `.values().annotate(Sum(...))` does
NOT — it `GROUP BY`s the inflated rows and multiplies sums by N×M (where N and
M are the counts of matching related rows). Fix pattern: use
`.filter(id__in=Model.objects.filter(m2m__field=X).values('id'))` to keep the
outer queryset JOIN-free. See `_build_report_context` in `core/views.py` and
`ReportContextFilterInflationTests` in `core/tests.py` for the reference
implementation (commit f1e246c, Apr 2026).

## PayrollAdjustment Type Handling
- **Bonus / Deduction** — standalone, require a linked Project
- **New Loan** — creates a `Loan` record (`loan_type='loan'`); has a "Pay Immediately" checkbox (checked by default) that auto-processes the loan (creates PayrollRecord, sends payslip to Spark, marks as paid). When unchecked, the loan sits in Pending Payments for the next pay cycle. Editing syncs loan amount/balance/reason; deleting cascades to Loan + unpaid repayments
- **Advance Payment** — **auto-processed immediately** (never sits in Pending): creates `Loan` (`loan_type='advance'`), creates PayrollRecord, sends payslip email, and auto-creates an "Advance Repayment" for the next salary. Requires a Project (for cost tracking) and at least one unpaid work log (otherwise use New Loan).
- **Overtime** — links to `WorkLog` via `adj.work_log` FK; managed by `price_overtime()` view
- **Loan Repayment** — links to `Loan` (loan_type='loan') via `adj.loan` FK; loan balance changes during payment processing
- **Advance Repayment** — auto-created when an advance is paid; deducts from advance balance during `process_payment()`. If partial repayment, remaining balance converts advance to regular loan (`loan_type` changes from 'advance' to 'loan'). Editable by admin (amount can be reduced before payday).

## Outstanding Payments Logic (Dashboard)
The dashboard's outstanding amount uses **per-worker** checking, not per-log:
- For each WorkLog, get the set of `paid_worker_ids` from linked PayrollRecords
- A worker is "unpaid for this log" only if their ID is NOT in that set
- This correctly handles partially-paid logs (e.g., one worker paid, another not)
- Unpaid adjustments: additive types increase outstanding, deductive types decrease it

## Commands
```bash
# Local development (SQLite)
set USE_SQLITE=true && python manage.py runserver 0.0.0.0:8000
# Or use run_dev.bat which sets the env var

python manage.py migrate                    # Apply database migrations
python manage.py setup_groups               # Create Admin + Work Logger permission groups
python manage.py setup_test_data            # Populate sample workers, projects, logs
python manage.py import_production_data     # Import real production data (14 workers)
python manage.py collectstatic              # Collect static files for production
python manage.py check                      # System check

# Run the test suite (sets env vars inline — works in Git Bash; on cmd.exe use `set` first)
USE_SQLITE=true DJANGO_DEBUG=true python manage.py test core.tests -v 2
```

## Development Workflow
- Active development branch: `ai-dev` (PR target: `master`)
- Local dev uses SQLite: set `USE_SQLITE=true` environment variable
- Preview server config: `.claude/launch.json` → runs `run_dev.bat`
- Admin check in views: `is_admin(request.user)` helper (top of views.py)
- "Unpaid" adjustment = `payroll_record__isnull=True` (no linked PayrollRecord)
- POST-only mutation views pattern: check `request.method != 'POST'` → redirect
- Template UI: Bootstrap 5 modals with dynamic JS, data-* attributes on clickable elements
- Atomic transactions: `process_payment()` uses `select_for_update()` to prevent duplicate payments
- Payslip Preview modal ("Worker Payment Hub"): shows earnings, adjustments, net pay, **plus** active loans/advances with inline repayment forms. Uses `refreshPreview()` JS function that re-fetches after AJAX repayment submission. Repayment POSTs to `add_repayment_ajax` which creates a PayrollAdjustment (balance deduction only happens during `process_payment()`)
- Advance Payment auto-processing: `add_adjustment` immediately creates PayrollRecord + sends payslip when an advance is created. Also auto-creates an "Advance Repayment" adjustment for the next salary cycle. Uses `_send_payslip_email()` helper (shared with `process_payment`)
- Advance-to-loan conversion: When an Advance Repayment is only partially paid, `process_payment` changes the Loan's `loan_type` from 'advance' to 'loan' so the remainder is tracked as a regular loan
- Split Payslip: Preview modal has checkboxes on work logs and adjustments (all checked by default). `process_payment()` accepts optional `selected_log_ids` / `selected_adj_ids` POST params to pay only selected items. Falls back to "pay all" if no IDs provided (backward compatible with the quick Pay button).
- Team Pay Schedules: Teams have optional `pay_frequency` + `pay_start_date` fields. `get_pay_period(team)` calculates current period boundaries by stepping forward from the anchor date. The preview modal shows a "Split at Pay Date" button that auto-unchecks items after the `cutoff_date` (end of last completed period — includes ALL overdue work, not just one period). `get_worker_active_team(worker)` returns the worker's first active team.
- Pay period calculation: `pay_start_date` is an anchor (never needs updating). Weekly=7 days, Fortnightly=14 days, Monthly=calendar month stepping. Uses `calendar.monthrange()` for month-length edge cases (no `dateutil` dependency).
- Batch Pay: "Batch Pay" button on payroll dashboard opens a modal with two radio modes — **"Until Last Paydate"** (default, splits at last completed pay period per team schedule) and **"Pay All"** (includes all unpaid items regardless of date). Preview fetches from `batch_pay_preview` with `?mode=schedule|all`. Workers without team pay schedules are skipped in schedule mode but included in Pay All mode. `batch_pay` POST endpoint processes each worker in independent atomic transactions; emails are sent after all payments complete. Uses `_process_single_payment()` shared helper (same logic as individual `process_payment`). Modal includes team filter dropdown and 3-option loan filter (All / With loans only / Without loans).
- Pending Payments Table: Shows overdue badges (red) for workers with unpaid work from completed pay periods, and loan badges (yellow) for workers with active loans/advances. Filter bar has: team dropdown, "Overdue only" checkbox, and loan dropdown (All Workers / With loans only / Without loans). Overdue detection uses `get_pay_period()` cutoff logic.
- Quick Adjust Button: Each pending payments row has an "Adjust" button (slider icon) that opens the Add Adjustment modal with that worker pre-checked and their most recent project pre-selected. The header "Add Adjustment" button resets the modal to a clean state. Uses `_quickAdjustOpen` flag to distinguish between the two open paths.
- Worker Lookup Modal: Clicking any worker name on the payroll dashboard (or using the "Worker Lookup" button) opens a modal with a comprehensive report card — amount payable, outstanding loans, paid this month/year, loans this year, recent activity (last payslip, loan, repayment, advance), active loans table, current project + days on project, PPE sizing, drivers license, and notes. Uses `worker_lookup_ajax` AJAX endpoint. Worker dropdown in modal allows switching workers without closing.
- Team & Project Management UIs: Friendlier alternatives to `/admin/core/team/` and `/admin/core/project/`. Reachable via the "Resources" dropdown in the topbar (admin only). **Team pages**: `/teams/` (list + search/filter), `/teams/<id>/` (detail with Profile/Pay Schedule/Workers/History tabs — Pay Schedule tab uses the existing `get_pay_period()` helper to show current + next 2 periods), `/teams/<id>/edit/` (single-page form for name, supervisor, pay schedule, and workers M2M). **Project pages**: `/projects/`, `/projects/<id>/` (tabs: Profile/Supervisors/Teams/Workers/History), `/projects/<id>/edit/` (form for name, description, dates, supervisors M2M). Uses `TeamForm` and `ProjectForm` from `core/forms.py` (both simple ModelForms, no inline formsets). Batch reports at `/teams/report/` and `/projects/report/` with CSV exports; PDF exports deferred as a follow-up. Dashboard "Manage Resources" card now has "Manage All Workers/Projects/Teams" footer links on each tab. Django admin remains fully functional as a fallback.
- Worker Management UI: A friendlier alternative to `/admin/core/worker/`. Reachable via the "Resources" topbar dropdown → Workers (admin-only). Pages: `/workers/` (list with search + status filter), `/workers/<id>/` (detail with Profile/Certifications/Warnings/History tabs), `/workers/<id>/edit/` or `/workers/new/` (single-page form with sections for Personal & Pay, PPE, Documents, Driver's License, plus inline formsets for certifications and warnings). Uses `WorkerForm`, `WorkerCertificateFormSet`, `WorkerWarningFormSet` from `core/forms.py`. The "+ Add Certification" / "+ Add Warning" buttons clone a `<template>` element via `content.cloneNode()` (DOM-safe, no innerHTML) and rewrite `__PREFIX__` in input names to the next formset index. File uploads validated at 5 MB max via `validate_max_5mb()` in `forms.py`. Django admin (`/admin/core/worker/`) remains fully functional as a fallback — both UIs coexist.
- Worker Batch Report: `/workers/report/` shows every worker with aggregated lifetime history — days worked, projects worked on, teams, first/last payslip dates, total paid, cert status (active/total + expired/expiring counts), warning count. Filter by status, project, team. CSV export via `/workers/report/csv/`, PDF via `/workers/report/pdf/` (landscape A4, same amber-accent typography as the payroll report). Built on the reusable `_build_worker_report_context()` helper which uses `annotate(Min/Max/Count/Sum)` + prefetch for efficient aggregation.
- Dashboard cert-expiry card: The admin dashboard shows a "Certifications Need Attention" stat card with count of expired + expiring-within-30-days certs (active workers only). Card is CONDITIONAL — renders only when count > 0, so it disappears when everything is in good standing. Clicking it goes to the worker batch report. Counts come from `index()` view adding `certs_expired_count`, `certs_expiring_count`, `certs_alert_total` to context.

## URL Routes
| Path | View | Purpose |
|------|------|---------|
| `/` | `index` | Dashboard (admin stats / supervisor work view) |
| `/attendance/log/` | `attendance_log` | Log daily work with date range support |
| `/history/` | `work_history` | Work logs table with filters |
| `/history/export/` | `export_work_log_csv` | Download filtered logs as CSV |
| `/workers/export/` | `export_workers_csv` | Admin: export all workers to CSV |
| `/workers/` | `worker_list` | Admin: friendly worker list with search + status filter |
| `/workers/new/` | `worker_edit` | Admin: blank worker-create form |
| `/workers/<id>/` | `worker_detail` | Admin: worker profile with profile/certs/warnings/history tabs |
| `/workers/<id>/edit/` | `worker_edit` | Admin: edit worker + inline cert/warning formsets |
| `/workers/report/` | `worker_batch_report` | Admin: aggregated roster report (days, projects, payslips, certs) |
| `/workers/report/csv/` | `worker_batch_report_csv` | Admin: batch report as CSV download |
| `/workers/report/pdf/` | `worker_batch_report_pdf` | Admin: batch report as PDF download |
| `/teams/` | `team_list` | Admin: friendly team list with search + status filter |
| `/teams/new/` | `team_edit` | Admin: blank team-create form |
| `/teams/<id>/` | `team_detail` | Admin: team profile with profile/pay schedule/workers/history tabs |
| `/teams/<id>/edit/` | `team_edit` | Admin: edit team (name, supervisor, pay schedule, workers M2M) |
| `/teams/report/` | `team_batch_report` | Admin: aggregated team report (HTML) |
| `/teams/report/csv/` | `team_batch_report_csv` | Admin: team batch report as CSV download |
| `/projects/` | `project_list` | Admin: friendly project list with search + status filter |
| `/projects/new/` | `project_edit` | Admin: blank project-create form |
| `/projects/<id>/` | `project_detail` | Admin: project profile with profile/supervisors/teams/workers/history tabs |
| `/projects/<id>/edit/` | `project_edit` | Admin: edit project (name, description, dates, supervisors M2M) |
| `/projects/report/` | `project_batch_report` | Admin: aggregated project report (HTML) |
| `/projects/report/csv/` | `project_batch_report_csv` | Admin: project batch report as CSV download |
| `/toggle/<model>/<id>/` | `toggle_active` | Admin: AJAX toggle active status |
| `/payroll/` | `payroll_dashboard` | Admin: pending payments, loans, charts |
| `/payroll/pay/<worker_id>/` | `process_payment` | Admin: process payment (atomic) |
| `/payroll/price-overtime/` | `price_overtime` | Admin: AJAX price unpriced OT entries |
| `/payroll/adjustment/add/` | `add_adjustment` | Admin: create adjustment |
| `/payroll/adjustment/<id>/edit/` | `edit_adjustment` | Admin: edit unpaid adjustment |
| `/payroll/adjustment/<id>/delete/` | `delete_adjustment` | Admin: delete unpaid adjustment |
| `/payroll/preview/<worker_id>/` | `preview_payslip` | Admin: AJAX JSON payslip preview (includes active loans) |
| `/payroll/worker-lookup/<worker_id>/` | `worker_lookup_ajax` | Admin: AJAX JSON worker report card |
| `/payroll/repayment/<worker_id>/` | `add_repayment_ajax` | Admin: AJAX add loan/advance repayment from preview |
| `/payroll/payslip/<pk>/` | `payslip_detail` | Admin: view completed payslip |
| `/receipts/create/` | `create_receipt` | Staff: expense receipt with line items |
| `/import-data/` | `import_data` | Setup: run import command from browser |
| `/payroll/batch-pay/preview/` | `batch_pay_preview` | Admin: AJAX JSON batch pay preview (`?mode=schedule\|all`) |
| `/payroll/batch-pay/` | `batch_pay` | Admin: POST process batch payments for multiple workers |
| `/run-migrate/` | `run_migrate` | Setup: run pending DB migrations from browser |

## Frontend Design Conventions
- **Dual-theme** (dark + light) driven by a single CSS variable set in `static/css/custom.css`.
  The theme is dark-first; the light theme is a set of var overrides inside a `:root.light` block.
  A sun/moon toggle in the topbar flips a class on `<html>` and persists the choice to localStorage.
- **CSS variables** in `static/css/custom.css` `:root` — always use `var(--name)`:
  - `--accent: #e8851a` (warm orange/amber, brand), `--accent-hover: #f59e0b`
  - `--primary-dark: #0f172a`, `--primary: #1e293b`
  - `--bg-card: #161921`, `--bg-card-hover: #1c2029` (elevated surfaces)
  - `--text-primary: #d8d8d8` (dark theme), `--text-secondary`, `--text-tertiary`
  - Light-theme overrides flip backgrounds to white/grey and accent to `#d97706`
- **Icons**: Font Awesome 6 only (`fas fa-*`). Do NOT use Bootstrap Icons (`bi bi-*`)
- **CTA buttons**: `btn-accent` (orange) for primary actions. `btn-primary` (dark slate) for modal Save/Submit
- **Page titles**: `{% block title %}Page Name | FoxFitt{% endblock %}`
- **Fonts**: Inter (body) + Poppins (headings) loaded in base.html via Google Fonts CDN
- **Cards**: Borderless with subtle shadow. Stat cards have coloured accent bars on the left.
- **Bootstrap tooltips**: Global init in `base.html` — any element with
  `data-bs-toggle="tooltip" title="..."` gets a tooltip automatically. Tooltips are themed
  via custom `--bs-tooltip-bg`/`--bs-tooltip-color` overrides in custom.css so they're
  readable in both light and dark modes (otherwise Bootstrap's default picks the wrong
  pair of body vars for dark mode).
- **Email/PDF payslips**: Worker name dominant — do NOT add prominent FoxFitt branding

## Static Assets & Cache-Busting (Cloudflare is in front)

Production traffic reaches the Flatlogic VM through **Cloudflare** (response headers
include `cf-ray`, `cf-cache-status`, and a `cache-control: max-age=14400`). Static
assets — including `custom.css` — are cached at Cloudflare's edge for up to 4 hours
per unique URL. This is great for performance and bad for deploys if the URL doesn't
change when the file does.

### How cache-busting works now
`base.html` loads CSS as:

```html
<link rel="stylesheet" href="{% static 'css/custom.css' %}?v={{ deployment_timestamp|default:'1' }}">
```

`deployment_timestamp` comes from `core/context_processors.py::project_context` as
`int(time.time())` — meaning every Django request generates a new query-string value.
Cloudflare treats each new `?v=...` value as a new URL → `cf-cache-status: MISS` →
fresh fetch from the VM. Users always see the latest CSS as soon as the Django
process restarts.

**Trade-off**: because the timestamp changes every second, CDN cache-hit rate on
CSS is effectively zero. For a low-traffic app this is fine. If traffic grows,
consider switching to a file-mtime-based token so the URL only changes when the
CSS actually changes.

### The pitfall this replaced
Pre-Apr 2026, the template used `{{ request.timestamp|default:'1.0' }}`. But
`request.timestamp` is **not** a Django request attribute — the variable always
fell back to the literal `'1.0'`. Every deploy's CSS URL resolved to the same
`custom.css?v=1.0`, so Cloudflare held onto a pre-redesign copy for hours while
the VM served the new one. Symptom was "the deploy worked but the page looks wrong"
that only a hard refresh in incognito temporarily fixed. Never use `request.timestamp`
in templates — it doesn't exist.

### When CSS changes don't appear on production
1. Confirm Django is rendering the new URL: `curl -s https://foxlog.flatlogic.app/ | grep -oE 'custom\.css\?v=[^"]+'` — the `v=` number should change per request (or at least per restart)
2. Confirm the CDN honours it: `curl -sI "https://foxlog.flatlogic.app/static/css/custom.css?cb=test$(date +%s)" | grep -i cache` — expect `cf-cache-status: MISS` then `HIT` on repeat
3. If the Django URL still looks like `?v=1.0` (constant), `deployment_timestamp` isn't being injected — check that `core.context_processors.project_context` is listed in `TEMPLATES[0]['OPTIONS']['context_processors']` in `config/settings.py`

### `collectstatic` is required after CSS/JS changes on production
Flatlogic's rebuild does NOT automatically run `collectstatic`. If new CSS is on
disk but the VM's `staticfiles/` hasn't been refreshed, Apache serves the old
collected copy. Have Gemini run `python3 manage.py collectstatic --noinput`
after any PR that touches `static/`.

### `staticfiles/` is NOT tracked in git (since Apr 2026)
`staticfiles/` is gitignored because it's a **build artifact** — `collectstatic`
regenerates it from `static/` and each installed-app's static dirs on every
deploy. Previously it was tracked, which caused two problems:

1. **Flatlogic-auto-noise commits.** Every time Gemini ran `collectstatic`,
   Flatlogic's web UI detected the modified files in `staticfiles/` and
   auto-committed them with a generic `Ver XX.YY` message (e.g. the notorious
   "Ver 30.04 Fix reports and add Supervisor" commit that diverged our branch
   from GitHub in Apr 2026). These commits reached gitea but not GitHub,
   creating recurring reconciliation pain.
2. **Misleading diffs in PRs.** Every CSS change showed up twice in git
   (once in `static/`, once in `staticfiles/`), doubling review surface.

**Deploy consequence**: after pulling a commit that modifies `static/` files,
Gemini MUST run `collectstatic` BEFORE restarting the service. If the pull
removed the `staticfiles/` directory from the working tree (which happens
the first time after the gitignore change), collectstatic recreates everything
from source. Brief window (~seconds) of possible 404s on static assets during
the deploy; acceptable at this scale.

**First-time migration note**: the commit that added `staticfiles/` to
`.gitignore` also removed all previously-tracked files via
`git rm -r --cached staticfiles/`. The VM's filesystem still holds the
directory — git just stops tracking it. On next deploy the pull will
delete the working-tree copies (because they no longer exist in the
commit tree), so `collectstatic --noinput` MUST run immediately after
pull to repopulate. After that, all is stable.

## PDF Generation (WeasyPrint)
Migrated from xhtml2pdf in Nov 2026. WeasyPrint is a browser-grade HTML→PDF renderer — it supports real CSS (flexbox, grid, `@font-face`, shadows, `border-radius`, proper cascade) that xhtml2pdf could not handle.

### Files
- `core/utils.py` — `render_to_pdf(template_src, context_dict)` is the single entry point; lazy-imports WeasyPrint, returns PDF bytes or `None` on failure
- `core/templates/core/pdf/report_pdf.html` — payroll report (complex layout, 600+ lines)
- `core/templates/core/pdf/payslip_pdf.html` — payslip (emailed to Spark Receipt after each payment)
- `core/templates/core/pdf/receipt_pdf.html` — expense receipt (emailed to Spark Receipt after each expense entry)
- `core/templatetags/format_tags.py::money` — South African space-separated currency formatting (`R 64 939.00`); use this instead of `floatformat:2`

### Callers
- `generate_report_pdf()` — downloads the report PDF to the browser
- `_send_payslip_email()` — attaches payslip PDF to Gmail SMTP email (called by `process_payment`, `add_adjustment` advance path, `batch_pay`)
- `create_receipt()` — attaches receipt PDF to Gmail SMTP email
- All three use the same `EmailMultiAlternatives.attach(filename, pdf_bytes, "application/pdf")` pattern — engine-agnostic

### Dependencies
- **Python package**: `weasyprint==68.1` (pinned in `requirements.txt`)
- **System libraries** (Pango, Cairo, GDK-PixBuf, FFI, shared-mime-info):
  - **Flatlogic/production (Debian)**: already installed on the platform image — confirmed via Flatlogic's Gemini
  - **Windows local dev**: install the GTK3 runtime via `winget install -e --id tschoonj.GTKForWindows --accept-package-agreements --accept-source-agreements --silent` (installs `C:\Program Files\GTK3-Runtime Win64\`)
  - **macOS local dev**: `brew install pango` (not currently used but documented for completeness)

### Windows DLL resolution quirk
Since Python 3.8, native DLLs are *not* loaded from PATH automatically — an explicit `os.add_dll_directory()` call is required. The `_ensure_gtk_on_windows()` helper in `core/utils.py` handles this automatically: it checks common GTK3 install paths on module load and registers the first one found. No-op on Linux/macOS.

If the report page returns "PDF generation failed", check the Django log for the underlying error:
- `cannot load library 'gobject-2.0-0'` → GTK3 runtime not installed (run the winget command above)
- `'super' object has no attribute 'transform'` → `weasyprint`/`pydyf` version mismatch; reinstall with `pip install --upgrade weasyprint==68.1`

### Template conventions
- **Modern CSS is fine** — flexbox (`display: flex`), grid (`display: grid; gap: 20pt`), `@font-face`, `box-shadow`, `border-radius` all render correctly
- **Fonts**: WeasyPrint can load web fonts. If we ever add `@font-face` blocks pointing to `static/fonts/Inter-*.ttf` and `Poppins-*.ttf`, the PDFs can use the same typography as the web app (currently the PDFs use Helvetica by default — upgrading to Inter/Poppins is optional follow-up work)
- **Page setup**: `@page { size: a4 portrait; margin: 2cm 1.8cm 1.6cm 1.8cm; }` — standard A4 with generous margins
- **`base_url`**: `render_to_pdf()` passes `settings.STATIC_ROOT or "."` as `base_url` so relative paths in `<img src="...">` and `@font-face src: url(...)` resolve against the collected static dir

### Known lint footguns (legacy from xhtml2pdf era)
- `report_pdf.html` still uses invisible `<table class="cols">` elements for two-column layout — these work fine under WeasyPrint but could be simplified to `display: grid; grid-template-columns: 1fr 1fr; gap: 20pt` as a future cleanup pass
- The `period-detail td { padding-top: 3pt; padding-bottom: 3pt; }` split-padding workaround (from a shorthand-collision bug with xhtml2pdf) is no longer needed; safe to replace with the `padding` shorthand when cleaning up the template

## Users, Roles & Permissions

Understanding who-can-do-what in this app requires grasping **three separate layers**
of Django auth that stack on top of each other:

### Layer 1 — Django's three built-in user flags

These live on the `auth.User` model and are the foundation. Every user has exactly one
combination of these three flags:

| Flag | What it means | Who should have it |
|---|---|---|
| `is_superuser=True` | **Bypasses every permission check.** Full access to everything everywhere, including Django admin. Created by `createsuperuser`. | Konrad (the owner), and one emergency-access account. That's it. |
| `is_staff=True` | Can log into `/admin/` (the built-in Django admin interface) and sees it. Does NOT grant any model permissions by itself — those come from groups or per-user permissions. | Konrad, and any "office admin" people who need full access to edit data via Django admin. Usually combined with `is_superuser` in this app. |
| (neither) | Regular user. Can log into the friendly app at `/` but cannot enter Django admin. Sees only what they've been explicitly given access to via group membership or supervisor assignments. | Site supervisors (Work Loggers), and any future non-admin roles. |

Key mental model: **`is_superuser` beats everything**. A superuser's permission groups
and assignments don't matter — they always see everything. Use superuser sparingly
so that regular permission paths get exercised and tested.

### Layer 2 — The two app-specific permission groups

Created by `python manage.py setup_groups` (a one-time command, safe to re-run; it
updates existing groups rather than duplicating them). Assignment happens in
`/admin/auth/group/` → add users to groups:

**Admin group** — grants every Django model permission (add/change/delete/view) on
all 10 core models (Worker, Project, Team, WorkLog, PayrollRecord, Loan,
PayrollAdjustment, ExpenseReceipt, ExpenseLineItem, and the new WorkerCertificate/
WorkerWarning via separate admin registration).

- **Practical effect**: only matters for `is_staff=True` users who are NOT
  superusers. For them, the Admin group is what lets them actually use Django
  admin (without it, they can log into /admin/ but see empty lists).
- For superusers, the Admin group is redundant (they bypass permissions anyway).
- For non-staff users, the Admin group is pointless (they can't reach Django
  admin at all).

**Work Logger group** — grants: add/change/view `WorkLog`; view-only on
`Project`, `Worker`, `Team`. Notably does NOT grant any Payroll permissions.

- **Practical effect**: this group is the app's signal for "this user is a site
  supervisor". The `is_supervisor()` helper in `views.py` explicitly checks for
  membership in this group — so adding someone here marks them as a supervisor,
  even if they don't own any teams or projects yet.
- Work Loggers typically have `is_staff=False` (no Django admin access). They
  use the friendly app UI at `/attendance/log/`, `/history/`, and the dashboard.

### Layer 3 — Implicit supervisor roles via model relationships

Two model fields independently grant "supervisor-ness" even without group membership:

- `Team.supervisor` (ForeignKey → User) — whoever this points to is a supervisor
  of that team. Set on the Team edit page or `/admin/core/team/`.
- `Project.supervisors` (ManyToManyField → User) — every user in this M2M is a
  supervisor of the project. Set on the Project edit page or
  `/admin/core/project/`.

The `is_supervisor()` helper treats any ONE of these as sufficient:

```python
def is_supervisor(user):
    return (
        user.supervised_teams.exists()          # Team.supervisor FK reverse
        or user.assigned_projects.exists()      # Project.supervisors M2M reverse
        or user.groups.filter(name='Work Logger').exists()
    )
```

So a user can become a supervisor via **any** of: Work Logger group, assigned to
a Team as supervisor, or added to a Project's supervisors M2M.

### The three permission-check helpers in `core/views.py`

All three are defined near the top of `views.py` (around line 47–67):

| Helper | Returns True if… | Used by |
|---|---|---|
| `is_admin(user)` | `is_staff=True` OR `is_superuser=True` | Every admin-only view (payroll, reports, worker/team/project management, CSV exports) |
| `is_supervisor(user)` | Supervises a team OR has assigned projects OR is in Work Logger group | Attendance logging, history page filtering |
| `is_staff_or_supervisor(user)` | `is_admin` OR `is_supervisor` | Views accessible to both tiers (dashboard shows different content per tier) |

**Critical**: `is_admin()` does NOT check for the "Admin" group. It checks the
Django `is_staff`/`is_superuser` flags. A user can be in the "Admin" permission
group but NOT be an admin as far as the app is concerned, and vice versa. The
group controls Django-admin model permissions; the flags control everything else.

### How views enforce permissions

1. **`@login_required`** is on every view except `import_data()` and
   `run_migrate()` (temporary setup endpoints).
2. **Admin-only views** call `is_admin(request.user)` at the top and return
   `HttpResponseForbidden("Admin access required.")` if false. Examples:
   everything under `/payroll/`, `/workers/*`, `/teams/*`, `/projects/*`,
   `/report/*`, `/workers/export/`.
3. **Supervisor-scoped data** uses `is_supervisor()` to gate access, then filters
   querysets by the user's `supervised_teams` / `assigned_projects`:
   - `work_history` — supervisors see only logs for their teams/projects
   - `AttendanceLogForm` — pre-filters `project` and `team` dropdowns by what
     the user can see; `workers` field is filtered by team membership
4. **Permission cascading** — a supervisor of a Team automatically "supervises"
   every worker in that team, and every project that team has worked on.
   This is implicit — there's no per-worker permission.

### The "Resources" dropdown supervisor picker

When editing a Team or Project via the friendly UI (`/teams/<id>/edit/` or
`/projects/<id>/edit/`), the Supervisor/Supervisors picker uses
`_supervisor_user_queryset()` in `core/forms.py`:

```python
User.objects.filter(is_active=True).order_by('username')
```

Any active user can be picked. The picker is deliberately NOT pre-filtered
by group/staff flags because `is_supervisor(user)` (views.py) grants
supervisor powers to anyone assigned to a team/project FK/M2M — so the
picker shouldn't be stricter than the permission model. Pre-Apr 2026 the
picker required Work Logger group membership, which hid valid supervisors
(see commit 0ceceeb for the fix + regression tests). Deactivated accounts
are still hidden.

### Typical user setups

| User | is_superuser | is_staff | Groups | Supervised Teams/Projects | What they can do |
|---|---|---|---|---|---|
| **admin** (Konrad) | ✓ | ✓ | — | — | Everything. Bypasses all checks. |
| **testadmin** (Flatlogic) | ✓ | ✓ | — | — | Same as above. |
| **eendman** | — | — | Work Logger | *(usually also supervises a team)* | Log work for assigned teams/projects, see their history. Cannot enter Django admin. Cannot view payroll or worker salary data via app UI. |
| **Office data-entry staff** (future) | — | ✓ | Admin | — | Can enter Django admin and CRUD all core models there. Does NOT see the payroll dashboard or worker salary UI (because `is_admin()` helper still returns True via `is_staff`, so actually yes they CAN see payroll — see note below). |
| **Inactive terminated employee** | — | — | — | — | Cannot log in if `is_active=False`. |

**Note on is_staff + payroll access**: `is_admin()` returns True for any
`is_staff` user. That means if you create an office-admin user with
`is_staff=True` but no superuser flag, they WILL see payroll and salary data
via the friendly UI. If you need a "Django admin only, no payroll UI" role,
we'd have to add a separate flag or group check — not currently supported.

### How to add a new supervisor (step by step)

1. Go to `/admin/auth/user/add/` and create the user with a username and
   password. **Uncheck "Staff status"** on the initial form (they don't need
   Django admin access).
2. (Optional) Add them to the **Work Logger** group if you want
   `is_supervisor(user)` to return True even without a team/project
   assignment. Not required for the picker to show them — the picker
   shows any active user (see commit 0ceceeb, Apr 2026).
3. (Optional) Assign them as the supervisor of one or more teams via
   `/teams/<id>/edit/` (Supervisor dropdown — they'll appear in the list
   because they're active).
4. (Optional) Add them to one or more projects via `/projects/<id>/edit/`
   (Supervisors M2M checklist).
5. They can now log in at `/accounts/login/` and will land on the Dashboard
   with a supervisor view — their teams + projects only.

### Common misconceptions (read these)

- **"Admin" group ≠ Django admin access.** Django admin access requires
  `is_staff=True`. The "Admin" group is just a permission bundle.
- **Work Logger doesn't need to supervise a team to be "a supervisor".**
  Group membership alone satisfies `is_supervisor()`.
- **Superuser bypasses the "Admin" group.** They don't need it.
- **Deactivating a user** (`is_active=False`) blocks login entirely; their
  team/project assignments remain in the DB for audit purposes but stop
  having any effect.
- **The `supervisor` field on WorkLog is historical, not authoritative.** It
  records who logged the work that day — not who currently supervises the
  workers.

## Authentication

- Django's built-in auth (`django.contrib.auth`)
- Login: `/accounts/login/` → redirects to `/` (home)
- Logout: POST to `/accounts/logout/` → redirects to login
- All views use `@login_required` except `import_data()` and `run_migrate()`
- No PIN auth in v5 (simplified from v2)
- Passwords: Django's default PBKDF2 hashing, no custom password policy
- Sessions: cookie-based, server-side session store (default Django)

## Django Admin Customisation

The `/admin/` interface is Django's built-in admin with two targeted customisations:

### Model registrations (`core/admin.py`)
- Every core model is registered with `list_display`, `list_filter`, `search_fields`, and
  (where relevant) `filter_horizontal` / `filter_vertical` for M2M pickers.
- `WorkerAdmin` has `WorkerCertificateInline` + `WorkerWarningInline` so you can edit a
  worker's certs and warnings inline on the worker change page.
- `WorkerCertificateAdmin` + `WorkerWarningAdmin` are also standalone (useful for
  "show me all expiring certs across all workers" type queries via list_filter).

### Template override — `core/templates/admin/base_site.html`
Extends `admin/base.html` and injects a small `<style>` block into every admin page.
Currently used for:
- Taller `FilteredSelectMultiple` widgets (the "Choose" / "Available" boxes used by
  `filter_horizontal` on Groups, WorkLogs etc.) — default Django height is ~16em which
  is too short for long permission lists; we set 30em (40em on tall screens).

Add more admin-only CSS tweaks inside that `<style>` block rather than polluting
`static/css/custom.css`.

### Why the override works — `TEMPLATES.DIRS` setting
Django's template loader tries `TEMPLATES[0].DIRS` before the app-dirs loader. Since
`django.contrib.admin` comes before `core` in `INSTALLED_APPS` (the standard order),
its `admin/base_site.html` would normally win. Adding `BASE_DIR / 'core' / 'templates'`
to `TEMPLATES[0]['DIRS']` in `config/settings.py` makes our override take priority
without reordering `INSTALLED_APPS` (which would risk subtle side effects on signals,
migrations, and admin URL registration).

## Backup & Restore (production safety net)

Flatlogic doesn't expose MySQL directly (no SSH, no `mysqldump`, no DB console).
Instead, the app ships two management commands + two admin-only browser URLs
that back up and restore every row via Django's ORM — platform-independent,
works anywhere Django does.

### Making a backup (before any risky deploy)

**From the browser (production)**:
1. Log in as admin
2. Visit `/backup-data/`
3. Browser downloads `foxlog_backup_<timestamp>.json` to your laptop
4. Move that file somewhere safe (Google Drive, local disk — NOT the repo)

**From the command line (local dev or SSH-able host)**:
```
python manage.py backup_data                        # → backups/foxlog_<timestamp>.json
python manage.py backup_data --output=custom.json   # → custom.json
```

Either method produces an identical JSON file covering:
- All auth tables: User, Group, Permission, ContentType (so accounts restore correctly)
- UserProfile, Project, Worker, Team, WorkLog, PayrollRecord, Loan, PayrollAdjustment,
  ExpenseReceipt, ExpenseLineItem, WorkerCertificate, WorkerWarning

File size is roughly 1 KB per row (14 workers / 100 work logs → ~90 KB).

### Restoring a backup

**From the browser**:
1. Log in as admin
2. Visit `/restore-data/`
3. Upload your `.json` file
4. Tick "Yes, I understand" checkbox
5. Click Restore — runs inside a database transaction, all-or-nothing

**Behaviour**:
- Rows with matching primary key are **UPDATED** (no duplicates)
- Rows with primary keys not yet in the DB are **INSERTED**
- Rows in the DB but NOT in the backup are **KEPT** (restore doesn't delete)
- If any row fails to load, the whole restore is rolled back (no partial state)

**For a clean restore** (wipe everything first, then load the backup):
```
python manage.py flush     # irreversible — deletes ALL data
python manage.py restore_data backup.json
```

### Recommended workflow before any deploy

1. `/backup-data/` on production before pushing the change
2. Push the change to `ai-dev`, let Flatlogic rebuild
3. Verify the new version works
4. If broken: restore from the backup you just took via `/restore-data/`
5. Delete the backup file from your laptop once you're confident the deploy is stable

### Files involved
- `core/management/commands/backup_data.py` — CLI command + reusable `build_backup_payload()` helper
- `core/management/commands/restore_data.py` — CLI command + reusable `restore_from_json_string()` helper
- `core/views.py::backup_data` — browser view that reuses the helper
- `core/views.py::restore_data` — browser view with minimal HTML upload UI
- URLs: `/backup-data/`, `/restore-data/` (both `@login_required` + `is_admin()` gated)

## Environment Variables
```
DJANGO_SECRET_KEY              # required in prod — startup fails without it
DJANGO_DEBUG                   # "true"/"false"; defaults to false; keep false in prod
HOST_FQDN, CSRF_TRUSTED_ORIGIN # trusted hostnames (scheme-less ok, auto-prefixed https://)
DB_NAME, DB_USER, DB_PASS, DB_HOST (default: 127.0.0.1), DB_PORT (default: 3306)
USE_SQLITE                     # "true" → use SQLite instead of MySQL (local dev only)
EMAIL_HOST_USER                # Gmail address — required for any outbound email
EMAIL_HOST_PASSWORD            # Gmail App Password (16 chars, no spaces/non-breaking-space)
DEFAULT_FROM_EMAIL             # Optional — falls back to EMAIL_HOST_USER if unset
SPARK_RECEIPT_EMAIL            # Optional — defaults to FoxFitt's Spark Receipt address
PROJECT_DESCRIPTION, PROJECT_IMAGE_URL  # Flatlogic branding
```

### Email fallback behaviour
`DEFAULT_FROM_EMAIL` is not strictly required — `config/settings.py` sets it as:

```python
DEFAULT_FROM_EMAIL = os.getenv("DEFAULT_FROM_EMAIL", "") or EMAIL_HOST_USER
```

…so if the env var is unset or empty, the "From" address on every outbound email
falls back to the authenticated Gmail address (which is always valid since we
send AS that account). Without this fallback, receipt and payslip emails fail
with `Invalid address ""`. If you want to send FROM a different display address
than the authenticated one (e.g. "FoxFitt Payroll <payroll@foxfitt.co.za>"),
set `DEFAULT_FROM_EMAIL` explicitly — but Gmail will likely rewrite it to the
authenticated user anyway unless you've configured a "Send mail as" alias.

### Where env vars live on Flatlogic
Flatlogic's platform has no env-var UI. Values are set in a `.env` file at
`BASE_DIR.parent / ".env"` on the VM (one level up from the repo). Edit via
Gemini/shell — the user cannot modify via Flatlogic's web editor because
`.env` is outside the project tree. The file is loaded by
`python-dotenv` in `config/settings.py` before any `os.getenv()` calls.

## Flatlogic/AppWizzy Deployment
- **Branches**: `ai-dev` = development (Flatlogic AI + Claude Code). `master` = deploy target.
- **Workflow**: Push to `ai-dev` → Flatlogic auto-detects → "Pull Latest" → app rebuilds (~5 min)
- **Deploy from Git** (Settings): Full rebuild from `master` — use for production
- **Migrations**: Sometimes run automatically during rebuild, but NOT always reliable. If you get "Unknown column" errors after pulling latest, visit `/run-migrate/` in the browser to apply pending migrations manually. This endpoint runs `python manage.py migrate` on the production MySQL database.
- **Static files**: Flatlogic's rebuild does NOT auto-run `collectstatic`. After CSS/JS changes have Gemini run `python3 manage.py collectstatic --noinput` + restart the service, otherwise Apache keeps serving the previously-collected copy.
- **Service**: The Django app runs as `django-dev.service` (systemd). Gemini restarts it via `sudo systemctl restart django-dev.service`. It runs `python manage.py runserver 0.0.0.0:8000` — a **development server**, not gunicorn/uwsgi (Flatlogic default, works fine at this scale).
- **CDN**: All production traffic goes through Cloudflare. Response headers show `cf-ray`/`cf-cache-status`. Static assets are cached at the edge for 4h — see "Static Assets & Cache-Busting" section for how the `deployment_timestamp` token breaks stale caches.
- **Never edit `ai-dev` directly on GitHub** — Flatlogic pushes overwrite it
- **Gemini gotcha**: Flatlogic's Gemini AI reads `__pycache__/*.pyc` and gets confused. Tell it: "Do NOT read .pyc files. Only work with .py source files."
- **Sequential workflow**: Don't edit in Flatlogic and Claude Code at the same time

### Git remotes on the VM
The Flatlogic VM has TWO git remotes, both kept in sync:
- `github` → `https://github.com/Konradzar/LabourPay_v5.git` (our canonical source)
- `gitea` → `https://gitea.flatlogic.app/admin/<id>-vm.git` (Flatlogic's internal mirror — the one the platform UI watches)

Any push the VM makes must go to BOTH: `git push github ai-dev && git push gitea ai-dev`.
If the two diverge, Flatlogic's dashboard can show a different commit than GitHub,
which silently confuses deploys. Flatlogic's UI occasionally commits as
`Flatlogic Bot <support@flatlogic.com>` (autosaves from the in-browser file editor) —
those commits land on gitea but don't propagate to GitHub unless someone pushes.

### VM-local safety branches
When doing risky deploys (model migrations, branch resets, history rewrites), we
create a safety branch on the VM at the pre-deploy HEAD so Gemini can
`git reset --hard <safety-branch>` + service-restart to roll back in ~60 seconds:

```
git branch pre-<purpose>-YYYYMMDD HEAD
git branch --list "pre-*"
```

Safety branches are VM-local — not pushed to GitHub by default. They're
single-use rollback anchors; delete after 7 days of confirmed stability via
`git branch -D pre-<purpose>-YYYYMMDD`.

### Workflow options going forward
Either works — pick one and stick to it per change to avoid divergence:
1. **Claude → GitHub → Flatlogic pulls**: Claude pushes to origin/ai-dev; you click "Pull Latest" in the Flatlogic UI (or ask Gemini to `git pull + push gitea + restart`).
2. **Flatlogic UI → GitHub**: edit in Flatlogic's file editor; click "Push to GitHub" in their UI; Claude pulls locally with `git pull origin ai-dev`.
**Don't mix** paths in the same change — that's how divergence (and the "Ver XX.YY screeeewup" commits) happen.

## Security Notes
- Production: `SESSION_COOKIE_SECURE=True`, `CSRF_COOKIE_SECURE=True`, `SameSite=None` (cross-origin for Flatlogic iframe)
- Local dev: Secure cookies disabled when `USE_SQLITE=true`
- X-Frame-Options middleware disabled (required for Flatlogic preview)
- Email App Password should be in env var, not hardcoded in settings.py

## Important Context
- The owner (Konrad) is not a developer — explain changes clearly and avoid unnecessary complexity
- This system handles real payroll for field workers — accuracy is critical
- `render_to_pdf()` uses lazy import of WeasyPrint to prevent app crash if library missing; on Windows it also auto-registers the GTK3 runtime's DLL directory so `ctypes.find_library()` can locate `gobject-2.0-0` (Python 3.8+ requires explicit `os.add_dll_directory()`)
- Django admin is available at `/admin/` with full model registration and search/filter