Adds a new CLAUDE.md section documenting the display/DB gap that Path A of the UX Polish Pass creates: user sees 'Loan' / 'Advance' / 'Advance Repaid' while DB stores 'New Loan' / 'Advance Payment' / 'Advance Repayment'. Includes a lookup table, the rule for when to use which (DB for logic, display for templates), and the failure symptom so future Claude sessions don't chase ghost filters. Ships BEFORE the rename so the doc is searchable from minute one. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
806 lines
61 KiB
Markdown
806 lines
61 KiB
Markdown
# FoxFitt LabourPay v5
|
||
|
||
## Coding Style
|
||
- Always add clear section header comments using the format: # === SECTION NAME ===
|
||
- Add plain English comments explaining what complex logic does
|
||
- The project owner is not a programmer — comments should be understandable by a non-technical person
|
||
- When creating or editing code, maintain the existing comment structure
|
||
- **Django template comments `{# ... #}` are SINGLE-LINE only.** Multi-line blocks need `{% comment %}...{% endcomment %}`. A `{#` on line N with no closing `#}` on the same line renders the whole block as literal text onto the page (and silently — no error). This bit us 4× during the Adjustments feature. Also: the literal tokens `{#` and `#}` cannot appear inside a `{% comment %}` block — they'll be parsed as a nested comment marker. Rephrase meta-notes about comment syntax OUTSIDE the block.
|
||
- **Duplicate `id=""` attributes cause silent bugs.** `document.getElementById()` returns only the FIRST match in DOM order, so adding a second element with an existing id silently steals the handler from the original. Grep the template before assigning any new id (caught `adjSelectAll` collision in Task 6 — header checkbox stole the Add-Adjustment modal's Select-All handler).
|
||
|
||
## Project Overview
|
||
Django payroll management system for FoxFitt Construction, a civil works contractor specializing in solar farm foundation installations. Manages field worker attendance, payroll processing, employee loans, and business expenses for solar farm projects.
|
||
|
||
This is v5 — a fresh export from Flatlogic/AppWizzy, rebuilt from the v2 codebase with simplified models and cleaner structure.
|
||
|
||
## Tech Stack
|
||
- Django 5.2.7, Python 3.13, MySQL (production on Flatlogic Cloud Run) / SQLite (local dev)
|
||
- Bootstrap 5.3.3 (CDN), Font Awesome 6.5.1 (CDN), Google Fonts (Inter + Poppins)
|
||
- WeasyPrint for PDF generation (payroll report, payslips, receipts) — migrated from xhtml2pdf; browser-grade HTML/CSS rendering with flexbox, grid, @font-face, shadows, and proper CSS cascade
|
||
- Gmail SMTP for automated document delivery
|
||
- Hosted on Flatlogic/AppWizzy platform (Apache on Debian, e2-micro VM)
|
||
|
||
## Project Structure
|
||
```
|
||
config/ — Django project settings, URLs, WSGI/ASGI
|
||
core/ — Single main app: ALL business logic, models, views, forms, templates
|
||
context_processors.py — Injects deployment_timestamp (cache-busting), Flatlogic branding vars
|
||
forms.py — AttendanceLogForm, PayrollAdjustmentForm, ExpenseReceiptForm + formset
|
||
models.py — All 10 database models
|
||
utils.py — render_to_pdf() helper (lazy WeasyPrint import + Windows GTK3 DLL registration)
|
||
views.py — All view functions (~52 functions, ~3,800 lines) — dashboard, attendance, payroll, reports, worker/team/project CRUD
|
||
forms.py — All form classes + validators (WorkerForm, TeamForm, ProjectForm, AttendanceLogForm, PayrollAdjustmentForm, ExpenseReceiptForm, WorkerCertificate/WarningFormSet, 5MB file validator)
|
||
admin.py — Django admin registrations for all core models + WorkerCertificate/Warning inlines on Worker
|
||
templatetags/ — format_tags.py: `money` (ZAR), `money_abs` (signed callers), `type_slug` (type→CSS class), `url_replace` (swap one query-param), `dictlookup`
|
||
management/commands/ — setup_groups, setup_test_data, import_production_data
|
||
templates/
|
||
base.html — App shell (topbar + mobile menu + bottom tab bar)
|
||
core/ — Page templates: index, attendance_log, work_history, payroll_dashboard,
|
||
report, create_receipt, payslip, login
|
||
Partials: _adjustment_row.html (shared row for flat + grouped Adjustments tab)
|
||
core/workers/ — 4 templates: list, detail, edit, batch_report
|
||
core/teams/ — 4 templates: list, detail, edit, batch_report
|
||
core/projects/— 4 templates: list, detail, edit, batch_report
|
||
core/pdf/ — 4 PDF templates: report_pdf, payslip_pdf, receipt_pdf, workers_report_pdf
|
||
core/email/ — 2 HTML email templates
|
||
admin/ — base_site.html override (adds admin CSS tweaks, e.g. taller M2M pickers)
|
||
ai/ — Flatlogic AI proxy client (not used in app logic)
|
||
static/css/ — custom.css (CSS variables, component styles, tooltip overrides)
|
||
staticfiles/ — Collected static assets (Bootstrap, admin) — NOT in git (build artifact, regenerated by collectstatic)
|
||
```
|
||
|
||
## Key Models
|
||
- **UserProfile** — extends Django User (OneToOne); minimal, no extra fields in v5
|
||
- **Project** — work sites with supervisor assignments (M2M User), start/end dates, active flag
|
||
- **Worker** — profiles with salary, `daily_rate` property (monthly_salary / 20), photo, ID doc, PPE sizing (shoe, overall top, pants, tshirt), drivers license (boolean + file upload)
|
||
- **Team** — groups of workers under a supervisor, with optional pay schedule (`pay_frequency`: weekly/fortnightly/monthly, `pay_start_date`: anchor date)
|
||
- **WorkLog** — daily attendance: date, project, team, workers (M2M), supervisor, overtime, `priced_workers` (M2M)
|
||
- **PayrollRecord** — completed payments linked to WorkLogs (M2M) and Worker (FK)
|
||
- **PayrollAdjustment** — bonuses, deductions, overtime, loans; linked to project (FK, optional), worker, and optionally to a Loan or WorkLog
|
||
- **Loan** — worker loans AND advances with principal, remaining_balance, `loan_type` ('loan' or 'advance')
|
||
- **ExpenseReceipt / ExpenseLineItem** — business expense records with VAT handling
|
||
- **WorkerCertificate** — per-worker certifications (Skills, PDP, First Aid, Medical, Work at Height) with `valid_until` expiry and optional document upload. Unique per (worker, cert_type). Has `is_expired` and `expires_soon` (≤30 days) properties.
|
||
- **WorkerWarning** — disciplinary records per worker with severity (verbal/written/final), reason, optional document. Ordered -date.
|
||
|
||
### Schema name-drifts to remember
|
||
Fields / accessors that differ from what you'd guess. Each has bitten multiple
|
||
sessions; grep `core/models.py` before using any field you haven't used before:
|
||
|
||
- `PayrollAdjustment.description` — NOT `reason`
|
||
- `log.adjustments_by_work_log` (reverse accessor for PayrollAdjustment.work_log FK) — NOT `payrolladjustment_set` (the FK has `related_name` set)
|
||
- `log.overtime_amount` (DecimalField, default 0.00) — NOT `log.overtime`
|
||
- `PayrollRecord.amount_paid` (DecimalField) + `PayrollRecord.work_logs` (M2M reverse) — NOT `total_amount` / `days_worked` (easy to guess wrong when writing test fixtures)
|
||
- `Loan.principal_amount` — NOT `principal`. `Loan.save()` auto-sets `remaining_balance = principal_amount` on create, so tests rarely need to pass both.
|
||
|
||
## UI-vs-DB naming drift (Apr 2026) — READ BEFORE WRITING FORMULAS
|
||
|
||
`PayrollAdjustment.type` is DISPLAYED to users with short labels,
|
||
but the raw string stored in the database is always the long
|
||
legacy value:
|
||
|
||
| What the user SEES | What the DATABASE stores |
|
||
|---|---|
|
||
| Bonus | `'Bonus'` |
|
||
| Overtime | `'Overtime'` |
|
||
| Deduction | `'Deduction'` |
|
||
| Loan Repayment | `'Loan Repayment'` |
|
||
| Loan | `'New Loan'` ← mismatch |
|
||
| Advance | `'Advance Payment'` ← mismatch |
|
||
| Advance Repaid | `'Advance Repayment'` ← mismatch |
|
||
|
||
When writing ANY formula, filter, comparison, ORM query, test
|
||
fixture, CSS class name, or `data-type=` attribute: use the
|
||
DATABASE value (left column of the model).
|
||
|
||
- `ADDITIVE_TYPES = ['Bonus', 'Overtime', 'New Loan', 'Advance Payment']`
|
||
in `views.py` uses DB values.
|
||
- `if adj.type == 'New Loan':` checks the DB value.
|
||
- `<span class="badge-type-{{ adj.type|type_slug }}">` produces
|
||
`.badge-type-new-loan` from the DB value.
|
||
- `<tr data-type="{{ adj.type }}">` emits the DB value.
|
||
- Tests use `PayrollAdjustment.objects.create(type='New Loan', ...)`.
|
||
|
||
Only user-facing template TEXT uses the short label — via
|
||
`{{ adj.get_type_display }}`, Django's built-in choices lookup.
|
||
The label mapping lives in `PayrollAdjustment.TYPE_CHOICES`
|
||
(`core/models.py`).
|
||
|
||
**How this happened:** originally the adjustment-creation dropdown
|
||
said "New Loan" because that's what the action meant (_"log a new
|
||
loan"_). That label then propagated into every other view — tables,
|
||
badges, reports. On 24 Apr 2026 we renamed the user-visible labels
|
||
to be shorter and cleaner BUT deliberately kept the database values
|
||
untouched — to avoid breaking historic rows, tests, and hardcoded
|
||
string comparisons across ~30 source locations.
|
||
|
||
**Symptom of getting this wrong:** code that filters for
|
||
`type='Loan'` returns zero rows. Fix: use `type='New Loan'`.
|
||
|
||
## Key Business Rules
|
||
- All business logic lives in the `core/` app — do not create additional Django apps
|
||
- Workers have a `daily_rate` property: `monthly_salary / Decimal('20.00')`
|
||
- Admin = `is_staff` or `is_superuser` (checked via `is_admin(user)` helper in views.py)
|
||
- Supervisors see only their assigned projects, teams, and workers
|
||
- Admins have full access to payroll, adjustments, and resource management
|
||
- WorkLog is the central attendance record — links workers to projects on specific dates
|
||
- Attendance logging includes conflict detection (prevents double-logging same worker+date+project)
|
||
- Loans have automated repayment deductions during payroll processing
|
||
- Cascading deletes use SET_NULL for supervisors/teams to preserve historical data
|
||
|
||
## Payroll Constants
|
||
Defined at top of views.py — used in dashboard calculations and payment processing:
|
||
- **ADDITIVE_TYPES** = `['Bonus', 'Overtime', 'New Loan', 'Advance Payment']` — increase worker's net pay
|
||
- **DEDUCTIVE_TYPES** = `['Deduction', 'Loan Repayment', 'Advance Repayment']` — decrease net pay
|
||
|
||
## Django ORM gotcha — M2M filter + aggregate inflation
|
||
Chained `.filter(m2m__field=X).filter(m2m__other=Y)` creates **separate JOIN
|
||
aliases**, producing a cartesian product of rows. `.aggregate(Sum(...))` dedupes
|
||
via subquery when `distinct()` is present; `.values().annotate(Sum(...))` does
|
||
NOT — it `GROUP BY`s the inflated rows and multiplies sums by N×M (where N and
|
||
M are the counts of matching related rows). Fix pattern: use
|
||
`.filter(id__in=Model.objects.filter(m2m__field=X).values('id'))` to keep the
|
||
outer queryset JOIN-free. See `_build_report_context` in `core/views.py` and
|
||
`ReportContextFilterInflationTests` in `core/tests.py` for the reference
|
||
implementation (commit f1e246c, Apr 2026).
|
||
|
||
## Django ORM gotcha — PayrollAdjustment project double-attribution
|
||
`PayrollAdjustment` has TWO project FKs: a direct `adj.project` and an
|
||
indirect `adj.work_log.project`. For every **Overtime** adjustment these
|
||
always point at the same project (see `price_overtime()` — it sets
|
||
BOTH). When rolling up "costs per project" you typically want the
|
||
OR-union — "adjustments where either FK points to project P".
|
||
|
||
- **Correct**: `Q(project_id__in=ids) | Q(work_log__project_id__in=ids)` filter
|
||
+ `.annotate(effective_project_id=Coalesce('project_id', 'work_log__project_id'))`
|
||
+ `.values('effective_project_id', ...).annotate(total=Sum('amount'))`.
|
||
Each row contributes to exactly ONE project.
|
||
- **WRONG**: two separate filtered querysets (one per FK) summed in
|
||
Python. Any row with BOTH FKs set (every Overtime) gets counted twice.
|
||
Bit us during the Apr 2026 perf pass — Coalesce fix is commit
|
||
`167c821`. Regression test: `PayrollDashboardAdjustmentAggregationTests`
|
||
in `core/tests.py`. See `payroll_dashboard()` in `core/views.py` for
|
||
the reference implementation on both the unpaid-outstanding card and
|
||
the paid-monthly stacked chart.
|
||
|
||
## PayrollAdjustment Type Handling
|
||
- **Bonus / Deduction** — standalone, require a linked Project
|
||
- **New Loan** — creates a `Loan` record (`loan_type='loan'`); has a "Pay Immediately" checkbox (checked by default) that auto-processes the loan (creates PayrollRecord, sends payslip to Spark, marks as paid). When unchecked, the loan sits in Pending Payments for the next pay cycle. Editing syncs loan amount/balance/reason; deleting cascades to Loan + unpaid repayments
|
||
- **Advance Payment** — **auto-processed immediately** (never sits in Pending): creates `Loan` (`loan_type='advance'`), creates PayrollRecord, sends payslip email, and auto-creates an "Advance Repayment" for the next salary. Requires a Project (for cost tracking) and at least one unpaid work log (otherwise use New Loan).
|
||
- **Overtime** — links to `WorkLog` via `adj.work_log` FK; managed by `price_overtime()` view
|
||
- **Loan Repayment** — links to `Loan` (loan_type='loan') via `adj.loan` FK; loan balance changes during payment processing
|
||
- **Advance Repayment** — auto-created when an advance is paid; deducts from advance balance during `process_payment()`. If partial repayment, remaining balance converts advance to regular loan (`loan_type` changes from 'advance' to 'loan'). Editable by admin (amount can be reduced before payday).
|
||
|
||
## Outstanding Payments Logic (Dashboard)
|
||
The dashboard's outstanding amount uses **per-worker** checking, not per-log:
|
||
- For each WorkLog, get the set of `paid_worker_ids` from linked PayrollRecords
|
||
- A worker is "unpaid for this log" only if their ID is NOT in that set
|
||
- This correctly handles partially-paid logs (e.g., one worker paid, another not)
|
||
- Unpaid adjustments: additive types increase outstanding, deductive types decrease it
|
||
|
||
## Payroll dashboard query-count baselines (post Apr 2026 perf pass)
|
||
Target ranges after `payroll_dashboard()` was optimized with batched
|
||
aggregates + `Prefetch(to_attr='active_workers_cached')` + Coalesce-based
|
||
project attribution (commits `61c485f` + `167c821`):
|
||
- `/` (admin dashboard) — ~15 queries
|
||
- `/payroll/?status=pending` — ~24
|
||
- `/payroll/?status=history` — ~24
|
||
- `/payroll/?status=loans` — ~25
|
||
- `/payroll/?status=adjustments` — ~32
|
||
|
||
If any of these jumps meaningfully (>50%) after a future change, an N+1
|
||
was reintroduced. Profile with Django Debug Toolbar (see Profiling
|
||
section below) to find it. The test suite does NOT have `assertNumQueries`
|
||
guards on these views — deliberate YAGNI for now, worth adding if
|
||
regressions become a pattern.
|
||
|
||
## Commands
|
||
```bash
|
||
# Local development (SQLite)
|
||
set USE_SQLITE=true && python manage.py runserver 0.0.0.0:8000
|
||
# Or use run_dev.bat which sets the env var
|
||
|
||
python manage.py migrate # Apply database migrations
|
||
python manage.py setup_groups # Create Admin + Work Logger permission groups
|
||
python manage.py setup_test_data # Populate sample workers, projects, logs
|
||
python manage.py import_production_data # Import real production data (14 workers)
|
||
python manage.py collectstatic # Collect static files for production
|
||
python manage.py check # System check
|
||
|
||
# Run the test suite (sets env vars inline — works in Git Bash; on cmd.exe use `set` first)
|
||
USE_SQLITE=true DJANGO_DEBUG=true python manage.py test core.tests -v 2
|
||
```
|
||
|
||
## Profiling locally — Django Debug Toolbar
|
||
Installed as a dev-only dependency in `requirements.txt` since Apr 2026.
|
||
Triple-gated in `config/settings.py`: only loads when **DEBUG=true AND
|
||
USE_SQLITE=true AND NOT running tests**. Never loads in production —
|
||
prod has neither flag, and the test-run gate exists because the toolbar
|
||
emits an E001 system-check error + breaks template rendering when
|
||
DEBUG=false (which Django forces during `manage.py test`).
|
||
|
||
To profile a page: start the dev server normally (`run_dev.bat` or
|
||
inline `USE_SQLITE=true DJANGO_DEBUG=true python manage.py runserver`),
|
||
log in as admin, navigate to any URL, click the toolbar tab on the
|
||
right edge. The **SQL panel** shows query count + highlights any
|
||
duplicate-query groups — the go-to tool for N+1 hunting. See the
|
||
"Payroll dashboard query-count baselines" section for expected
|
||
numbers on hot pages.
|
||
|
||
## Development Workflow
|
||
- Active development branch: `ai-dev` (PR target: `master`)
|
||
- Local dev uses SQLite: set `USE_SQLITE=true` environment variable
|
||
- Preview server config: `.claude/launch.json` → runs `run_dev.bat`
|
||
- Admin check in views: `is_admin(request.user)` helper (top of views.py)
|
||
- "Unpaid" adjustment = `payroll_record__isnull=True` (no linked PayrollRecord)
|
||
- POST-only mutation views pattern: check `request.method != 'POST'` → redirect
|
||
- Template UI: Bootstrap 5 modals with dynamic JS, data-* attributes on clickable elements
|
||
- Atomic transactions: `process_payment()` uses `select_for_update()` to prevent duplicate payments
|
||
- Payslip Preview modal ("Worker Payment Hub"): shows earnings, adjustments, net pay, **plus** active loans/advances with inline repayment forms. Uses `refreshPreview()` JS function that re-fetches after AJAX repayment submission. Repayment POSTs to `add_repayment_ajax` which creates a PayrollAdjustment (balance deduction only happens during `process_payment()`)
|
||
- Advance Payment auto-processing: `add_adjustment` immediately creates PayrollRecord + sends payslip when an advance is created. Also auto-creates an "Advance Repayment" adjustment for the next salary cycle. Uses `_send_payslip_email()` helper (shared with `process_payment`)
|
||
- Advance-to-loan conversion: When an Advance Repayment is only partially paid, `process_payment` changes the Loan's `loan_type` from 'advance' to 'loan' so the remainder is tracked as a regular loan
|
||
- Split Payslip: Preview modal has checkboxes on work logs and adjustments (all checked by default). `process_payment()` accepts optional `selected_log_ids` / `selected_adj_ids` POST params to pay only selected items. Falls back to "pay all" if no IDs provided (backward compatible with the quick Pay button).
|
||
- Team Pay Schedules: Teams have optional `pay_frequency` + `pay_start_date` fields. `get_pay_period(team)` calculates current period boundaries by stepping forward from the anchor date. The preview modal shows a "Split at Pay Date" button that auto-unchecks items after the `cutoff_date` (end of last completed period — includes ALL overdue work, not just one period). `get_worker_active_team(worker)` returns the worker's first active team.
|
||
- Pay period calculation: `pay_start_date` is an anchor (never needs updating). Weekly=7 days, Fortnightly=14 days, Monthly=calendar month stepping. Uses `calendar.monthrange()` for month-length edge cases (no `dateutil` dependency).
|
||
- Batch Pay: "Batch Pay" button on payroll dashboard opens a modal with two radio modes — **"Until Last Paydate"** (default, splits at last completed pay period per team schedule) and **"Pay All"** (includes all unpaid items regardless of date). Preview fetches from `batch_pay_preview` with `?mode=schedule|all`. Workers without team pay schedules are skipped in schedule mode but included in Pay All mode. `batch_pay` POST endpoint processes each worker in independent atomic transactions; emails are sent after all payments complete. Uses `_process_single_payment()` shared helper (same logic as individual `process_payment`). Modal includes team filter dropdown and 3-option loan filter (All / With loans only / Without loans).
|
||
- Pending Payments Table: Shows overdue badges (red) for workers with unpaid work from completed pay periods, and loan badges (yellow) for workers with active loans/advances. Filter bar has: team dropdown, "Overdue only" checkbox, and loan dropdown (All Workers / With loans only / Without loans). Overdue detection uses `get_pay_period()` cutoff logic.
|
||
- Quick Adjust Button: Each pending payments row has an "Adjust" button (slider icon) that opens the Add Adjustment modal with that worker pre-checked and their most recent project pre-selected. The header "Add Adjustment" button resets the modal to a clean state. Uses `_quickAdjustOpen` flag to distinguish between the two open paths.
|
||
- Worker Lookup Modal: Clicking any worker name on the payroll dashboard (or using the "Worker Lookup" button) opens a modal with a comprehensive report card — amount payable, outstanding loans, paid this month/year, loans this year, recent activity (last payslip, loan, repayment, advance), active loans table, current project + days on project, PPE sizing, drivers license, and notes. Uses `worker_lookup_ajax` AJAX endpoint. Worker dropdown in modal allows switching workers without closing.
|
||
- Team & Project Management UIs: Friendlier alternatives to `/admin/core/team/` and `/admin/core/project/`. Reachable via the "Resources" dropdown in the topbar (admin only). **Team pages**: `/teams/` (list + search/filter), `/teams/<id>/` (detail with Profile/Pay Schedule/Workers/History tabs — Pay Schedule tab uses the existing `get_pay_period()` helper to show current + next 2 periods), `/teams/<id>/edit/` (single-page form for name, supervisor, pay schedule, and workers M2M). **Project pages**: `/projects/`, `/projects/<id>/` (tabs: Profile/Supervisors/Teams/Workers/History), `/projects/<id>/edit/` (form for name, description, dates, supervisors M2M). Uses `TeamForm` and `ProjectForm` from `core/forms.py` (both simple ModelForms, no inline formsets). Batch reports at `/teams/report/` and `/projects/report/` with CSV exports; PDF exports deferred as a follow-up. Dashboard "Manage Resources" card now has "Manage All Workers/Projects/Teams" footer links on each tab. Django admin remains fully functional as a fallback.
|
||
- Worker Management UI: A friendlier alternative to `/admin/core/worker/`. Reachable via the "Resources" topbar dropdown → Workers (admin-only). Pages: `/workers/` (list with search + status filter), `/workers/<id>/` (detail with Profile/Certifications/Warnings/History tabs), `/workers/<id>/edit/` or `/workers/new/` (single-page form with sections for Personal & Pay, PPE, Documents, Driver's License, plus inline formsets for certifications and warnings). Uses `WorkerForm`, `WorkerCertificateFormSet`, `WorkerWarningFormSet` from `core/forms.py`. The "+ Add Certification" / "+ Add Warning" buttons clone a `<template>` element via `content.cloneNode()` (DOM-safe, no innerHTML) and rewrite `__PREFIX__` in input names to the next formset index. File uploads validated at 5 MB max via `validate_max_5mb()` in `forms.py`. Django admin (`/admin/core/worker/`) remains fully functional as a fallback — both UIs coexist.
|
||
- Worker Batch Report: `/workers/report/` shows every worker with aggregated lifetime history — days worked, projects worked on, teams, first/last payslip dates, total paid, cert status (active/total + expired/expiring counts), warning count. Filter by status, project, team. CSV export via `/workers/report/csv/`, PDF via `/workers/report/pdf/` (landscape A4, same amber-accent typography as the payroll report). Built on the reusable `_build_worker_report_context()` helper which uses `annotate(Min/Max/Count/Sum)` + prefetch for efficient aggregation.
|
||
- Dashboard cert-expiry card: The admin dashboard shows a "Certifications Need Attention" stat card with count of expired + expiring-within-30-days certs (active workers only). Card is CONDITIONAL — renders only when count > 0, so it disappears when everything is in good standing. Clicking it goes to the worker batch report. Counts come from `index()` view adding `certs_expired_count`, `certs_expiring_count`, `certs_alert_total` to context.
|
||
- **Inline Filters on the Report page**: `/report/` has three pill-buttons (Date / Projects / Teams) in a sticky strip. Clicking a pill opens an inline popover with the editor for that filter. Popover's OK rebuilds the URL and navigates — no "dirty state", no global Apply. Date popover has a Single/Range/Custom mode toggle; Projects + Teams use Choices.js multi-select. Bidirectional project↔team cross-filter disables workers/projects that never paired in the selected date range. Context key `project_team_pairs_json` is consumed via `|json_script` (raw Python list — NEVER `json.dumps` it first; the filter does the serialisation and double-encoding silently breaks `.forEach(...)`). Deleted the old `_report_config_modal.html`; Dashboard "Generate Report" button is now a plain link with the current month pre-filled.
|
||
- **Adjustments tab** (`/payroll/?status=adjustments`): the 4th payroll-dashboard tab, next to Pending / History / Loans. Browse every `PayrollAdjustment` across all workers with 5 pill-style filters (Type / Workers / Teams / Status / Date — all popover-checkbox/radio, no native `<select>`s). Semantic colour badges per type: 7 types × 2 themes = 14 `--badge-*-bg/fg` CSS tokens, with `+15%` saturation siblings for `Loan Repayment` / `Advance Repayment` so "money coming back" reads as hotter than "money going out". Group-by toggle (Flat / By Type / By Worker) with collapsible sections; By-Type headers get a 4px left-border in the matching badge colour via `[data-type="..."]` attribute selectors. Sortable column headers (Date / Worker / Amount / Status) toggle `?sort=X&order=asc|desc`. Bulk-delete via row checkboxes + floating action bar → `POST /payroll/adjustments/bulk-delete/`. Row actions reuse existing modals (Worker Lookup, Preview Payslip, Edit adjustment) — clicking a worker name or paid-row eye icon opens a modal rather than navigating away; project name links to `/projects/<id>/#history` (a tiny hash-based tab-activation helper in `projects/detail.html` activates whichever tab matches the URL hash).
|
||
- **Adjustment cascade helper**: `_delete_adjustment_with_cascade(adj)` in `core/views.py` is the single authority for "delete this adjustment, cleaning up linked objects". Returns `(ok: bool, reason: str|None)` — `reason` is `'paid'` or `'has_paid_repayments'`. For `New Loan`/`Advance Payment` it deletes the linked `Loan` + any still-unpaid repayment adjustments (aborts if any are already paid). For `Overtime` it removes the worker from `work_log.priced_workers`. Both `delete_adjustment` (single-row) AND `bulk_delete_adjustments` delegate to this helper, so bulk and single-row have identical semantics. Without it, bulk-delete was silently orphaning `Loan` rows (was a critical bug caught in code review — see `test_bulk_delete_cascades_new_loan`).
|
||
- **Pill-popover filter pattern** (used by both Inline Filters on `/report/` AND the Adjustments tab's 5 filters): each filter is a `.filter-pill-wrap` containing a `<button class="filter-pill filter-pill--editable">` + a hidden `<div class="filter-popover">`. Committed state lives in hidden `<input>`s inside a `.adj-hidden-inputs[data-adj-filter="X"]` container (rewritten on popover OK via `replaceChildren()` + `createElement` — XSS-safe, no `innerHTML`). Popover OK commits + closes; Cancel/Esc/click-outside revert + close. Reusable CSS classes live in `static/css/custom.css` under `/* === Inline Filters === */`.
|
||
|
||
## URL Routes
|
||
| Path | View | Purpose |
|
||
|------|------|---------|
|
||
| `/` | `index` | Dashboard (admin stats / supervisor work view) |
|
||
| `/attendance/log/` | `attendance_log` | Log daily work with date range support |
|
||
| `/history/` | `work_history` | Work logs table with filters |
|
||
| `/history/export/` | `export_work_log_csv` | Download filtered logs as CSV |
|
||
| `/workers/export/` | `export_workers_csv` | Admin: export all workers to CSV |
|
||
| `/workers/` | `worker_list` | Admin: friendly worker list with search + status filter |
|
||
| `/workers/new/` | `worker_edit` | Admin: blank worker-create form |
|
||
| `/workers/<id>/` | `worker_detail` | Admin: worker profile with profile/certs/warnings/history tabs |
|
||
| `/workers/<id>/edit/` | `worker_edit` | Admin: edit worker + inline cert/warning formsets |
|
||
| `/workers/report/` | `worker_batch_report` | Admin: aggregated roster report (days, projects, payslips, certs) |
|
||
| `/workers/report/csv/` | `worker_batch_report_csv` | Admin: batch report as CSV download |
|
||
| `/workers/report/pdf/` | `worker_batch_report_pdf` | Admin: batch report as PDF download |
|
||
| `/teams/` | `team_list` | Admin: friendly team list with search + status filter |
|
||
| `/teams/new/` | `team_edit` | Admin: blank team-create form |
|
||
| `/teams/<id>/` | `team_detail` | Admin: team profile with profile/pay schedule/workers/history tabs |
|
||
| `/teams/<id>/edit/` | `team_edit` | Admin: edit team (name, supervisor, pay schedule, workers M2M) |
|
||
| `/teams/report/` | `team_batch_report` | Admin: aggregated team report (HTML) |
|
||
| `/teams/report/csv/` | `team_batch_report_csv` | Admin: team batch report as CSV download |
|
||
| `/projects/` | `project_list` | Admin: friendly project list with search + status filter |
|
||
| `/projects/new/` | `project_edit` | Admin: blank project-create form |
|
||
| `/projects/<id>/` | `project_detail` | Admin: project profile with profile/supervisors/teams/workers/history tabs |
|
||
| `/projects/<id>/edit/` | `project_edit` | Admin: edit project (name, description, dates, supervisors M2M) |
|
||
| `/projects/report/` | `project_batch_report` | Admin: aggregated project report (HTML) |
|
||
| `/projects/report/csv/` | `project_batch_report_csv` | Admin: project batch report as CSV download |
|
||
| `/toggle/<model>/<id>/` | `toggle_active` | Admin: AJAX toggle active status |
|
||
| `/payroll/` | `payroll_dashboard` | Admin: pending payments, loans, charts |
|
||
| `/payroll/?status=adjustments` | `payroll_dashboard` | Admin: browse ALL payroll adjustments (filter by type, worker, team, status, date; group-by type/worker; bulk-delete unpaid; row actions open existing modals) |
|
||
| `/payroll/adjustments/bulk-delete/` | `bulk_delete_adjustments` | Admin: POST-only; delete multiple unpaid adjustments in one shot via fetch() with X-CSRFToken cookie |
|
||
| `/payroll/pay/<worker_id>/` | `process_payment` | Admin: process payment (atomic) |
|
||
| `/payroll/price-overtime/` | `price_overtime` | Admin: AJAX price unpriced OT entries |
|
||
| `/payroll/adjustment/add/` | `add_adjustment` | Admin: create adjustment |
|
||
| `/payroll/adjustment/<id>/edit/` | `edit_adjustment` | Admin: edit unpaid adjustment |
|
||
| `/payroll/adjustment/<id>/delete/` | `delete_adjustment` | Admin: delete unpaid adjustment |
|
||
| `/payroll/preview/<worker_id>/` | `preview_payslip` | Admin: AJAX JSON payslip preview (includes active loans) |
|
||
| `/payroll/worker-lookup/<worker_id>/` | `worker_lookup_ajax` | Admin: AJAX JSON worker report card |
|
||
| `/payroll/repayment/<worker_id>/` | `add_repayment_ajax` | Admin: AJAX add loan/advance repayment from preview |
|
||
| `/payroll/payslip/<pk>/` | `payslip_detail` | Admin: view completed payslip |
|
||
| `/receipts/create/` | `create_receipt` | Staff: expense receipt with line items |
|
||
| `/import-data/` | `import_data` | Setup: run import command from browser |
|
||
| `/payroll/batch-pay/preview/` | `batch_pay_preview` | Admin: AJAX JSON batch pay preview (`?mode=schedule\|all`) |
|
||
| `/payroll/batch-pay/` | `batch_pay` | Admin: POST process batch payments for multiple workers |
|
||
| `/run-migrate/` | `run_migrate` | Setup: run pending DB migrations from browser |
|
||
|
||
## Frontend Design Conventions
|
||
- **Dual-theme** (dark + light) driven by a single CSS variable set in `static/css/custom.css`.
|
||
The theme is dark-first; the light theme is a set of var overrides inside a `[data-theme="light"]` block.
|
||
A sun/moon toggle in the topbar flips the `data-theme` attribute on `<html>` and persists the choice to localStorage.
|
||
- **CSS variables** in `static/css/custom.css` `:root` — always use `var(--name)`:
|
||
- `--accent: #e8851a` (warm orange/amber, brand), `--accent-hover: #f59e0b`
|
||
- `--primary-dark: #0f172a`, `--primary: #1e293b`
|
||
- `--bg-card: #161921`, `--bg-card-hover: #1c2029` (elevated surfaces)
|
||
- `--text-primary: #d8d8d8` (dark theme), `--text-secondary`, `--text-tertiary`
|
||
- Light-theme overrides flip backgrounds to white/grey and accent to `#d97706`
|
||
- **Icons**: Font Awesome 6 only (`fas fa-*`). Do NOT use Bootstrap Icons (`bi bi-*`)
|
||
- **CTA buttons**: `btn-accent` (orange) for primary actions. `btn-primary` (dark slate) for modal Save/Submit
|
||
- **Page titles**: `{% block title %}Page Name | FoxFitt{% endblock %}`
|
||
- **Fonts**: Inter (body) + Poppins (headings) loaded in base.html via Google Fonts CDN
|
||
- **Cards**: Borderless with subtle shadow. Stat cards have coloured accent bars on the left.
|
||
- **Bootstrap tooltips**: Global init in `base.html` — any element with
|
||
`data-bs-toggle="tooltip" title="..."` gets a tooltip automatically. Tooltips are themed
|
||
via custom `--bs-tooltip-bg`/`--bs-tooltip-color` overrides in custom.css so they're
|
||
readable in both light and dark modes (otherwise Bootstrap's default picks the wrong
|
||
pair of body vars for dark mode).
|
||
- **Email/PDF payslips**: Worker name dominant — do NOT add prominent FoxFitt branding
|
||
|
||
## Static Assets & Cache-Busting (Cloudflare is in front)
|
||
|
||
Production traffic reaches the Flatlogic VM through **Cloudflare** (response headers
|
||
include `cf-ray`, `cf-cache-status`, and a `cache-control: max-age=14400`). Static
|
||
assets — including `custom.css` — are cached at Cloudflare's edge for up to 4 hours
|
||
per unique URL. This is great for performance and bad for deploys if the URL doesn't
|
||
change when the file does.
|
||
|
||
### How cache-busting works now
|
||
`base.html` loads CSS as:
|
||
|
||
```html
|
||
<link rel="stylesheet" href="{% static 'css/custom.css' %}?v={{ deployment_timestamp|default:'1' }}">
|
||
```
|
||
|
||
`deployment_timestamp` comes from `core/context_processors.py::project_context`
|
||
via `_compute_cache_bust_token()` — returns the mtime of
|
||
`static/css/custom.css` as an integer. The token only changes when the
|
||
CSS file is modified, so Cloudflare's edge cache holds each version for
|
||
its full 4h TTL and repeat page loads in a session hit the browser
|
||
cache (304 Not Modified). Deploys that include a CSS change bump the
|
||
mtime → new token → cache busts. Pre-24-Apr-2026 this was
|
||
`int(time.time())` per-request, which defeated the CDN cache entirely
|
||
(effectively 0% hit rate on CSS). Degraded-mode fallback: if
|
||
`custom.css` isn't on disk (e.g., fresh container before
|
||
`collectstatic`), the function falls back to the old per-request
|
||
timestamp rather than crashing.
|
||
|
||
### The pitfall this replaced
|
||
Pre-Apr 2026, the template used `{{ request.timestamp|default:'1.0' }}`. But
|
||
`request.timestamp` is **not** a Django request attribute — the variable always
|
||
fell back to the literal `'1.0'`. Every deploy's CSS URL resolved to the same
|
||
`custom.css?v=1.0`, so Cloudflare held onto a pre-redesign copy for hours while
|
||
the VM served the new one. Symptom was "the deploy worked but the page looks wrong"
|
||
that only a hard refresh in incognito temporarily fixed. Never use `request.timestamp`
|
||
in templates — it doesn't exist.
|
||
|
||
### When CSS changes don't appear on production
|
||
1. Confirm Django is rendering a stable URL: `curl -s https://foxlog.flatlogic.app/ | grep -oE 'custom\.css\?v=[^"]+'` — run it twice; the `v=` number must be IDENTICAL across requests. Under the mtime-based token (see previous subsection), the number only changes after `static/css/custom.css` is edited. If it DOES change every request, the fallback branch of `_compute_cache_bust_token()` is active (the CSS file couldn't be stat'd) — check the file exists and is readable.
|
||
2. Confirm the CDN honours it: `curl -sI "https://foxlog.flatlogic.app/static/css/custom.css?cb=test$(date +%s)" | grep -i cache` — expect `cf-cache-status: MISS` then `HIT` on repeat
|
||
3. If you've just deployed a CSS change and the old file is still showing: confirm `collectstatic` ran on the VM after pull (Flatlogic doesn't auto-run it). The token is read from `static/css/custom.css` (the SOURCE file) — so editing the source and pushing DOES bump the token, and Cloudflare correctly misses its edge cache on the next request. What breaks is the VM's response to that miss: Apache serves the OLD bytes from `staticfiles/` because `collectstatic` hasn't refreshed the collected copy. You'll see a URL with a new `?v=...` value (confirming the token bumped) but stale bytes behind it. Fix: `python3 manage.py collectstatic --noinput`, then restart the service.
|
||
4. If `deployment_timestamp` isn't being injected at all (the `?v=` query string is missing from the rendered URL): check that `core.context_processors.project_context` is listed in `TEMPLATES[0]['OPTIONS']['context_processors']` in `config/settings.py`.
|
||
|
||
### `collectstatic` is required after CSS/JS changes on production
|
||
Flatlogic's rebuild does NOT automatically run `collectstatic`. If new CSS is on
|
||
disk but the VM's `staticfiles/` hasn't been refreshed, Apache serves the old
|
||
collected copy. Have Gemini run `python3 manage.py collectstatic --noinput`
|
||
after any PR that touches `static/`.
|
||
|
||
### `staticfiles/` is NOT tracked in git (since Apr 2026)
|
||
`staticfiles/` is gitignored because it's a **build artifact** — `collectstatic`
|
||
regenerates it from `static/` and each installed-app's static dirs on every
|
||
deploy. Previously it was tracked, which caused two problems:
|
||
|
||
1. **Flatlogic-auto-noise commits.** Every time Gemini ran `collectstatic`,
|
||
Flatlogic's web UI detected the modified files in `staticfiles/` and
|
||
auto-committed them with a generic `Ver XX.YY` message (e.g. the notorious
|
||
"Ver 30.04 Fix reports and add Supervisor" commit that diverged our branch
|
||
from GitHub in Apr 2026). These commits reached gitea but not GitHub,
|
||
creating recurring reconciliation pain.
|
||
2. **Misleading diffs in PRs.** Every CSS change showed up twice in git
|
||
(once in `static/`, once in `staticfiles/`), doubling review surface.
|
||
|
||
**Deploy consequence**: after pulling a commit that modifies `static/` files,
|
||
Gemini MUST run `collectstatic` BEFORE restarting the service. If the pull
|
||
removed the `staticfiles/` directory from the working tree (which happens
|
||
the first time after the gitignore change), collectstatic recreates everything
|
||
from source. Brief window (~seconds) of possible 404s on static assets during
|
||
the deploy; acceptable at this scale.
|
||
|
||
**First-time migration note**: the commit that added `staticfiles/` to
|
||
`.gitignore` also removed all previously-tracked files via
|
||
`git rm -r --cached staticfiles/`. The VM's filesystem still holds the
|
||
directory — git just stops tracking it. On next deploy the pull will
|
||
delete the working-tree copies (because they no longer exist in the
|
||
commit tree), so `collectstatic --noinput` MUST run immediately after
|
||
pull to repopulate. After that, all is stable.
|
||
|
||
## PDF Generation (WeasyPrint)
|
||
Migrated from xhtml2pdf in Nov 2026. WeasyPrint is a browser-grade HTML→PDF renderer — it supports real CSS (flexbox, grid, `@font-face`, shadows, `border-radius`, proper cascade) that xhtml2pdf could not handle.
|
||
|
||
### Files
|
||
- `core/utils.py` — `render_to_pdf(template_src, context_dict)` is the single entry point; lazy-imports WeasyPrint, returns PDF bytes or `None` on failure
|
||
- `core/templates/core/pdf/report_pdf.html` — payroll report (complex layout, 600+ lines)
|
||
- `core/templates/core/pdf/payslip_pdf.html` — payslip (emailed to Spark Receipt after each payment)
|
||
- `core/templates/core/pdf/receipt_pdf.html` — expense receipt (emailed to Spark Receipt after each expense entry)
|
||
- `core/templatetags/format_tags.py::money` — South African space-separated currency formatting (`R 64 939.00`); use this instead of `floatformat:2`
|
||
|
||
### Callers
|
||
- `generate_report_pdf()` — downloads the report PDF to the browser
|
||
- `_send_payslip_email()` — attaches payslip PDF to Gmail SMTP email (called by `process_payment`, `add_adjustment` advance path, `batch_pay`)
|
||
- `create_receipt()` — attaches receipt PDF to Gmail SMTP email
|
||
- All three use the same `EmailMultiAlternatives.attach(filename, pdf_bytes, "application/pdf")` pattern — engine-agnostic
|
||
|
||
### Dependencies
|
||
- **Python package**: `weasyprint==68.1` (pinned in `requirements.txt`)
|
||
- **System libraries** (Pango, Cairo, GDK-PixBuf, FFI, shared-mime-info):
|
||
- **Flatlogic/production (Debian)**: already installed on the platform image — confirmed via Flatlogic's Gemini
|
||
- **Windows local dev**: install the GTK3 runtime via `winget install -e --id tschoonj.GTKForWindows --accept-package-agreements --accept-source-agreements --silent` (installs `C:\Program Files\GTK3-Runtime Win64\`)
|
||
- **macOS local dev**: `brew install pango` (not currently used but documented for completeness)
|
||
|
||
### Windows DLL resolution quirk
|
||
Since Python 3.8, native DLLs are *not* loaded from PATH automatically — an explicit `os.add_dll_directory()` call is required. The `_ensure_gtk_on_windows()` helper in `core/utils.py` handles this automatically: it checks common GTK3 install paths on module load and registers the first one found. No-op on Linux/macOS.
|
||
|
||
If the report page returns "PDF generation failed", check the Django log for the underlying error:
|
||
- `cannot load library 'gobject-2.0-0'` → GTK3 runtime not installed (run the winget command above)
|
||
- `'super' object has no attribute 'transform'` → `weasyprint`/`pydyf` version mismatch; reinstall with `pip install --upgrade weasyprint==68.1`
|
||
|
||
### Template conventions
|
||
- **Modern CSS is fine** — flexbox (`display: flex`), grid (`display: grid; gap: 20pt`), `@font-face`, `box-shadow`, `border-radius` all render correctly
|
||
- **Fonts**: WeasyPrint can load web fonts. If we ever add `@font-face` blocks pointing to `static/fonts/Inter-*.ttf` and `Poppins-*.ttf`, the PDFs can use the same typography as the web app (currently the PDFs use Helvetica by default — upgrading to Inter/Poppins is optional follow-up work)
|
||
- **Page setup**: `@page { size: a4 portrait; margin: 2cm 1.8cm 1.6cm 1.8cm; }` — standard A4 with generous margins
|
||
- **`base_url`**: `render_to_pdf()` passes `settings.STATIC_ROOT or "."` as `base_url` so relative paths in `<img src="...">` and `@font-face src: url(...)` resolve against the collected static dir
|
||
|
||
### Known lint footguns (legacy from xhtml2pdf era)
|
||
- `report_pdf.html` still uses invisible `<table class="cols">` elements for two-column layout — these work fine under WeasyPrint but could be simplified to `display: grid; grid-template-columns: 1fr 1fr; gap: 20pt` as a future cleanup pass
|
||
- The `period-detail td { padding-top: 3pt; padding-bottom: 3pt; }` split-padding workaround (from a shorthand-collision bug with xhtml2pdf) is no longer needed; safe to replace with the `padding` shorthand when cleaning up the template
|
||
|
||
## Users, Roles & Permissions
|
||
|
||
Understanding who-can-do-what in this app requires grasping **three separate layers**
|
||
of Django auth that stack on top of each other:
|
||
|
||
### Layer 1 — Django's three built-in user flags
|
||
|
||
These live on the `auth.User` model and are the foundation. Every user has exactly one
|
||
combination of these three flags:
|
||
|
||
| Flag | What it means | Who should have it |
|
||
|---|---|---|
|
||
| `is_superuser=True` | **Bypasses every permission check.** Full access to everything everywhere, including Django admin. Created by `createsuperuser`. | Konrad (the owner), and one emergency-access account. That's it. |
|
||
| `is_staff=True` | Can log into `/admin/` (the built-in Django admin interface) and sees it. Does NOT grant any model permissions by itself — those come from groups or per-user permissions. | Konrad, and any "office admin" people who need full access to edit data via Django admin. Usually combined with `is_superuser` in this app. |
|
||
| (neither) | Regular user. Can log into the friendly app at `/` but cannot enter Django admin. Sees only what they've been explicitly given access to via group membership or supervisor assignments. | Site supervisors (Work Loggers), and any future non-admin roles. |
|
||
|
||
Key mental model: **`is_superuser` beats everything**. A superuser's permission groups
|
||
and assignments don't matter — they always see everything. Use superuser sparingly
|
||
so that regular permission paths get exercised and tested.
|
||
|
||
### Layer 2 — The two app-specific permission groups
|
||
|
||
Created by `python manage.py setup_groups` (a one-time command, safe to re-run; it
|
||
updates existing groups rather than duplicating them). Assignment happens in
|
||
`/admin/auth/group/` → add users to groups:
|
||
|
||
**Admin group** — grants every Django model permission (add/change/delete/view) on
|
||
all 10 core models (Worker, Project, Team, WorkLog, PayrollRecord, Loan,
|
||
PayrollAdjustment, ExpenseReceipt, ExpenseLineItem, and the new WorkerCertificate/
|
||
WorkerWarning via separate admin registration).
|
||
|
||
- **Practical effect**: only matters for `is_staff=True` users who are NOT
|
||
superusers. For them, the Admin group is what lets them actually use Django
|
||
admin (without it, they can log into /admin/ but see empty lists).
|
||
- For superusers, the Admin group is redundant (they bypass permissions anyway).
|
||
- For non-staff users, the Admin group is pointless (they can't reach Django
|
||
admin at all).
|
||
|
||
**Work Logger group** — grants: add/change/view `WorkLog`; view-only on
|
||
`Project`, `Worker`, `Team`. Notably does NOT grant any Payroll permissions.
|
||
|
||
- **Practical effect**: this group is the app's signal for "this user is a site
|
||
supervisor". The `is_supervisor()` helper in `views.py` explicitly checks for
|
||
membership in this group — so adding someone here marks them as a supervisor,
|
||
even if they don't own any teams or projects yet.
|
||
- Work Loggers typically have `is_staff=False` (no Django admin access). They
|
||
use the friendly app UI at `/attendance/log/`, `/history/`, and the dashboard.
|
||
|
||
### Layer 3 — Implicit supervisor roles via model relationships
|
||
|
||
Two model fields independently grant "supervisor-ness" even without group membership:
|
||
|
||
- `Team.supervisor` (ForeignKey → User) — whoever this points to is a supervisor
|
||
of that team. Set on the Team edit page or `/admin/core/team/`.
|
||
- `Project.supervisors` (ManyToManyField → User) — every user in this M2M is a
|
||
supervisor of the project. Set on the Project edit page or
|
||
`/admin/core/project/`.
|
||
|
||
The `is_supervisor()` helper treats any ONE of these as sufficient:
|
||
|
||
```python
|
||
def is_supervisor(user):
|
||
return (
|
||
user.supervised_teams.exists() # Team.supervisor FK reverse
|
||
or user.assigned_projects.exists() # Project.supervisors M2M reverse
|
||
or user.groups.filter(name='Work Logger').exists()
|
||
)
|
||
```
|
||
|
||
So a user can become a supervisor via **any** of: Work Logger group, assigned to
|
||
a Team as supervisor, or added to a Project's supervisors M2M.
|
||
|
||
### The three permission-check helpers in `core/views.py`
|
||
|
||
All three are defined near the top of `views.py` (around line 47–67):
|
||
|
||
| Helper | Returns True if… | Used by |
|
||
|---|---|---|
|
||
| `is_admin(user)` | `is_staff=True` OR `is_superuser=True` | Every admin-only view (payroll, reports, worker/team/project management, CSV exports) |
|
||
| `is_supervisor(user)` | Supervises a team OR has assigned projects OR is in Work Logger group | Attendance logging, history page filtering |
|
||
| `is_staff_or_supervisor(user)` | `is_admin` OR `is_supervisor` | Views accessible to both tiers (dashboard shows different content per tier) |
|
||
|
||
**Critical**: `is_admin()` does NOT check for the "Admin" group. It checks the
|
||
Django `is_staff`/`is_superuser` flags. A user can be in the "Admin" permission
|
||
group but NOT be an admin as far as the app is concerned, and vice versa. The
|
||
group controls Django-admin model permissions; the flags control everything else.
|
||
|
||
### How views enforce permissions
|
||
|
||
1. **`@login_required`** is on every view except `import_data()` and
|
||
`run_migrate()` (temporary setup endpoints).
|
||
2. **Admin-only views** call `is_admin(request.user)` at the top and return
|
||
`HttpResponseForbidden("Admin access required.")` if false. Examples:
|
||
everything under `/payroll/`, `/workers/*`, `/teams/*`, `/projects/*`,
|
||
`/report/*`, `/workers/export/`.
|
||
3. **Supervisor-scoped data** uses `is_supervisor()` to gate access, then filters
|
||
querysets by the user's `supervised_teams` / `assigned_projects`:
|
||
- `work_history` — supervisors see only logs for their teams/projects
|
||
- `AttendanceLogForm` — pre-filters `project` and `team` dropdowns by what
|
||
the user can see; `workers` field is filtered by team membership
|
||
4. **Permission cascading** — a supervisor of a Team automatically "supervises"
|
||
every worker in that team, and every project that team has worked on.
|
||
This is implicit — there's no per-worker permission.
|
||
|
||
### The "Resources" dropdown supervisor picker
|
||
|
||
When editing a Team or Project via the friendly UI (`/teams/<id>/edit/` or
|
||
`/projects/<id>/edit/`), the Supervisor/Supervisors picker uses
|
||
`_supervisor_user_queryset()` in `core/forms.py`:
|
||
|
||
```python
|
||
User.objects.filter(is_active=True).order_by('username')
|
||
```
|
||
|
||
Any active user can be picked. The picker is deliberately NOT pre-filtered
|
||
by group/staff flags because `is_supervisor(user)` (views.py) grants
|
||
supervisor powers to anyone assigned to a team/project FK/M2M — so the
|
||
picker shouldn't be stricter than the permission model. Pre-Apr 2026 the
|
||
picker required Work Logger group membership, which hid valid supervisors
|
||
(see commit 0ceceeb for the fix + regression tests). Deactivated accounts
|
||
are still hidden.
|
||
|
||
### Typical user setups
|
||
|
||
| User | is_superuser | is_staff | Groups | Supervised Teams/Projects | What they can do |
|
||
|---|---|---|---|---|---|
|
||
| **admin** (Konrad) | ✓ | ✓ | — | — | Everything. Bypasses all checks. |
|
||
| **testadmin** (Flatlogic) | ✓ | ✓ | — | — | Same as above. |
|
||
| **eendman** | — | — | Work Logger | *(usually also supervises a team)* | Log work for assigned teams/projects, see their history. Cannot enter Django admin. Cannot view payroll or worker salary data via app UI. |
|
||
| **Office data-entry staff** (future) | — | ✓ | Admin | — | Can enter Django admin and CRUD all core models there. Does NOT see the payroll dashboard or worker salary UI (because `is_admin()` helper still returns True via `is_staff`, so actually yes they CAN see payroll — see note below). |
|
||
| **Inactive terminated employee** | — | — | — | — | Cannot log in if `is_active=False`. |
|
||
|
||
**Note on is_staff + payroll access**: `is_admin()` returns True for any
|
||
`is_staff` user. That means if you create an office-admin user with
|
||
`is_staff=True` but no superuser flag, they WILL see payroll and salary data
|
||
via the friendly UI. If you need a "Django admin only, no payroll UI" role,
|
||
we'd have to add a separate flag or group check — not currently supported.
|
||
|
||
### How to add a new supervisor (step by step)
|
||
|
||
1. Go to `/admin/auth/user/add/` and create the user with a username and
|
||
password. **Uncheck "Staff status"** on the initial form (they don't need
|
||
Django admin access).
|
||
2. (Optional) Add them to the **Work Logger** group if you want
|
||
`is_supervisor(user)` to return True even without a team/project
|
||
assignment. Not required for the picker to show them — the picker
|
||
shows any active user (see commit 0ceceeb, Apr 2026).
|
||
3. (Optional) Assign them as the supervisor of one or more teams via
|
||
`/teams/<id>/edit/` (Supervisor dropdown — they'll appear in the list
|
||
because they're active).
|
||
4. (Optional) Add them to one or more projects via `/projects/<id>/edit/`
|
||
(Supervisors M2M checklist).
|
||
5. They can now log in at `/accounts/login/` and will land on the Dashboard
|
||
with a supervisor view — their teams + projects only.
|
||
|
||
### Common misconceptions (read these)
|
||
|
||
- **"Admin" group ≠ Django admin access.** Django admin access requires
|
||
`is_staff=True`. The "Admin" group is just a permission bundle.
|
||
- **Work Logger doesn't need to supervise a team to be "a supervisor".**
|
||
Group membership alone satisfies `is_supervisor()`.
|
||
- **Superuser bypasses the "Admin" group.** They don't need it.
|
||
- **Deactivating a user** (`is_active=False`) blocks login entirely; their
|
||
team/project assignments remain in the DB for audit purposes but stop
|
||
having any effect.
|
||
- **The `supervisor` field on WorkLog is historical, not authoritative.** It
|
||
records who logged the work that day — not who currently supervises the
|
||
workers.
|
||
|
||
## Authentication
|
||
|
||
- Django's built-in auth (`django.contrib.auth`)
|
||
- Login: `/accounts/login/` → redirects to `/` (home)
|
||
- Logout: POST to `/accounts/logout/` → redirects to login
|
||
- All views use `@login_required` except `import_data()` and `run_migrate()`
|
||
- No PIN auth in v5 (simplified from v2)
|
||
- Passwords: Django's default PBKDF2 hashing, no custom password policy
|
||
- Sessions: cookie-based, server-side session store (default Django)
|
||
|
||
## Django Admin Customisation
|
||
|
||
The `/admin/` interface is Django's built-in admin with two targeted customisations:
|
||
|
||
### Model registrations (`core/admin.py`)
|
||
- Every core model is registered with `list_display`, `list_filter`, `search_fields`, and
|
||
(where relevant) `filter_horizontal` / `filter_vertical` for M2M pickers.
|
||
- `WorkerAdmin` has `WorkerCertificateInline` + `WorkerWarningInline` so you can edit a
|
||
worker's certs and warnings inline on the worker change page.
|
||
- `WorkerCertificateAdmin` + `WorkerWarningAdmin` are also standalone (useful for
|
||
"show me all expiring certs across all workers" type queries via list_filter).
|
||
|
||
### Template override — `core/templates/admin/base_site.html`
|
||
Extends `admin/base.html` and injects a small `<style>` block into every admin page.
|
||
Currently used for:
|
||
- Taller `FilteredSelectMultiple` widgets (the "Choose" / "Available" boxes used by
|
||
`filter_horizontal` on Groups, WorkLogs etc.) — default Django height is ~16em which
|
||
is too short for long permission lists; we set 30em (40em on tall screens).
|
||
|
||
Add more admin-only CSS tweaks inside that `<style>` block rather than polluting
|
||
`static/css/custom.css`.
|
||
|
||
### Why the override works — `TEMPLATES.DIRS` setting
|
||
Django's template loader tries `TEMPLATES[0].DIRS` before the app-dirs loader. Since
|
||
`django.contrib.admin` comes before `core` in `INSTALLED_APPS` (the standard order),
|
||
its `admin/base_site.html` would normally win. Adding `BASE_DIR / 'core' / 'templates'`
|
||
to `TEMPLATES[0]['DIRS']` in `config/settings.py` makes our override take priority
|
||
without reordering `INSTALLED_APPS` (which would risk subtle side effects on signals,
|
||
migrations, and admin URL registration).
|
||
|
||
## Backup & Restore (production safety net)
|
||
|
||
Flatlogic doesn't expose MySQL directly (no SSH, no `mysqldump`, no DB console).
|
||
Instead, the app ships two management commands + two admin-only browser URLs
|
||
that back up and restore every row via Django's ORM — platform-independent,
|
||
works anywhere Django does.
|
||
|
||
### Making a backup (before any risky deploy)
|
||
|
||
**From the browser (production)**:
|
||
1. Log in as admin
|
||
2. Visit `/backup-data/`
|
||
3. Browser downloads `foxlog_backup_<timestamp>.json` to your laptop
|
||
4. Move that file somewhere safe (Google Drive, local disk — NOT the repo)
|
||
|
||
**From the command line (local dev or SSH-able host)**:
|
||
```
|
||
python manage.py backup_data # → backups/foxlog_<timestamp>.json
|
||
python manage.py backup_data --output=custom.json # → custom.json
|
||
```
|
||
|
||
Either method produces an identical JSON file covering:
|
||
- All auth tables: User, Group, Permission, ContentType (so accounts restore correctly)
|
||
- UserProfile, Project, Worker, Team, WorkLog, PayrollRecord, Loan, PayrollAdjustment,
|
||
ExpenseReceipt, ExpenseLineItem, WorkerCertificate, WorkerWarning
|
||
|
||
File size is roughly 1 KB per row (14 workers / 100 work logs → ~90 KB).
|
||
|
||
### Restoring a backup
|
||
|
||
**From the browser**:
|
||
1. Log in as admin
|
||
2. Visit `/restore-data/`
|
||
3. Upload your `.json` file
|
||
4. Tick "Yes, I understand" checkbox
|
||
5. Click Restore — runs inside a database transaction, all-or-nothing
|
||
|
||
**Behaviour**:
|
||
- Rows with matching primary key are **UPDATED** (no duplicates)
|
||
- Rows with primary keys not yet in the DB are **INSERTED**
|
||
- Rows in the DB but NOT in the backup are **KEPT** (restore doesn't delete)
|
||
- If any row fails to load, the whole restore is rolled back (no partial state)
|
||
|
||
**For a clean restore** (wipe everything first, then load the backup):
|
||
```
|
||
python manage.py flush # irreversible — deletes ALL data
|
||
python manage.py restore_data backup.json
|
||
```
|
||
|
||
### Recommended workflow before any deploy
|
||
|
||
1. `/backup-data/` on production before pushing the change
|
||
2. Push the change to `ai-dev`, let Flatlogic rebuild
|
||
3. Verify the new version works
|
||
4. If broken: restore from the backup you just took via `/restore-data/`
|
||
5. Delete the backup file from your laptop once you're confident the deploy is stable
|
||
|
||
### Files involved
|
||
- `core/management/commands/backup_data.py` — CLI command + reusable `build_backup_payload()` helper
|
||
- `core/management/commands/restore_data.py` — CLI command + reusable `restore_from_json_string()` helper
|
||
- `core/views.py::backup_data` — browser view that reuses the helper
|
||
- `core/views.py::restore_data` — browser view with minimal HTML upload UI
|
||
- URLs: `/backup-data/`, `/restore-data/` (both `@login_required` + `is_admin()` gated)
|
||
|
||
## Environment Variables
|
||
```
|
||
DJANGO_SECRET_KEY # required in prod — startup fails without it
|
||
DJANGO_DEBUG # "true"/"false"; defaults to false; keep false in prod
|
||
HOST_FQDN, CSRF_TRUSTED_ORIGIN # trusted hostnames (scheme-less ok, auto-prefixed https://)
|
||
DB_NAME, DB_USER, DB_PASS, DB_HOST (default: 127.0.0.1), DB_PORT (default: 3306)
|
||
USE_SQLITE # "true" → use SQLite instead of MySQL (local dev only)
|
||
EMAIL_HOST_USER # Gmail address — required for any outbound email
|
||
EMAIL_HOST_PASSWORD # Gmail App Password (16 chars, no spaces/non-breaking-space)
|
||
DEFAULT_FROM_EMAIL # Optional — falls back to EMAIL_HOST_USER if unset
|
||
SPARK_RECEIPT_EMAIL # Optional — defaults to FoxFitt's Spark Receipt address
|
||
PROJECT_DESCRIPTION, PROJECT_IMAGE_URL # Flatlogic branding
|
||
```
|
||
|
||
### Email fallback behaviour
|
||
`DEFAULT_FROM_EMAIL` is not strictly required — `config/settings.py` sets it as:
|
||
|
||
```python
|
||
DEFAULT_FROM_EMAIL = os.getenv("DEFAULT_FROM_EMAIL", "") or EMAIL_HOST_USER
|
||
```
|
||
|
||
…so if the env var is unset or empty, the "From" address on every outbound email
|
||
falls back to the authenticated Gmail address (which is always valid since we
|
||
send AS that account). Without this fallback, receipt and payslip emails fail
|
||
with `Invalid address ""`. If you want to send FROM a different display address
|
||
than the authenticated one (e.g. "FoxFitt Payroll <payroll@foxfitt.co.za>"),
|
||
set `DEFAULT_FROM_EMAIL` explicitly — but Gmail will likely rewrite it to the
|
||
authenticated user anyway unless you've configured a "Send mail as" alias.
|
||
|
||
### Where env vars live on Flatlogic
|
||
Flatlogic's platform has no env-var UI. Values are set in a `.env` file at
|
||
`BASE_DIR.parent / ".env"` on the VM (one level up from the repo). Edit via
|
||
Gemini/shell — the user cannot modify via Flatlogic's web editor because
|
||
`.env` is outside the project tree. The file is loaded by
|
||
`python-dotenv` in `config/settings.py` before any `os.getenv()` calls.
|
||
|
||
## Flatlogic/AppWizzy Deployment
|
||
- **Branches**: `ai-dev` = development (Flatlogic AI + Claude Code). `master` = deploy target.
|
||
- **Workflow**: Push to `ai-dev` → Flatlogic auto-detects → "Pull Latest" → app rebuilds (~5 min)
|
||
- **Deploy from Git** (Settings): Full rebuild from `master` — use for production
|
||
- **Migrations**: Sometimes run automatically during rebuild, but NOT always reliable. If you get "Unknown column" errors after pulling latest, visit `/run-migrate/` in the browser to apply pending migrations manually. This endpoint runs `python manage.py migrate` on the production MySQL database.
|
||
- **Static files**: Flatlogic's rebuild does NOT auto-run `collectstatic`. After CSS/JS changes have Gemini run `python3 manage.py collectstatic --noinput` + restart the service, otherwise Apache keeps serving the previously-collected copy.
|
||
- **Service**: The Django app runs as `django-dev.service` (systemd). Gemini restarts it via `sudo systemctl restart django-dev.service`. It runs `python manage.py runserver 0.0.0.0:8000` — a **development server**, not gunicorn/uwsgi (Flatlogic default, works fine at this scale).
|
||
- **CDN**: All production traffic goes through Cloudflare. Response headers show `cf-ray`/`cf-cache-status`. Static assets are cached at the edge for 4h — see "Static Assets & Cache-Busting" section for how the `deployment_timestamp` token breaks stale caches.
|
||
- **Never edit `ai-dev` directly on GitHub** — Flatlogic pushes overwrite it
|
||
- **Gemini gotcha**: Flatlogic's Gemini AI reads `__pycache__/*.pyc` and gets confused. Tell it: "Do NOT read .pyc files. Only work with .py source files."
|
||
- **Sequential workflow**: Don't edit in Flatlogic and Claude Code at the same time
|
||
|
||
### Git remotes on the VM
|
||
The Flatlogic VM has TWO git remotes, both kept in sync:
|
||
- `github` → `https://github.com/Konradzar/LabourPay_v5.git` (our canonical source)
|
||
- `gitea` → `https://gitea.flatlogic.app/admin/<id>-vm.git` (Flatlogic's internal mirror — the one the platform UI watches)
|
||
|
||
Any push the VM makes must go to BOTH: `git push github ai-dev && git push gitea ai-dev`.
|
||
If the two diverge, Flatlogic's dashboard can show a different commit than GitHub,
|
||
which silently confuses deploys. Flatlogic's UI occasionally commits as
|
||
`Flatlogic Bot <support@flatlogic.com>` (autosaves from the in-browser file editor) —
|
||
those commits land on gitea but don't propagate to GitHub unless someone pushes.
|
||
|
||
### VM-local safety branches
|
||
When doing risky deploys (model migrations, branch resets, history rewrites), we
|
||
create a safety branch on the VM at the pre-deploy HEAD so Gemini can
|
||
`git reset --hard <safety-branch>` + service-restart to roll back in ~60 seconds:
|
||
|
||
```
|
||
git branch pre-<purpose>-YYYYMMDD HEAD
|
||
git branch --list "pre-*"
|
||
```
|
||
|
||
Safety branches are VM-local — not pushed to GitHub by default. They're
|
||
single-use rollback anchors; delete after 7 days of confirmed stability via
|
||
`git branch -D pre-<purpose>-YYYYMMDD`.
|
||
|
||
### Workflow options going forward
|
||
Either works — pick one and stick to it per change to avoid divergence:
|
||
1. **Claude → GitHub → Flatlogic pulls**: Claude pushes to origin/ai-dev; you click "Pull Latest" in the Flatlogic UI (or ask Gemini to `git pull + push gitea + restart`).
|
||
2. **Flatlogic UI → GitHub**: edit in Flatlogic's file editor; click "Push to GitHub" in their UI; Claude pulls locally with `git pull origin ai-dev`.
|
||
**Don't mix** paths in the same change — that's how divergence (and the "Ver XX.YY screeeewup" commits) happen.
|
||
|
||
## Security Notes
|
||
- Production: `SESSION_COOKIE_SECURE=True`, `CSRF_COOKIE_SECURE=True`, `SameSite=None` (cross-origin for Flatlogic iframe)
|
||
- Local dev: Secure cookies disabled when `USE_SQLITE=true`
|
||
- X-Frame-Options middleware disabled (required for Flatlogic preview)
|
||
- Email App Password should be in env var, not hardcoded in settings.py
|
||
|
||
## Important Context
|
||
- The owner (Konrad) is not a developer — explain changes clearly and avoid unnecessary complexity
|
||
- This system handles real payroll for field workers — accuracy is critical
|
||
- `render_to_pdf()` uses lazy import of WeasyPrint to prevent app crash if library missing; on Windows it also auto-registers the GTK3 runtime's DLL directory so `ctypes.find_library()` can locate `gobject-2.0-0` (Python 3.8+ requires explicit `os.add_dll_directory()`)
|
||
- Django admin is available at `/admin/` with full model registration and search/filter
|