Compare commits

...

16 Commits

Author SHA1 Message Date
Konrad du Plessis
804e286af7 docs: production-status phrasing absorbs doc-only drift between origin and prod
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-12 22:36:22 +02:00
Konrad du Plessis
4f5f1bbe13 docs: audit bundle deployed & verified — close breadcrumbs, park leftovers, record gitea-auth break + autosave-diff rule
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-12 22:35:51 +02:00
Konrad du Plessis
abfae69606 fix: keep Flatlogic's internal preview domain in ALLOWED_HOSTS
The 29 May incident recovery added fox-fitt-payroll-7de4.dev.flatlogic.app
to ALLOWED_HOSTS as a VM-side Flatlogic-editor autosave (commit 98f66e9,
broken indentation). The 12 Jun audit deploy's git reset --hard discarded
it, which would 400 the platform's preview iframe. Committed properly so
it survives every future reset-based deploy.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-12 22:27:08 +02:00
Konrad du Plessis
e3e5bdeb78 docs: audit bundle pushed to origin — breadcrumb now tracks the pending prod deploy
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-12 21:23:23 +02:00
Konrad du Plessis
2d3cc43984 fix: run_dev.bat works from any launch directory (cd to its own folder first)
The bat assumed it was started from the repo root — python couldn't
find manage.py when the Claude preview runner (or any other launcher)
started it from elsewhere. cd /d "%~dp0" pins it to the script's own
directory. .claude/launch.json (gitignored) now also passes the bat's
absolute path so cmd finds it regardless of working directory.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-12 19:03:09 +02:00
Konrad du Plessis
94b6ec08e1 docs: audit-fix breadcrumb (HARD STOP), resolve stale SiteReport push status, fix views.py size
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-12 18:07:20 +02:00
Konrad du Plessis
921bdb6b73 chore: money-handling + template hygiene (audit items 11-16)
- money filter: format via Decimal(str(...)) instead of float — money
  never passes through binary floating point on its way to the screen
- preview_payslip / worker_lookup_ajax: accumulate totals in Decimal,
  convert to float only at the JsonResponse boundary, so previews are
  bit-identical to what _process_single_payment records
- price_overtime: explicit .quantize(0.01) instead of letting the DB
  engine silently round the 3rd decimal place
- stacked-chart wage rollup: Decimal(str(salary)) — on SQLite dev the
  value arrives as float and Decimal(float) keeps the binary noise
- payroll_dashboard fmt(): en-ZA space-separated thousands like every
  other money helper on the page (was the one comma-format outlier)
- base.html admin gate: '(auth and staff) or superuser' precedence trap
  replaced with 'staff or superuser' — exact same truth table (both
  flags False on AnonymousUser), matches server-side is_admin()
- attendance_log.html: worker day-rates now ship via the house
  json_script pattern instead of |safe-rendering a Python dict repr
  into the script block

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-12 18:04:01 +02:00
Konrad du Plessis
541b8973c7 perf: kill per-row queries on the History tab and Batch Pay preview
- paid_records (History tab) now prefetches work_logs + adjustments:
  the template shows a day-count and loops adjustments per row, which
  fired 2 queries per visible record (~100 on a long history).
- batch_pay_preview replaces the per-worker get_worker_active_team()
  call (worker.teams.filter(...).first() — bypasses prefetch, 1 query
  per worker) with the same batched membership-dict pattern
  payroll_dashboard already uses, and reads the unpaid-adjustments
  check from the existing filtered prefetch instead of .exists().

Also includes (committed earlier in 25910b2 but noting for the record):
the /report/ worker-breakdown loop's per-worker-per-type aggregates
were replaced by one GROUP BY dict (audit fix #7).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-12 17:58:56 +02:00
Konrad du Plessis
25910b2861 fix: worker batch report 'Total Paid' was inflated by the work-log join (+ kill its N+1s)
_build_worker_report_context computed Sum('payroll_records__amount_paid')
in the SAME .annotate() as counts over work_logs and warnings. Django
joins ALL those relations into one query, so each payroll row was
duplicated once per work-log row — the lifetime Total Paid column on
/workers/report/ (HTML, CSV and PDF) was multiplied by the worker's
log count (3 logs + one R100 payslip displayed R300). The distinct=True
counts were immune, which is why the suite never caught it. Verified by
a controlled experiment before fixing; regression test added.

Restructure: payroll aggregates stay in one annotate (same join — safe);
days-worked, warnings and project names move to three batched GROUP BY
dicts; teams + certificates are prefetched. Also removes the ~4 queries
per worker the loop used to fire (audit findings #8).

Display-only bug — payment processing was never affected.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-12 17:56:39 +02:00
Konrad du Plessis
f0f3938621 fix: worker report views survive stale/typo'd ?project= and ?team= params
All three worker batch-report views (HTML / CSV / PDF) passed raw
query-string values into queryset filters — ?team=abc raised ValueError
deep in the ORM, and the PDF view's display-name lookups raised
Project/Team.DoesNotExist on a deleted id (stale bookmark) → 500s.

New _int_param_or_none() sanitizer coerces filter params at the view
boundary (junk degrades to 'no filter'); the PDF's display-name lookups
fall back to 'All Projects'/'All Teams' on DoesNotExist. Template
dropdowns already compare via |stringformat so int params are safe.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-12 17:50:48 +02:00
Konrad du Plessis
4d029dd6e5 fix: cap attendance date range at 31 days (year-typo flood guard)
AttendanceLogForm.clean only checked end >= start — a typo'd year in
the end-date field created a WorkLog per worker per day for the whole
span (365+ days) in one submit, with no way back but manual deletion.
Ranges longer than 31 days (a full calendar month) are now rejected
with a message pointing at the year fields.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-12 17:47:27 +02:00
Konrad du Plessis
81753695a1 fix: refuse payments where deductions exceed earnings (no negative PayrollRecords)
_process_single_payment wrote logs_amount + adj_amount straight into
PayrollRecord.amount_paid with no floor — a selection where deductions
beat earnings (easy via the split-payslip checkboxes: untick the work
logs, leave a big loan repayment ticked) recorded a NEGATIVE payment,
emailed a negative payslip to Spark Receipt, and deducted the loan.

Per Konrad's decision (12 Jun 2026): REFUSE such payments. New
DeductionsExceedEarningsError raised inside the atomic block (full
rollback — loan balance and adjustments untouched); process_payment
shows a clear error toast, batch_pay names the refused worker in its
summary. Exactly-zero net stays allowed (a repayment consuming the
whole wage legitimately settles a loan with a R 0.00 payslip).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-12 17:45:39 +02:00
Konrad du Plessis
7ce3bfb232 fix: Batch Pay modal — filters no longer silently re-tick unticked workers; surface server errors
Two payment-safety fixes in the Batch Pay modal JS:

1. Changing the team/loan filter force-checked every visible row,
   silently re-selecting workers the admin had deliberately unticked
   (untick a disputed worker -> change filter -> Confirm & Pay All pays
   them anyway). Filters now only EXCLUDE (hidden rows untick); visible
   rows keep the admin's manual choice, and Select All reflects the
   real state instead of being forced on.

2. The batch-pay fetch() redirected to the dashboard on ANY HTTP
   response — fetch only rejects on network failure, so a 500 (batch
   died partway; each worker pays in its own transaction) looked like
   success. Now checks resp.ok and tells the admin to verify the
   History tab before retrying.

JS-only change; needs manual verification in the browser (no JS test
harness in this project).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-12 17:42:39 +02:00
Konrad du Plessis
cfc78b72ad fix: email failure after payment no longer shows a 500 to interactive callers
_send_payslip_email caught email errors, queued a warning toast, then
re-raised unconditionally — but only batch_pay catches that re-raise.
The Pay button (process_payment) and the three Pay-Immediately paths in
add_adjustment called it bare, so an SMTP failure AFTER the payment
committed replaced the warning with a 500 error page (the 28 May 2026
incident behaviour — admin can't tell the payment saved, may pay twice).

Re-raise is now conditional on suppress_messages=True (the batch path,
which counts failures in its summary); interactive callers get the
warning toast as intended. Regression tests cover both sides.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-12 17:41:20 +02:00
Konrad du Plessis
14ab8d0f76 docs: capture 27-29 May incident lessons (two-tier env precedence, --insecure, SSH access) + gitignore .claude.local.md
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-12 17:36:45 +02:00
Konrad du Plessis
663b7d98ba docs: park 2 Manager/Salaried follow-ups (Spark Receipt payslip verify + monthly_salary display)
(3) PENDING: Konrad testing 17 May whether Spark Receipt handles the
is_salary payslip layout for variable-pay managers. (4) Deferred:
monthly_salary required field is misleading on manager report rows
(inert for pay; display-only tidy-up parked).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-17 06:25:25 +02:00
12 changed files with 811 additions and 114 deletions

1
.gitignore vendored
View File

@ -16,6 +16,7 @@ media/
# Claude Code / IDE
.claude/
.claude.local.md
.vscode/
.idea/

229
CLAUDE.md
View File

@ -3,25 +3,66 @@
## What's mid-flight — read this first
**Parked / deferred work:** see `docs/plans/parked-work.md`.
**Production status (17 May 2026):** ✅ **fully caught up & verified.**
The 36-commit bundle (Manager/Salaried Pay + pay-type filter + Salary
auto-scope picker + Pay Salary dashboard quick action) is **deployed
and confirmed working on production** (`https://foxlog.flatlogic.app/`,
Konrad verified 17 May 2026). `origin/ai-dev` HEAD `80d96d7` == prod
(the only delta over the functional tip `4c25011` is doc breadcrumbs).
Migrations `0016`/`0017` applied; `static/css/custom.css` collected.
**Production status (12 Jun 2026):** ✅ **fully caught up & verified.**
Prod is at `abfae69` (deployed 12 Jun 2026 via fetch + `reset --hard`
+ restart; Konrad browser-verified same day). Any delta between
`origin/ai-dev` and prod beyond that is doc-only breadcrumb commits —
functionally identical; they ride along on the next real deploy.
Live on prod: the Manager/Salaried bundle (17 May), the SiteReport
removal (migration `0018` confirmed applied), the **12 Jun audit-fix
bundle** (see breadcrumb below), and the Flatlogic preview domain in
`ALLOWED_HOSTS` (`abfae69`). Migrations applied through `0018`;
static collected (no changes since May). **Known platform nit:** the
VM can no longer push to Flatlogic's gitea mirror (auth failed since
the May incident — see "Git remotes on the VM" below).
**🔧 In progress — local only, NOT pushed (HARD STOP):** removal of
the "Log Today's Work" / **SiteReport** feature (Konrad wants to
rethink it from scratch separately — work mix is shifting).
**Implemented locally — Tasks 1-3 complete:** model/table/UI/routes
deleted, migration `0018_delete_sitereport` drops `core_sitereport`,
post-attendance flow now returns to the dashboard, suite **193 OK**.
Still un-pushed and under a HARD STOP — nothing reaches origin until
Konrad verifies locally (destructive migration on the daily-use
attendance path). Design knowledge preserved for a future rebuild in
the capture doc `docs/plans/2026-05-17-site-report-removed-capture.md`;
see also the parked rebuild entry in `docs/plans/parked-work.md`.
**🔥 Incident 27-29 May 2026 (now closed) — what future sessions
need to know:** Cloudflare Tunnel error 1033 (27 May) → suspected
SSH-activate side-effect wiped `/home/ubuntu/executor/.env` → payment
500s with `ValueError: Invalid address ""` (28 May) → Flatlogic AI
agent returned only `AI agent failed with exit code -1` for hours
before Erik restored it (29 May) → discovered
`/etc/flatcloud/python-secrets.env` overrides `.env` via a systemd
drop-in → restored Gmail credentials in the secrets file → flipping
`DJANGO_DEBUG=false` killed static-file serving (runserver requires
`--insecure` for that with `DEBUG=False`) → added `--insecure` to the
systemd unit. **Three lessons baked into the sections below:**
(1) "Where env vars live on Flatlogic" — two-tier env file
precedence; (2) "Flatlogic/AppWizzy Deployment" — `--insecure` is
now required in the systemd unit; (3) new "SSH access on the VM"
section — direct VM shell is now available, key in Konrad's
password manager. **Strategic side note:** SSH access closes the
#1 risk identified in the platform-risk memo at
`C:\Users\konra\.claude\plans\prancy-painting-brook.md` (off-platform
backup of `media/` is now feasible via `rsync`).
**SiteReport removal — DEPLOYED (confirmed 12 Jun 2026).** Migration
`0018_delete_sitereport` shows `[X]` applied on production and the
code lineage was already on the VM. Fully resolved — nothing pending.
Design knowledge for a future rebuild lives in
`docs/plans/2026-05-17-site-report-removed-capture.md`; see also the
parked rebuild entry in `docs/plans/parked-work.md`.
**✅ 12 Jun 2026 audit-fix bundle — DEPLOYED & VERIFIED on
production** (`14ab8d0..abfae69`, 13 commits). A comprehensive
technical audit (4 parallel review agents + manual verification of
every finding) fixed: email-failure 500 after committed payments
(the 28 May incident class — interactive callers now get the warning
toast; only batch re-raises), Batch Pay modal silently re-ticking
unticked workers + swallowing server errors, **payments with
deductions > earnings now REFUSED** (Konrad's decision — no negative
PayrollRecords; exactly-zero net still allowed; see
`DeductionsExceedEarningsError` in `core/views.py`), attendance date
range capped at 31 days, worker-report views survive junk query
params (`_int_param_or_none`), **worker batch report's lifetime
"Total Paid" was inflated by the work-log join** (fixed +
`WorkerReportLifetimeTotalsTests`), report-page N+1s killed, money
display paths standardised on Decimal. Suite **206 OK**; browser
checks + prod smoke test passed (Konrad, 12 Jun 2026). The deploy
also surfaced + properly committed a VM-side autosave fix
(`abfae69`: Flatlogic preview domain in `ALLOWED_HOSTS`) — see the
"reset --hard" warning in the Deployment section. Small leftovers
parked in `docs/plans/parked-work.md` ("Audit leftovers").
**🧊 Backburner — do NOT start in `ai-dev`:** Phase A.2 (manual
JournalEntry UI) and Phase B (Letterly inbound webhook) are
@ -64,7 +105,7 @@ core/ — Single main app: ALL business logic, models, views, forms,
forms.py — AttendanceLogForm, PayrollAdjustmentForm, ExpenseReceiptForm + formset
models.py — All 10 database models
utils.py — render_to_pdf() helper (lazy WeasyPrint import + Windows GTK3 DLL registration)
views.py — All view functions (~52 functions, ~3,800 lines) — dashboard, attendance, payroll, reports, worker/team/project CRUD
views.py — All view functions (~6,000 lines) — dashboard, attendance, payroll, reports, worker/team/project CRUD
forms.py — All form classes + validators (WorkerForm, TeamForm, ProjectForm, AttendanceLogForm, PayrollAdjustmentForm, ExpenseReceiptForm, WorkerCertificate/WarningFormSet, 5MB file validator)
admin.py — Django admin registrations for all core models + WorkerCertificate/Warning inlines on Worker
templatetags/ — format_tags.py: `money` (ZAR), `money_abs` (signed callers), `type_slug` (type→CSS class), `url_replace` (swap one query-param), `dictlookup`
@ -334,6 +375,16 @@ section below) to find it. The test suite does NOT have `assertNumQueries`
guards on these views — deliberate YAGNI for now, worth adding if
regressions become a pattern.
**Jun 2026 audit note:** the history tab (`?status=history`), `/report/`,
`/workers/report/`, and the Batch Pay preview all had per-row query
loops removed (prefetches + batched GROUP BY dicts) — their query
counts should now be FLAT regardless of row count. The same audit also
fixed a real aggregation bug: mixing `Sum('payroll_records__...')` with
counts over OTHER relations in one `.annotate()` multiplies the Sum by
the join row count (see `WorkerReportLifetimeTotalsTests`) — keep
multi-valued aggregates on separate querysets / one relation per
annotate.
## Commands
```bash
# Local development (SQLite)
@ -889,12 +940,63 @@ than the authenticated one (e.g. "FoxFitt Payroll <payroll@foxfitt.co.za>"),
set `DEFAULT_FROM_EMAIL` explicitly — but Gmail will likely rewrite it to the
authenticated user anyway unless you've configured a "Send mail as" alias.
### Where env vars live on Flatlogic
Flatlogic's platform has no env-var UI. Values are set in a `.env` file at
`BASE_DIR.parent / ".env"` on the VM (one level up from the repo). Edit via
Gemini/shell — the user cannot modify via Flatlogic's web editor because
`.env` is outside the project tree. The file is loaded by
`python-dotenv` in `config/settings.py` before any `os.getenv()` calls.
### Where env vars live on Flatlogic — TWO files, second wins (learned the hard way 29 May 2026)
Flatlogic's platform has no env-var UI. The Django service reads env
from **two** files at runtime, loaded in this order:
1. **`/home/ubuntu/executor/.env`** — at `BASE_DIR.parent / ".env"` on
the VM, one level up from the repo. Loaded by `python-dotenv` in
`config/settings.py` at process startup. User-editable via
Gemini/shell (or now SSH).
2. **`/etc/flatcloud/python-secrets.env`** — root-owned, managed by
Flatlogic's platform image. Loaded by **systemd** itself via a
drop-in unit at `/etc/systemd/system/django-dev.service.d/flatcloud-env.conf`
that has an `EnvironmentFile=/etc/flatcloud/python-secrets.env`
directive.
**Precedence:** when both files set the same key, **the systemd-loaded
secrets file wins** because systemd injects its variables into the
process environment BEFORE `manage.py runserver` starts, and
`python-dotenv`'s default `load_dotenv()` does NOT override existing
env vars. So `.env` only "wins" for keys the secrets file doesn't set.
**Practical consequence (the bit that cost hours on 29 May 2026):**
editing `.env` to fix `EMAIL_HOST_USER`/`EMAIL_HOST_PASSWORD`/
`DJANGO_DEBUG` had ZERO effect because all three were also set in
`python-secrets.env` (often to wrong values — Flatlogic's recovery
template installs **AWS SES placeholder credentials** that conflict
with our Gmail SMTP config). The fix is always: edit the secrets file
with `sudo`, then `sudo systemctl restart django-dev.service`.
**Diagnostic command to confirm both files are loaded and in what
order:**
```bash
systemctl show django-dev.service --property=EnvironmentFiles
```
**Recovery playbook if email/payments break after a platform incident:**
1. `sudo grep EMAIL_HOST /etc/flatcloud/python-secrets.env` — does it
show AWS-shaped values (long base64-ish keys) instead of
`konrad@foxfitt.co.za` + a Gmail App Password? That's the symptom.
2. `sudo nano /etc/flatcloud/python-secrets.env` — set:
- `EMAIL_HOST=smtp.gmail.com`
- `EMAIL_PORT=587`
- `EMAIL_USE_TLS=True`
- `EMAIL_HOST_USER=konrad@foxfitt.co.za`
- `EMAIL_HOST_PASSWORD=<16-char Gmail App Password, no spaces>`
- `DEFAULT_FROM_EMAIL=konrad@foxfitt.co.za`
- `DJANGO_DEBUG=false`
3. `sudo systemctl restart django-dev.service`
4. Verify: `sudo systemctl status django-dev.service` shows
`active (running)`; first test payment delivers a payslip to Spark
Receipt.
**Security note:** the secrets file contains the Gmail App Password,
`DJANGO_SECRET_KEY`, and DB credentials. Never `cat` it into the
Flatlogic AI agent chat or this Claude session — values leak into
transcripts. Use `grep -c KEY_NAME` to confirm presence without
printing the value.
## Flatlogic/AppWizzy Deployment
- **Branches**: `ai-dev` = development (Flatlogic AI + Claude Code). `master` = deploy target.
@ -902,8 +1004,10 @@ Gemini/shell — the user cannot modify via Flatlogic's web editor because
- **Deploy from Git** (Settings): Full rebuild from `master` — use for production
- **Migrations**: Sometimes run automatically during rebuild, but NOT always reliable. If you get "Unknown column" errors after pulling latest, visit `/run-migrate/` in the browser to apply pending migrations manually. This endpoint runs `python manage.py migrate` on the production MySQL database.
- **Static files**: Flatlogic's rebuild does NOT auto-run `collectstatic`. After CSS/JS changes have Gemini run `python3 manage.py collectstatic --noinput` + restart the service, otherwise Apache keeps serving the previously-collected copy.
- **Service**: The Django app runs as `django-dev.service` (systemd). Gemini restarts it via `sudo systemctl restart django-dev.service`. It runs `python manage.py runserver 0.0.0.0:8000` — a **development server**, not gunicorn/uwsgi (Flatlogic default, works fine at this scale).
- **⚠ DEPLOY ORDERING — pull THEN restart, not the reverse.** Production runs `DEBUG=False`, so Django uses the **cached template loader**: every `.html` template is compiled into memory once at process start and is NEVER re-read from disk until the process restarts. Symptom of getting this wrong: "I pulled the code, `git log` shows the right commit, but the page still looks old." Cause: the `restart` happened *before* the code reached the target commit (e.g. Flatlogic auto-pulled afterward, or Gemini pulled after restarting). **Fix: restart AGAIN, after confirming `git log --oneline -1` is at the target commit.** Correct deploy order is ALWAYS: (1) `git fetch github ai-dev && git reset --hard github/ai-dev`, (2) `/run-migrate/` if there are new migrations, (3) `collectstatic` if `static/` changed, (4) `sudo systemctl restart django-dev.service` **last**. Template-only changes still need the restart (cached loader) — unlike local dev where `DEBUG=True` re-reads templates per request. Bit us 15 May 2026: 14 commits of template fixes were "invisible" on prod until a second restart. `git reset --hard github/ai-dev` (not `git pull`) is preferred because the VM accumulates Flatlogic-editor autosave commits that make a plain pull conflict.
- **Service**: The Django app runs as `django-dev.service` (systemd). Gemini restarts it via `sudo systemctl restart django-dev.service`. It runs `python manage.py runserver 0.0.0.0:8000 --insecure` — a **development server**, not gunicorn/uwsgi (Flatlogic default, works fine at this scale).
- **⚠ The `--insecure` flag on runserver is REQUIRED in production (added 29 May 2026).** With `DEBUG=False` (the correct production state), Django's `runserver` refuses to serve `/static/` files by default — every CSS/JS request returns 404, and the dashboard renders as plain unstyled HTML. The `--insecure` flag explicitly opts in to serving static files even with DEBUG off. **If you ever see "everything works but the page looks unstyled" after a deploy:** check the `ExecStart=` line in `/etc/systemd/system/django-dev.service` (or its drop-in directory) — if `--insecure` is missing, add it, then `sudo systemctl daemon-reload && sudo systemctl restart django-dev.service`. The proper long-term fix is an Apache `Alias /static/ → staticfiles/` directive that bypasses Django entirely, but `--insecure` is a stable workaround.
- **⚠ Cloudflare HIT-caches 404 responses for ~4h.** If a static-file URL returned 404 at any point, Cloudflare will keep serving that 404 even after you fix the underlying problem. To verify a fix without waiting for the TTL: append a random query string (`?cb=$(date +%s)`) — that's a cache key Cloudflare hasn't seen, so it fetches from origin. The Flatlogic preview iframe sometimes shows cached-working CSS while a fresh browser tab shows the cached 404; trust the browser tab, not the iframe.
- **⚠ DEPLOY ORDERING — pull THEN restart, not the reverse.** Production runs `DEBUG=False`, so Django uses the **cached template loader**: every `.html` template is compiled into memory once at process start and is NEVER re-read from disk until the process restarts. Symptom of getting this wrong: "I pulled the code, `git log` shows the right commit, but the page still looks old." Cause: the `restart` happened *before* the code reached the target commit (e.g. Flatlogic auto-pulled afterward, or Gemini pulled after restarting). **Fix: restart AGAIN, after confirming `git log --oneline -1` is at the target commit.** Correct deploy order is ALWAYS: (1) `git fetch github ai-dev && git reset --hard github/ai-dev`, (2) `/run-migrate/` if there are new migrations, (3) `collectstatic` if `static/` changed, (4) `sudo systemctl restart django-dev.service` **last**. Template-only changes still need the restart (cached loader) — unlike local dev where `DEBUG=True` re-reads templates per request. Bit us 15 May 2026: 14 commits of template fixes were "invisible" on prod until a second restart. `git reset --hard github/ai-dev` (not `git pull`) is preferred because the VM accumulates Flatlogic-editor autosave commits that make a plain pull conflict. **⚠ BEFORE any reset --hard: `git show --stat` every VM-local commit being discarded, and anchor it with a `pre-*` safety branch.** An "Autosave" commit can contain a REAL fix — on 12 Jun 2026 autosave `98f66e9` held the only copy of the Flatlogic preview domain in `ALLOWED_HOSTS` (a 29 May incident-recovery edit); we read the diff, anchored it, and re-committed it properly via GitHub (`abfae69`). Never discard a VM commit unread.
- **CDN**: All production traffic goes through Cloudflare. Response headers show `cf-ray`/`cf-cache-status`. Static assets are cached at the edge for 4h — see "Static Assets & Cache-Busting" section for how the `deployment_timestamp` token breaks stale caches.
- **Never edit `ai-dev` directly on GitHub** — Flatlogic pushes overwrite it
- **Gemini gotcha**: Flatlogic's Gemini AI reads `__pycache__/*.pyc` and gets confused. Tell it: "Do NOT read .pyc files. Only work with .py source files."
@ -920,6 +1024,14 @@ which silently confuses deploys. Flatlogic's UI occasionally commits as
`Flatlogic Bot <support@flatlogic.com>` (autosaves from the in-browser file editor) —
those commits land on gitea but don't propagate to GitHub unless someone pushes.
**⚠ gitea auth BROKEN since the May 2026 incident (found 12 Jun 2026):**
the VM gets `Authentication failed` pushing to gitea — credentials
presumably rotated/lost in the platform recovery. Until Erik/Flatlogic
support restores them, deploys are GitHub→VM one-way and the Flatlogic
dashboard may show a stale commit; don't burn time re-diagnosing a
gitea push failure. Once fixed, re-sync with `git push gitea ai-dev`
(may need `--force` — gitea last saw the discarded autosave `98f66e9`).
### VM-local safety branches
When doing risky deploys (model migrations, branch resets, history rewrites), we
create a safety branch on the VM at the pre-deploy HEAD so Gemini can
@ -940,6 +1052,69 @@ Either works — pick one and stick to it per change to avoid divergence:
2. **Flatlogic UI → GitHub**: edit in Flatlogic's file editor; click "Push to GitHub" in their UI; Claude pulls locally with `git pull origin ai-dev`.
**Don't mix** paths in the same change — that's how divergence (and the "Ver XX.YY screeeewup" commits) happen.
## SSH access on the VM (added 29 May 2026)
Direct SSH access was activated during the 27-29 May incident
recovery. **The key and SSH command are stored in Konrad's password
manager — they MUST NOT be committed to git or pasted into any AI
agent chat.** SSH gives full root-equivalent access to the production
VM; treat it like a vault credential.
**When to use SSH:**
- Flatlogic AI agent is broken or unresponsive (happened on 29 May
2026 — "AI agent failed with exit code -1" for several hours).
- Need to inspect logs in real time (`journalctl -f`,
`tail -f /var/log/...`).
- Need to run a true `mysqldump` for a full database backup
(`/backup-data/` only dumps via the Django ORM; mysqldump is more
complete).
- Need to `rsync` the `media/` directory off-platform for backup
(closes the platform-risk memo's #1 gap — see
`C:\Users\konra\.claude\plans\prancy-painting-brook.md`).
- Need to escape a Cloudflare outage (the SSH host:port is direct, not
proxied through Cloudflare).
**When NOT to use SSH:**
- Routine deploys — keep using the GitHub→Flatlogic pull workflow.
- Anything the Flatlogic AI agent can do — the agent is normally
faster, safer, and produces a chat-transcript audit trail.
**⚠ DO NOT click the "Activate SSH" / "Deactivate SSH" button again
casually.** The strong hypothesis from the 27 May incident is that
clicking it triggered Flatlogic's platform-side recovery process,
which wiped `/home/ubuntu/executor/.env`. SSH is already active —
leave the button alone. If a deactivation is ever genuinely needed,
take a fresh `/backup-data/` first and allocate 30 minutes for
potential recovery.
**Connection (do this from Konrad's laptop only):**
```bash
# In Git Bash (Windows) or terminal (Mac/Linux). Key path will
# differ depending on where Konrad stored the key. NEVER paste the
# actual key path or the host IP into a Claude/AI session.
chmod 600 <path-to-key> # one-time, only needed on a fresh download
ssh -i <path-to-key> -p <port> <user>@<host>
```
**Useful off-platform backup commands** (run from laptop, not on VM):
```bash
# Full SQL dump of the production DB → laptop
ssh -i <key> -p <port> <user>@<host> \
"mysqldump --single-transaction <db_name>" > foxlog_$(date +%Y%m%d).sql
# Sync uploaded media (photos, ID docs, certs, warnings) → laptop
rsync -avz -e "ssh -i <key> -p <port>" \
<user>@<host>:/home/ubuntu/executor/workspace/media/ \
./foxlog_media_backup/
```
These two commands together produce a **truly complete off-platform
backup** (DB + uploaded files) — something `/backup-data/` cannot do
because Django's ORM dump doesn't include `media/`. Run them weekly,
store in a non-Flatlogic location (Google Drive / external disk /
S3-compatible bucket), and the single biggest off-platform-readiness
gap closes without leaving Flatlogic.
## Security Notes
- Production: `SESSION_COOKIE_SECURE=True`, `CSRF_COOKIE_SECURE=True`, `SameSite=None` (cross-origin for Flatlogic iframe)
- Local dev: Secure cookies disabled when `USE_SQLITE=true`

View File

@ -51,6 +51,11 @@ ALLOWED_HOSTS = [
"127.0.0.1",
"localhost",
"foxlog.flatlogic.app",
# Flatlogic's internal preview domain for this app — the platform's
# dashboard iframe loads the site through it. First added 29 May 2026
# as a VM-side autosave edit during the incident recovery; committed
# properly here so it survives `git reset --hard` deploys.
"fox-fitt-payroll-7de4.dev.flatlogic.app",
os.getenv("HOST_FQDN", ""),
]

View File

@ -132,6 +132,16 @@ class AttendanceLogForm(forms.ModelForm):
if start_date and end_date and end_date < start_date:
raise forms.ValidationError('End date cannot be before start date.')
# === GUARD: cap the range length (audit fix, Jun 2026) ===
# A typo'd year in the end date (2027 instead of 2026) would
# otherwise create hundreds of WorkLogs in one submit. 31 days
# covers a full calendar month — the longest real logging period.
if start_date and end_date and (end_date - start_date).days > 31:
raise forms.ValidationError(
'Date range cannot exceed 31 days — check the year on both '
'dates. Log longer periods in separate submissions.'
)
return cleaned_data

View File

@ -404,7 +404,11 @@
{# === WORK LOG PAYROLL MODAL — click handler + safe DOM builder === #}
{# Builds the modal body from JSON via createElement + textContent. #}
{% if user.is_authenticated and user.is_staff or user.is_superuser %}
{# staff-or-superuser matches the server-side is_admin() helper. The old #}
{# "is_authenticated and is_staff or is_superuser" parsed as "(auth AND #}
{# staff) OR superuser" — template `and` binds tighter than `or`. Both #}
{# flags are False on AnonymousUser, so this simpler form is exact. #}
{% if user.is_staff or user.is_superuser %}
<script>
document.addEventListener('DOMContentLoaded', function() {
var modalEl = document.getElementById('workLogPayrollModal');
@ -636,7 +640,11 @@ document.addEventListener('DOMContentLoaded', function() {
{# === WORK LOG PAYROLL MODAL (admin-only) === #}
{# Hidden by default. Any element with data-log-id anywhere in the app #}
{# triggers this modal. Fetches JSON and builds the DOM safely. #}
{% if user.is_authenticated and user.is_staff or user.is_superuser %}
{# staff-or-superuser matches the server-side is_admin() helper. The old #}
{# "is_authenticated and is_staff or is_superuser" parsed as "(auth AND #}
{# staff) OR superuser" — template `and` binds tighter than `or`. Both #}
{# flags are False on AnonymousUser, so this simpler form is exact. #}
{% if user.is_staff or user.is_superuser %}
<div class="modal fade" id="workLogPayrollModal" tabindex="-1" aria-labelledby="workLogPayrollModalLabel" aria-hidden="true">
<div class="modal-dialog modal-lg modal-dialog-scrollable">
<div class="modal-content">

View File

@ -216,6 +216,10 @@
</div>
<!-- === JavaScript: Team auto-select + Cost estimator === -->
{# Worker day-rates as a safe JSON island (house json_script pattern — #}
{# never |safe-render data into a <script> block). Admin-only, matching #}
{# the cost-estimator block below that consumes it. #}
{% if is_admin %}{{ worker_rates_json|json_script:"workerRatesData" }}{% endif %}
<script>
document.addEventListener('DOMContentLoaded', function() {
@ -241,7 +245,9 @@ document.addEventListener('DOMContentLoaded', function() {
{% if is_admin %}
// === ESTIMATED COST CALCULATOR (Admin Only) ===
const workerRates = {{ worker_rates_json|safe }};
// Rates come from the #workerRatesData json_script island above —
// parsing JSON is XSS-safe regardless of what the data contains.
const workerRates = JSON.parse(document.getElementById('workerRatesData').textContent);
const startDateInput = document.querySelector('[name="date"]');
const endDateInput = document.querySelector('[name="end_date"]');
const satCheckbox = document.querySelector('[name="include_saturday"]');

View File

@ -1413,8 +1413,13 @@ document.addEventListener('DOMContentLoaded', function() {
const workerChartData = JSON.parse(document.getElementById('workerChartJson').textContent);
// === HELPER: Format currency ===
// en-ZA = space-separated thousands (R 1 234.56) — matches the server's
// `money` filter and the formatMoney/formatRand helpers further down.
// The old comma regex was the one outlier on the page.
function fmt(val) {
return 'R ' + parseFloat(val).toFixed(2).replace(/\B(?=(\d{3})+(?!\d))/g, ',');
return 'R ' + Number(val).toLocaleString('en-ZA', {
minimumFractionDigits: 2, maximumFractionDigits: 2
});
}
// === HELPER: Create a table cell with text ===
@ -3266,10 +3271,16 @@ document.addEventListener('DOMContentLoaded', function() {
});
// --- Shared filter function (team + loan filters combined) ---
// SAFETY RULE: a filter change may EXCLUDE workers from the
// batch (hidden rows are unticked) but must NEVER silently
// re-tick a worker the admin deliberately unticked — that
// would pay someone the admin chose to skip. Use Select All
// to re-include everyone after changing filters.
function applyBatchFilters() {
var selectedTeam = filterSelect.value;
var loanMode = batchLoanFilter ? batchLoanFilter.value : '';
var rows = tbody.querySelectorAll('tr');
var allChecked = true;
for (var i = 0; i < rows.length; i++) {
var row = rows[i];
var teamMatch = !selectedTeam || row.dataset.team === selectedTeam;
@ -3278,13 +3289,17 @@ document.addEventListener('DOMContentLoaded', function() {
|| (loanMode === 'without' && row.dataset.hasLoan !== 'true');
if (teamMatch && loanMatch) {
row.style.display = '';
row.querySelector('.batch-worker-cb').checked = true;
// keep the admin's manual tick/untick choice
if (!row.querySelector('.batch-worker-cb').checked) {
allChecked = false;
}
} else {
row.style.display = 'none';
row.querySelector('.batch-worker-cb').checked = false;
}
}
selectAllCb.checked = true;
// Reflect the real state of the visible rows (not forced on)
selectAllCb.checked = allChecked;
updateBatchSummary(data, summary);
}
@ -3374,17 +3389,27 @@ document.addEventListener('DOMContentLoaded', function() {
'X-CSRFToken': '{{ csrf_token }}',
},
body: JSON.stringify({ workers: workers }),
}).then(function() {
}).then(function(resp) {
// fetch() does NOT reject on HTTP errors (4xx/5xx) — only on
// network failure. A 500 here can mean the batch died partway
// (each worker pays in its own transaction), so surface it
// instead of redirecting as if everything succeeded.
if (!resp.ok) {
throw new Error('server returned ' + resp.status);
}
// Redirect to refresh page and show Django success messages
window.location.href = '/payroll/';
}).catch(function() {
}).catch(function(err) {
btn.disabled = false;
while (btn.firstChild) btn.removeChild(btn.firstChild);
var retryIcon = document.createElement('i');
retryIcon.className = 'fas fa-money-bill-wave me-1';
btn.appendChild(retryIcon);
btn.appendChild(document.createTextNode('Confirm & Pay All'));
alert('Batch payment failed. Please try again.');
alert('Batch payment may have partially failed ('
+ (err && err.message ? err.message : 'network error')
+ '). Check the History tab to see which workers were paid '
+ 'before retrying — do not blindly re-pay everyone.');
});
});
}

View File

@ -2,6 +2,8 @@
# Number formatting filters for South African currency display.
# Usage: {% load format_tags %} then {{ value|money }}
from decimal import Decimal, InvalidOperation
from django import template
register = template.Library()
@ -16,10 +18,13 @@ def money(value):
22500 22 500.00
400.0 400.00
-300.00 -300.00
Formats via Decimal (not float) so money values are never coerced
through binary floating point exact at any magnitude.
"""
try:
num = float(value)
except (ValueError, TypeError):
num = Decimal(str(value))
except (InvalidOperation, ValueError, TypeError):
return value
# Python's :, format gives comma separators — swap commas for spaces

View File

@ -3,7 +3,9 @@
# determines, for each worker on a log, whether they were paid for it.
import datetime
import json
from decimal import Decimal
from unittest import mock
from django.conf import settings
from django.contrib.auth.models import User
@ -3679,3 +3681,258 @@ class ManagerSalariedPayUITests(TestCase):
# guard test).
self.assertNotIn('Salary Details', body)
self.assertNotIn('Salary Amount:', body)
# =============================================================================
# === AUDIT FIX #1 — EMAIL FAILURE AFTER PAYMENT MUST NOT 500 ===
# Regression tests for the 28 May 2026 incident class: SMTP breaks, the
# payment commits, and the admin must see a warning toast — NOT an error
# page (which hides that the payment DID go through and invites a
# dangerous second click of Pay).
# =============================================================================
class PaymentEmailFailureTests(TestCase):
"""Email failure after a successful payment: warn the admin, never crash."""
def setUp(self):
self.admin = User.objects.create_user(
username='emailfail_admin', password='x', is_staff=True)
self.client.force_login(self.admin)
self.project = Project.objects.create(name='EF Project')
self.worker = Worker.objects.create(
name='EF Worker', id_number='EF-1', monthly_salary=Decimal('8000'))
log = WorkLog.objects.create(
date=datetime.date(2026, 6, 1), project=self.project,
supervisor=self.admin)
log.workers.add(self.worker)
def _broken_smtp(self):
# Simulate the 28 May 2026 state: every send() raises (bad creds /
# SMTP unreachable / empty DEFAULT_FROM_EMAIL).
return mock.patch(
'django.core.mail.EmailMultiAlternatives.send',
side_effect=Exception('SMTP down'))
def test_individual_payment_survives_email_failure(self):
with self._broken_smtp():
resp = self.client.post(
reverse('process_payment', args=[self.worker.id]), follow=True)
# The payment was saved...
self.assertEqual(
PayrollRecord.objects.filter(worker=self.worker).count(), 1)
# ...and the admin landed back on a normal page (not a 500)
self.assertEqual(resp.status_code, 200)
msgs = [str(m) for m in resp.context['messages']]
self.assertTrue(
any('email delivery failed' in m for m in msgs),
f'expected an email-failure warning toast, got: {msgs}')
def test_batch_pay_still_counts_email_failures(self):
# The batch path NEEDS the helper to re-raise internally so it can
# count failures — this guards the other side of the conditional.
with self._broken_smtp():
resp = self.client.post(
reverse('batch_pay'),
data=json.dumps({'workers': [{'worker_id': self.worker.id}]}),
content_type='application/json',
follow=True)
self.assertEqual(
PayrollRecord.objects.filter(worker=self.worker).count(), 1)
self.assertEqual(resp.status_code, 200)
msgs = [str(m) for m in resp.context['messages']]
self.assertTrue(
any('1 email(s) failed to send' in m for m in msgs),
f'expected batch summary to count the email failure, got: {msgs}')
# =============================================================================
# === AUDIT FIX #4 — REFUSE PAYMENTS WHERE DEDUCTIONS EXCEED EARNINGS ===
# Konrad's decision (12 Jun 2026): when a payment selection's deductions
# are bigger than its earnings (e.g. a R1,500 loan repayment against
# R400 of work), the payment is REFUSED with a clear message — never
# recorded as a negative PayrollRecord, never clamped to zero. The loan
# balance must stay untouched and the adjustment must stay unpaid so the
# admin can fix the selection and retry.
# =============================================================================
class NegativePaymentGuardTests(TestCase):
"""Deductions > earnings: refuse, leave everything unpaid and intact."""
def setUp(self):
self.admin = User.objects.create_user(
username='negpay_admin', password='x', is_staff=True)
self.client.force_login(self.admin)
self.project = Project.objects.create(name='NP Project')
# Daily rate = 8000 / 20 = R400; one logged day = R400 earnings.
self.worker = Worker.objects.create(
name='NP Worker', id_number='NP-1', monthly_salary=Decimal('8000'))
log = WorkLog.objects.create(
date=datetime.date(2026, 6, 2), project=self.project,
supervisor=self.admin)
log.workers.add(self.worker)
# A R1,500 repayment dwarfs the R400 of earnings → net would be -1100.
self.loan = Loan.objects.create(
worker=self.worker, principal_amount=Decimal('1500.00'),
loan_type='loan')
self.repayment = PayrollAdjustment.objects.create(
worker=self.worker, type='Loan Repayment',
amount=Decimal('1500.00'), date=datetime.date(2026, 6, 2),
loan=self.loan)
def test_individual_payment_refused_not_recorded_negative(self):
resp = self.client.post(
reverse('process_payment', args=[self.worker.id]), follow=True)
self.assertEqual(resp.status_code, 200)
# Nothing was recorded...
self.assertEqual(
PayrollRecord.objects.filter(worker=self.worker).count(), 0)
# ...the loan balance is untouched and the repayment still pending...
self.loan.refresh_from_db()
self.repayment.refresh_from_db()
self.assertEqual(self.loan.remaining_balance, Decimal('1500.00'))
self.assertIsNone(self.repayment.payroll_record)
# ...and the admin was told why.
msgs = [str(m) for m in resp.context['messages']]
self.assertTrue(
any('exceed' in m.lower() for m in msgs),
f'expected a deductions-exceed-earnings refusal message, got: {msgs}')
def test_batch_pay_refuses_and_reports(self):
resp = self.client.post(
reverse('batch_pay'),
data=json.dumps({'workers': [{'worker_id': self.worker.id}]}),
content_type='application/json',
follow=True)
self.assertEqual(resp.status_code, 200)
self.assertEqual(
PayrollRecord.objects.filter(worker=self.worker).count(), 0)
self.loan.refresh_from_db()
self.assertEqual(self.loan.remaining_balance, Decimal('1500.00'))
msgs = [str(m) for m in resp.context['messages']]
self.assertTrue(
any('NP Worker' in m and 'exceed' in m.lower() for m in msgs),
f'expected the batch summary to name the refused worker, got: {msgs}')
def test_zero_net_payment_still_allowed(self):
# Repayment EXACTLY equal to earnings is legitimate (settles the
# loan with the whole wage) — only strictly-negative is refused.
self.repayment.amount = Decimal('400.00')
self.repayment.save()
resp = self.client.post(
reverse('process_payment', args=[self.worker.id]), follow=True)
self.assertEqual(resp.status_code, 200)
record = PayrollRecord.objects.get(worker=self.worker)
self.assertEqual(record.amount_paid, Decimal('0.00'))
self.loan.refresh_from_db()
self.assertEqual(self.loan.remaining_balance, Decimal('1100.00'))
# =============================================================================
# === AUDIT FIX #5 — ATTENDANCE DATE RANGE CAP ===
# A typo'd year in the end-date field (2027 instead of 2026) would create
# hundreds of WorkLogs in a single submit. The form now rejects ranges
# longer than 31 days; one calendar month is the longest real logging
# period, anything longer is almost certainly a typo.
# =============================================================================
class AttendanceDateRangeCapTests(TestCase):
"""The attendance form refuses date ranges longer than 31 days."""
def setUp(self):
from core.forms import AttendanceLogForm
self.form_cls = AttendanceLogForm
self.admin = User.objects.create_user(
username='rangecap_admin', password='x', is_staff=True)
def _range_errors(self, start, end):
# Other required fields (project, workers) are deliberately left
# blank — we only care about the non-field errors clean() raises.
form = self.form_cls({'date': start, 'end_date': end}, user=self.admin)
form.is_valid()
return [str(e) for e in form.non_field_errors()]
def test_range_over_31_days_is_rejected(self):
errs = self._range_errors('2026-01-01', '2026-02-02') # 32-day span
self.assertTrue(any('31 days' in e for e in errs), errs)
def test_year_typo_is_rejected(self):
errs = self._range_errors('2026-01-05', '2027-01-05') # the real footgun
self.assertTrue(any('31 days' in e for e in errs), errs)
def test_range_of_exactly_31_days_is_allowed(self):
errs = self._range_errors('2026-01-01', '2026-02-01') # 31-day diff
self.assertFalse(any('31 days' in e for e in errs), errs)
# =============================================================================
# === AUDIT FIX #6 — WORKER REPORT VIEWS MUST SURVIVE BAD QUERY PARAMS ===
# Stale bookmarks and hand-edited URLs (?project=99999 after a delete,
# ?team=abc from a typo) hit all three worker-report views (HTML / CSV /
# PDF). Bad filter values must degrade to "no filter", never 500.
# render_to_pdf is mocked so these tests don't depend on the local GTK3
# install — the crashes under test happen BEFORE PDF rendering anyway.
# =============================================================================
class WorkerReportBadParamsTests(TestCase):
"""?project= / ?team= junk degrades gracefully on all 3 report views."""
def setUp(self):
self.admin = User.objects.create_user(
username='pdfguard_admin', password='x', is_staff=True)
self.client.force_login(self.admin)
def _get_pdf(self, params):
with mock.patch('core.utils.render_to_pdf',
return_value=b'%PDF-1.4 fake'):
return self.client.get(reverse('worker_batch_report_pdf'), params)
def test_pdf_nonexistent_project_id_falls_back(self):
resp = self._get_pdf({'project': '99999'})
self.assertEqual(resp.status_code, 200)
def test_pdf_non_numeric_team_falls_back(self):
resp = self._get_pdf({'team': 'abc'})
self.assertEqual(resp.status_code, 200)
def test_html_non_numeric_project_falls_back(self):
resp = self.client.get(reverse('worker_batch_report'), {'project': 'abc'})
self.assertEqual(resp.status_code, 200)
def test_csv_non_numeric_team_falls_back(self):
resp = self.client.get(reverse('worker_batch_report_csv'), {'team': 'abc'})
self.assertEqual(resp.status_code, 200)
# =============================================================================
# === AUDIT FINDING (Jun 2026) — WORKER REPORT LIFETIME TOTAL INFLATION ===
# _build_worker_report_context computed Sum('payroll_records__amount_paid')
# in the SAME .annotate() as counts over work_logs and warnings. Django
# builds ONE query with all the joins, so each payroll record row is
# duplicated once per work-log row — the lifetime "Total Paid" column was
# multiplied by the worker's log count (3 logs + one R100 payslip showed
# R300). The distinct=True counts were immune, which is why it hid.
# =============================================================================
class WorkerReportLifetimeTotalsTests(TestCase):
"""total_paid_lifetime must not be multiplied by the work-log join."""
def test_total_paid_not_inflated_by_work_log_joins(self):
from core.views import _build_worker_report_context
admin = User.objects.create_user(username='infl_admin', is_staff=True)
p = Project.objects.create(name='Infl Project')
w = Worker.objects.create(
name='Infl Worker', id_number='INF-1',
monthly_salary=Decimal('4000'))
for d in (1, 2, 3): # 3 logs → 3 join rows
wl = WorkLog.objects.create(
date=datetime.date(2026, 6, d), project=p, supervisor=admin)
wl.workers.add(w)
PayrollRecord.objects.create(
worker=w, amount_paid=Decimal('100.00'),
date=datetime.date(2026, 6, 4)) # ONE payslip of R100
rows = _build_worker_report_context()
row = next(r for r in rows if r['worker'].id == w.id)
self.assertEqual(row['total_paid_lifetime'], Decimal('100.00'))
self.assertEqual(row['days_worked'], 3)
self.assertEqual(row['payslip_count'], 1)

View File

@ -1645,6 +1645,19 @@ def worker_edit(request, worker_id=None):
# === WORKER BATCH REPORT ===
# =============================================================
def _int_param_or_none(raw):
"""Coerce a raw query-string value to int, or None if absent/invalid.
Stale bookmarks and hand-edited URLs (?project=abc, ?team= after a
delete) must degrade to "no filter" never crash the view with a
ValueError deep inside a queryset. (Audit fix, Jun 2026.)
"""
try:
return int(raw)
except (TypeError, ValueError):
return None
def _build_worker_report_context(status=None, project_id=None, team_id=None):
"""Build the per-worker aggregation list used by HTML / CSV / PDF views.
@ -1653,38 +1666,71 @@ def _build_worker_report_context(status=None, project_id=None, team_id=None):
and warning counts. All aggregates are computed in a single query
via annotate/prefetch to avoid N+1 database hits.
"""
workers = Worker.objects.all()
base = Worker.objects.all()
if status == 'active':
workers = workers.filter(active=True)
base = base.filter(active=True)
elif status == 'inactive':
workers = workers.filter(active=False)
base = base.filter(active=False)
if project_id:
workers = workers.filter(work_logs__project_id=project_id).distinct()
base = base.filter(work_logs__project_id=project_id).distinct()
if team_id:
workers = workers.filter(teams__id=team_id).distinct()
base = base.filter(teams__id=team_id).distinct()
workers = workers.annotate(
_days_worked=Count('work_logs__date', distinct=True),
# === AGGREGATION SAFETY (audit fix, Jun 2026) ===
# All four annotations below ride the SAME payroll_records join, so
# they can't multiply each other. The previous version also folded
# work_logs and warnings counts into this one annotate — Django then
# builds a single query joining ALL the relations, and every payroll
# row is duplicated once per work-log row. Result: the lifetime
# "Total Paid" Sum was multiplied by the worker's log count (the
# distinct=True counts were immune, which is why it went unnoticed).
# work_logs / warnings / projects are aggregated in their own batched
# GROUP BY queries below — same numbers, no shared-join cross product,
# and no per-worker queries inside the loop either.
workers = base.annotate(
_first_payslip_date=Min('payroll_records__date'),
_last_payslip_date=Max('payroll_records__date'),
_total_paid_lifetime=Sum('payroll_records__amount_paid'),
_payslip_count=Count('payroll_records', distinct=True),
_active_warnings=Count('warnings', distinct=True),
).order_by('name')
).prefetch_related('teams', 'certificates').order_by('name')
worker_id_qs = base.values('id')
# Days worked per worker (distinct dates across their work logs)
days_by_worker = dict(
WorkLog.objects.filter(workers__id__in=worker_id_qs)
.values('workers__id')
.annotate(days=Count('date', distinct=True))
.values_list('workers__id', 'days')
)
# Warning count per worker
warnings_by_worker = dict(
WorkerWarning.objects.filter(worker_id__in=worker_id_qs)
.values('worker_id')
.annotate(n=Count('id'))
.values_list('worker_id', 'n')
)
# Distinct project names each worker has appeared on
projects_by_worker = {}
for wid, pname in (
WorkLog.objects.filter(workers__id__in=worker_id_qs)
.values_list('workers__id', 'project__name')
.distinct()
.order_by('project__name')
):
projects_by_worker.setdefault(wid, []).append(pname)
today = timezone.localdate()
thirty_days_out = today + datetime.timedelta(days=30)
rows = []
for w in workers:
projects = list(
Project.objects.filter(work_logs__workers=w).distinct().values_list('name', flat=True)
)
teams = list(w.teams.values_list('name', flat=True))
projects = projects_by_worker.get(w.id, [])
teams = [t.name for t in w.teams.all()] # prefetched — no query
certs = w.certificates.all()
certs_total = certs.count()
certs = list(w.certificates.all()) # prefetched — no query
certs_total = len(certs)
certs_active = 0
certs_expiring = 0
certs_expired = 0
@ -1703,7 +1749,7 @@ def _build_worker_report_context(status=None, project_id=None, team_id=None):
'worker': w,
'projects': projects,
'teams': teams,
'days_worked': w._days_worked or 0,
'days_worked': days_by_worker.get(w.id, 0),
'first_payslip_date': w._first_payslip_date,
'last_payslip_date': w._last_payslip_date,
'total_paid_lifetime': w._total_paid_lifetime or Decimal('0.00'),
@ -1712,7 +1758,7 @@ def _build_worker_report_context(status=None, project_id=None, team_id=None):
'certs_active': certs_active,
'certs_expiring': certs_expiring,
'certs_expired': certs_expired,
'warnings_count': w._active_warnings or 0,
'warnings_count': warnings_by_worker.get(w.id, 0),
})
return rows
@ -1724,8 +1770,8 @@ def worker_batch_report(request):
return HttpResponseForbidden("Admin access required.")
status = request.GET.get('status') or 'all'
project_id = request.GET.get('project') or None
team_id = request.GET.get('team') or None
project_id = _int_param_or_none(request.GET.get('project'))
team_id = _int_param_or_none(request.GET.get('team'))
rows = _build_worker_report_context(status=status, project_id=project_id, team_id=team_id)
@ -1749,8 +1795,8 @@ def worker_batch_report_csv(request):
return HttpResponseForbidden("Admin access required.")
status = request.GET.get('status') or 'all'
project_id = request.GET.get('project') or None
team_id = request.GET.get('team') or None
project_id = _int_param_or_none(request.GET.get('project'))
team_id = _int_param_or_none(request.GET.get('team'))
rows = _build_worker_report_context(status=status, project_id=project_id, team_id=team_id)
@ -1788,18 +1834,27 @@ def worker_batch_report_pdf(request):
from .utils import render_to_pdf
status = request.GET.get('status') or 'all'
project_id = request.GET.get('project') or None
team_id = request.GET.get('team') or None
project_id = _int_param_or_none(request.GET.get('project'))
team_id = _int_param_or_none(request.GET.get('team'))
rows = _build_worker_report_context(status=status, project_id=project_id, team_id=team_id)
# Filter names are display-only — a deleted project/team (stale
# bookmark) degrades to the unfiltered label instead of crashing.
try:
project_name = Project.objects.get(id=project_id).name if project_id else 'All Projects'
except Project.DoesNotExist:
project_name = 'All Projects'
try:
team_name = Team.objects.get(id=team_id).name if team_id else 'All Teams'
except Team.DoesNotExist:
team_name = 'All Teams'
context = {
'rows': rows,
'status': status,
'project_name': (
Project.objects.get(id=project_id).name if project_id else 'All Projects'
),
'team_name': Team.objects.get(id=team_id).name if team_id else 'All Teams',
'project_name': project_name,
'team_name': team_name,
'now': timezone.now(),
'total_workers': len(rows),
}
@ -2537,20 +2592,24 @@ def _build_report_context(start_date, end_date, project_ids=None, team_ids=None)
.values_list('workers__id', 'days')
)
# Adjustment totals per (worker, type) — ONE GROUP BY query instead of
# a per-worker-per-type .aggregate() inside the loop below (which fired
# workers × types separate queries, ~112 on a busy month — audit fix,
# Jun 2026). Same numbers, computed once.
adj_by_worker_type = {
(row['worker_id'], row['type']): row['total'] or Decimal('0.00')
for row in adjustments.values('worker_id', 'type').annotate(total=Sum('amount'))
}
worker_breakdown = []
for wr in worker_records:
w_adjs = adjustments.filter(worker_id=wr['worker__id'])
# Per-type amounts for this worker (only for types that exist in the period).
# Each `adj_values` entry is {'amount': Decimal, 'is_deductive': bool}
# so the template can render "-R 500.00" for deductive types.
adj_values = []
for adj_type in active_adj_types:
amt = w_adjs.filter(type=adj_type).aggregate(
t=Sum('amount'))['t'] or Decimal('0.00')
adj_values.append({
'amount': amt,
'is_deductive': adj_type in DEDUCTIVE_TYPES,
})
adj_values = [{
'amount': adj_by_worker_type.get((wr['worker__id'], adj_type), Decimal('0.00')),
'is_deductive': adj_type in DEDUCTIVE_TYPES,
} for adj_type in active_adj_types]
worker_breakdown.append({
'name': wr['worker__name'],
@ -3035,9 +3094,13 @@ def payroll_dashboard(request):
pending_adj_sub_total += worker_adj_sub
# --- Payment history ---
# prefetch work_logs + adjustments: the History tab template shows
# "{{ record.work_logs.count }} days" and loops record.adjustments.all
# per row — without the prefetch that's 2 queries per visible record
# (audit fix #9, Jun 2026).
paid_records = PayrollRecord.objects.select_related(
'worker'
).order_by('-date', '-id')
).prefetch_related('work_logs', 'adjustments').order_by('-date', '-id')
# --- Recent payments total (last 60 days, inclusive) ---
# 60-day window math: subtract 59 days, not 60. With `>=` that yields
@ -3182,7 +3245,11 @@ def payroll_dashboard(request):
key = (row['project_id'], row['month'].year, row['month'].month)
if key not in project_month_wage:
continue
daily = Decimal(salary) / Decimal('20.00')
# Decimal(str(...)) — on SQLite (local dev) .values() returns a
# float here, and Decimal(float) keeps the float's binary noise;
# str() round-trips cleanly on both backends (MySQL already
# returns Decimal, where this is a no-op).
daily = Decimal(str(salary)) / Decimal('20.00')
project_month_wage[key] += daily * row['worker_count']
# --- 4. Per-project × per-month paid-adjustment net (ONE GROUP BY) ---
@ -3577,10 +3644,23 @@ def payroll_dashboard(request):
# and handles loan repayment deductions — all inside an atomic transaction.
# =============================================================================
class DeductionsExceedEarningsError(Exception):
"""A payment selection's deductions are bigger than its earnings.
Business rule (Konrad, 12 Jun 2026): such payments are REFUSED we
never record a negative PayrollRecord and never clamp to zero. The
admin reduces the deduction (e.g. a smaller loan repayment) or adds
more work days to the selection, then retries. Raising inside the
atomic block below rolls everything back, so the loan balance and
all adjustments stay exactly as they were.
"""
def _process_single_payment(worker_id, selected_log_ids=None, selected_adj_ids=None):
"""
Process payment for one worker inside an atomic transaction.
Returns (payroll_record, log_count, logs_amount) on success, or None if nothing to pay.
Raises DeductionsExceedEarningsError if the net would be negative.
- worker_id: the Worker's PK
- selected_log_ids: list of WorkLog IDs to include (None = all unpaid)
@ -3623,6 +3703,17 @@ def _process_single_payment(worker_id, selected_log_ids=None, selected_adj_ids=N
total_amount = logs_amount + adj_amount
# === GUARD: never record a negative payment ===
# Deductions larger than earnings means there is nothing to pay
# out — refuse instead of writing a negative amount to the books.
# (Exactly zero IS allowed: a repayment that consumes the whole
# wage legitimately settles a loan with a R 0.00 payslip.)
if total_amount < Decimal('0.00'):
raise DeductionsExceedEarningsError(
f'{worker.name}: deductions (R {-adj_amount:,.2f}) exceed '
f'earnings (R {logs_amount:,.2f}) for this selection'
)
# Create the PayrollRecord
payroll_record = PayrollRecord.objects.create(
worker=worker,
@ -3679,11 +3770,19 @@ def process_payment(request, worker_id):
selected_log_ids = [int(x) for x in request.POST.getlist('selected_log_ids') if x.isdigit()]
selected_adj_ids = [int(x) for x in request.POST.getlist('selected_adj_ids') if x.isdigit()]
result = _process_single_payment(
worker_id,
selected_log_ids=selected_log_ids or None,
selected_adj_ids=selected_adj_ids or None,
)
try:
result = _process_single_payment(
worker_id,
selected_log_ids=selected_log_ids or None,
selected_adj_ids=selected_adj_ids or None,
)
except DeductionsExceedEarningsError as e:
messages.error(
request,
f'Payment refused — {e}. Reduce the deduction amount or include '
f'more work days, then try again.'
)
return redirect('payroll_dashboard')
if result is None:
messages.warning(request, f'No pending payments for {worker.name} — already paid or nothing owed.')
@ -3804,14 +3903,20 @@ def _send_payslip_email(request, worker, payroll_record, log_count, logs_amount,
f'Payslip emailed successfully.'
)
except Exception as e:
# Payment is saved — just warn that email failed
if not suppress_messages:
messages.warning(
request,
f'Payment of R {total_amount:,.2f} processed for {worker.name}, '
f'but email delivery failed: {str(e)}'
)
raise # Re-raise so batch_pay can count failures
# Payment is saved — the email is the only thing that failed.
# Batch mode (suppress_messages=True) re-raises so batch_pay can
# count failures in its summary. Interactive callers (the Pay
# button, Pay-Immediately adjustments) get a warning toast
# instead — a raise here would show the admin a 500 page and
# hide the fact that the payment DID go through (the 28 May
# 2026 incident behaviour).
if suppress_messages:
raise # batch_pay catches this and counts it
messages.warning(
request,
f'Payment of R {total_amount:,.2f} processed for {worker.name}, '
f'but email delivery failed: {str(e)}'
)
else:
# No SPARK_RECEIPT_EMAIL configured — just show success
if not suppress_messages:
@ -3855,8 +3960,22 @@ def batch_pay_preview(request):
),
).order_by('name')
# === Active team per worker — ONE batched membership query ===
# get_worker_active_team() runs worker.teams.filter(...).first() per
# worker, which bypasses the prefetch cache and fired a query per row
# (audit fix #10, Jun 2026). Same dict pattern as payroll_dashboard.
active_team_by_id = {t.id: t for t in Team.objects.filter(active=True)}
worker_active_team = {}
for membership in Team.workers.through.objects.filter(
team_id__in=active_team_by_id.keys()
).values('team_id', 'worker_id'):
wid = membership['worker_id']
if wid in worker_active_team:
continue
worker_active_team[wid] = active_team_by_id[membership['team_id']]
for worker in active_workers:
team = get_worker_active_team(worker)
team = worker_active_team.get(worker.id)
# --- In 'schedule' mode, skip workers without a pay schedule ---
if mode == 'schedule':
@ -3869,7 +3988,9 @@ def batch_pay_preview(request):
has_unpaid = True
break
if not has_unpaid:
has_unpaid = worker.adjustments.filter(payroll_record__isnull=True).exists()
# the adjustments prefetch above is already filtered to
# unpaid rows — bool() reads the cache, no extra query
has_unpaid = bool(worker.adjustments.all())
if has_unpaid:
skipped.append({
@ -4002,11 +4123,17 @@ def batch_pay(request):
errors.append(f'Worker ID {worker_id} not found or inactive.')
continue
result = _process_single_payment(
worker_id,
selected_log_ids=log_ids or None,
selected_adj_ids=adj_ids or None,
)
try:
result = _process_single_payment(
worker_id,
selected_log_ids=log_ids or None,
selected_adj_ids=adj_ids or None,
)
except DeductionsExceedEarningsError as e:
# Refused, nothing recorded — name the worker in the summary
# so the admin knows who still needs attention.
errors.append(f'Payment refused — {e}.')
continue
if result is None:
continue # Nothing to pay — silently skip
@ -4066,7 +4193,12 @@ def price_overtime(request):
rate_pct = Decimal(pct)
# Calculate: daily_rate × overtime_fraction × (rate_percentage / 100)
amount = worker.daily_rate * worklog.overtime_amount * (rate_pct / Decimal('100'))
# Quantize to cents explicitly — 875.00 × 0.25 × 1.5 gives
# 328.125, and without this the DB engine rounds the extra
# decimal place silently (engine-dependent half-up/half-even).
amount = (
worker.daily_rate * worklog.overtime_amount * (rate_pct / Decimal('100'))
).quantize(Decimal('0.01'))
if amount > 0:
PayrollAdjustment.objects.create(
@ -4701,7 +4833,10 @@ def preview_payslip(request, worker_id):
unpaid_logs.sort(key=lambda x: x['date'])
log_count = len(unpaid_logs)
log_amount = float(log_count * worker.daily_rate)
# Keep the running totals in Decimal — float() happens only at the
# JsonResponse boundary below, so the preview math is bit-identical
# to what _process_single_payment will actually record.
log_amount = log_count * worker.daily_rate
# Find pending adjustments — include ID and date for split payslip
pending_adjs = worker.adjustments.filter(
@ -4709,10 +4844,10 @@ def preview_payslip(request, worker_id):
).select_related('project')
adjustments_list = []
adj_total = 0.0
adj_total = Decimal('0.00')
for adj in pending_adjs:
sign = '+' if adj.type in ADDITIVE_TYPES else '-'
adj_total += float(adj.amount) if adj.type in ADDITIVE_TYPES else -float(adj.amount)
adj_total += adj.amount if adj.type in ADDITIVE_TYPES else -adj.amount
adjustments_list.append({
'id': adj.id,
# 'type' keeps the raw DB value so any JS that uses it as an
@ -4770,10 +4905,10 @@ def preview_payslip(request, worker_id):
'worker_id_number': worker.id_number,
'day_rate': float(worker.daily_rate),
'days_worked': log_count,
'log_amount': log_amount,
'log_amount': float(log_amount),
'adjustments': adjustments_list,
'adj_total': adj_total,
'net_pay': log_amount + adj_total,
'adj_total': float(adj_total),
'net_pay': float(log_amount + adj_total),
'logs': unpaid_logs,
'active_loans': loans_list,
'pay_period': pay_period,
@ -4806,18 +4941,20 @@ def worker_lookup_ajax(request, worker_id):
if worker.id not in paid_worker_ids:
unpaid_log_count += 1
log_amount = float(unpaid_log_count * worker.daily_rate)
# Decimal accumulation — float() only when the value enters the JSON
# response, so the report card matches the payment engine's numbers.
log_amount = unpaid_log_count * worker.daily_rate
# Net adjustment total: additive types increase pay, deductive types decrease it
pending_adjs = worker.adjustments.filter(payroll_record__isnull=True)
adj_total = 0.0
adj_total = Decimal('0.00')
for adj in pending_adjs:
if adj.type in ADDITIVE_TYPES:
adj_total += float(adj.amount)
adj_total += adj.amount
elif adj.type in DEDUCTIVE_TYPES:
adj_total -= float(adj.amount)
adj_total -= adj.amount
amount_payable = log_amount + adj_total
amount_payable = float(log_amount + adj_total)
# === OUTSTANDING LOANS ===
# Total remaining balance across all active loans and advances

View File

@ -1,12 +1,45 @@
# Parked / deferred work
> Updated 15 May 2026. A small index of features that are designed,
> Updated 12 Jun 2026. A small index of features that are designed,
> half-built, blocked on input, or pending an operator step. When a
> fresh session opens, glance here first to see what's already on
> the workbench.
---
## 🔧 Audit leftovers (12 Jun 2026) — small, deliberate deferrals
The 12 Jun 2026 technical-audit bundle (deployed; see CLAUDE.md
breadcrumb) deliberately left these on the bench:
1. **Team/project batch-report per-row queries.**
`_build_team_report_context` and `_build_project_report_context`
(`core/views.py`) still run a per-team/per-project query loop
(~2 queries × ~6 rows on `/teams/report/` and `/projects/report/`).
Same disease as the fixed worker report; skipped because the pages
are rarely used and the fix pattern is already proven (copy the
batched-GROUP-BY-dict restructure from
`_build_worker_report_context`). Do it next time those pages feel
slow or get touched anyway.
2. **JS vs server decimal-separator inconsistency (cosmetic).**
Browser-side money helpers use `toLocaleString('en-ZA')` → comma
decimals (`R 2 400,00`); server-side `money` filter renders dot
decimals (`R 2 400.00`). Pre-dates the audit; harmonising means
choosing ONE convention and touching both sides. Pure cosmetics —
decide when something user-facing forces the question.
3. **Stale VM safety branches.** Five `pre-*` branches on the VM
(four from Apr 2026 + `pre-audit-deploy-20260612`). After a few
days of confirmed stability post-12-Jun, tell the Flatlogic agent
to delete them all (the 20260612 one anchors autosave `98f66e9`,
whose only real content is now properly committed as `abfae69`).
4. **gitea mirror auth broken** — operator step: Konrad asks Erik
(Flatlogic support) to restore the VM's gitea credentials; then
re-sync with `git push gitea ai-dev` (may need `--force`). Until
then deploys are GitHub→VM one-way. Details in CLAUDE.md "Git
remotes on the VM".
---
## ⏸ Paused — ready to execute (not started, not pushed)
### Site-progress logging — rebuild from scratch (parked)
@ -20,11 +53,10 @@ schema-as-Python pattern, recovery pointers, and the now-superseded
`2026-05-15-post-attendance-flow-v2-*` prior thinking, which stays on
disk).
**Removal status (local only, HARD STOP — not pushed):** Tasks 1-3
done — model/table/UI/routes deleted, migration
`0018_delete_sitereport` drops `core_sitereport`, suite **193 OK**.
Un-pushed pending Konrad's local verification (destructive migration
on the daily-use attendance path).
**Removal status: SHIPPED.** Pushed to origin and confirmed deployed
on production (migration `0018_delete_sitereport` shows `[X]` applied
— verified 12 Jun 2026 during the audit-bundle deploy). Nothing
pending; only the future rebuild remains parked.
---
@ -113,6 +145,38 @@ the 36-commit push `d7015b9..4c25011`.
the Manager/Salaried tests cannot perturb it). Low priority:
investigate a possible date/locale or response-rendering
nondeterminism in that test. **NOT introduced by this feature.**
3. **⏳ PENDING OPERATOR VERIFICATION — Spark Receipt payslip
handling for Salary payments (Konrad testing 17 May 2026).**
Manager/Salaried `Salary` payments email a payslip PDF to Spark
Receipt (`_send_payslip_email`, `SPARK_RECEIPT_EMAIL`) using the
`is_salary` clean single-line layout (no work-log/days table —
mirrors `is_advance`/`is_loan`). Konrad's managers (Konrad,
Christiaan, Fitz) have **variable** monthly pay — there is no fixed
salary; the amount is entered manually per Salary adjustment.
UNKNOWN: whether Spark Receipt's downstream system parses/accepts
this Salary payslip format the same as a normal daily-worker
payslip. Konrad is testing a real Salary payment **later today
(17 May 2026)**. Until confirmed, this is an OPEN external-
integration verification, NOT an in-app defect. If Spark Receipt
mishandles it → follow-up to adjust the `is_salary` payslip
template / email body (`pdf/payslip_pdf.html`,
`email/payslip_email.html`, `core/payslip.html`).
4. **Deferred display tidy-up — `monthly_salary` is misleading for
fixed-pay managers.** `Worker.monthly_salary` is a required field
(no null/blank) but means nothing for variable-pay managers. It is
**inert for their pay/cost** (managers never reach a `WorkLog`, so
`daily_rate` / all WorkLog-derived math never includes them) yet
still **displays** as "monthly salary" + a computed "daily rate"
on the worker detail page, worker batch report, and CSV/PDF
exports — authoritative-looking but meaningless numbers on manager
rows. Pragmatic interim: enter a representative figure (or 0) for
managers. Optional future tidy-up (small, display-only, no pay/cost
impact): hide/relabel the monthly-salary field for `pay_type='fixed'`
in `WorkerForm` + `workers/edit.html`, and suppress/relabel the
monthly/daily columns for managers in `workers/detail.html`,
`workers/batch_report.html`, the workers CSV export, and
`pdf/workers_report_pdf.html`. Low priority — flagged by Konrad
17 May 2026.
> Note: the **Site-progress logging — rebuild from scratch** parked
> entry above is unaffected by this Manager/Salaried entry (it was

View File

@ -1,5 +1,9 @@
@echo off
REM === Local dev launcher ===
REM cd to this script's own folder first (%~dp0 = the .bat's directory) so
REM "python manage.py ..." works no matter where the launcher was started
REM from (the Claude preview runner starts cmd outside the project root).
cd /d "%~dp0"
REM USE_SQLITE=true -> use SQLite, skip MySQL requirement, relax prod-only checks
REM DJANGO_DEBUG=true -> make Django's dev server auto-serve /static/ files so
REM CSS/JS/images load without needing collectstatic (prod