admin/38686-vm
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: admin:8f495064c309d8351c22c2ad97b9f19054698df9
Branches Tags
admin:master
admin:ai-dev
..
compare: admin:503eff67a0867ec8dae0124712b92002a02398b4
Branches Tags
admin:ai-dev
admin:master

No commits in common. "8f495064c309d8351c22c2ad97b9f19054698df9" and "503eff67a0867ec8dae0124712b92002a02398b4" have entirely different histories.
8f495064c3 ... 503eff67a0

9 changed files with 83 additions and 1177 deletions
Show Stats Expand all files Collapse all files

27
CLAUDE.md
View File

@ -242,18 +242,16 @@ change when the file does.
<link rel="stylesheet" href="{% static 'css/custom.css' %}?v={{ deployment_timestamp|default:'1' }}">
```
`deployment_timestamp` comes from `core/context_processors.py::project_context`
via `_compute_cache_bust_token()` — returns the mtime of
`static/css/custom.css` as an integer. The token only changes when the
CSS file is modified, so Cloudflare's edge cache holds each version for
its full 4h TTL and repeat page loads in a session hit the browser
cache (304 Not Modified). Deploys that include a CSS change bump the
mtime → new token → cache busts. Pre-24-Apr-2026 this was
`int(time.time())` per-request, which defeated the CDN cache entirely
(effectively 0% hit rate on CSS). Degraded-mode fallback: if
`custom.css` isn't on disk (e.g., fresh container before
`collectstatic`), the function falls back to the old per-request
timestamp rather than crashing.
`deployment_timestamp` comes from `core/context_processors.py::project_context` as
`int(time.time())` — meaning every Django request generates a new query-string value.
Cloudflare treats each new `?v=...` value as a new URL → `cf-cache-status: MISS` →
fresh fetch from the VM. Users always see the latest CSS as soon as the Django
process restarts.
**Trade-off**: because the timestamp changes every second, CDN cache-hit rate on
CSS is effectively zero. For a low-traffic app this is fine. If traffic grows,
consider switching to a file-mtime-based token so the URL only changes when the
CSS actually changes.
### The pitfall this replaced
Pre-Apr 2026, the template used `{{ request.timestamp|default:'1.0' }}`. But
@ -265,10 +263,9 @@ that only a hard refresh in incognito temporarily fixed. Never use `request.time
in templates — it doesn't exist.
### When CSS changes don't appear on production
1. Confirm Django is rendering a stable URL: `curl -s https://foxlog.flatlogic.app/ | grep -oE 'custom\.css\?v=[^"]+'` — run it twice; the `v=` number must be IDENTICAL across requests. Under the mtime-based token (see previous subsection), the number only changes after `static/css/custom.css` is edited. If it DOES change every request, the fallback branch of `_compute_cache_bust_token()` is active (the CSS file couldn't be stat'd) — check the file exists and is readable.
1. Confirm Django is rendering the new URL: `curl -s https://foxlog.flatlogic.app/ | grep -oE 'custom\.css\?v=[^"]+'` — the `v=` number should change per request (or at least per restart)
2. Confirm the CDN honours it: `curl -sI "https://foxlog.flatlogic.app/static/css/custom.css?cb=test$(date +%s)" | grep -i cache` — expect `cf-cache-status: MISS` then `HIT` on repeat
3. If you've just deployed a CSS change and the old file is still showing: confirm `collectstatic` ran on the VM after pull (Flatlogic doesn't auto-run it). The token is read from `static/css/custom.css` (the SOURCE file) — so editing the source and pushing DOES bump the token, and Cloudflare correctly misses its edge cache on the next request. What breaks is the VM's response to that miss: Apache serves the OLD bytes from `staticfiles/` because `collectstatic` hasn't refreshed the collected copy. You'll see a URL with a new `?v=...` value (confirming the token bumped) but stale bytes behind it. Fix: `python3 manage.py collectstatic --noinput`, then restart the service.
4. If `deployment_timestamp` isn't being injected at all (the `?v=` query string is missing from the rendered URL): check that `core.context_processors.project_context` is listed in `TEMPLATES[0]['OPTIONS']['context_processors']` in `config/settings.py`.
3. If the Django URL still looks like `?v=1.0` (constant), `deployment_timestamp` isn't being injected — check that `core.context_processors.project_context` is listed in `TEMPLATES[0]['OPTIONS']['context_processors']` in `config/settings.py`
### `collectstatic` is required after CSS/JS changes on production
Flatlogic's rebuild does NOT automatically run `collectstatic`. If new CSS is on

40
config/settings.py
View File

@ -300,43 +300,3 @@ if os.getenv('USE_SQLITE', 'false').lower() == 'true':
CSRF_COOKIE_SECURE = False
SESSION_COOKIE_SAMESITE = 'Lax'
CSRF_COOKIE_SAMESITE = 'Lax'
# === DEV-ONLY: Django Debug Toolbar ===
# Loaded ONLY when BOTH DEBUG=true AND USE_SQLITE=true AND we're not
# running tests. This is a deliberately strict gate — the toolbar
# exposes query internals, settings, and request state that should
# never appear in production. The USE_SQLITE half of the check acts
# as a belt-and-suspenders guard against accidentally enabling DEBUG
# on production (which would be its own serious problem, but at least
# wouldn't leak toolbar data).
#
# Test-run skip: Django forces DEBUG=False during `manage.py test`,
# which makes the toolbar emit its own E001 system-check error AND
# leaves template tags referencing the unregistered `djdt` URL
# namespace — both fatal to the test suite. Detecting the test
# command up-front and skipping the install entirely is cleaner than
# trying to work around both symptoms.
import sys as _sys
_IS_RUNNING_TESTS = 'test' in _sys.argv
if DEBUG and _IS_DEV and not _IS_RUNNING_TESTS:
try:
import debug_toolbar # noqa: F401 — probe for installed package
except ImportError:
pass
else:
INSTALLED_APPS += ['debug_toolbar']
# Insert the middleware as early as possible in the chain so it
# captures every request, but AFTER SecurityMiddleware (standard
# recommendation in the toolbar's install docs).
MIDDLEWARE.insert(1, 'debug_toolbar.middleware.DebugToolbarMiddleware')
INTERNAL_IPS = ['127.0.0.1', 'localhost']
DEBUG_TOOLBAR_CONFIG = {
# Don't auto-collapse the SQL panel — the SQL count is the
# main thing we check on every page.
'SHOW_COLLAPSED': False,
# This callback is invoked per-request by the toolbar's middleware.
# Since we only REACH this config block when the triple gate
# (DEBUG + _IS_DEV + not running tests) already passed at settings
# load time, there's nothing to re-check here — just return True.
'SHOW_TOOLBAR_CALLBACK': lambda request: True,
}

12
config/urls.py
View File

@ -11,14 +11,4 @@ urlpatterns = [
if settings.DEBUG:
urlpatterns += static(settings.STATIC_URL, document_root=settings.STATIC_ROOT)
urlpatterns += static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT)
# === DEV-ONLY: Django Debug Toolbar URL include ===
# Matches the conditional load in settings.py. No-op in prod.
# The settings.py gate already guarantees debug_toolbar is importable
# by the time we get here, so no try/except is needed. If we ever
# downgrade to a toolbar version older than v4.0 (which introduced
# debug_toolbar_urls), swap this for `path('__debug__/', include(debug_toolbar.urls))`.
if 'debug_toolbar' in settings.INSTALLED_APPS:
from debug_toolbar.toolbar import debug_toolbar_urls
urlpatterns += debug_toolbar_urls()
urlpatterns += static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT)

41
core/context_processors.py
View File

@ -1,44 +1,13 @@
# === core/context_processors.py ===
# Globals injected into every template render.
#
# `deployment_timestamp` is a cache-bust token on our CSS URL. Historically
# it was `int(time.time())` — a new value every second — which defeated
# Cloudflare's edge cache because every page load saw a different `?v=...`
# string. We now tie the token to `custom.css`'s mtime, so the URL
# changes ONLY when the CSS actually changes. Cloudflare can hold the
# file for its full 4h TTL, and users on repeat visits hit the browser
# cache (304 Not Modified).
import os
import time
from pathlib import Path
from django.conf import settings
# Path to the file whose mtime drives the cache-bust token. Module-level
# constant so tests can monkey-patch it to simulate "file missing".
_CSS_PATH_FOR_TOKEN = Path(settings.BASE_DIR) / 'static' / 'css' / 'custom.css'
def _compute_cache_bust_token():
"""Return an integer cache-bust token.
Normal path: returns the CSS file's mtime (an integer).
Fallback: if the file can't be stat'd (doesn't exist / permission
error / disk issue), returns the current wall-clock time in seconds
— that degrades to the PRE-FIX behaviour (new token per request)
rather than crashing the whole request cycle.
"""
try:
return int(os.path.getmtime(_CSS_PATH_FOR_TOKEN))
except (OSError, FileNotFoundError):
return int(time.time())
def project_context(request):
"""Adds project-specific environment variables to the template context globally."""
"""
Adds project-specific environment variables to the template context globally.
"""
return {
"project_description": os.getenv("PROJECT_DESCRIPTION", ""),
"project_image_url": os.getenv("PROJECT_IMAGE_URL", ""),
# Cache-busts static assets — see _compute_cache_bust_token().
"deployment_timestamp": _compute_cache_bust_token(),
# Used for cache-busting static assets
"deployment_timestamp": int(time.time()),
}

142
core/tests.py
View File

@ -5,7 +5,6 @@
import datetime
from decimal import Decimal
from django.conf import settings
from django.contrib.auth.models import User
from django.test import TestCase
from django.urls import reverse
@ -1238,144 +1237,3 @@ class AdjustmentsTabTests(TestCase):
self.assertIn((self.team.id, self.w2.id), pair_set)
self.assertIn((self.team2.id, self.w1.id), pair_set)
self.assertIn((self.team2.id, self.w2.id), pair_set)
# === Cache-bust token tests ===
# The deployment_timestamp context variable controls the ?v=... query
# string on our CSS URL. It MUST only change when custom.css changes
# — otherwise Cloudflare cache-HIT rate on the CSS drops to zero and
# every page reload re-fetches 64 KB from the VM.
class CacheBustTokenTests(TestCase):
"""Regression tests for the mtime-based cache-bust token."""
def test_token_is_an_integer(self):
"""Token must be int (templates cast to str; float would show a dot)."""
from core.context_processors import project_context
ctx = project_context(request=None)
self.assertIsInstance(ctx['deployment_timestamp'], int)
def test_token_is_stable_across_two_calls(self):
"""Critical property: two back-to-back calls return the same
token — because custom.css hasn't changed between them. This
is the entire point of the mtime-based approach."""
from core.context_processors import project_context
t1 = project_context(request=None)['deployment_timestamp']
t2 = project_context(request=None)['deployment_timestamp']
self.assertEqual(t1, t2)
def test_token_falls_back_if_file_missing(self):
"""If static/css/custom.css is somehow missing (fresh
container pre-collectstatic), we must NOT crash. We fall back
to int(time.time()) so every page still renders."""
import core.context_processors as cp
try:
# Monkey-patch so the function sees a guaranteed-missing path.
# (The function itself is not patched — only the path constant.)
cp._CSS_PATH_FOR_TOKEN = cp.Path('/definitely/does/not/exist.css')
# Should return an int and NOT raise.
token = cp._compute_cache_bust_token()
self.assertIsInstance(token, int)
finally:
# Reset the module-level path constant so other tests (or reruns)
# get the real CSS file back.
cp._CSS_PATH_FOR_TOKEN = cp.Path(settings.BASE_DIR) / 'static' / 'css' / 'custom.css'
# === REGRESSION: adjustment project-attribution double-count ===
# Ticket summary: commit 61c485f replaced a per-project loop using SQL
# OR (`Q(project=P) | Q(work_log__project=P)`) with two SEPARATE
# filter+GROUP BY queries that were summed in Python. Any adjustment
# with BOTH `project_id` AND `work_log.project_id` set matched both
# queries and got counted TWICE.
#
# Every Overtime adjustment fits that shape — `price_overtime()` in
# views.py sets both FKs to the same project. So every unpaid
# Overtime silently inflated the outstanding-costs dashboard by its
# own amount. This test locks in the "count once" behaviour.
class PayrollDashboardAdjustmentAggregationTests(TestCase):
"""Regression tests for the per-project adjustment aggregation used by
the payroll dashboard. An adjustment with both `project` and
`work_log.project` set must contribute its amount ONCE to that
project, not twice."""
def setUp(self):
# Admin so the payroll_dashboard view accepts us (is_admin helper
# returns True for is_staff OR is_superuser).
self.admin = User.objects.create_user(
username='double-count-admin',
password='pw',
is_staff=True,
is_superuser=True,
)
self.client.force_login(self.admin)
# One active project, one worker, one fresh work log.
self.project = Project.objects.create(
name='Double-Count Test Site',
start_date=datetime.date(2026, 4, 1),
active=True,
)
self.worker = Worker.objects.create(
name='DC Worker',
id_number='DC1',
monthly_salary=Decimal('10000'),
)
self.work_log = WorkLog.objects.create(
date=datetime.date(2026, 4, 23),
project=self.project,
supervisor=self.admin,
)
# NOTE: deliberately no workers on the log — we do NOT want
# unpaid-log wage cost to pollute this test; we only want to
# measure the adjustment contribution.
# THE KEY SHAPE: one unpaid adjustment with BOTH FKs set to the
# same project. This mirrors how price_overtime() creates every
# Overtime adjustment (adj.project = worklog.project, and
# adj.work_log = worklog).
PayrollAdjustment.objects.create(
worker=self.worker,
type='Overtime', # additive → contributes positively
amount=Decimal('500'),
project=self.project,
work_log=self.work_log,
date=datetime.date(2026, 4, 23),
description='Regression-test Overtime',
)
def test_overtime_with_both_project_and_work_log_counts_once(self):
"""An unpaid Overtime with BOTH project + work_log.project set
must contribute its R500 amount once to outstanding-costs —
not R1000.
Before the Coalesce fix, the batched aggregates summed the
direct-project group + the work_log-project group separately
in Python, so an Overtime adjustment landed in both and got
double-counted. After the fix (Coalesce('project_id',
'work_log__project_id')) each adjustment row is attributed to
exactly one effective project.
"""
response = self.client.get(reverse('payroll_dashboard'))
self.assertEqual(response.status_code, 200)
outstanding = response.context['outstanding_project_costs']
# Find the entry for our test project by name (the shape is
# {'name': str, 'cost': Decimal} — no 'id' key).
ours = next(
(p for p in outstanding if p['name'] == self.project.name),
None,
)
self.assertIsNotNone(
ours,
"Test project should appear in outstanding_project_costs "
"(its unpaid Overtime is non-zero).",
)
self.assertEqual(
ours['cost'],
Decimal('500'),
"Overtime adjustment with both project + work_log.project "
"FKs set must count ONCE (R500), not twice (R1000). If "
"this fails with R1000 the project-attribution double-"
"count bug has reappeared.",
)

278
core/views.py
View File

@ -12,7 +12,7 @@ from django.shortcuts import render, redirect, get_object_or_404
from django.utils import timezone
from django.db import transaction
from django.db.models import Sum, Count, Q, F, Prefetch, Max, Min
from django.db.models.functions import Coalesce, TruncMonth
from django.db.models.functions import TruncMonth
from django.contrib import messages
from django.contrib.auth.decorators import login_required
from django.http import JsonResponse, HttpResponseForbidden, HttpResponse
@ -2617,30 +2617,6 @@ def payroll_dashboard(request):
pending_adj_sub_total = Decimal('0.00') # Unpaid deductive adjustments
all_ot_data = [] # For the Price Overtime modal
# === PRE-COMPUTED LOOKUPS — avoid per-worker SELECTs in the loop below ===
# Previously the loop fired:
# - one `Loan.objects.filter(worker=w, active=True).exists()` per worker
# - one `worker.teams.filter(active=True).first()` per worker (via
# get_worker_active_team) — which fires a fresh SELECT even though
# active_workers was prefetched, because `.filter()` bypasses the
# prefetch cache.
# We batch both into dict lookups keyed by worker_id.
workers_with_active_loan = set(
Loan.objects.filter(active=True).values_list('worker_id', flat=True).distinct()
)
# Map worker_id → first active Team instance (mirrors get_worker_active_team).
# We load every active team once, then walk the through-table to find the
# first active team per worker.
active_team_by_id = {t.id: t for t in Team.objects.filter(active=True)}
worker_active_team = {}
for membership in Team.workers.through.objects.filter(
team_id__in=active_team_by_id.keys()
).values('team_id', 'worker_id'):
wid = membership['worker_id']
if wid in worker_active_team:
continue
worker_active_team[wid] = active_team_by_id[membership['team_id']]
for worker in active_workers:
# Find unpaid work logs for this worker.
# A log is "unpaid for this worker" if no PayrollRecord links
@ -2692,8 +2668,7 @@ def payroll_dashboard(request):
# --- Overdue detection ---
# A worker is "overdue" if they have unpaid work from a completed pay period.
# Uses their team's pay schedule to determine the cutoff date.
# PERF: team lookup via pre-computed dict (no per-worker SELECT).
team = worker_active_team.get(worker.id)
team = get_worker_active_team(worker)
team_name = team.name if team else ''
earliest_unpaid = min((l.date for l in unpaid_logs), default=None) if unpaid_logs else None
is_overdue = False
@ -2703,8 +2678,7 @@ def payroll_dashboard(request):
cutoff = period_start - datetime.timedelta(days=1)
is_overdue = earliest_unpaid <= cutoff
# PERF: loan membership via pre-computed set (no per-worker SELECT).
has_loan = worker.id in workers_with_active_loan
has_loan = Loan.objects.filter(worker=worker, active=True).exists()
# Most recent project — used by the "Adjust" button to pre-select project
last_project_id = unpaid_logs[-1].project_id if unpaid_logs else None
@ -2744,16 +2718,31 @@ def payroll_dashboard(request):
# --- Outstanding cost per project ---
# Check per-worker: a WorkLog is "unpaid for worker X" if no PayrollRecord
# links BOTH that log AND that worker. This handles partially-paid logs.
#
# PERF: materialise the active-project list once and reuse it for both
# the outstanding-costs loop and the chart-data loop below. Previously
# each loop re-queried `Project.objects.filter(active=True)`, firing the
# same SELECT twice per dashboard render.
active_projects_list = list(Project.objects.filter(active=True))
active_project_ids = [p.id for p in active_projects_list]
outstanding_project_costs = []
for project in Project.objects.filter(active=True):
project_outstanding = Decimal('0.00')
# Unpaid work log costs — check each worker individually
for log in project.work_logs.prefetch_related('payroll_records', 'workers').all():
paid_worker_ids = {pr.worker_id for pr in log.payroll_records.all()}
for w in log.workers.all():
if w.id not in paid_worker_ids:
project_outstanding += w.daily_rate
# Unpaid adjustments for this project
unpaid_adjs = PayrollAdjustment.objects.filter(
payroll_record__isnull=True
).filter(Q(project=project) | Q(work_log__project=project))
for adj in unpaid_adjs:
if adj.type in ADDITIVE_TYPES:
project_outstanding += adj.amount
elif adj.type in DEDUCTIVE_TYPES:
project_outstanding -= adj.amount
if project_outstanding != 0:
outstanding_project_costs.append({
'name': project.name,
'cost': project_outstanding,
})
# === CHART DATE-WINDOW SETUP (moved up so the batched queries below can
# also use it) ===
# --- Chart data: last 6 months ---
today = timezone.now().date()
chart_months = []
for i in range(5, -1, -1):
@ -2767,81 +2756,6 @@ def payroll_dashboard(request):
chart_labels = [
datetime.date(y, m, 1).strftime('%b %Y') for y, m in chart_months
]
six_months_ago_date = datetime.date(chart_months[0][0], chart_months[0][1], 1)
# === BATCHED AGGREGATES: one SQL query per concept instead of per-project ===
# Previously we looped over each active project and issued:
# - 1 SELECT of WorkLog (with workers prefetch) per project
# - 1 SELECT of PayrollAdjustment (unpaid) per project
# - 1 SELECT of WorkLog (workers prefetch) per project × 6 months
# - 1 SELECT of PayrollAdjustment (paid) per project × 6 months
# On a ~7-project dataset that's ~7+7+42+42 ≈ 98 SQL round-trips.
# The rewrite replaces those with 4 GROUP-BY queries that return
# project_id (and month, where relevant) → total, plus one query for
# per-log paid-worker sets.
# --- 1. Unpaid-work-log cost per project ---
# We can't do pure SQL aggregation for this because a WorkLog can be
# partially paid (one worker of two). We still need per-log inspection,
# BUT we can load all unpaid-or-partially-paid logs + their workers +
# payroll_records in a bounded set of queries using prefetch_related
# rather than looping one project at a time.
project_outstanding_map = {pid: Decimal('0.00') for pid in active_project_ids}
all_project_logs = WorkLog.objects.filter(
project_id__in=active_project_ids
).prefetch_related('payroll_records', 'workers')
for log in all_project_logs:
paid_worker_ids = {pr.worker_id for pr in log.payroll_records.all()}
for w in log.workers.all():
if w.id not in paid_worker_ids:
project_outstanding_map[log.project_id] += w.daily_rate
# --- 2. Unpaid-adjustment net per project (ONE GROUP BY via Coalesce) ---
# Each unpaid adjustment is attributed to exactly ONE project: its
# direct FK (`project_id`) if set, otherwise its work_log's project.
# This mirrors the original OR-filter semantics
# (`Q(project=P) | Q(work_log__project=P)`) which naturally dedupes:
# a row with BOTH FKs pointing at the same project matches ONCE.
#
# Why Coalesce matters: every Overtime adjustment is created by
# price_overtime() with BOTH adj.project AND adj.work_log.project set
# to the same project. A naive "filter by direct FK + filter by
# work_log FK + sum both maps in Python" approach double-counts
# every Overtime row. Coalesce picks ONE effective project per row
# so each amount contributes to the outstanding-costs card once.
unpaid_adj_rows = (
PayrollAdjustment.objects
.filter(payroll_record__isnull=True)
.filter(
Q(project_id__in=active_project_ids)
| Q(work_log__project_id__in=active_project_ids)
)
.annotate(
effective_project_id=Coalesce('project_id', 'work_log__project_id')
)
.values('effective_project_id', 'type')
.annotate(total=Sum('amount'))
)
for row in unpaid_adj_rows:
pid = row['effective_project_id']
# Only contribute to active projects we're tracking.
if pid not in project_outstanding_map:
continue
total = row['total'] or Decimal('0.00')
if row['type'] in ADDITIVE_TYPES:
project_outstanding_map[pid] += total
elif row['type'] in DEDUCTIVE_TYPES:
project_outstanding_map[pid] -= total
outstanding_project_costs = []
for project in active_projects_list:
cost = project_outstanding_map[project.id]
if cost != 0:
outstanding_project_costs.append({
'name': project.name,
'cost': cost,
})
# Monthly payroll totals
paid_by_month_qs = PayrollRecord.objects.annotate(
@ -2853,72 +2767,28 @@ def payroll_dashboard(request):
}
chart_totals = [paid_by_month.get((y, m), 0) for y, m in chart_months]
# --- 3. Per-project × per-month work-log cost (for stacked bar chart) ---
# Aggregate worker×log rows directly in SQL: one GROUP BY
# (project_id, year, month) returns all we need.
project_month_wage = {
(pid, y, m): Decimal('0.00')
for pid in active_project_ids for y, m in chart_months
}
wage_rows = WorkLog.objects.filter(
project_id__in=active_project_ids,
date__gte=six_months_ago_date,
).annotate(month=TruncMonth('date')).values(
'project_id', 'month', 'workers__monthly_salary'
).annotate(worker_count=Count('workers'))
# Each row = one (project, month, distinct salary) with how many workers
# at that salary were logged. Multiply by daily_rate (salary / 20) × count.
for row in wage_rows:
salary = row['workers__monthly_salary']
if salary is None:
continue
key = (row['project_id'], row['month'].year, row['month'].month)
if key not in project_month_wage:
continue
daily = Decimal(salary) / Decimal('20.00')
project_month_wage[key] += daily * row['worker_count']
# --- 4. Per-project × per-month paid-adjustment net (ONE GROUP BY) ---
# Same Coalesce trick as site #2: pick ONE effective project per
# adjustment row so we don't double-count the stacked-chart bars
# for Overtime (which always has both FKs pointing at the same
# project — see note on the unpaid block above).
paid_adj_rows = (
PayrollAdjustment.objects
.filter(payroll_record__isnull=False, date__gte=six_months_ago_date)
.filter(
Q(project_id__in=active_project_ids)
| Q(work_log__project_id__in=active_project_ids)
)
.annotate(
effective_project_id=Coalesce('project_id', 'work_log__project_id'),
month=TruncMonth('date'),
)
.values('effective_project_id', 'month', 'type')
.annotate(total=Sum('amount'))
)
# Accumulate add/sub per (project, year, month) keys in Python.
paid_adj_add = {}
paid_adj_sub = {}
for row in paid_adj_rows:
pid = row['effective_project_id']
if pid not in project_outstanding_map: # only active projects
continue
key = (pid, row['month'].year, row['month'].month)
total = row['total'] or Decimal('0.00')
if row['type'] in ADDITIVE_TYPES:
paid_adj_add[key] = paid_adj_add.get(key, Decimal('0.00')) + total
elif row['type'] in DEDUCTIVE_TYPES:
paid_adj_sub[key] = paid_adj_sub.get(key, Decimal('0.00')) + total
# Per-project monthly costs (for stacked bar chart)
project_chart_data = []
for project in active_projects_list:
for project in Project.objects.filter(active=True):
monthly_data = []
for y, m in chart_months:
key = (project.id, y, m)
month_cost = project_month_wage.get(key, Decimal('0.00'))
month_cost += paid_adj_add.get(key, Decimal('0.00'))
month_cost -= paid_adj_sub.get(key, Decimal('0.00'))
month_cost = Decimal('0.00')
month_logs = project.work_logs.filter(
date__year=y, date__month=m
).prefetch_related('workers')
for log in month_logs:
for w in log.workers.all():
month_cost += w.daily_rate
# Include paid adjustments for this project in this month
paid_adjs = PayrollAdjustment.objects.filter(
payroll_record__isnull=False,
date__year=y, date__month=m,
).filter(Q(project=project) | Q(work_log__project=project))
for adj in paid_adjs:
if adj.type in ADDITIVE_TYPES:
month_cost += adj.amount
elif adj.type in DEDUCTIVE_TYPES:
month_cost -= adj.amount
monthly_data.append(float(month_cost))
if any(v > 0 for v in monthly_data):
project_chart_data.append({
@ -2931,9 +2801,9 @@ def payroll_dashboard(request):
# This powers the "By Worker" toggle on the Monthly Payroll Totals chart.
# Only ~14 workers x 6 months = tiny dataset, so we embed it all as JSON
# and switching between workers is instant (no server round-trips).
#
# `six_months_ago_date` is already defined above (hoisted next to the
# date-window setup) and reused here.
# Starting date for the 6-month window (first day of the oldest chart month)
six_months_ago_date = datetime.date(chart_months[0][0], chart_months[0][1], 1)
# Query 1: Total amount paid per worker per month.
# Uses database-level grouping — one query for ALL workers at once.
@ -2978,13 +2848,8 @@ def payroll_dashboard(request):
# Base pay is reverse-engineered from the net total:
# amount_paid = base + overtime + bonus + new_loan - deduction - loan_repayment - advance
# So: base = amount_paid - overtime - bonus - new_loan + deduction + loan_repayment + advance
#
# PERF: reuse `active_workers` (already loaded + cached at the top of the
# function) instead of re-querying Worker.objects.filter(active=True).
# Same ordered row-set; saves an SQL round-trip. The unused prefetches
# on `active_workers` are already materialised so they cost nothing extra.
worker_chart_data = {}
for worker in active_workers:
for worker in Worker.objects.filter(active=True).order_by('name'):
months_data = []
has_any_data = False
@ -3038,25 +2903,16 @@ def payroll_dashboard(request):
)['total'] or Decimal('0.00')
# --- Active projects and workers for modal dropdowns ---
# `active_workers` is reused (already loaded + evaluated by the workers_data
# loop). For the modal-dropdown context key we alias it as `all_workers`
# so the template name stays descriptive.
all_workers = active_workers
active_projects = Project.objects.filter(active=True).order_by('name')
all_teams = Team.objects.filter(active=True).prefetch_related(
# PERF: prefetch only the active workers so the template's
# `team.workers.all` (and our map below) already filters to active
# without re-querying. Using `.filter()` on the plain `workers`
# accessor bypasses Django's prefetch cache and fires one SELECT
# per team — an N+1 we need to avoid.
Prefetch('workers', queryset=Worker.objects.filter(active=True), to_attr='active_workers_cached')
).order_by('name')
all_workers = Worker.objects.filter(active=True).order_by('name')
all_teams = Team.objects.filter(active=True).prefetch_related('workers').order_by('name')
# Team-workers map for auto-selecting workers when a team is picked.
# Uses the prefetched `active_workers_cached` list — no extra queries.
# Team-workers map for auto-selecting workers when a team is picked
team_workers_map = {}
for team in all_teams:
team_workers_map[str(team.id)] = [w.id for w in team.active_workers_cached]
team_workers_map[str(team.id)] = list(
team.workers.filter(active=True).values_list('id', flat=True)
)
# NOTE: Pass raw Python objects here, NOT json.dumps() strings.
# The template uses Django's |json_script filter which handles
@ -3170,17 +3026,10 @@ def payroll_dashboard(request):
# main sort key has ties (e.g. two adjustments on the same date).
adjustments = adjustments.order_by(sort_field, '-id')
# --- Pagination: 50 rows per page (flat view only) ---
# PERF: build the paginator first so we can reuse its cached `count`
# for the "Total adjustments" stat card below — avoids a duplicate
# `SELECT COUNT(*) FROM core_payrolladjustment`.
paginator = Paginator(adjustments, 50)
adj_page = paginator.get_page(request.GET.get('page', 1))
# --- Stats cards (all computed BEFORE pagination cuts the rows) ---
# --- Stats cards (all computed BEFORE pagination) ---
# These numbers always reflect what the current filter produces,
# not just what fits on the current page.
adj_total_count = paginator.count
adj_total_count = adjustments.count()
unpaid_qs = adjustments.filter(payroll_record__isnull=True)
adj_unpaid_count = unpaid_qs.count()
adj_unpaid_sum = unpaid_qs.aggregate(
@ -3205,6 +3054,10 @@ def payroll_dashboard(request):
if group_by in ('type', 'worker'):
adj_groups = _group_adjustments(list(adjustments), group_by)
# --- Pagination: 50 rows per page (flat view only) ---
paginator = Paginator(adjustments, 50)
adj_page = paginator.get_page(request.GET.get('page', 1))
# --- Everything the Adjustments tab template will need ---
context.update({
'adj_page': adj_page,
@ -3230,11 +3083,8 @@ def payroll_dashboard(request):
# 'adjustment_types' context var (which is TYPE_CHOICES tuples
# used by the Add/Edit adjustment modals).
'adj_type_choices': list(ADDITIVE_TYPES) + list(DEDUCTIVE_TYPES),
# PERF: reuse `all_workers`/`all_teams` (already cached above for
# the Add-Adjustment modal) — same row-set, same ordering, so no
# need to re-query the database for the filter popovers.
'all_workers_for_filter': all_workers,
'all_teams_for_filter': all_teams,
'all_workers_for_filter': Worker.objects.filter(active=True).order_by('name'),
'all_teams_for_filter': Team.objects.filter(active=True).order_by('name'),
# Task 4 will use this to decide +/- signs on each row.
'additive_types': list(ADDITIVE_TYPES),
# === CROSS-FILTER SOURCE: (team_id, worker_id) PAIRS ===

175
docs/plans/2026-04-24-perf-quick-wins-design.md
View File

@ -1,175 +0,0 @@
# Perf Quick-Wins Pass — Design (24 Apr 2026)
## Origin
Konrad, after a long stretch of feature work (Inline Filters + Adjustments
tab + filter-bar v2):
> _"the app feel a bit sluggish especially changing between main spaces.
> Go through the app systematically and look for bugs and un optimized
> code. systematically go through the code and expertly and thoroughly
> review and fix it."_
Presented three scopes (A quick-wins / B focused dashboard pass / C full
systematic audit). Konrad picked **A — quick-wins first**, on the
principle that perf work is notorious for "big rewrite that didn't help."
If A moves the needle, we can stop; if not, we escalate with evidence.
## Goal
Make navigation between main spaces (Dashboard ↔ Payroll ↔ Workers ↔
Report ↔ Teams ↔ Projects) feel snappier. Ship in 1-3 commits. No
architecture changes. Every change individually revertible.
## Who it's for
Everyone who uses the app — most immediately Konrad, who navigates
between Dashboard and Payroll dozens of times a day.
## What we already know (pre-measurement)
- `payroll_dashboard.html` is 213 KB / 4,147 lines — all 4 tabs rendered
server-side even when only one is visible. Addressed in plan B, not A.
- `deployment_timestamp` context var is `int(time.time())` per-request
→ `custom.css?v=<timestamp>` is a new URL every second → Cloudflare
edge-cache HIT rate on CSS is effectively 0 → every page load fetches
64 KB of CSS from the VM. Documented as a trade-off in CLAUDE.md.
This is almost certainly the biggest single contributor to the
"heavy navigation" feel.
- 49 `select_related`/`prefetch_related` calls vs 91
`.all()/.first()/.count()` calls in `views.py`. Not damning but worth
pointing at hot-path views.
## Scope — 4 changes, in order
### 1. Fix `deployment_timestamp` to bust cache only on real deploys
**File:** `core/context_processors.py`
Today:
```python
'deployment_timestamp': int(time.time()),
```
After:
```python
# Cache-bust token tied to the CSS file's mtime — only changes when
# custom.css actually changes. Falls back to int(time.time()) if the
# file isn't on disk yet (fresh container, pre-collectstatic).
try:
_css_path = settings.BASE_DIR / 'static' / 'css' / 'custom.css'
_token = int(os.path.getmtime(_css_path))
except (OSError, FileNotFoundError):
_token = int(time.time())
```
Effect: the `?v=...` query string stays constant across requests until
`custom.css` is modified. Cloudflare can finally hold the file at its
edge for its full 4h TTL. Repeat navigation within a session drops from
"fetch 64 KB from VM" to "304 Not Modified" from the browser cache,
after the first hit in a 4h window.
**Degraded-mode guarantee:** if the file is missing (shouldn't happen in
normal dev or prod, but could in a fresh container), we degrade to
today's behaviour (per-request timestamp) rather than crash.
### 2. Profile + fix N+1 on the two busiest pages
**Pages:** `/` (dashboard) and `/payroll/` (payroll dashboard — all 4
tabs — Pending, History, Loans, Adjustments).
**Tool:** Django Debug Toolbar, added to `requirements.txt` as a
dev-only dependency. Gated in `config/settings.py` so it only
initialises when `DJANGO_DEBUG=true` AND `USE_SQLITE=true` (never
loads in prod).
**Process:**
1. Install toolbar, confirm the SQL panel loads on `/`.
2. Navigate to `/`, read the SQL tab: flag any query count > ~20,
any row with `+N duplicate queries`, any view of the queryset
that could be answered with `select_related`/`prefetch_related`/
`annotate(Count/Sum)`.
3. Fix each flag with the minimal ORM change. One fix = one commit.
4. Re-run, confirm query count dropped, confirm no test regressions.
5. Repeat for `/payroll/?status=pending`, `/payroll/?status=history`,
`/payroll/?status=loans`, `/payroll/?status=adjustments`.
**Likely suspects** (prediction — to be confirmed by toolbar):
- **Dashboard cert-expiry card** — aggregates expired/expiring certs
across active workers. If it loops in Python instead of `annotate`-
plus-filter, that's an N+1.
- **Pending payments table** — worker + team + overdue calc per row.
The overdue check calls `get_pay_period(team)` per worker; if teams
aren't prefetched we're firing one SELECT per row.
- **Adjustments tab groupings** — we fixed `worker.teams.first()` →
`worker.teams.all()` once already (commit `06b3315`); worth
double-checking grouped view for similar patterns.
**Out of scope for this step:** any fix that requires a template
rewrite. If something needs more than a `.select_related()` /
`.prefetch_related()` / `.annotate()` tweak, it goes on the plan-B list.
### 3. Double-check WeasyPrint is not eager-imported anywhere
**File:** `core/utils.py`, `core/views.py`.
We already lazy-import WeasyPrint in `render_to_pdf()` (per CLAUDE.md).
I'll grep to confirm nothing else on the app-boot path imports
`weasyprint` at module level. If anything does, move it into a function
body. 10 minutes, zero-risk.
### 4. Commit message includes before/after measurement
The final commit's message records:
- Page size bytes (DOM serialized) for `/` and `/payroll/` before & after
- Network request count on a cold cache hit
- SQL query count on both pages
If the numbers don't materially improve after steps 1-3, I stop. We
don't press on to plan B without evidence that plan A helped (or at
least surfaced what's actually slow).
## What I will NOT touch in this pass
- Splitting `payroll_dashboard.html` into tab partials
- Any refactoring of `views.py` or extraction of helpers
- Any visual / UX change
- Tests — query-count changes don't break the existing tests (they
assert URL contract + output shape, not query plans). If a test
genuinely needs updating because I materially changed a view's
behaviour, I'll note why in the commit
## Risks + rollback
All four changes are individually revertible. Biggest risks:
- **mtime-based token misfires on fresh containers** — mitigated by
try/except fallback to today's behaviour
- **A `select_related` fix changes query semantics** — e.g., eager
loading a nullable FK that used to be accessed lazily-with-None. Low
risk on Django's ORM, but the test suite (65 tests, all passing at
HEAD `503eff6`) will catch any behavioural regression
- **Django Debug Toolbar pulled in in prod** — mitigated by double-gate
(DEBUG=true AND USE_SQLITE=true) in `config/settings.py`
Rollback: `git revert <sha>` on the offending commit. No data, schema,
or URL-contract impact.
## Out of scope (explicit non-goals)
- Plan B / C items (template splitting, written baseline doc, whole-app
measurement)
- Moving CDN assets to local / self-hosted
- Changing Flatlogic's `runserver` → gunicorn
- Turning on HTTP/2 push, service workers, or other frontend perf tooling
- Any refactor that requires a migration
## Next step
Generate an implementation plan via the writing-plans skill
(task-by-task, bite-sized steps) and then execute via
subagent-driven-development. Auto mode is active — proceed
continuously, no mid-execution checkpoints (plan A is 4 small
mechanical changes; a checkpoint adds overhead without value).
Ship alongside current `ai-dev` HEAD (`503eff6`) in the same branch.

542
docs/plans/2026-04-24-perf-quick-wins-plan.md
View File

@ -1,542 +0,0 @@
# Perf Quick-Wins Pass — Implementation Plan
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development to implement this plan task-by-task. Auto mode is active — execute continuously, no mid-execution checkpoints.
**Goal:** Make main-space navigation feel snappier via targeted ORM + static-asset fixes. No architecture change.
**Architecture:** Four small mechanical changes: (1) mtime-based CSS cache-bust token so Cloudflare can actually cache `custom.css`; (2) install & gate Django Debug Toolbar as a dev-only profiling tool; (3) fix any N+1 hotspots the toolbar surfaces on `/` and `/payroll/` via targeted `select_related`/`prefetch_related`/`annotate`; (4) record before/after numbers in the final commit message.
**Tech Stack:** Django 5.2.7 ORM; `django-debug-toolbar` (new dev-only dep); `os.path.getmtime` for static-asset cache-busting.
**Design doc:** `docs/plans/2026-04-24-perf-quick-wins-design.md` (committed as `d1490c4`).
**Starting HEAD:** `d1490c4` on branch `ai-dev` (clean working tree).
**Expected net change:** ~100-200 lines across 5-6 files + 1 requirements bump.
---
## Task 1: mtime-based cache-bust token + regression test
**Goal:** The `deployment_timestamp` context var changes only when `static/css/custom.css` actually changes, not on every HTTP request.
**Files:**
- Modify: `core/context_processors.py` (replace `int(time.time())` with `_compute_cache_bust_token()`)
- Test: `core/tests.py` (add `CacheBustTokenTests` class)
**Step 1: Write the failing tests**
Append to `core/tests.py` (find a spot after existing context-processor / URL-contract tests; conventional place is at the bottom of the file):
```python
# === Cache-bust token tests ===
# The deployment_timestamp context variable controls the ?v=... query
# string on our CSS URL. It MUST only change when custom.css changes
# — otherwise Cloudflare cache-HIT rate on the CSS drops to zero and
# every page reload re-fetches 64 KB from the VM.
class CacheBustTokenTests(TestCase):
"""Regression tests for the mtime-based cache-bust token."""
def test_token_is_an_integer(self):
"""Token must be int (templates cast to str; float would show a dot)."""
from core.context_processors import project_context
ctx = project_context(request=None)
self.assertIsInstance(ctx['deployment_timestamp'], int)
def test_token_is_stable_across_two_calls(self):
"""Critical property: two back-to-back calls return the same
token — because custom.css hasn't changed between them. This
is the entire point of the mtime-based approach."""
from core.context_processors import project_context
t1 = project_context(request=None)['deployment_timestamp']
t2 = project_context(request=None)['deployment_timestamp']
self.assertEqual(t1, t2)
def test_token_falls_back_if_file_missing(self):
"""If static/css/custom.css is somehow missing (fresh
container pre-collectstatic), we must NOT crash. We fall back
to int(time.time()) so every page still renders."""
import core.context_processors as cp
original = cp._compute_cache_bust_token
try:
# Monkey-patch so the function sees a guaranteed-missing path.
cp._CSS_PATH_FOR_TOKEN = cp.Path('/definitely/does/not/exist.css')
# Should return an int and NOT raise.
token = cp._compute_cache_bust_token()
self.assertIsInstance(token, int)
finally:
# Reset the module-level cache so other tests get the real value.
cp._CSS_PATH_FOR_TOKEN = cp.Path(settings.BASE_DIR) / 'static' / 'css' / 'custom.css'
```
Add the `from django.conf import settings` import at the top of `core/tests.py` if not already present.
**Step 2: Run the tests to verify they fail**
```bash
USE_SQLITE=true DJANGO_DEBUG=true python manage.py test core.tests.CacheBustTokenTests -v 2
```
Expected: 3 failures (function `_compute_cache_bust_token` doesn't exist).
**Step 3: Implement the token function**
Replace the entire contents of `core/context_processors.py` with:
```python
# === core/context_processors.py ===
# Globals injected into every template render.
#
# `deployment_timestamp` is a cache-bust token on our CSS URL. Historically
# it was `int(time.time())` — a new value every second — which defeated
# Cloudflare's edge cache because every page load saw a different `?v=...`
# string. We now tie the token to `custom.css`'s mtime, so the URL
# changes ONLY when the CSS actually changes. Cloudflare can hold the
# file for its full 4h TTL, and users on repeat visits hit the browser
# cache (304 Not Modified).
import os
import time
from pathlib import Path
from django.conf import settings
# Path to the file whose mtime drives the cache-bust token. Module-level
# constant so tests can monkey-patch it to simulate "file missing".
_CSS_PATH_FOR_TOKEN = Path(settings.BASE_DIR) / 'static' / 'css' / 'custom.css'
def _compute_cache_bust_token():
"""Return an integer cache-bust token.
Normal path: returns the CSS file's mtime (an integer).
Fallback: if the file can't be stat'd (doesn't exist / permission
error / disk issue), returns the current wall-clock time in seconds
— that degrades to the PRE-FIX behaviour (new token per request)
rather than crashing the whole request cycle.
"""
try:
return int(os.path.getmtime(_CSS_PATH_FOR_TOKEN))
except (OSError, FileNotFoundError):
return int(time.time())
def project_context(request):
"""Adds project-specific environment variables to the template context globally."""
return {
"project_description": os.getenv("PROJECT_DESCRIPTION", ""),
"project_image_url": os.getenv("PROJECT_IMAGE_URL", ""),
# Cache-busts static assets — see _compute_cache_bust_token().
"deployment_timestamp": _compute_cache_bust_token(),
}
```
**Step 4: Run the tests to verify they pass**
```bash
USE_SQLITE=true DJANGO_DEBUG=true python manage.py test core.tests.CacheBustTokenTests -v 2
```
Expected: 3 passes.
**Step 5: Run the full test suite to catch any regression**
```bash
USE_SQLITE=true DJANGO_DEBUG=true python manage.py test core.tests -v 2
```
Expected: 68 passes (65 pre-existing + 3 new). If anything else fails, stop and investigate.
**Step 6: Update CLAUDE.md's "Static Assets & Cache-Busting" note**
In `CLAUDE.md`, find the section "How cache-busting works now" — specifically the paragraph beginning "`deployment_timestamp` comes from `core/context_processors.py::project_context` as `int(time.time())`..." and replace with:
```markdown
`deployment_timestamp` comes from `core/context_processors.py::project_context`
via `_compute_cache_bust_token()` — returns the mtime of
`static/css/custom.css` as an integer. The token only changes when the
CSS file is modified, so Cloudflare's edge cache holds each version for
its full 4h TTL and repeat page loads in a session hit the browser
cache (304 Not Modified). Deploys that include a CSS change bump the
mtime → new token → cache busts. Pre-24-Apr-2026 this was
`int(time.time())` per-request, which defeated the CDN cache entirely
(effectively 0% hit rate on CSS). Degraded-mode fallback: if
`custom.css` isn't on disk (e.g., fresh container before
`collectstatic`), the function falls back to the old per-request
timestamp rather than crashing.
```
**Step 7: Commit**
```bash
git add core/context_processors.py core/tests.py CLAUDE.md
git commit -m "$(cat <<'EOF'
perf(cache): mtime-based CSS cache-bust token
deployment_timestamp was int(time.time()) per-request, giving every
page load a new ?v=... query string on custom.css. Cloudflare treats
each unique URL as a new resource, so the CSS was fetched from the VM
on every page load — 64 KB over the wire per navigation.
Token now tied to static/css/custom.css mtime. The URL only changes
when the CSS actually changes, so Cloudflare can hold the file for
its full 4h TTL. Degraded-mode fallback preserves today's behaviour
if the file isn't on disk.
3 new CacheBustTokenTests; all 68 tests pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
EOF
)"
```
---
## Task 2: Install & gate Django Debug Toolbar (dev-only)
**Goal:** Be able to see SQL query count, duplicate-query warnings, and execution times on any page — but ONLY when DEBUG + USE_SQLITE, never in prod.
**Files:**
- Modify: `requirements.txt` (add one line)
- Modify: `config/settings.py` (conditional INSTALLED_APPS, MIDDLEWARE, INTERNAL_IPS, DEBUG_TOOLBAR_CONFIG)
- Modify: `config/urls.py` (conditional `debug_toolbar.urls` include)
**Step 1: Add the dependency**
Append to `requirements.txt`:
```
django-debug-toolbar==6.0.0
```
Then install:
```bash
pip install django-debug-toolbar==6.0.0
```
Expected: clean install, one new package.
**Step 2: Gate it in `config/settings.py`**
At the very bottom of `config/settings.py` (after the SQLite override block), append:
```python
# === DEV-ONLY: Django Debug Toolbar ===
# Loaded ONLY when BOTH DEBUG=true AND USE_SQLITE=true. This is a
# deliberately strict gate — the toolbar exposes query internals,
# settings, and request state that should never appear in production.
# The USE_SQLITE half of the check acts as a belt-and-suspenders guard
# against accidentally enabling DEBUG on production (which would be its
# own serious problem, but at least wouldn't leak toolbar data).
if DEBUG and _IS_DEV:
try:
import debug_toolbar # noqa: F401 — probe for installed package
except ImportError:
pass
else:
INSTALLED_APPS += ['debug_toolbar']
# Insert the middleware as early as possible in the chain so it
# captures every request, but AFTER SecurityMiddleware (standard
# recommendation in the toolbar's install docs).
MIDDLEWARE.insert(1, 'debug_toolbar.middleware.DebugToolbarMiddleware')
INTERNAL_IPS = ['127.0.0.1', 'localhost']
DEBUG_TOOLBAR_CONFIG = {
# Don't auto-collapse the SQL panel — the SQL count is the
# main thing we check on every page.
'SHOW_COLLAPSED': False,
# Explicit check so the toolbar ONLY renders when the hosting
# flags are still set (guards against stale cached pages).
'SHOW_TOOLBAR_CALLBACK': lambda request: DEBUG and _IS_DEV,
}
```
**Step 3: Gate it in `config/urls.py`**
Read `config/urls.py` first to see the current shape. Then at the bottom (after `urlpatterns` is defined), append:
```python
# === DEV-ONLY: Django Debug Toolbar URL include ===
# Matches the conditional load in settings.py. No-op in prod.
from django.conf import settings
if settings.DEBUG and getattr(settings, '_IS_DEV_DEBUG_TOOLBAR_ENABLED', False) is False:
# (settings.py sets no such flag; gate on INSTALLED_APPS membership.)
pass
if 'debug_toolbar' in settings.INSTALLED_APPS:
from debug_toolbar.toolbar import debug_toolbar_urls
urlpatterns += debug_toolbar_urls()
```
Note: `debug_toolbar_urls()` is the v6.x API (replaces the older pattern-include). If that import fails in v6, fall back to:
```python
if 'debug_toolbar' in settings.INSTALLED_APPS:
import debug_toolbar
urlpatterns += [path('__debug__/', include(debug_toolbar.urls))]
```
The implementer should try `debug_toolbar_urls()` first; if it errors, switch to the include form and move on.
**Step 4: Manual verification**
Start the dev server, load `/`, confirm the toolbar appears. If the page 500s or the toolbar doesn't render, stop and investigate before profiling.
```bash
run_dev.bat # or manually: set USE_SQLITE=true && set DJANGO_DEBUG=true && python manage.py runserver 0.0.0.0:8000
```
Then open `http://127.0.0.1:8000/` in a browser and confirm the toolbar tab appears on the right edge. Click SQL panel, note the query count.
Run Django's system check too, to catch any app-config issue:
```bash
USE_SQLITE=true DJANGO_DEBUG=true python manage.py check
```
Expected: `System check identified no issues (0 silenced).`
**Step 5: Run the full test suite**
```bash
USE_SQLITE=true DJANGO_DEBUG=true python manage.py test core.tests -v 2
```
Expected: still 68 passes. Toolbar shouldn't affect tests.
**Step 6: Commit**
```bash
git add requirements.txt config/settings.py config/urls.py
git commit -m "$(cat <<'EOF'
chore(dev): add Django Debug Toolbar (dev-only, DEBUG+USE_SQLITE gated)
Double-gated install: only loads when DEBUG=true AND USE_SQLITE=true,
never in prod. Lets us profile SQL query counts on the dashboard and
payroll pages before attacking N+1 hotspots.
requirements.txt adds django-debug-toolbar==6.0.0
config/settings.py conditionally appends to INSTALLED_APPS + MIDDLEWARE
config/urls.py conditionally includes __debug__ route
No behavioural change to production.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
EOF
)"
```
---
## Task 3: Profile & fix N+1 on `/` (dashboard)
**Goal:** Confirm the dashboard's SQL query count and fix any N+1 the toolbar surfaces.
**Files:**
- Modify: `core/views.py` (the `index()` view — starts line ~353, `certs_expired/expiring` aggregation around lines ~406-413)
- Possibly modify: `core/templates/core/index.html` (only if a template access is firing queries)
**Step 1: Baseline measurement**
Start the dev server. Log in as an admin user. Open `/`. In the Debug Toolbar panel:
- Note **total SQL query count** on the page
- Note **total SQL time**
- Expand the SQL tab, scan for any group marked "N+1" or "N duplicate queries"
- Note **DOM size / page weight** from DevTools Network tab (the HTML document itself, not total transfer)
Record these four numbers. They go into the Task 4 commit message.
**Step 2: Identify each problem**
The design doc predicts three suspects:
1. **Cert-expiry card** — two separate COUNT queries at views.py:406 and views.py:409. Could become one `.aggregate()` call.
2. **Stat cards** — if any stat card loops over a queryset in the template instead of using `.count()` on the queryset, that's an N+1.
3. **Pending payments table** (if it renders on this page for admins) — worker/team/log per row.
For each "N duplicate queries" group the toolbar shows, inspect the Python side:
- Find the view function (use the traceback in the toolbar if available)
- Decide whether the fix is `select_related` (FK), `prefetch_related` (M2M / reverse FK), or `annotate` (aggregate)
- Apply the minimal fix
**Step 3: Apply fixes**
For each confirmed N+1, edit `core/views.py` with the minimal change. Example shape (only as an illustration — apply whatever the toolbar surfaces):
```python
# BEFORE
workers = Worker.objects.filter(active=True)
for w in workers:
for cert in w.workercertificate_set.all(): # N+1 — one query per worker
...
# AFTER
workers = Worker.objects.filter(active=True).prefetch_related('workercertificate_set')
for w in workers:
for cert in w.workercertificate_set.all(): # reuses prefetched cache
...
```
For the cert counts specifically (views.py:406-413), the current code is:
```python
certs_expired_count = WorkerCertificate.objects.filter(
worker__is_active=True, valid_until__lt=today
).count()
certs_expiring_count = WorkerCertificate.objects.filter(
worker__is_active=True, valid_until__gte=today, valid_until__lte=today + timedelta(days=30)
).count()
certs_alert_total = certs_expired_count + certs_expiring_count
```
This is TWO queries, not an N+1. If the toolbar shows them as hot, consolidate into one `.aggregate()`:
```python
from django.db.models import Count, Q
_cert_counts = WorkerCertificate.objects.filter(
worker__is_active=True
).aggregate(
expired=Count('id', filter=Q(valid_until__lt=today)),
expiring=Count('id', filter=Q(valid_until__gte=today, valid_until__lte=today + timedelta(days=30))),
)
certs_expired_count = _cert_counts['expired']
certs_expiring_count = _cert_counts['expiring']
certs_alert_total = certs_expired_count + certs_expiring_count
```
But ONLY apply this if the toolbar flags it. Per YAGNI, don't pre-optimise. Two simple COUNTs with an index on `valid_until` are plenty fast.
**Step 4: Re-measure**
Reload `/`. The SQL query count should drop. If it didn't, your fix didn't hit the real hot path — revert that edit and look again at the toolbar output.
**Step 5: Run the full test suite**
```bash
USE_SQLITE=true DJANGO_DEBUG=true python manage.py test core.tests -v 2
```
Expected: 68 passes. A change in the ORM query plan does NOT typically move test counts, but a bug in a `prefetch_related` pattern can. If any test fails, revert and rethink.
**Step 6: Commit**
Only commit if ≥1 N+1 was actually found and fixed. If the toolbar showed `/` at ~20 queries with no N+1s flagged, skip Task 3 entirely and note that in the final commit message.
```bash
git add core/views.py [any template files touched]
git commit -m "$(cat <<'EOF'
perf(dashboard): fix N+1 on [specific problem identified]
Debug Toolbar showed [before-count] queries on /, including [N
duplicates of query X]. Added [select_related/prefetch_related/annotate]
to [view function] to fold those into one.
After: [after-count] queries. Test suite unaffected (68/68 pass).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
EOF
)"
```
---
## Task 4: Profile & fix N+1 on `/payroll/` (all 4 tabs) + before/after summary
**Goal:** Same as Task 3, applied to the payroll dashboard's four tabs, plus a final commit that captures the full before/after picture.
**Files:**
- Modify: `core/views.py` (the `payroll_dashboard()` view — starts line ~2593; each status branch: `pending`, `history`, `loans`, `adjustments`)
- Possibly: `core/templates/core/payroll_dashboard.html` (only if a template access fires queries)
**Step 1: Baseline measurement — all 4 tabs**
For each of the four URLs, record query count + SQL time:
- `/payroll/` (defaults to pending)
- `/payroll/?status=history`
- `/payroll/?status=loans`
- `/payroll/?status=adjustments`
**Step 2: Identify N+1s per tab**
Expected suspects (design doc predictions):
- **Pending tab** — for every worker in the outstanding list, `get_pay_period(team)` is called per worker. If `team` isn't prefetched via `select_related('team')` or `prefetch_related('teams')` somewhere upstream, each call hits the DB. Look at views.py around line 2676 (the pending-payments loop where `get_pay_period` is called).
- **History tab** — PayrollRecord list view; each row needs worker + work_logs + adjustments. Check for `prefetch_related('work_logs', 'payrolladjustment_set')` on the base queryset.
- **Loans tab** — Loan list with repayment adjustments. Check for `prefetch_related('payrolladjustment_set')` on active loans.
- **Adjustments tab** — we already fixed `worker.teams.first()` → `worker.teams.all()` in commit `06b3315`; confirm no new N+1 introduced by the sort / group work done in later Adjustments commits.
**Step 3: Apply fixes**
Same pattern as Task 3. One fix = one toolbar re-measure. Don't batch multiple fixes into one change — it's much harder to attribute the win (or the regression).
**Step 4: Confirm WeasyPrint stays lazy-imported**
One-line grep-and-document check (design doc Task 3):
```bash
grep -rn "^import weasyprint\|^from weasyprint" core/ config/
```
Expected: no matches. WeasyPrint should only be imported inside the body of `render_to_pdf()` in `core/utils.py`. If any module-level import exists, move it inside the function.
**Step 5: Run the full test suite once more**
```bash
USE_SQLITE=true DJANGO_DEBUG=true python manage.py test core.tests -v 2
```
Expected: 68 passes.
**Step 6: Final commit with before/after summary**
Even if Task 4 found no N+1s (unlikely but possible), this commit records the measurement. If Task 3 or 4 DID find real fixes, they're each in their own commit already; this is the "closing the pass" commit.
```bash
# (Only include files in git add IF you actually changed them in this task.)
git add core/views.py [any templates touched]
git commit -m "$(cat <<'EOF'
perf(payroll): [specific fix] + quick-wins pass summary
Fixed [N+1 description] on the [pending|history|loans|adjustments] tab
by [select_related / prefetch_related / annotate added to queryset X].
Perf Quick-Wins Pass A — before/after:
/ N queries, Tms SQL → M queries, Sms SQL
/payroll/ N queries, Tms SQL → M queries, Sms SQL (pending)
/payroll/?status=history N → M queries
/payroll/?status=loans N → M queries
/payroll/?status=adjustments N → M queries
Cache-bust token (d1490c4): CSS is now browser-cached across sessions
and Cloudflare holds it at the edge for 4h — biggest felt improvement
expected from this pass.
WeasyPrint confirmed still lazy-imported (module-level grep clean).
Test suite: 68/68.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
EOF
)"
```
---
## Final acceptance checklist
Before declaring the pass complete, the controller (this session) verifies:
- [ ] All commits on `ai-dev` from `d1490c4` → HEAD show `perf(...)` or `chore(dev):` prefix
- [ ] `USE_SQLITE=true DJANGO_DEBUG=true python manage.py test core.tests -v 2` reports 68/68 passing
- [ ] Django Debug Toolbar renders on `/` with USE_SQLITE=true DEBUG=true — and does NOT render when DEBUG=false (quick toggle test)
- [ ] The final commit message contains concrete before/after query counts (not placeholders)
- [ ] Working tree clean, branch ready to push
---
## What's NOT in this plan (explicit non-goals)
- Splitting `payroll_dashboard.html` into tab partials — plan B territory
- Refactoring any view's body structure
- Any visual/UX change
- Adding a perf regression test harness — if we want that, it's a separate design
Rollback: `git revert <sha>` on any individual commit. No data, schema, or URL-contract impact in any task.

3
requirements.txt
View File

@ -2,5 +2,4 @@ Django==5.2.7
mysqlclient==2.2.7
python-dotenv==1.1.1
pillow==12.1.1
weasyprint==68.1
django-debug-toolbar==6.0.0 # dev-only — gated in config/settings.py, never active in prod
weasyprint==68.1