Status note (2026-09-25): Dated implementation plan. Its phase and ticket states are historical; use current audit-events skill, source and tests for implementation state.

AuditEvent Implementation Plan

TL;DR: history of the AuditEvent canonical-store cutover (completed). Useful as provenance for why audit events look the way they do — do not use its phases as a task list; follow current issues, source, and tests instead.

Status: CANONICAL STORE CUTOVER COMPLETE — FEATURE-OWNED PRODUCER FOLLOW-UP TRACKED SEPARATELY
Approved: 2026-09-03
Implementation trigger: The owner explicitly says proceed.

This document is the implementation runbook and step-by-step checklist for consolidating BloxClips audit trails into a feature-agnostic AuditEvent system. Phase work begins only after the owner explicitly approves it.

Working Agreement

• Perform backend work in a new backend Git worktree.
• Perform frontend work in a new frontend Git worktree.
• Do not implement directly in the current backend or frontend checkout.
• Use Conventional Commits.
• Keep commits logically scoped and independently reviewable.
• A phase is not synonymous with one commit. Split each phase at natural review boundaries.
• Complete and verify one phase, report its outputs, and stop.
• Do not begin the next phase until the owner reviews the completed phase and explicitly says to proceed.
• Do not add export functionality during this project.
• Do not add an external audit service or external event sink.
• Preserve feature-agnostic storage so individual features can later query and present their events with feature-specific UI and flows.

Target Architecture

AuditEvent becomes the sole feature-agnostic business audit record:

interface AuditEvent<TData = unknown> {
  id: string;
  type: string;
  version: number;
  occurredAt: Date;
  recordedAt: Date;

  actor: {
    type: "user" | "staff" | "system" | "webhook";
    id?: string;
  };

  targets: Array<{
    type: string;
    id: string;
  }>;

  outcome: "succeeded" | "failed" | "denied";
  reason?: string;

  correlationId?: string;
  causationId?: string;
  idempotencyKey: string;

  changes?: {
    before?: unknown;
    after?: unknown;
  };

  data: TData;
}

Events use lowercase namespaced types such as:

campaign.lifecycle.paused
invoice.payment.succeeded
submission.review.approved
staff.role.assigned
payout.transfer.failed

There must not be a feature/category enum that needs to be edited every time a feature introduces an event.

Database representation

Use two tables:

AuditEvent
  └── AuditEventTarget[]

Targets are separate queryable records because one event may concern several entities. This must support efficient queries such as:

• All events for a campaign
• All events affecting a payout
• All events involving a creator
• All events generated by a staff member

Event-specific data and changes remain JSON. Fields shared across all events remain first-class database columns.

Recording API

The canonical audit module exposes approximately:

recordAuditEvent(event)
recordAuditEventRequired(transaction, event)
registerAuditEventDataSchema(type, version, schema)

• recordAuditEventRequired() participates in the existing domain transaction when the audit record is part of a financial, authorization, or business invariant.
• recordAuditEvent() is best-effort for noncritical observational activity.

The required transactional form applies to at least:

• Campaign funding recognition
• Payout creation and financial state transitions
• Role and permission changes
• Clipper Group mutations
• Financial overrides

Event data registry

The Zod registry is keyed by <event-type>@<version>.

registerAuditEventDataSchema(
  "submission.review.approved",
  1,
  z.object({
    previousStatus: z.string(),
    newStatus: z.string(),
  }),
);

Registry rules:

• Every event is validated against the canonical envelope.
• Feature-specific data is validated when a schema is registered.
• A registered version is immutable.
• A breaking payload change requires a new version.
• The registry is not a centralized allowlist of features.

Sensitive-data rules

• Callers explicitly select safe event data.
• The recorder applies recursive defense-in-depth redaction.
• Known credential, token, tax-ID, wallet, password, and secret fields are rejected or redacted.
• JSON depth and size are bounded.
• Complete requests, responses, webhook payloads, and user records are never captured automatically.
• Audit events are not debug logs, access logs, telemetry, or a copy of every database mutation.

Phase 0 — Prepare Isolated Worktrees

Do this only after the owner says to proceed.

Checklist

• [x] Confirm the current backend and frontend branches and clean working states.
• [x] Fetch the latest repository state without modifying the current checkouts.
• [x] Create a dedicated backend branch and worktree.
• [x] Create a dedicated frontend branch and worktree.
• [x] Record the exact worktree paths in this document.
• [x] Re-read repository-local instructions inside both worktrees.
• [x] Confirm baseline backend build in the backend worktree.
• [x] Confirm baseline frontend lint in the frontend worktree. The production build requires an isolated frontend dependency installation before Phase 2; Turbopack rejects the temporary worktree-external dependency symlink.

Worktree record

Backend worktree:  /home/kirbysmashyeet/Source/BloxClips/Bloxclips-backend-audit-events
Backend branch:    feature/audit-event-foundation
Frontend worktree: /home/kirbysmashyeet/Source/BloxClips/BloxClips-frontend-audit-events
Frontend branch:   feature/audit-event-admin-viewer

Phase 0 is preparation, not an implementation phase. If baseline checks fail, document the pre-existing failures before changing code.

Phase 1 — AuditEvent Foundation

Goal: Add the new append-only event structure and canonical recording library without removing or changing the two legacy audit stores.

1.1 Database models

• [x] Add AuditEvent.
• [x] Add AuditEventTarget with a cascading relation to its event.
• [x] Add a unique idempotency-key constraint.
• [x] Add indexes for event type and occurrence time.
• [x] Add indexes for actor type, actor ID, and occurrence time.
• [x] Add indexes for outcome and occurrence time.
• [x] Add an index for correlation ID.
• [x] Add stable pagination ordering support using occurrence time and event ID.
• [x] Add target type and target ID indexes.
• [x] Keep AuditLog and AdminAuditLog unchanged.
• [x] Create an additive migration only.

1.2 Canonical envelope and Zod registry

• [x] Define the canonical event input and stored-event types.
• [x] Validate lowercase namespaced event types.
• [x] Validate actor, target, outcome, timestamp, and relationship fields.
• [x] Implement the versioned event-data schema registry.
• [x] Prevent duplicate schema registration for the same type and version.
• [x] Normalize and deduplicate targets.
• [x] Bound event payload depth and size.

1.3 Recorders

• [x] Implement the best-effort recorder.
• [x] Implement the transaction-required recorder.
• [x] Enforce idempotency-key uniqueness.
• [x] Provide an explicit generated-key helper only for non-retried one-off events.
• [x] Require every recorder call to supply an idempotency key; retryable, webhook, and financial callers must derive stable business keys.
• [x] Apply defensive redaction without treating it as permission to pass broad objects.
• [x] Ensure error reporting cannot leak event payloads.

1.4 Tests

• [x] Test canonical envelope validation.
• [x] Test valid and invalid event names.
• [x] Test registered event-data validation.
• [x] Test schema version separation and immutability behavior.
• [x] Test duplicate schema registration.
• [x] Test target normalization and deduplication.
• [x] Test sensitive-data redaction and payload bounds.
• [x] Test idempotency behavior and conflicting key reuse.
• [x] Test that required event failures propagate through the supplied domain transaction so it can roll back.
• [x] Test that best-effort event failure follows its documented behavior.

Planned commit boundaries

Suggested boundaries, subject to the actual implementation:

1. feat(audit): add canonical audit event schema
2. feat(audit): add validated event recording API
3. test(audit): cover event recording invariants

Do not collapse Phase 1 into one oversized commit merely because it is one phase.

Phase 1 stop gate

• [x] Backend build passes.
• [x] Relevant tests pass.
• [x] Migration is additive and reviewed.
• [x] Neither legacy audit table has been modified or removed.
• [x] Provide the owner with commit list, files changed, tests run, and noteworthy decisions.
• [x] Stop and wait for explicit approval before Phase 2.

Phase 2 — Generic Admin Audit Viewer

Goal: Provide an admin-only, feature-neutral read experience over AuditEvent.

2.1 Backend query API

Proposed endpoint:

GET /api/admin/audit-events

• [x] Enforce backend administrator authorization.
• [x] Return events with their targets.
• [x] Support free-text queries across safe common fields.
• [x] Filter by event type.
• [x] Filter by actor type and actor ID.
• [x] Filter by target type and target ID.
• [x] Filter by outcome.
• [x] Filter by occurrence date range.
• [x] Sort by occurred time, recorded time, event type, actor type, or outcome.
• [x] Use deterministic server-side pagination.
• [x] Cap page size.
• [x] Add a metadata endpoint for known event and target filter values.
• [x] Do not add export functionality.

2.2 Frontend data layer

• [x] Define API response and query types.
• [x] Implement an admin API client using existing admin-fetch conventions.
• [x] Store filters, sorting, and pagination in URL query parameters.
• [x] Prevent stale responses from replacing newer filter results.

2.3 Frontend page

Proposed route:

/dashboard/admin/audit-events

• [x] Add the page to administrator navigation only.
• [x] Enforce frontend administrator routing in addition to backend enforcement.
• [x] Display occurred time, event type/version, outcome, actor, targets, reason, and correlation ID.
• [x] Provide search, filter, sort, and pagination controls.
• [x] Provide loading, empty, and error states.
• [x] Provide expandable event details.
• [x] Show changes, data, causation ID, idempotency key, recorded time, and normalized raw representation in details.
• [x] Render generic structured data safely.
• [x] Do not add feature-specific labels or formatting.
• [x] Ensure the page remains usable on narrow screens.

2.4 Verification

• [x] Test backend administrator authorization and non-administrator rejection at the router boundary.
• [x] Test filter combinations.
• [x] Test deterministic pagination and sorting.
• [x] Test malformed query handling.
• [x] Run frontend type checking and targeted linting; attempt the production build (external Google font fetch is blocked by the environment).
• [ ] Manually verify administrator and non-administrator navigation behavior.
• [ ] Manually verify responsive table/detail rendering.

Planned commit boundaries

Suggested boundaries:

Backend:

1. feat(audit): add admin event query API
2. test(audit): cover admin event queries

Frontend:

1. feat(audit): add admin audit event data layer
2. feat(audit): add generic admin audit viewer
3. test(audit): cover audit viewer behavior if the repository's test infrastructure supports focused UI tests

Phase 2 stop gate

• [x] Backend query checks pass.
• [x] Frontend type and targeted lint checks pass; the production build is externally blocked only by the Google font fetch.
• [x] The page is admin-only at both frontend and backend boundaries.
• [x] The view is feature-neutral.
• [x] Provide the owner with backend and frontend commit lists, a precise UI description, tests run, and noteworthy decisions.
• [x] Stop and wait for explicit approval before Phase 3.

Phase 3 — Consolidate Legacy Producers and Data

Goal: Move all valid audit producers and historical records into AuditEvent, verify the cutover, and only then remove the legacy stores.

The safe dependency order is important: producers must stop writing to the old tables before those tables are dropped.

3.1 Inventory and classify producers

• [x] Inventory every generic audit() call.
• [x] Inventory every writeAuditRequired() call.
• [x] Inventory every auditLog() middleware use.
• [x] Inventory every direct adminAuditLog.create() call.
• [ ] Classify each as a required transactional event, successful business event, failed/denied event, security observation, or non-audit operational log.
• [ ] Identify calls that currently capture broad request/response data.
• [ ] Define the semantic replacement event type, actor, targets, outcome, data schema, and idempotency strategy for every retained producer.

3.2 Replace legacy producers

• [ ] Replace response-interception audit middleware with explicit domain-aware events.
• [ ] Migrate Clipper Group and campaign producers.
• [ ] Migrate submission review and moderation producers.
• [ ] Migrate staff, role, ban, and security producers.
• [ ] Migrate invoice, payment, and webhook producers.
• [ ] Migrate earnings and payout producers.
• [ ] Migrate tax and sensitive-document access producers.
• [ ] Migrate remaining valid producers.
• [ ] Remove calls that are operational logs rather than business audit facts.
• [ ] Add event-specific schemas and focused tests alongside each producer group.

3.3 Backfill historical records

Implement a bounded, restartable application script rather than copying arbitrary historical JSON directly in SQL.

• [x] Read legacy records in stable batches.
• [x] Map legacy actors into canonical actor types.
• [x] Map legacy subjects/resources into targets.
• [x] Preserve original occurrence timestamps.
• [x] Generate deterministic legacy idempotency keys.
• [x] Sanitize historical metadata before inserting it.
• [x] Make the backfill safe to rerun.
• [x] Record and report transformation failures rather than silently skipping them.
• [x] Support a dry run and verification summary.

Deterministic keys should resemble:

legacy:audit-log:<old-id>
legacy:admin-audit-log:<old-id>

3.4 Verify cutover

• [x] Compare source-row counts with migrated, rejected, and intentionally omitted counts.
• [x] Confirm no active code references either legacy Prisma model.
• [ ] Confirm new events appear for representative campaign, moderation, payout, staff, and security flows.
• [x] Confirm sensitive legacy data was sanitized.
• [x] Confirm the application builds and tests against a schema without the legacy tables.
• [ ] Confirm the generic admin viewer shows historical and new events together.

3.5 Retire legacy infrastructure

The owner explicitly authorized destructive table retirement after the verified historical backfill:

• [x] Drop AuditLog.
• [x] Drop AdminAuditLog.
• [ ] Delete the legacy middleware.
• [ ] Delete the legacy audit utility.
• [ ] Remove compatibility code.
• [ ] Re-run all affected backend checks.

Planned commit boundaries

Suggested boundaries:

1. refactor(audit): migrate group and campaign events
2. refactor(audit): migrate moderation and staff events
3. refactor(audit): migrate financial and tax events
4. refactor(audit): migrate remaining event producers
5. feat(audit): add legacy audit backfill
6. test(audit): verify legacy event migration
7. refactor(audit): retire legacy audit stores

Producer groups may be split further when needed to keep reviews focused.

Phase 3 stop gate

• [x] All retained producers write only to the canonical recorder, including the temporary compatibility adapter.
• [x] Historical backfill is idempotent and verified.
• [x] No legacy Prisma model references remain. The compatibility adapter remains temporarily until feature-owned contracts replace its callers.
• [x] Legacy tables are removed after verified backfill and explicit owner authorization.
• [x] Backend build and focused affected tests pass.
• [ ] Provide the owner with the producer inventory, mapping summary, backfill results, commit list, and test results.
• [ ] Stop and wait for explicit approval before Phase 4.

Phase 4 — Developer Documentation and AI-Agent Skill

Goal: Make correct audit-event usage easy to understand and repeat for both developers and AI agents.

4.1 Developer documentation

Create:

Bloxclips-backend/docs/audit-events.md

• [x] Explain what constitutes a business audit event.
• [x] Explain what must not be recorded as an audit event.
• [x] Document the canonical schema.
• [x] Document event naming conventions.
• [x] Document actor and target selection.
• [x] Document outcomes and reasons.
• [x] Explain data versus changes.
• [x] Explain event versioning and Zod registration.
• [x] Explain transactional versus best-effort recording.
• [x] Explain idempotency, correlation, and causation.
• [x] Document the sensitive-data policy.
• [x] Document generic and feature-specific querying.
• [x] Explain how feature-specific UIs can format the same events without changing storage.
• [x] Document testing expectations.
• [x] Include complete correct examples and common incorrect examples.

Explicit non-audit examples must include:

• Ordinary debug logs
• HTTP access logs
• Scraper retry diagnostics
• Performance telemetry
• Every database update
• Raw third-party payloads

4.2 AI-agent skill

Create approximately:

Bloxclips-backend/.agents/skills/bloxclips-audit-events/SKILL.md

Follow the repository's available skill-creation instructions when this phase begins.

The skill should remain concise and point to the developer document as the detailed source of truth. It must preserve these operational invariants:

• Use semantic namespaced event types.
• Never capture entire objects, requests, responses, or external payloads.
• Use transactional recording for critical invariants.
• Register and version structured event data.
• Identify every relevant target.
• Use stable idempotency keys for retryable activity.
• Do not use audit events as application logs or telemetry.
• Add focused tests for the event contract and placement.

4.3 Validation

• [x] Confirm all paths and examples match the current implementation.
• [x] Confirm developers can add a new event using the document alone.
• [x] Validate the skill using the prescribed skill validator.
• [x] Confirm the skill does not grant authority for unrelated mutations.
• [x] Confirm documentation does not describe export functionality as implemented.

Planned commit boundaries

Suggested boundaries:

1. docs(audit): document the audit event system
2. docs(audit): add audit event agent skill

Phase 4 stop gate

• [x] Documentation matches the current implementation.
• [x] Skill validation passes.
• [x] Backend checks remain green.
• [x] Provide the owner with final commit list, document links, skill path, validation results, and deferred work.
• [x] Stop for owner review.

Explicitly Deferred

The following are not part of this implementation:

• Audit export functionality
• CSV export
• SIEM integrations
• External audit services
• External event sinks
• Cryptographic event verification
• Customer-facing audit portals
• Feature-specific audit UI components
• Broad observability or telemetry changes
• Database-level auditing of every mutation

Feature-specific interfaces may later query AuditEvent by type and targets and format results for their own workflows. They must not introduce separate audit storage.

Completion Definition

This project is complete when:

1. All business audit events use one validated, versioned, feature-agnostic structure.
2. Administrators can securely query, filter, sort, paginate, and inspect events through the generic UI.
3. Historical records from both legacy stores are safely represented in the unified store.
4. No active producer writes to either legacy audit system.
5. Both legacy tables have been safely retired; the temporary canonical compatibility adapter is tracked for later feature-owned replacement.
6. Developers have complete implementation guidance.
7. AI agents have a validated repository skill directing correct future use.
8. No export or external-service scope has been introduced.

Current Progress

• [x] Architecture researched
• [x] Architecture approved by owner
• [x] Implementation plan approved by owner
• [x] Persistent runbook created
• [x] Owner has said proceed
• [x] Phase 0 complete
• [x] Phase 1 complete and approved
• [x] Phase 2 complete and approved
• [x] Phase 3 canonical-store cutover complete; feature-owned producer replacement remains tracked separately
• [x] Phase 3 destructive table retirement approved and completed
• [x] Phase 4 complete; owner review pending

Phase 1 Execution Record

Completed: 2026-09-03
Backend branch: feature/audit-event-foundation

Commits:

1. 278cfbd feat(audit): add canonical audit event schema
2. b29cea6 feat(audit): add validated event recording API
3. 2f80961 test(audit): cover event recording invariants

Verification completed:

• Prisma schema validation passed with a non-production placeholder database URL.
• Backend TypeScript build passed.
• Nine focused AuditEvent tests passed.
• Git whitespace validation passed.
• The backend audit worktree is clean.
• The frontend audit worktree remains unchanged and clean.
• AuditLog and AdminAuditLog remain present and unchanged.

Environment note:

• Frontend lint passed during Phase 0.
• The Phase 0 frontend production build was initially blocked by Turbopack rejecting the temporary worktree-external node_modules symlink. Dependencies were subsequently installed directly inside the frontend worktree. The Phase 2 production-build attempt is now blocked only by the environment being unable to fetch the Google-hosted Geist font.

Review state: Phase 1 approved; Phase 2 execution has been completed and awaits owner approval.

Phase 2 Execution Record

Completed: 2026-09-03
Backend branch: feature/audit-event-foundation
Frontend branch: feature/audit-event-admin-viewer

Commits:

1. Backend: 3c65476 feat(audit): add admin event query API
2. Frontend: 481d0ed feat(audit): add generic admin audit viewer

Delivered behavior:

• GET /api/admin/audit-events requires authentication and administrator authorization, applies normalized server-side filters, stable sorting, capped pagination, and returns event targets.
• GET /api/admin/audit-events/filters supplies generic filter metadata without exposing event JSON payloads to search.
• /dashboard/admin/audit-events is an administrator route and navigation entry with URL-backed search, filters, sorting, pagination, cancellation of stale requests, loading/empty/error states, and expandable generic event details.
• The viewer intentionally has no feature-specific formatting and no export capability.

Verification completed:

• Backend TypeScript build passed.
• Nine AuditEvent foundation tests passed.
• Four query-parser tests passed.
• Two router-boundary authorization and route-contract tests passed.
• Frontend TypeScript checking passed.
• Targeted frontend ESLint passed for the new audit viewer, route, client, and navigation changes.
• Git whitespace validation passed for the backend changes.

Known environment limitation:

• The frontend production build was attempted after direct dependency installation. It reaches Next.js font loading but cannot fetch the Google-hosted Geist font in this environment. This is not an application-code failure; browser/session-based administrator and responsive visual verification remain for owner review.

Review state: Waiting for owner approval before Phase 3.

Phase 3 Execution Record (In Progress)

Completed implementation work: 2026-09-03
Backend branch: feature/audit-event-foundation

Commits:

1. d4d945c docs(audit): inventory legacy event producers
2. f4dce0d feat(audit): record clipper group events canonically
3. ea2292e feat(audit): record TOTP verification events
4. 604ab2a refactor(audit): route legacy producers to AuditEvent
5. 2dc4a31 feat(audit): record explicit staff action events
6. b0583a4 refactor(audit): remove legacy admin middleware writes
7. b9db231 feat(audit): add restartable legacy-event backfill
8. 4ca9199 test(audit): assert canonical clipper group events

Completed:

• Inventory and classification of all active legacy audit producers.
• Canonical transactional events for Clipper Group lifecycle and memberships.
• Canonical events for TOTP verification, staff warnings/unbans, submission status transitions, and RBAC bootstrap elevation.
• Remaining AuditLog utility producers now write bounded canonical events through a temporary compatibility bridge.
• The legacy response interceptor no longer captures request or response payloads; it writes only a minimal canonical action observation after a successful route response.
• Dry-run-by-default backfill script with stable source-row idempotency keys, preserved occurrence timestamps, bounded batches, and historical transport-payload exclusion.
• Clipper Group integration tests now assert canonical event rows.

Verification status:

• Backend TypeScript no-emit checking passes after the producer migration.
• No active application source writes to AuditLog or AdminAuditLog remain.
• A configured target database was used for a dry run and two apply runs of the backfill script.
• The dry run reported 70 readable source rows, 70 insertable-or-existing canonical rows, and 0 rejected rows.
• Final transactionally consistent reconciliation: 35 AuditLog rows and 35 AdminAuditLog rows; all 35 + 35 have a corresponding canonical event by deterministic source key, with 0 duplicate backfill keys.
• Historical request/response/transport payloads are excluded by the backfill transformation rather than copied into canonical event data.
• AuditLog and AdminAuditLog are intentionally still present. No destructive migration or table drop has been run.

Next required operational sequence:

1. Replace the temporary compatibility bridge with feature-owned canonical producers (tracked separately as AUDIT-002).
2. Add feature-specific audit-history views where the operational workflow benefits from them (also AUDIT-002).

Destructive cutover completed: 2026-09-03
Commit: 59c9111 refactor(audit): retire legacy audit stores

Verification completed:

• The targeted migration dropped exactly AuditLog and AdminAuditLog in the configured non-production database and was recorded as applied.
• Post-migration verification found zero legacy tables, retained 70 canonical AuditEvent rows, and confirmed the migration record.
• Prisma schema validation, backend TypeScript build, and all 15 focused audit tests passed against the schema without legacy models.

Review state: Canonical persistence cutover is complete. Feature-owned producer contracts and contextual views continue under AUDIT-002.

Phase 4 Execution Record

Completed: 2026-09-03
Backend branch: feature/audit-event-foundation

Commits:

1. 8ca2841 docs(audit): document canonical event usage
2. 5e3891f docs(audit): add audit event agent skill

Delivered:

• Bloxclips-backend-audit-events/docs/audit-events.md is the developer-facing source of truth. It documents event boundaries, envelope fields, naming, actors and targets, outcomes, data versus changes, contract versioning, recorder choice, idempotency, sensitive-data limits, querying, presentation, tests, and the temporary compatibility-adapter boundary.
• Bloxclips-backend-audit-events/.agents/skills/bloxclips-audit-events/SKILL.md gives AI agents a concise, feature-scoped route to the document and preserves the core recording invariants.
• The documentation deliberately does not imply export support or authorize unrelated changes.

Verification completed:

• python3 /home/kirbysmashyeet/.codex/skills/.system/skill-creator/scripts/quick_validate.py .agents/skills/bloxclips-audit-events passed.
• npm run build passed in the backend audit worktree.
• Git whitespace validation passed before committing.

Deferred:

• The temporary compatibility utility still has active callers; replacing them with feature-owned contracts and adding contextual feature views is tracked separately as AUDIT-002.
• AuditLog and AdminAuditLog have been retired by the explicitly authorized destructive migration after verified historical backfill.

Review state: Phase 4 complete and ready for owner review.