wp-agentic-writer/docs/architecture/PLUGIN_AUDIT_FOLLOWUP_2026-05-24.md

WP Agentic Writer Follow-up Audit

Status: COMPLETE / SUPERSEDED
Completion marker date: 2026-05-24
Retrace audit: docs/architecture/PLUGIN_AUDIT_RETRACE_2026-05-24.md

This follow-up audit has been implementation-traced. Remaining work should be tracked from the retrace audit to avoid reopening duplicate findings that are already closed.

Audit date: 2026-05-24
Baseline audited: docs/architecture/PLUGIN_AUDIT_REPORT_2026-05-22.md
Definition of Done audited: docs/DEFINITION_OF_DONE.md
Scope: implementation trace, UI/UX, system architecture, conversation context/history, cost tracking, provider/model routing, migration safety, security, data lifecycle, and process gaps.

Executive Summary

The 2026-05-22 audit has been partially implemented. The strongest improvements are: conversation table creation was added to activation, PHP 7.4 streaming incompatibilities were removed, a Context Service exists, several session endpoints now check ownership, and the frontend attempts legacy chat migration.

However, the plugin is not yet out of the "fix A, break B" risk zone. The highest-risk previous defect, the OpenRouter model cache shape conflict, is still open. Conversation history is still split between post meta and the session table. Several post-scoped REST routes still rely only on edit_posts, not edit_post for the target post. Provider fallback remains silent to the user, cost tracking is still not a reliable ledger, and model defaults remain inconsistent across activation, settings, providers, and JS.

Current readiness: improved beta, but still not production-safe for multi-user editorial sites or cost-sensitive usage.

Verification Performed

• Static traced the previous audit against current PHP, JS, docs, and migration code.
• Ran PHP syntax checks across plugin PHP files with php -l; no syntax errors detected.
• Ran JS syntax checks with node -c assets/js/sidebar.js and node -c assets/js/settings-v2.js; no syntax errors detected.
• Did not run a live WordPress browser workflow in this pass, so runtime workflow findings remain static-analysis based.

Previous Audit Status Trace

Previous finding	Status	Evidence	Remaining action
Conversation table migration not wired	Partially fixed	Activation calls wpaw_create_conversations_table() in wp-agentic-writer.php:182-184; versioned table creation also calls it in wp-agentic-writer.php:234-238.	Make conversation table creation independent of main wpaw_db_version; use wpaw_conversations_db_version everywhere.
OpenRouter model cache conflicting shapes	Open	Full model objects and ID lists still share wpaw_openrouter_models in includes/class-openrouter-provider.php:105-177 and includes/class-openrouter-provider.php:248-255.	Split cache keys and harden validation.
PHP 7.4 incompatible str_starts_with()	Fixed	rg found no remaining str_starts_with; streaming checks use strpos.	Add a CI lint job under PHP 7.4 or raise the PHP requirement if future code uses PHP 8 APIs.
Conversation endpoints lack per-session ownership	Partially fixed	GET/PUT/DELETE/messages endpoints now call current_user_can_access() in includes/class-gutenberg-sidebar.php:7283-7411.	Add post ownership checks to list/create/link-post/writing-state routes and strengthen session IDs.
Two context stores compete	Open	Chat still writes _wpaw_chat_history and session messages in includes/class-gutenberg-sidebar.php:996-1026 and includes/class-gutenberg-sidebar.php:1139-1168.	Make wpaw_conversations.messages the only message authority.
Silent provider fallback	Open	Provider Manager still returns OpenRouter on missing/unreachable provider in includes/class-provider-manager.php:35-50.	Return provider metadata and warning to backend response and UI.
Cost tracking setting does not stop tracking or enforce budget	Open	add_request() always inserts cost rows in includes/class-cost-tracker.php:58-75.	Clarify setting semantics, add policy guardrails, track provider/session/status.
REST route contracts too loose	Open	Routes mostly lack args schemas, including core routes in includes/class-gutenberg-sidebar.php:302-813.	Add route schemas and contract tests.
Main backend class too large	Open	includes/class-gutenberg-sidebar.php still owns route registration, workflow, context, providers, SEO/GEO, image, and session handlers.	Extract REST controllers and workflow/context/provider/cost services.
Admin settings depend on external CDNs	Open	CDN assets still load in includes/class-settings-v2.php:67-75.	Bundle vendor assets locally or use WP-native components.
Uninstall incomplete and duplicated	Open	Main uninstall omits conversations/custom models/meta in wp-agentic-writer.php:269-278; uninstall.php:12-21 is a second, smaller cleanup path.	Consolidate uninstall and add a data-retention option.
Image generation partially integrated	Open	Image variants save cost locally but do not emit wp_aw_after_api_request in includes/class-image-manager.php:482-522.	Ledger image costs through Cost Tracker and add image lifecycle states.
Settings defaults and model labels inconsistent	Open	Defaults diverge across activation, provider, settings PHP, and JS. Examples: wp-agentic-writer.php:140-142, includes/class-openrouter-provider.php:34-69, includes/class-settings-v2.php:105-111, assets/js/settings-v2.js:26-38.	Create one model preset registry and migrate legacy execution_model.
Debug logging too noisy	Partially fixed	wpaw_debug_log() exists in includes/class-gutenberg-sidebar.php:20-27, but console logs remain in assets/js/sidebar.js and assets/js/settings-v2.js; some error_log() calls remain.	Gate frontend logging and remove prompt/response/debug traces from production.

Critical Findings

P0: OpenRouter Model Cache Conflict Still Breaks Valid Models

get_cached_models() stores full OpenRouter model objects in wpaw_openrouter_models, while validate_model_availability() reads the same transient as a flat list of model IDs. If the settings page refreshes model data first, validation compares a string model ID against arrays and can reject valid models.

Evidence:

• Full object cache: includes/class-openrouter-provider.php:105-177
• ID-list validation using the same key: includes/class-openrouter-provider.php:248-255
• Strict in_array() against the potentially wrong shape: includes/class-openrouter-provider.php:258-265

Impact:

• Settings model refresh can break chat streaming or image generation.
• "Model unavailable" errors can be false negatives.
• This directly preserves the old "fix models UI, break generation" loop.

Recommended fix:

• Rename full object cache to wpaw_openrouter_model_objects.
• Use wpaw_openrouter_model_ids for validation.
• Make validation normalize arrays safely if old transient data exists.
• Delete both old transients on model refresh.

P0: Conversation Storage Is Still Not a Single Source of Truth

The Definition of Done says conversation messages are authoritative in wpaw_conversations.messages via Context Service, but current chat flow still writes legacy _wpaw_chat_history and session messages.

Evidence:

• DoD requires session-table message authority in docs/DEFINITION_OF_DONE.md:17-31.
• Non-stream chat writes legacy post meta and session table in includes/class-gutenberg-sidebar.php:996-1026.
• Stream chat writes legacy post meta and session table in includes/class-gutenberg-sidebar.php:1139-1168.
• Legacy history endpoint still returns _wpaw_chat_history via get_post_chat_history() at includes/class-gutenberg-sidebar.php:1217-1233.
• Context Service claims legacy history is "migrated to session table on read" in includes/class-context-service.php:21-25, but get_context() does not call migration in includes/class-context-service.php:62-87.
• Migration leaves legacy meta in place in includes/class-context-service.php:299-300.

Impact:

• Chat, sessions, migration, clear context, and resume can diverge.
• Users can see old chat resurrect after session migration.
• Costs and context summaries may reference different conversation state.

Recommended fix:

• Stop writing _wpaw_chat_history for new messages.
• Make /chat-history/{post_id} a migration-only compatibility route or remove it after migration.
• Have Context_Service::get_context() perform one-time migration when legacy history exists.
• Delete or mark migrated legacy meta after a successful migration.

P0: Post-scoped REST Authorization Is Still Incomplete

The base permission callback is current_user_can('edit_posts'), which is too broad for routes that read or write a specific post. Some conversation endpoints now check session ownership, but several post-scoped handlers still do not check the target post.

Evidence:

• Global route permission is only edit_posts in includes/class-gutenberg-sidebar.php:906-908.
• /conversations/post/{post_id} returns a post-linked session without edit_post check in includes/class-gutenberg-sidebar.php:7217-7226.
• handle_create_conversation() accepts arbitrary post_id without checking edit_post in includes/class-gutenberg-sidebar.php:7255-7261.
• Writing state reads and writes post meta without edit_post checks in includes/class-gutenberg-sidebar.php:823-887.
• handle_link_conversation_to_post() checks target post edit permission, but does not first verify access to the source session in includes/class-gutenberg-sidebar.php:7453-7482.

Impact:

• Any user who can edit posts may read or modify state for posts they should not access.
• Session linking can attach another user's known session ID to an editable post.
• Editorial privacy is weak on multi-author sites.

Recommended fix:

• Add current_user_can('edit_post', $post_id) to every post-scoped read/write handler.
• Add current_user_can_access($session_id) before link-post.
• Add route-level args with validation for post_id and session_id.

P0: Conversation Table Migration Is Better, But Still Version-fragile

Activation now creates the conversation table, but version checks still mix the main DB version and conversation-specific version. Existing installs with wpaw_db_version=1.1.0 but no conversation table can still be skipped by wp_agentic_writer_maybe_create_tables().

Evidence:

• Conversation table creation records wpaw_conversations_db_version in includes/class-conversation-migration.php:43-47.
• wpaw_run_migrations() still checks wpaw_db_version in includes/class-conversation-migration.php:67-72.
• Main table creation only runs when wpaw_db_version < 1.1.0 in wp-agentic-writer.php:223-242.
• Cleanup cron is scheduled at include time in includes/class-conversation-migration.php:94-99, independent of table readiness.

Impact:

• A site previously marked upgraded can still miss wpaw_conversations.
• Cron may run SQL against a missing table.
• Reinstall and upgrade behavior remains hard to reason about.

Recommended fix:

• Change migration runner to read and update only wpaw_conversations_db_version.
• Always call a lightweight ensure_conversations_table() on plugin load or before conversation DB access.
• Unschedule wpaw_cleanup_old_sessions on deactivation.

High Priority Findings

P1: Provider Fallback Violates Provider Transparency Contract

The DoD requires actual provider/model metadata plus warnings in every AI response. Current provider routing still falls back to OpenRouter without surfacing that to the UI.

Evidence:

• DoD provider metadata requirement: docs/DEFINITION_OF_DONE.md:53-66.
• Fallback behavior: includes/class-provider-manager.php:35-50.
• OpenRouter chat responses include model, but not provider or warnings, in includes/class-openrouter-provider.php:505-513 and includes/class-openrouter-provider.php:739-747.

Impact:

• A user choosing local/private generation may unknowingly send prompts to OpenRouter.
• Sidebar cost/provider labels can be wrong.
• Debugging model routing remains confusing.

Recommended fix:

• Replace raw provider return with a Provider_Selection_Result containing provider instance, provider name, selected provider, actual provider, and warnings.
• Include provider, selected_provider, fallback_used, and warnings in all AI responses.
• Add a setting for fallback policy: fail closed, ask user, or auto fallback.

P1: Cost Tracker Is Still a Log, Not a Reliable Ledger

Cost tracking inserts rows, but it does not honor the tracking setting, record failures, record provider/session IDs, or enforce budget limits.

Evidence:

• DoD cost integrity requires success and failed calls to be intentionally recorded in docs/DEFINITION_OF_DONE.md:68-75.
• Cost table schema lacks provider/session/status/error columns in wp-agentic-writer.php:198-209.
• WP_Agentic_Writer_Cost_Tracker::add_request() unconditionally inserts in includes/class-cost-tracker.php:58-75.
• Image generation returns variant costs but does not call the ledger hook in includes/class-image-manager.php:482-522.

Impact:

• "Cost tracking disabled" is ambiguous because backend records still happen.
• Failed paid attempts are invisible.
• Costs cannot be reconciled by provider or conversation.

Recommended fix:

• Define whether cost_tracking_enabled means "show UI" or "store records"; rename or enforce accordingly.
• Add columns: provider, session_id, request_id, status, error_code, currency, raw_usage.
• Add preflight estimates and optional hard monthly budget enforcement before remote calls.

P1: Settings Cost Log Queries Load Too Much Data

The V2 AJAX cost log computes pagination after loading all matching rows. On real sites this can become slow.

Evidence:

• ajax_get_cost_log_data() counts rows, then fetches all matching records in includes/class-settings-v2.php:645-652.
• It groups in PHP and paginates after formatting in includes/class-settings-v2.php:654-703.

Impact:

• Cost log can degrade admin performance as usage grows.
• Large histories increase memory usage and can time out.

Recommended fix:

• Push grouping and pagination into SQL.
• Add indexes for created_at, action, model, and composite reporting queries.

P1: Writing State Routes Can Leak or Modify Other Users' Draft State

The new writing-state endpoints read and write post meta but rely only on the broad edit_posts permission.

Evidence:

• Routes registered at includes/class-gutenberg-sidebar.php:624-641.
• Read/write handlers do not call current_user_can('edit_post', $post_id) in includes/class-gutenberg-sidebar.php:823-887.

Impact:

• A contributor/editor can potentially inspect or change workflow state for a post outside their permissions.
• Pause/resume state can be corrupted across users.

Recommended fix:

• Require edit_post for the specific post in both handlers.
• Sanitize status against an allowlist: idle, in_progress, paused, completed, failed.
• Consider moving writing state into the conversation/session record to reduce post meta sprawl.

Medium Priority Findings

P2: Context Service Exists But Is Not the Unified Interface

WP_Agentic_Writer_Context_Service is a useful foundation, but most generation paths still assemble context directly in class-gutenberg-sidebar.php. Static search shows only chat persistence and migration use it.

Evidence:

• Context Service methods exist in includes/class-context-service.php.
• Current references from sidebar are limited to message append and migration in includes/class-gutenberg-sidebar.php:1009, includes/class-gutenberg-sidebar.php:1151, and includes/class-gutenberg-sidebar.php:7526.
• DoD requires all generation paths to use it in docs/DEFINITION_OF_DONE.md:29-52.

Opportunity:

• Make Context_Service::build_ai_context() the only path for chat, plan, write, refine, SEO/GEO, image prompt generation, and suggestions.
• Let REST handlers pass request data to Workflow Service, not directly assemble prompts.

P2: Session IDs Are Short and Non-cryptographic

Session IDs are generated with substr(md5(uniqid(wp_rand(), true)), 0, 16).

Evidence:

• includes/class-conversation-manager.php:63-65

Impact:

• The attack surface is still small, but this is weaker than modern token generation.
• Since session IDs gate private editorial context, stronger IDs are cheap insurance.

Recommended fix:

• Use bin2hex(random_bytes(16)) where available with a WP fallback, or wp_generate_uuid4().
• Expand DB column from VARCHAR(32) if needed.

P2: Model Defaults Are Still Fragmented

The implementation made some defaults cheaper, but there is still no single model registry.

Evidence:

• Activation still stores planning_model and legacy execution_model in wp-agentic-writer.php:140-142.
• Provider defaults differ in includes/class-openrouter-provider.php:34-69.
• Settings V2 has multiple fallback sets in includes/class-settings-v2.php:105-111, includes/class-settings-v2.php:228-273, and includes/class-settings-v2.php:990-995.
• JS has another set in assets/js/settings-v2.js:26-38.

Impact:

• Fresh install, upgraded install, saved settings, and JS reset can produce different model choices.
• Support/debugging becomes harder because "default" depends on code path.

Recommended fix:

• Add WP_Agentic_Writer_Model_Registry.
• Expose the registry to JS through localization.
• Migrate execution_model to writing_model and stop storing both.

P2: Debug Logging Is Partly Gated, Frontend Logging Is Not

Backend logging improved in some places, but frontend debug logs remain numerous and some backend providers still log operational details.

Evidence:

• wpaw_debug_log() is gated in includes/class-gutenberg-sidebar.php:20-27.
• Frontend migration/session/clarity logs remain in assets/js/sidebar.js:308-322, assets/js/sidebar.js:5619-5630, and assets/js/sidebar.js:6130-6157.
• Settings debug logs remain in assets/js/settings-v2.js:53-130 and cost log logs in assets/js/settings-v2.js:390-503.

Impact:

• Browser console can expose topics, local backend behavior, model state, and debug-only workflows.
• Debug noise hides real errors for users.

Recommended fix:

• Add wpAgenticWriter.debug and wpawSettingsV2.debug.
• Wrap all console logging behind a tiny logger utility.
• Keep console.error only for actionable user-visible failures.

P2: Admin UX Still Has External Runtime Dependencies

Settings V2 still uses CDN Bootstrap and Select2.

Evidence:

• includes/class-settings-v2.php:67-75

Impact:

• Settings UI can break offline, under CSP, or in restricted enterprise admin environments.
• External admin CDNs create privacy and supply-chain risk.

Recommended fix:

• Bundle vendor files locally or rebuild the settings UI with WordPress components.

P2: Uninstall and Data Lifecycle Are Not Clean

The main uninstall hook and uninstall.php disagree. Neither fully removes conversations, versions, custom models, all post meta, transients, or cron events.

Evidence:

• Main uninstall: wp-agentic-writer.php:269-294
• Separate uninstall file: uninstall.php:12-21

Impact:

• Reinstalls can inherit stale model settings, chat sessions, and DB versions.
• Testing fresh install behavior remains unreliable.

Recommended fix:

• Choose one uninstall path.
• Add an admin setting for "delete all plugin data on uninstall".
• Delete all plugin options, transients, scheduled hooks, tables, upload temp files, and known post/user meta.

UI/UX Opportunities

Make Context Visible

Users need a compact "What the agent knows" panel: active session, linked post, focus keyword, language, plan status, message count, provider, model, and estimated cost. This directly reduces the confusion when a conversation resumes with unexpected memory.

Make Provider Execution Explicit

Before any paid/private-sensitive action, show the selected provider and the actual provider health. If fallback will happen, ask or show a visible warning. After the call, show the actual provider/model used.

Add Workflow State Instead of Hidden Modes

The sidebar has chat, sessions, planning, writing, cost, SEO, clarification, resume, and suggestions. These should map to one visible state machine:

1. Context
2. Plan
3. Write
4. Review
5. Publish Assist

Each state should have one clear primary action and one clear recovery action.

Add Review/Accept Safety for High-impact Edits

Generated blocks and refinements should have a review layer for diff/accept/reject, especially for article-wide and multi-pass edits.

Definition of Done Compliance

DoD area	Current compliance	Notes
Storage layer declaration	Partial	DoD exists, but current code still uses dual chat storage.
Context Service usage	Failing	Not all generation paths use Context Service.
Provider transparency	Failing	Responses do not consistently include provider or fallback warnings.
Cost record integrity	Failing	Failed calls and image calls are not consistently ledgered.
Workflow tests	Unknown	Syntax checks pass, but no end-to-end workflow evidence found.
No double source of truth	Failing	Conversation messages still exist in post meta and sessions.
Migration safety	Partial	Conversation table creation added, but versioning remains inconsistent.
Security contract	Partial	Some session endpoints fixed, post-scoped routes remain broad.
Changelog policy	Failing	CHANGELOG.md was not found while DoD requires it.

Recommended Next Work Queue

Do First

1. Split OpenRouter model cache keys and flush old transient data.
2. Enforce edit_post and session access checks on all post/session routes.
3. Make conversation messages session-table only for new writes.
4. Fix conversation migration versioning to use wpaw_conversations_db_version independently.

Do Second

1. Implement provider selection metadata and fallback warnings.
2. Upgrade cost tracking into a provider/session-aware ledger.
3. Centralize model defaults in a registry.
4. Gate frontend and backend debug logging.

Do Third

1. Extract REST controllers and workflow services from class-gutenberg-sidebar.php.
2. Bundle admin dependencies locally.
3. Consolidate uninstall/data retention behavior.
4. Add end-to-end workflow tests for Chat -> Plan -> Write, Stop -> Resume, and Clear Context -> New Plan.

Completion Marker

The original 2026-05-22 audit is now marked complete/superseded. Remaining work should be tracked from this follow-up audit only, to avoid duplicate jobs debt.