wp-agentic-writer/docs/architecture/PLUGIN_AUDIT_RETRACE_EIGHTH_PASS_2026-05-25.md

WP Agentic Writer Eighth Retrace Audit

Status: COMPLETE / RETRACED
Completion marker: 2026-05-26
Follow-up report: docs/architecture/PLUGIN_AUDIT_RETRACE_NINTH_PASS_2026-05-26.md

Audit date: 2026-05-25
Baseline retraced: docs/architecture/PLUGIN_AUDIT_RETRACE_SEVENTH_PASS_2026-05-25.md
Scope: eighth pass after seventh-retrace implementation, covering conversation history migration, provider transparency, cost tracking, model defaults, UI/UX, and release readiness.

Executive Summary

The seventh-pass implementation made real progress:

• WP_Agentic_Writer_Cost_Tracker::add_request() now defaults unknown providers to unknown instead of openrouter.
• The previously sampled successful AI cost hooks now pass provider/session/status metadata.
• Previously missing provider metadata was added to more backend responses, including execution, regeneration, multi-pass refinement, and article refinement.
• The legacy /chat-history backend path now attempts to migrate/read conversation-backed data instead of always returning raw _wpaw_chat_history.
• PHP and JavaScript syntax checks pass.

However, the plugin is still not audit-clean. The most important remaining issue is conversation continuity: the sidebar still depends on the deprecated /chat-history route, and that route now appears internally inconsistent because it reads _wpaw_active_session_id, but no writer for that meta key was found. That can make legacy chat migration look successful in storage while the sidebar receives an empty history.

Verification Performed

• PHP syntax check across plugin PHP files: passed.
• node -c assets/js/sidebar.js: passed.
• node -c assets/js/settings-v2.js: passed.
• node -c assets/js/sidebar-utils.js: passed.
• Static retrace of seventh-pass findings against current code.
• Static sweep of chat-history migration, provider metadata, cost hook metadata, failed AI paths, sidebar provider UI, and model defaults.
• No live WordPress editor/browser workflow was run in this pass.

Seventh-Pass Status Trace

Seventh-pass item	Current status	Evidence
Sidebar dependency on /chat-history	Still open	assets/js/sidebar.js:644-668 still fetches /chat-history/${postId}.
Backend /chat-history compatibility	Partially fixed, has a new continuity bug	get_post_chat_history() migrates/reads sessions, but depends on _wpaw_active_session_id at includes/class-gutenberg-sidebar.php:1359-1389; no writer for that meta key was found.
Cost hook default provider	Fixed	add_request() now defaults provider to unknown at includes/class-cost-tracker.php:120-124.
Successful AI cost hook metadata	Improved	Previously sampled routes now pass provider/session/status, for example execution at includes/class-gutenberg-sidebar.php:3269-3281, regeneration at includes/class-gutenberg-sidebar.php:3748-3760, summarize at includes/class-gutenberg-sidebar.php:6438-6450, and article refinement at includes/class-gutenberg-sidebar.php:7005-7017.
Provider metadata on backend responses	Improved but inconsistent	More routes include provider_metadata, but chat still uses top-level fields and the sidebar does not render either shape.
Failed AI cost attempts	Still open	Several is_wp_error() branches still return without recording an error-status attempt.
Model registry/default unification	Still open	No central model registry found; defaults remain duplicated.
WordPress editor browser pass	Still open	Syntax checks passed, but no editor workflow was verified.

Remaining Findings

P1: Deprecated Chat-History Compatibility Can Return Empty Data After Migration

The sidebar still loads chat history through the deprecated route:

• assets/js/sidebar.js:644-668 fetches ${wpAgenticWriter.apiUrl}/chat-history/${postId} and seeds messages from the response.

The backend route now tries to be smarter:

• handle_get_chat_history() calls get_post_chat_history() at includes/class-gutenberg-sidebar.php:1300-1326.
• get_post_chat_history() checks _wpaw_chat_history_migrated, then tries to read _wpaw_active_session_id and load context from that id at includes/class-gutenberg-sidebar.php:1359-1368.
• If legacy history exists and is not migrated, it calls migrate_legacy_chat_history() at includes/class-gutenberg-sidebar.php:1378-1380.
• After migration, it again tries _wpaw_active_session_id at includes/class-gutenberg-sidebar.php:1382-1389.

The problem: a search found no writer for _wpaw_active_session_id; it appears only in this read path. migrate_legacy_chat_history() returns the migrated session id at includes/class-context-service.php:272-324, but get_post_chat_history() ignores that return value.

Impact:

• A legacy post can be migrated into the conversation table and have _wpaw_chat_history deleted, but /chat-history can still return an empty array.
• Because the sidebar still depends on /chat-history, users can perceive this as "chat history disappeared" even though the session data exists.
• This is exactly the kind of context/history regression that keeps the audit loop alive.

Recommended fix:

• Best: stop using /chat-history in the sidebar and hydrate from the canonical conversation/session context endpoint.
• If the compatibility endpoint remains, capture the return value from migrate_legacy_chat_history() and load that session directly.
• For already migrated posts, use WP_Agentic_Writer_Conversation_Manager::get_sessions_for_post( $post_id ) instead of _wpaw_active_session_id, or write _wpaw_active_session_id consistently when sessions are created/selected.
• Add a regression test for a legacy post with _wpaw_chat_history and no _wpaw_active_session_id; expected result: /chat-history returns migrated messages and the canonical session id.

P1: Provider Transparency Is Still Not A Single End-To-End Contract

The backend now has a shared metadata helper:

• build_provider_metadata() returns provider, selected_provider, fallback_used, warnings, and model at includes/class-gutenberg-sidebar.php:956-963.

More routes now include provider metadata:

• Plan generation at includes/class-gutenberg-sidebar.php:2049-2058.
• Execution at includes/class-gutenberg-sidebar.php:3283-3292.
• Regeneration at includes/class-gutenberg-sidebar.php:3762-3770.
• Meta description at includes/class-gutenberg-sidebar.php:6276-6285.
• Multi-pass refinement at includes/class-gutenberg-sidebar.php:6930-6939.
• Article refinement at includes/class-gutenberg-sidebar.php:7019-7028.

But the contract is still not end to end:

• Chat still returns top-level provider, selected_provider, fallback_used, and warnings at includes/class-gutenberg-sidebar.php:1067-1071, while other routes use nested provider_metadata.
• assets/js/sidebar.js does not reference provider_metadata, fallback_used, selected_provider, or warnings; its only provider references are web-search availability checks at assets/js/sidebar.js:5983-6000.

Impact:

• API consumers still need two response shapes.
• Users still do not see actual provider/fallback behavior in the editor.
• The Definition of Done says UI must show actual provider used, but that is still not implemented.

Recommended fix:

• Choose one response shape and apply it to all AI endpoints. If keeping provider_metadata, also make chat use that envelope.
• Update sidebar state/rendering to show actual provider, model, fallback, and warnings near cost/status feedback.
• Add a static check that every provider-backed route returns the same metadata shape.

P2: Failed AI Calls Still Usually Do Not Record Error-Status Cost Attempts

Successful cost tracking improved substantially, but failed attempts are still mostly invisible:

• Clarity fallback returns cost 0 without recording a failed provider attempt at includes/class-gutenberg-sidebar.php:4100-4110.
• Clarity JSON parse fallback returns cost 0 without an error-status record at includes/class-gutenberg-sidebar.php:4117-4132.
• Regeneration returns regeneration_error without cost/error tracking at includes/class-gutenberg-sidebar.php:3738-3746.
• Multi-pass refinement returns the provider error directly at includes/class-gutenberg-sidebar.php:6910-6914.
• Article refinement returns the provider error directly at includes/class-gutenberg-sidebar.php:6996-7000.

Impact:

• Provider reliability and failure rates are undercounted.
• Users may see a failure, but admins do not get a durable cost/attempt trail explaining which provider/model failed.
• Fallback behavior is harder to audit because failures and successes are not represented consistently.

Recommended fix:

• Add one helper, for example track_ai_cost( $post_id, $response, $action, $provider_result, $session_id = '', $status = 'success' ).
• Call it for both success and failure paths, with status = 'error', cost = 0, and available provider/model data for failures.
• Keep user-facing errors unchanged, but always record the attempt where a provider request was actually made.

P2: Cost Tracking Is Better But Still Not Structurally Guarded

The direct default-provider bug was fixed:

• add_request() now defaults provider to unknown at includes/class-cost-tracker.php:124.

The previous high-risk success hooks sampled in this pass now include provider/session/status. That is good. The remaining structural issue is that cost tracking is still done through repeated raw do_action( 'wp_aw_after_api_request', ... ) calls across the route class.

Impact:

• New AI routes can still accidentally omit provider/session/status, or skip failed-attempt tracking.
• The system relies on manual discipline instead of a reusable contract.

Recommended fix:

• Wrap the hook with a local helper and use that helper everywhere.
• Add a simple static test or lint script that rejects direct do_action( 'wp_aw_after_api_request' outside the helper.

P2: Model Defaults Are Still Fragmented

No central model registry was found. Defaults remain spread across:

• Activation defaults in wp-agentic-writer.php:140-142.
• Sidebar defaults in includes/class-gutenberg-sidebar.php:278-283.
• Settings defaults and fallbacks in includes/class-settings.php and includes/class-settings-v2.php.
• OpenRouter provider defaults in includes/class-openrouter-provider.php:34-69.
• JavaScript presets in assets/js/settings-v2.js:35-56.
• Wrapper fallback model groups in includes/class-wp-ai-client-wrapper.php:94-100.
• Image manager fallbacks in includes/class-image-manager.php:185-249.

Impact:

• Defaults can drift between activation, runtime provider behavior, settings UI, cost estimation, and JS presets.
• Model-related fixes remain easy to regress.

Recommended fix:

• Create one PHP model registry for task defaults, labels, capabilities, provider support, pricing hints, and deprecation status.
• Localize JS presets from that registry.
• Add a consistency check that activation/settings/provider defaults match the registry.

P2: Editor UI/UX Still Needs Browser Verification

Syntax checks are clean, but no live editor workflow was run.

Impact:

• The remaining issues are heavily UI-dependent: chat hydration, session continuity, provider warning display, streaming completion, and cost feedback.
• Static checks cannot validate editor package compatibility, REST nonce behavior, layout, or user-visible provider/cost states.

Recommended fix:

• Run a WordPress editor browser pass after the next implementation pass.
• Verify sidebar open/persist, chat reload continuity, plan/write/refine cost updates, provider warning display, and unauthorized post failures.

Recommended Next Work

1. Remove the sidebar dependency on /chat-history, or repair the compatibility route by using the migrated session id directly.
2. Add frontend rendering for provider/model/fallback/warnings.
3. Normalize provider metadata to one response shape across chat and non-chat AI actions.
4. Add failed-attempt cost tracking with status = 'error'.
5. Wrap cost tracking in a helper and stop using raw hook calls directly.
6. Consolidate model defaults into a registry.
7. Run the WordPress editor browser workflow pass.

Current Verdict

The seventh-pass implementation is partially proper and materially better than the previous state. Successful cost attribution and backend provider metadata coverage improved.

The plugin is still not audit-clean. The highest-priority remaining fix is chat/context continuity: the active sidebar history loader still depends on a deprecated route, and that route can return empty results after migration because it relies on _wpaw_active_session_id without any discovered writer.