feat: consolidate docs, backend/session infra, and settings updates
This commit is contained in:
Dwindi Ramadhana
199
docs/architecture/PLUGIN_AUDIT_RETRACE_EIGHTH_PASS_2026-05-25.md
Normal file
199
docs/architecture/PLUGIN_AUDIT_RETRACE_EIGHTH_PASS_2026-05-25.md
Normal file
@@ -0,0 +1,199 @@
|
||||
# WP Agentic Writer Eighth Retrace Audit
|
||||
|
||||
Status: COMPLETE / RETRACED
|
||||
Completion marker: 2026-05-26
|
||||
Follow-up report: `docs/architecture/PLUGIN_AUDIT_RETRACE_NINTH_PASS_2026-05-26.md`
|
||||
|
||||
Audit date: 2026-05-25
|
||||
Baseline retraced: `docs/architecture/PLUGIN_AUDIT_RETRACE_SEVENTH_PASS_2026-05-25.md`
|
||||
Scope: eighth pass after seventh-retrace implementation, covering conversation history migration, provider transparency, cost tracking, model defaults, UI/UX, and release readiness.
|
||||
|
||||
## Executive Summary
|
||||
|
||||
The seventh-pass implementation made real progress:
|
||||
|
||||
- `WP_Agentic_Writer_Cost_Tracker::add_request()` now defaults unknown providers to `unknown` instead of `openrouter`.
|
||||
- The previously sampled successful AI cost hooks now pass provider/session/status metadata.
|
||||
- Previously missing provider metadata was added to more backend responses, including execution, regeneration, multi-pass refinement, and article refinement.
|
||||
- The legacy `/chat-history` backend path now attempts to migrate/read conversation-backed data instead of always returning raw `_wpaw_chat_history`.
|
||||
- PHP and JavaScript syntax checks pass.
|
||||
|
||||
However, the plugin is still not audit-clean. The most important remaining issue is conversation continuity: the sidebar still depends on the deprecated `/chat-history` route, and that route now appears internally inconsistent because it reads `_wpaw_active_session_id`, but no writer for that meta key was found. That can make legacy chat migration look successful in storage while the sidebar receives an empty history.
|
||||
|
||||
## Verification Performed
|
||||
|
||||
- PHP syntax check across plugin PHP files: passed.
|
||||
- `node -c assets/js/sidebar.js`: passed.
|
||||
- `node -c assets/js/settings-v2.js`: passed.
|
||||
- `node -c assets/js/sidebar-utils.js`: passed.
|
||||
- Static retrace of seventh-pass findings against current code.
|
||||
- Static sweep of chat-history migration, provider metadata, cost hook metadata, failed AI paths, sidebar provider UI, and model defaults.
|
||||
- No live WordPress editor/browser workflow was run in this pass.
|
||||
|
||||
## Seventh-Pass Status Trace
|
||||
|
||||
| Seventh-pass item | Current status | Evidence |
|
||||
|---|---:|---|
|
||||
| Sidebar dependency on `/chat-history` | Still open | `assets/js/sidebar.js:644-668` still fetches `/chat-history/${postId}`. |
|
||||
| Backend `/chat-history` compatibility | Partially fixed, has a new continuity bug | `get_post_chat_history()` migrates/reads sessions, but depends on `_wpaw_active_session_id` at `includes/class-gutenberg-sidebar.php:1359-1389`; no writer for that meta key was found. |
|
||||
| Cost hook default provider | Fixed | `add_request()` now defaults provider to `unknown` at `includes/class-cost-tracker.php:120-124`. |
|
||||
| Successful AI cost hook metadata | Improved | Previously sampled routes now pass provider/session/status, for example execution at `includes/class-gutenberg-sidebar.php:3269-3281`, regeneration at `includes/class-gutenberg-sidebar.php:3748-3760`, summarize at `includes/class-gutenberg-sidebar.php:6438-6450`, and article refinement at `includes/class-gutenberg-sidebar.php:7005-7017`. |
|
||||
| Provider metadata on backend responses | Improved but inconsistent | More routes include `provider_metadata`, but chat still uses top-level fields and the sidebar does not render either shape. |
|
||||
| Failed AI cost attempts | Still open | Several `is_wp_error()` branches still return without recording an error-status attempt. |
|
||||
| Model registry/default unification | Still open | No central model registry found; defaults remain duplicated. |
|
||||
| WordPress editor browser pass | Still open | Syntax checks passed, but no editor workflow was verified. |
|
||||
|
||||
## Remaining Findings
|
||||
|
||||
### P1: Deprecated Chat-History Compatibility Can Return Empty Data After Migration
|
||||
|
||||
The sidebar still loads chat history through the deprecated route:
|
||||
|
||||
- `assets/js/sidebar.js:644-668` fetches `${wpAgenticWriter.apiUrl}/chat-history/${postId}` and seeds `messages` from the response.
|
||||
|
||||
The backend route now tries to be smarter:
|
||||
|
||||
- `handle_get_chat_history()` calls `get_post_chat_history()` at `includes/class-gutenberg-sidebar.php:1300-1326`.
|
||||
- `get_post_chat_history()` checks `_wpaw_chat_history_migrated`, then tries to read `_wpaw_active_session_id` and load context from that id at `includes/class-gutenberg-sidebar.php:1359-1368`.
|
||||
- If legacy history exists and is not migrated, it calls `migrate_legacy_chat_history()` at `includes/class-gutenberg-sidebar.php:1378-1380`.
|
||||
- After migration, it again tries `_wpaw_active_session_id` at `includes/class-gutenberg-sidebar.php:1382-1389`.
|
||||
|
||||
The problem: a search found no writer for `_wpaw_active_session_id`; it appears only in this read path. `migrate_legacy_chat_history()` returns the migrated session id at `includes/class-context-service.php:272-324`, but `get_post_chat_history()` ignores that return value.
|
||||
|
||||
Impact:
|
||||
|
||||
- A legacy post can be migrated into the conversation table and have `_wpaw_chat_history` deleted, but `/chat-history` can still return an empty array.
|
||||
- Because the sidebar still depends on `/chat-history`, users can perceive this as "chat history disappeared" even though the session data exists.
|
||||
- This is exactly the kind of context/history regression that keeps the audit loop alive.
|
||||
|
||||
Recommended fix:
|
||||
|
||||
- Best: stop using `/chat-history` in the sidebar and hydrate from the canonical conversation/session context endpoint.
|
||||
- If the compatibility endpoint remains, capture the return value from `migrate_legacy_chat_history()` and load that session directly.
|
||||
- For already migrated posts, use `WP_Agentic_Writer_Conversation_Manager::get_sessions_for_post( $post_id )` instead of `_wpaw_active_session_id`, or write `_wpaw_active_session_id` consistently when sessions are created/selected.
|
||||
- Add a regression test for a legacy post with `_wpaw_chat_history` and no `_wpaw_active_session_id`; expected result: `/chat-history` returns migrated messages and the canonical session id.
|
||||
|
||||
### P1: Provider Transparency Is Still Not A Single End-To-End Contract
|
||||
|
||||
The backend now has a shared metadata helper:
|
||||
|
||||
- `build_provider_metadata()` returns `provider`, `selected_provider`, `fallback_used`, `warnings`, and `model` at `includes/class-gutenberg-sidebar.php:956-963`.
|
||||
|
||||
More routes now include provider metadata:
|
||||
|
||||
- Plan generation at `includes/class-gutenberg-sidebar.php:2049-2058`.
|
||||
- Execution at `includes/class-gutenberg-sidebar.php:3283-3292`.
|
||||
- Regeneration at `includes/class-gutenberg-sidebar.php:3762-3770`.
|
||||
- Meta description at `includes/class-gutenberg-sidebar.php:6276-6285`.
|
||||
- Multi-pass refinement at `includes/class-gutenberg-sidebar.php:6930-6939`.
|
||||
- Article refinement at `includes/class-gutenberg-sidebar.php:7019-7028`.
|
||||
|
||||
But the contract is still not end to end:
|
||||
|
||||
- Chat still returns top-level `provider`, `selected_provider`, `fallback_used`, and `warnings` at `includes/class-gutenberg-sidebar.php:1067-1071`, while other routes use nested `provider_metadata`.
|
||||
- `assets/js/sidebar.js` does not reference `provider_metadata`, `fallback_used`, `selected_provider`, or `warnings`; its only provider references are web-search availability checks at `assets/js/sidebar.js:5983-6000`.
|
||||
|
||||
Impact:
|
||||
|
||||
- API consumers still need two response shapes.
|
||||
- Users still do not see actual provider/fallback behavior in the editor.
|
||||
- The Definition of Done says UI must show actual provider used, but that is still not implemented.
|
||||
|
||||
Recommended fix:
|
||||
|
||||
- Choose one response shape and apply it to all AI endpoints. If keeping `provider_metadata`, also make chat use that envelope.
|
||||
- Update sidebar state/rendering to show actual provider, model, fallback, and warnings near cost/status feedback.
|
||||
- Add a static check that every provider-backed route returns the same metadata shape.
|
||||
|
||||
### P2: Failed AI Calls Still Usually Do Not Record Error-Status Cost Attempts
|
||||
|
||||
Successful cost tracking improved substantially, but failed attempts are still mostly invisible:
|
||||
|
||||
- Clarity fallback returns cost `0` without recording a failed provider attempt at `includes/class-gutenberg-sidebar.php:4100-4110`.
|
||||
- Clarity JSON parse fallback returns cost `0` without an error-status record at `includes/class-gutenberg-sidebar.php:4117-4132`.
|
||||
- Regeneration returns `regeneration_error` without cost/error tracking at `includes/class-gutenberg-sidebar.php:3738-3746`.
|
||||
- Multi-pass refinement returns the provider error directly at `includes/class-gutenberg-sidebar.php:6910-6914`.
|
||||
- Article refinement returns the provider error directly at `includes/class-gutenberg-sidebar.php:6996-7000`.
|
||||
|
||||
Impact:
|
||||
|
||||
- Provider reliability and failure rates are undercounted.
|
||||
- Users may see a failure, but admins do not get a durable cost/attempt trail explaining which provider/model failed.
|
||||
- Fallback behavior is harder to audit because failures and successes are not represented consistently.
|
||||
|
||||
Recommended fix:
|
||||
|
||||
- Add one helper, for example `track_ai_cost( $post_id, $response, $action, $provider_result, $session_id = '', $status = 'success' )`.
|
||||
- Call it for both success and failure paths, with `status = 'error'`, `cost = 0`, and available provider/model data for failures.
|
||||
- Keep user-facing errors unchanged, but always record the attempt where a provider request was actually made.
|
||||
|
||||
### P2: Cost Tracking Is Better But Still Not Structurally Guarded
|
||||
|
||||
The direct default-provider bug was fixed:
|
||||
|
||||
- `add_request()` now defaults provider to `unknown` at `includes/class-cost-tracker.php:124`.
|
||||
|
||||
The previous high-risk success hooks sampled in this pass now include provider/session/status. That is good. The remaining structural issue is that cost tracking is still done through repeated raw `do_action( 'wp_aw_after_api_request', ... )` calls across the route class.
|
||||
|
||||
Impact:
|
||||
|
||||
- New AI routes can still accidentally omit provider/session/status, or skip failed-attempt tracking.
|
||||
- The system relies on manual discipline instead of a reusable contract.
|
||||
|
||||
Recommended fix:
|
||||
|
||||
- Wrap the hook with a local helper and use that helper everywhere.
|
||||
- Add a simple static test or lint script that rejects direct `do_action( 'wp_aw_after_api_request'` outside the helper.
|
||||
|
||||
### P2: Model Defaults Are Still Fragmented
|
||||
|
||||
No central model registry was found. Defaults remain spread across:
|
||||
|
||||
- Activation defaults in `wp-agentic-writer.php:140-142`.
|
||||
- Sidebar defaults in `includes/class-gutenberg-sidebar.php:278-283`.
|
||||
- Settings defaults and fallbacks in `includes/class-settings.php` and `includes/class-settings-v2.php`.
|
||||
- OpenRouter provider defaults in `includes/class-openrouter-provider.php:34-69`.
|
||||
- JavaScript presets in `assets/js/settings-v2.js:35-56`.
|
||||
- Wrapper fallback model groups in `includes/class-wp-ai-client-wrapper.php:94-100`.
|
||||
- Image manager fallbacks in `includes/class-image-manager.php:185-249`.
|
||||
|
||||
Impact:
|
||||
|
||||
- Defaults can drift between activation, runtime provider behavior, settings UI, cost estimation, and JS presets.
|
||||
- Model-related fixes remain easy to regress.
|
||||
|
||||
Recommended fix:
|
||||
|
||||
- Create one PHP model registry for task defaults, labels, capabilities, provider support, pricing hints, and deprecation status.
|
||||
- Localize JS presets from that registry.
|
||||
- Add a consistency check that activation/settings/provider defaults match the registry.
|
||||
|
||||
### P2: Editor UI/UX Still Needs Browser Verification
|
||||
|
||||
Syntax checks are clean, but no live editor workflow was run.
|
||||
|
||||
Impact:
|
||||
|
||||
- The remaining issues are heavily UI-dependent: chat hydration, session continuity, provider warning display, streaming completion, and cost feedback.
|
||||
- Static checks cannot validate editor package compatibility, REST nonce behavior, layout, or user-visible provider/cost states.
|
||||
|
||||
Recommended fix:
|
||||
|
||||
- Run a WordPress editor browser pass after the next implementation pass.
|
||||
- Verify sidebar open/persist, chat reload continuity, plan/write/refine cost updates, provider warning display, and unauthorized post failures.
|
||||
|
||||
## Recommended Next Work
|
||||
|
||||
1. Remove the sidebar dependency on `/chat-history`, or repair the compatibility route by using the migrated session id directly.
|
||||
2. Add frontend rendering for provider/model/fallback/warnings.
|
||||
3. Normalize provider metadata to one response shape across chat and non-chat AI actions.
|
||||
4. Add failed-attempt cost tracking with `status = 'error'`.
|
||||
5. Wrap cost tracking in a helper and stop using raw hook calls directly.
|
||||
6. Consolidate model defaults into a registry.
|
||||
7. Run the WordPress editor browser workflow pass.
|
||||
|
||||
## Current Verdict
|
||||
|
||||
The seventh-pass implementation is partially proper and materially better than the previous state. Successful cost attribution and backend provider metadata coverage improved.
|
||||
|
||||
The plugin is still not audit-clean. The highest-priority remaining fix is chat/context continuity: the active sidebar history loader still depends on a deprecated route, and that route can return empty results after migration because it relies on `_wpaw_active_session_id` without any discovered writer.
|
||||
Reference in New Issue
Block a user