feat: consolidate docs, backend/session infra, and settings updates
This commit is contained in:
Dwindi Ramadhana
233
docs/architecture/PLUGIN_AUDIT_RETRACE_SIXTH_PASS_2026-05-25.md
Normal file
233
docs/architecture/PLUGIN_AUDIT_RETRACE_SIXTH_PASS_2026-05-25.md
Normal file
@@ -0,0 +1,233 @@
|
||||
# WP Agentic Writer Sixth Retrace Audit
|
||||
|
||||
Status: COMPLETE / RETRACED
|
||||
Completion marker: 2026-05-25
|
||||
Follow-up report: `docs/architecture/PLUGIN_AUDIT_RETRACE_SEVENTH_PASS_2026-05-25.md`
|
||||
|
||||
Audit date: 2026-05-25
|
||||
Baseline retraced: `docs/architecture/PLUGIN_AUDIT_RETRACE_FIFTH_PASS_2026-05-25.md`
|
||||
Scope: sixth pass after fifth-retrace implementation, covering REST authorization, UI/UX risk, conversation context/history, provider transparency, cost attribution, model defaults, and release readiness.
|
||||
|
||||
## Executive Summary
|
||||
|
||||
The fifth-pass implementation closed the main P0 defects that were causing the "fix A, lose B" cycle:
|
||||
|
||||
- Permission checks for the previously flagged post-scoped handlers now happen before post config, meta, content, provider, streaming, or cost work.
|
||||
- Utility routes that accept `postId` for cost/context attribution now validate `edit_post` before using that post id.
|
||||
- Legacy chat migration now has stronger migrate-on-read behavior and deletes old `_wpaw_chat_history` after migration.
|
||||
- PHP and JavaScript syntax checks pass.
|
||||
|
||||
I did not find a new P0 blocker in this retrace. The remaining gaps are narrower and mostly about consistency contracts: provider metadata is still only complete in chat, migrate-on-read can still return a stale `session_id`, legacy chat history is still exposed through a deprecated route, model defaults remain fragmented, and browser-level UI compatibility has not been verified.
|
||||
|
||||
## Verification Performed
|
||||
|
||||
- PHP syntax check across plugin PHP files: passed.
|
||||
- `node -c assets/js/sidebar.js`: passed.
|
||||
- `node -c assets/js/settings-v2.js`: passed.
|
||||
- Static retrace of fifth-pass findings against current code.
|
||||
- Static sweep of provider metadata, context migration, cost tracker, model defaults, and legacy chat history surfaces.
|
||||
- No live WordPress editor browser workflow was run in this pass.
|
||||
|
||||
## Fifth-Pass Status Trace
|
||||
|
||||
| Fifth-pass item | Current status | Evidence |
|
||||
|---|---:|---|
|
||||
| `handle_revise_plan()` permission ordering | Fixed | `postId` is extracted and checked before post config/meta/provider work in `includes/class-gutenberg-sidebar.php:2023-2057`. |
|
||||
| `handle_block_refine()` permission ordering | Fixed | `edit_post` is checked before post config/provider work in `includes/class-gutenberg-sidebar.php:4207-4236`. |
|
||||
| `handle_refine_from_chat()` permission ordering | Fixed | `edit_post` is checked before post config/streaming work in `includes/class-gutenberg-sidebar.php:4863-4893`. |
|
||||
| `handle_generate_meta()` permission ordering | Fixed | `edit_post` is checked before `get_post()` in `includes/class-gutenberg-sidebar.php:6085-6108`. |
|
||||
| `handle_check_clarity()` permission ordering | Fixed | `edit_post` is checked before resolving post config in `includes/class-gutenberg-sidebar.php:3811-3839`. |
|
||||
| `handle_summarize_context()` post-id authorization | Fixed | The route validates `edit_post` before provider/cost work in `includes/class-gutenberg-sidebar.php:6266-6278`. |
|
||||
| `handle_detect_intent()` post-id authorization | Fixed | The route validates `edit_post` before provider/cost work in `includes/class-gutenberg-sidebar.php:6374-6388`. |
|
||||
| `handle_refine_multi_pass()` post-id authorization | Fixed | The route validates `edit_post` before provider/cost work in `includes/class-gutenberg-sidebar.php:6756-6770`. |
|
||||
| Migrate-on-read session recovery | Improved, still has one response-identity edge case | `get_context()` uses the migrated id for lookup, but returns the original `$session_id` in the response at `includes/class-context-service.php:72-90`. |
|
||||
| Provider metadata response contract | Still open | Chat responses include provider transparency; many non-chat responses still omit it. |
|
||||
| Legacy `/chat-history` endpoint | Still open | The deprecated route remains registered and still returns legacy post meta in `includes/class-gutenberg-sidebar.php:346-354` and `includes/class-gutenberg-sidebar.php:1282-1308`. |
|
||||
| Deprecated cost wrapper | Still open as low-risk debt | `record_usage()` is deprecated but still hard-codes provider `openrouter` in `includes/class-cost-tracker.php:176-186`. |
|
||||
| Model registry/default unification | Still open | Defaults remain spread across activation, settings, providers, JS presets, wrappers, and image manager. |
|
||||
| Sidebar browser compatibility | Unverified | Syntax passes, but no WordPress editor/browser pass was run. |
|
||||
|
||||
## Remaining Findings
|
||||
|
||||
### P1: Non-Chat AI Responses Still Do Not Expose Provider Transparency
|
||||
|
||||
Chat now satisfies the provider transparency contract by returning `provider`, `selected_provider`, `fallback_used`, and `warnings` in normal and streaming completion paths at `includes/class-gutenberg-sidebar.php:1049-1053` and `includes/class-gutenberg-sidebar.php:1220-1227`.
|
||||
|
||||
The same contract is still missing from many non-chat AI endpoints:
|
||||
|
||||
- Plan generation tracks the actual provider but returns only `plan`, `cost`, and `web_search_results` at `includes/class-gutenberg-sidebar.php:1992-2013`.
|
||||
- Plan revision tracks the actual provider but returns only `plan` and `cost` at `includes/class-gutenberg-sidebar.php:2153-2172`.
|
||||
- Block refinement returns `blocks`, `blockId`, and `cost` only at `includes/class-gutenberg-sidebar.php:4338-4355`.
|
||||
- Multi-pass refinement returns `pass`, `refined_content`, and `cost` only at `includes/class-gutenberg-sidebar.php:6808-6825`.
|
||||
|
||||
Impact:
|
||||
|
||||
- Users and support logs can see provider/fallback behavior for chat, but not for planning, writing, refinement, clarity, SEO/meta, or utility AI actions.
|
||||
- When fallback routing or local/cloud routing differs by task, the UI cannot explain why costs, quality, or latency changed.
|
||||
- Cost records may be accurate internally while the user-visible response stays opaque.
|
||||
|
||||
Recommended fix:
|
||||
|
||||
- Add one shared response metadata helper, for example `build_provider_metadata( $provider_result )`.
|
||||
- Add the same keys to every non-streaming AI response:
|
||||
- `provider`
|
||||
- `selected_provider`
|
||||
- `fallback_used`
|
||||
- `warnings`
|
||||
- Add equivalent completion metadata to every streaming AI action, not only chat.
|
||||
- Add focused REST response tests for plan, revise, block refine, refine multi-pass, clarity, and meta generation.
|
||||
|
||||
### P1: Migrated Context Can Return Messages From One Session But Report Another Session ID
|
||||
|
||||
`get_context()` now does the important migration recovery step: when the requested session does not exist and legacy post history exists, it migrates the legacy history and then fetches the effective migrated session id.
|
||||
|
||||
The remaining problem is the response identity:
|
||||
|
||||
- Migration returns `$migrated_session_id` at `includes/class-context-service.php:72`.
|
||||
- `$effective_session_id` is used to fetch `$session` at `includes/class-context-service.php:74-75`.
|
||||
- The returned context still uses the original `$session_id` at `includes/class-context-service.php:89-90`.
|
||||
|
||||
Impact:
|
||||
|
||||
- A client can receive migrated messages/context but keep using an empty or stale session id.
|
||||
- That can fragment follow-up chat into a different session, making "conversation memory disappeared" bugs look random.
|
||||
- This is especially risky during migration from legacy `_wpaw_chat_history` to conversation sessions because it happens only for older posts.
|
||||
|
||||
Recommended fix:
|
||||
|
||||
- After successful migration lookup, set `$session_id = $effective_session_id` before returning context.
|
||||
- Prefer returning `$session['session_id'] ?? $effective_session_id` if the session object carries its canonical id.
|
||||
- Add a migration test where `get_context( 'missing-session', $post_id )` creates a new session and verifies that the response `session_id` equals the migrated session id.
|
||||
|
||||
### P2: Deprecated `/chat-history` Still Exposes Legacy Post-Meta History
|
||||
|
||||
The legacy route is still registered:
|
||||
|
||||
- `GET /wp-agentic-writer/v1/chat-history/(?P<post_id>\d+)` remains active at `includes/class-gutenberg-sidebar.php:346-354`.
|
||||
- `handle_get_chat_history()` still returns values from `_wpaw_chat_history` with a `deprecated` marker at `includes/class-gutenberg-sidebar.php:1282-1308`.
|
||||
- `get_post_chat_history()` still reads `_wpaw_chat_history` directly at `includes/class-gutenberg-sidebar.php:1336-1346`.
|
||||
|
||||
Impact:
|
||||
|
||||
- This is not an immediate permission issue because the route checks `edit_post`.
|
||||
- It is still product debt because there are now two observable history APIs: legacy post meta and conversation sessions.
|
||||
- UI or integrations can keep depending on the old endpoint and bypass the new context/session model.
|
||||
|
||||
Recommended fix:
|
||||
|
||||
- Replace the response body with a conversation-session-backed compatibility payload, or return a structured `410`/deprecation response after one release window.
|
||||
- Track any client code still calling `/chat-history` before removal.
|
||||
- Add a regression test proving no active UI path uses this endpoint.
|
||||
|
||||
### P2: Deprecated Cost Wrapper Still Defaults Provider To OpenRouter
|
||||
|
||||
`record_usage()` is now explicitly deprecated, which is good. It still calls `record_usage_full()` with provider `openrouter` at `includes/class-cost-tracker.php:176-186`.
|
||||
|
||||
Impact:
|
||||
|
||||
- Any remaining caller using the legacy wrapper can create misleading provider attribution.
|
||||
- This is lower risk than before because newer paths use provider metadata, but it can still pollute reporting.
|
||||
|
||||
Recommended fix:
|
||||
|
||||
- Change the default provider to `legacy_unknown` or `unknown`.
|
||||
- Log a debug warning when this wrapper is called so remaining callers can be eliminated.
|
||||
- Search for all direct calls to `record_usage()` and migrate them to `record_usage_full()`.
|
||||
|
||||
### P2: Model Defaults Are Still Fragmented Across Runtime Surfaces
|
||||
|
||||
Model defaults are still declared in multiple places with conflicting generations and ids:
|
||||
|
||||
- Activation defaults in `wp-agentic-writer.php:140-142`.
|
||||
- Sidebar defaults in `includes/class-gutenberg-sidebar.php:276-283`.
|
||||
- Settings defaults and sanitization fallbacks in `includes/class-settings.php` and `includes/class-settings-v2.php`.
|
||||
- Provider property defaults in `includes/class-openrouter-provider.php:34-69`.
|
||||
- JavaScript presets in `assets/js/settings-v2.js:35-56`.
|
||||
- Wrapper fallback groups in `includes/class-wp-ai-client-wrapper.php:94-100`.
|
||||
- Image-manager fallbacks in `includes/class-image-manager.php:185-249`.
|
||||
|
||||
Impact:
|
||||
|
||||
- Changing a default model in one layer can silently diverge from another layer.
|
||||
- UI presets, provider runtime defaults, activation defaults, and cost estimation can disagree.
|
||||
- This is exactly the kind of area where fixing A can regress B.
|
||||
|
||||
Recommended fix:
|
||||
|
||||
- Create one PHP model registry for task defaults, labels, capabilities, pricing hints, provider support, and deprecation status.
|
||||
- Generate or localize the JS presets from that registry instead of hard-coding them independently.
|
||||
- Add a consistency test that checks all task defaults exist in the registry and that activation/settings/provider fallbacks match it.
|
||||
|
||||
### P2: Sidebar UI Needs A Real WordPress Editor Compatibility Pass
|
||||
|
||||
`assets/js/sidebar.js` and `assets/js/settings-v2.js` both pass syntax checks, but syntax is not enough for the editor UI.
|
||||
|
||||
Impact:
|
||||
|
||||
- WordPress package availability, `wp.editPost.PluginSidebar` compatibility, REST nonce behavior, streaming UI states, and cost display states can still fail only inside the block editor.
|
||||
- The plugin has many user-facing states now: chat, planning, refinement, clarity, model routing, cost display, warnings, and session history. A static pass cannot validate their interaction quality.
|
||||
|
||||
Recommended fix:
|
||||
|
||||
- Run a browser pass inside the WordPress editor with the plugin enabled.
|
||||
- Cover at minimum:
|
||||
- Sidebar opens and persists.
|
||||
- Chat session continues after reload.
|
||||
- Provider/fallback warnings render where metadata exists.
|
||||
- Cost display updates after chat, plan, refine, and meta-generation actions.
|
||||
- Unauthorized post access fails cleanly without partial UI mutation.
|
||||
- Model settings changes are reflected in generated requests.
|
||||
|
||||
## System-Level Opportunities
|
||||
|
||||
### 1. Add A Shared Guard For Post-Scoped REST Requests
|
||||
|
||||
The fifth-pass fixes were successful, but they were route-by-route. To keep this from regressing, add a helper such as:
|
||||
|
||||
```php
|
||||
private function require_post_permission_from_request( WP_REST_Request $request, $field = 'postId' ) {
|
||||
$post_id = absint( $request->get_param( $field ) );
|
||||
if ( $post_id > 0 && ! $this->check_post_permission( $post_id ) ) {
|
||||
return new WP_Error( 'forbidden', __( 'You do not have permission to access this post.', 'wp-agentic-writer' ), array( 'status' => 403 ) );
|
||||
}
|
||||
return $post_id;
|
||||
}
|
||||
```
|
||||
|
||||
Then use it immediately after request parsing in every route that reads, writes, streams, or attributes cost to a post.
|
||||
|
||||
### 2. Standardize AI Response Envelopes
|
||||
|
||||
The plugin needs one response shape for all AI actions:
|
||||
|
||||
```json
|
||||
{
|
||||
"data": {},
|
||||
"cost": 0,
|
||||
"provider": "openrouter",
|
||||
"selected_provider": "openrouter",
|
||||
"fallback_used": false,
|
||||
"warnings": []
|
||||
}
|
||||
```
|
||||
|
||||
The exact shape can differ, but the metadata keys should not. This will make UI, debugging, and cost review dramatically simpler.
|
||||
|
||||
### 3. Treat Conversation Session ID As A Canonical Contract
|
||||
|
||||
The remaining migration edge case is a sign that `session_id` should be treated as a canonical response contract. Any endpoint that creates, migrates, clears, or resumes a session should return the active session id and the UI should update local state from that value.
|
||||
|
||||
## Recommended Next Work
|
||||
|
||||
1. Fix `get_context()` so migrated reads return the effective session id.
|
||||
2. Add provider metadata to every non-chat AI response and stream completion event.
|
||||
3. Replace or retire `/chat-history` compatibility behavior.
|
||||
4. Change deprecated cost wrapper provider attribution from `openrouter` to `legacy_unknown`.
|
||||
5. Start model registry consolidation.
|
||||
6. Run the WordPress editor browser pass before calling the audit chain fully closed.
|
||||
|
||||
## Current Verdict
|
||||
|
||||
The fifth-pass implementation is proper for the critical authorization and cost-attribution issues it targeted. I would mark the fifth-pass P0 items complete.
|
||||
|
||||
The plugin is not fully audit-clean yet. The next highest-value work is now chat/context correctness plus response transparency: fix the migrated `session_id` edge case and make provider/cost metadata consistent across every AI action.
|
||||
Reference in New Issue
Block a user