feat: consolidate docs, backend/session infra, and settings updates
This commit is contained in:
Dwindi Ramadhana
@@ -0,0 +1,208 @@
|
||||
# WP Agentic Writer Seventh Retrace Audit
|
||||
|
||||
Status: COMPLETE / RETRACED
|
||||
Completion marker: 2026-05-25
|
||||
Follow-up report: `docs/architecture/PLUGIN_AUDIT_RETRACE_EIGHTH_PASS_2026-05-25.md`
|
||||
|
||||
Audit date: 2026-05-25
|
||||
Baseline retraced: `docs/architecture/PLUGIN_AUDIT_RETRACE_SIXTH_PASS_2026-05-25.md`
|
||||
Scope: seventh pass after sixth-retrace implementation, covering provider transparency, conversation context/history, cost tracking, model defaults, UI/UX, and release readiness.
|
||||
|
||||
## Executive Summary
|
||||
|
||||
The sixth-pass implementation fixed two important backend issues:
|
||||
|
||||
- `WP_Agentic_Writer_Context_Service::get_context()` now carries an `$effective_session_id` and returns it, so migrate-on-read no longer returns messages from one session while reporting the original stale session id.
|
||||
- Deprecated `WP_Agentic_Writer_Cost_Tracker::record_usage()` now records provider as `unknown` instead of hard-coding `openrouter`.
|
||||
|
||||
I did not find a new P0 authorization blocker. The remaining problems are still meaningful because they affect user trust, cost accuracy, and conversation continuity:
|
||||
|
||||
- The sidebar still actively calls deprecated `/chat-history`, which reads legacy `_wpaw_chat_history` instead of the authoritative conversation table.
|
||||
- Provider transparency was added only partially: some responses now include nested `provider_metadata`, but other AI endpoints still omit it, chat uses a different top-level shape, and the sidebar does not render provider/fallback metadata.
|
||||
- Cost tracking still defaults missing provider arguments to `openrouter`, and many AI cost hooks still omit provider/session/status arguments.
|
||||
- Model defaults remain fragmented across PHP activation, settings, providers, JavaScript presets, wrappers, and image helpers.
|
||||
- Browser-level WordPress editor verification is still not done.
|
||||
|
||||
## Verification Performed
|
||||
|
||||
- PHP syntax check across plugin PHP files: passed.
|
||||
- `node -c assets/js/sidebar.js`: passed.
|
||||
- `node -c assets/js/settings-v2.js`: passed.
|
||||
- `node -c assets/js/sidebar-utils.js`: passed.
|
||||
- Static retrace of sixth-pass findings against current code.
|
||||
- Static sweep of provider metadata, cost hooks, context migration, sidebar history loading, and model defaults.
|
||||
- No live WordPress editor/browser workflow was run in this pass.
|
||||
|
||||
## Sixth-Pass Status Trace
|
||||
|
||||
| Sixth-pass item | Current status | Evidence |
|
||||
|---|---:|---|
|
||||
| Migrated context returns effective session id | Fixed | `$effective_session_id` is initialized, updated after migration, and returned as `session_id` in `includes/class-context-service.php:62-92`. |
|
||||
| Deprecated `record_usage()` provider attribution | Fixed narrowly | The deprecated wrapper now passes `unknown` at `includes/class-cost-tracker.php:176-186`. |
|
||||
| Provider metadata helper | Partially fixed | `build_provider_metadata()` exists at `includes/class-gutenberg-sidebar.php:956-963`, and several responses now include `provider_metadata`. |
|
||||
| Provider transparency response contract | Still partial | Chat uses top-level keys, several non-chat routes use nested `provider_metadata`, and some AI routes still omit metadata entirely. |
|
||||
| Legacy `/chat-history` endpoint | Still open, now confirmed active in UI | Backend still reads `_wpaw_chat_history`, and `assets/js/sidebar.js` still calls `/chat-history/{postId}` on load. |
|
||||
| Cost tracker provider/session/status integrity | Still open | The cost hook accepts 9 args, but many call sites still pass only 7 args, causing provider fallback to the hook default. |
|
||||
| Model registry/default unification | Still open | No model registry was found; defaults remain duplicated across runtime surfaces. |
|
||||
| WordPress editor browser pass | Still open | Syntax checks passed, but no editor workflow was verified. |
|
||||
|
||||
## Remaining Findings
|
||||
|
||||
### P1: Sidebar Still Loads Conversation State From Deprecated Post Meta
|
||||
|
||||
The backend still registers and serves the legacy route:
|
||||
|
||||
- `/chat-history/(?P<post_id>\d+)` remains registered in `includes/class-gutenberg-sidebar.php:346-354`.
|
||||
- `handle_get_chat_history()` still returns `messages` from `get_post_chat_history()` in `includes/class-gutenberg-sidebar.php:1300-1326`.
|
||||
- `get_post_chat_history()` still reads `_wpaw_chat_history` directly in `includes/class-gutenberg-sidebar.php:1354-1364`.
|
||||
|
||||
More importantly, this is still an active frontend dependency:
|
||||
|
||||
- `assets/js/sidebar.js:644-668` calls `${wpAgenticWriter.apiUrl}/chat-history/${postId}` and seeds `messages` from that response.
|
||||
|
||||
Impact:
|
||||
|
||||
- The Definition of Done says conversation messages are authoritative in `wpaw_conversations`, but the sidebar can still hydrate UI state from legacy post meta.
|
||||
- Migrated or newly created sessions can look empty while old post-meta history appears, or old history can override the expected session model on first render.
|
||||
- This keeps two mental models alive: "chat belongs to a post meta array" and "chat belongs to a conversation session."
|
||||
|
||||
Recommended fix:
|
||||
|
||||
- Replace the sidebar history load with a conversation-session/context endpoint.
|
||||
- If compatibility is still needed, make `/chat-history` return session-backed data only, not raw `_wpaw_chat_history`.
|
||||
- Add a regression check that `assets/js/sidebar.js` has no `/chat-history` fetch after migration.
|
||||
|
||||
### P1: Cost Provider Attribution Is Still Wrong For Many AI Actions
|
||||
|
||||
The deprecated `record_usage()` wrapper was fixed, but the main action hook still defaults missing provider data to OpenRouter:
|
||||
|
||||
- `add_action( 'wp_aw_after_api_request', ..., 10, 9 )` expects provider/session/status arguments in `includes/class-cost-tracker.php:48-50`.
|
||||
- `add_request()` still defaults `$provider = 'openrouter'` in `includes/class-cost-tracker.php:124`.
|
||||
|
||||
Many AI routes still call `do_action( 'wp_aw_after_api_request', ... )` with only the first seven arguments, so the cost table will store `openrouter` even when the actual provider was local backend, Codex, or another fallback:
|
||||
|
||||
- Clarity check at `includes/class-gutenberg-sidebar.php:4096-4106`.
|
||||
- Meta description at `includes/class-gutenberg-sidebar.php:6217-6228`.
|
||||
- Summarize context at `includes/class-gutenberg-sidebar.php:6393-6402`.
|
||||
- Multi-pass refinement at `includes/class-gutenberg-sidebar.php:6868-6877`.
|
||||
- Article refinement at `includes/class-gutenberg-sidebar.php:6954-6963`.
|
||||
- Execution total at `includes/class-gutenberg-sidebar.php:3244-3253`.
|
||||
- Regeneration at `includes/class-gutenberg-sidebar.php:3716-3725`.
|
||||
|
||||
Impact:
|
||||
|
||||
- The visible response can say one provider while `wpaw_cost_tracking.provider` records another.
|
||||
- Cost review, provider debugging, local/cloud usage reporting, and fallback analysis become unreliable.
|
||||
- This directly undermines the cost counter/tracker area the audit chain is trying to stabilize.
|
||||
|
||||
Recommended fix:
|
||||
|
||||
- Change the hook default provider from `openrouter` to `unknown`.
|
||||
- Add provider/session/status to every AI `wp_aw_after_api_request` call.
|
||||
- Add a small helper like `track_ai_cost( $post_id, $response, $action, $provider_result, $session_id = '', $status = 'success' )` so new routes cannot omit metadata accidentally.
|
||||
- Add one test or static check that no AI cost hook call has only seven payload arguments.
|
||||
|
||||
### P1: Provider Transparency Is Present But Inconsistent And Not Surfaced In UI
|
||||
|
||||
A shared helper now exists:
|
||||
|
||||
- `build_provider_metadata()` returns `provider`, `selected_provider`, `fallback_used`, `warnings`, and `model` at `includes/class-gutenberg-sidebar.php:956-963`.
|
||||
|
||||
Several responses now include nested `provider_metadata`, for example:
|
||||
|
||||
- Plan generation at `includes/class-gutenberg-sidebar.php:2024-2033`.
|
||||
- Plan revision at `includes/class-gutenberg-sidebar.php:2189-2197`.
|
||||
- Block refinement at `includes/class-gutenberg-sidebar.php:4384-4393`.
|
||||
- Summarize context at `includes/class-gutenberg-sidebar.php:6404-6414`.
|
||||
|
||||
But the contract is not yet consistent:
|
||||
|
||||
- Chat responses still write top-level `provider`, `selected_provider`, `fallback_used`, and `warnings` at `includes/class-gutenberg-sidebar.php:1067-1071`.
|
||||
- Some non-chat AI responses still omit provider metadata entirely, including execution response at `includes/class-gutenberg-sidebar.php:3255-3260`, regenerate block response at `includes/class-gutenberg-sidebar.php:3727-3732`, and article refinement response at `includes/class-gutenberg-sidebar.php:6965-6970`.
|
||||
- `assets/js/sidebar.js` has no handling for `provider_metadata`, `fallback_used`, or `warnings`, so the user still cannot see fallback/provider behavior in the editor.
|
||||
|
||||
Impact:
|
||||
|
||||
- API consumers must handle two provider shapes: top-level metadata for chat and nested metadata elsewhere.
|
||||
- Some workflows still provide no provider transparency.
|
||||
- The Definition of Done requires the UI to show actual provider used, but the sidebar does not yet render it.
|
||||
|
||||
Recommended fix:
|
||||
|
||||
- Pick one response shape and enforce it everywhere. The existing Definition of Done examples use top-level `provider`, `model`, `cost`, and `warnings`.
|
||||
- Include the same provider fields in all AI success responses and all stream completion events.
|
||||
- Update the sidebar to render actual provider and fallback warnings in a compact status line near cost.
|
||||
- Add a static response-contract checklist for every provider-backed route.
|
||||
|
||||
### P2: Failed AI Calls Still Usually Do Not Record Failed Cost Attempts
|
||||
|
||||
The Definition of Done says failed calls should record an attempt with error status. Many routes still return on provider errors without a cost/status record:
|
||||
|
||||
- Clarity check falls back to default questions and returns cost `0` without recording a failed provider attempt in `includes/class-gutenberg-sidebar.php:4054-4071`.
|
||||
- Regenerate block returns a `regeneration_error` without cost tracking in `includes/class-gutenberg-sidebar.php:3706-3714`.
|
||||
- Multi-pass refinement returns the provider error directly in `includes/class-gutenberg-sidebar.php:6862-6866`.
|
||||
- Article refinement returns the provider error directly in `includes/class-gutenberg-sidebar.php:6945-6949`.
|
||||
|
||||
Impact:
|
||||
|
||||
- Reliability metrics undercount provider failures.
|
||||
- Users may see no cost but also no durable audit trail explaining why generation failed.
|
||||
- It becomes harder to debug whether failures are model, provider, prompt, network, or quota related.
|
||||
|
||||
Recommended fix:
|
||||
|
||||
- Use the same cost helper for failed attempts with `status = 'error'`, `cost = 0`, and available provider/model data.
|
||||
- Preserve the user-facing error, but write a cost/event row for observability.
|
||||
|
||||
### P2: Model Defaults Are Still Fragmented
|
||||
|
||||
No central model registry was found. Defaults remain spread across:
|
||||
|
||||
- Activation defaults in `wp-agentic-writer.php:140-142`.
|
||||
- Sidebar defaults in `includes/class-gutenberg-sidebar.php:278-283`.
|
||||
- Settings defaults and fallbacks in `includes/class-settings.php` and `includes/class-settings-v2.php`.
|
||||
- OpenRouter provider defaults in `includes/class-openrouter-provider.php:34-69`.
|
||||
- JavaScript presets in `assets/js/settings-v2.js:35-56`.
|
||||
- Wrapper fallback model groups in `includes/class-wp-ai-client-wrapper.php:94-100`.
|
||||
- Image manager fallbacks in `includes/class-image-manager.php:185-249`.
|
||||
|
||||
Impact:
|
||||
|
||||
- A model default can be changed in one layer while another layer silently keeps the old value.
|
||||
- Cost estimates, UI presets, activation defaults, and provider runtime defaults can disagree.
|
||||
|
||||
Recommended fix:
|
||||
|
||||
- Create one PHP model registry for task defaults, labels, capabilities, provider support, and deprecation status.
|
||||
- Localize JS presets from the registry.
|
||||
- Add a consistency check that activation/settings/provider defaults match the registry.
|
||||
|
||||
### P2: Editor UI/UX Still Needs Browser Verification
|
||||
|
||||
The syntax checks are clean, but no live WordPress editor pass was run.
|
||||
|
||||
Impact:
|
||||
|
||||
- The exact areas still changing, chat hydration, session persistence, provider warnings, streaming completion, and cost display, are frontend workflow problems as much as backend problems.
|
||||
- Static checks cannot validate `wp.editPost` package availability, REST nonce behavior, editor state persistence, or whether new provider metadata appears to users.
|
||||
|
||||
Recommended fix:
|
||||
|
||||
- Run a browser pass in the block editor after the next implementation pass.
|
||||
- Verify sidebar open/persist, chat reload continuity, plan/write/refine cost updates, provider warning display, and unauthorized post failures.
|
||||
|
||||
## Recommended Next Work
|
||||
|
||||
1. Remove the sidebar dependency on `/chat-history`; load active conversation/session context instead.
|
||||
2. Replace raw `do_action( 'wp_aw_after_api_request', ... )` calls with one cost helper that always records provider/session/status.
|
||||
3. Standardize provider metadata response shape and add it to the remaining AI endpoints.
|
||||
4. Render provider/fallback metadata in the sidebar near cost/status feedback.
|
||||
5. Change `add_request()` default provider from `openrouter` to `unknown`.
|
||||
6. Start model registry consolidation after the chat/context and cost contracts are stable.
|
||||
7. Run the WordPress editor browser workflow pass.
|
||||
|
||||
## Current Verdict
|
||||
|
||||
The sixth-pass implementation is partially proper. It fixed the migrated session id edge case and the deprecated wrapper's misleading provider default.
|
||||
|
||||
It is not audit-clean yet. The biggest remaining risk is now cross-layer consistency: the backend moved toward session/provider/cost contracts, but the sidebar and cost ledger still have old assumptions that can make context history and provider cost reporting disagree with what actually happened.
|
||||
Reference in New Issue
Block a user